Commit Graph
66844 Commits
Author SHA1 Message Date
Nate Coraor 03d9eb63b8 Create version 23.0.4 v23.0.4 2023-06-30 17:13:57 -04:00
Marius van den Beek 34e758001c Merge pull request #16354 from natefoo/packages-pin-pydantic
[23.0] Pin pydantic in packages to <2
2023-06-30 21:52:20 +02:00
Nate Coraor 492b2ee64b Pin fastapi to <0.99 2023-06-30 15:20:22 -04:00
Nate Coraor 10fe1c885f Also pin pydantic in pyproject.toml 2023-06-30 14:55:58 -04:00
Nate Coraor ab1e0485d7 Pin pydantic in packages to <2 2023-06-30 14:51:18 -04:00
Marius van den Beek 9893cb3346 Merge pull request #16349 from mvdbeek/redact_name_in_private_user_role
[23.0] Redact private role name and description when purging user
2023-06-30 14:46:53 +02:00
mvdbeek 910677274e Redact private role name when purging user
Fixes
```
UniqueViolation: duplicate key value violates unique constraint "ix_role_name"
DETAIL:  Key (name)=(email@example.com) already exists.

  File "sqlalchemy/engine/base.py", line 1905, in _execute_context
    self.dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ix_role_name"
DETAIL:  Key (name)=(email@example.com) already exists.

[SQL: INSERT INTO role (create_time, update_time, name, description, type, deleted) VALUES (%(create_time)s, %(update_time)s, %(name)s, %(description)s, %(type)s, %(deleted)s) RETURNING role.id]
[parameters: {'create_time': datetime.datetime(2023, 6, 28, 22, 15, 51,
947910), 'update_time': datetime.datetime(2023, 6, 28, 22, 15, 51,
947918), 'name': 'email@example.com', 'description': 'Private Role for
email@example.com', 'type': <types.PRIVATE: 'private'>, 'deleted': False}]
(Background on this error at: https://sqlalche.me/e/14/gkpj)
  File "galaxy/web/framework/decorators.py", line 337, in decorator
    rval = func(self, trans, *args, **kwargs)
  File "galaxy/webapps/galaxy/controllers/user.py", line 142, in login
    return self.__validate_login(trans, payload, **kwd)
  File "galaxy/webapps/galaxy/controllers/user.py", line 199, in __validate_login
    trans.handle_user_login(user)
  File "galaxy/webapps/base/webapp.py", line 891, in handle_user_login
    self.app.security_agent.create_user_role(user, self.app)
  File "galaxy/model/security.py", line 738, in create_user_role
    self.get_private_user_role(user, auto_create=True)
  File "galaxy/model/security.py", line 765, in get_private_user_role
    return self.create_private_user_role(user)
  File "galaxy/model/security.py", line 748, in create_private_user_role
    user.attempt_create_private_role()
  File "galaxy/model/__init__.py", line 1105, in attempt_create_private_role
    with transaction(session):
  File "contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "galaxy/model/base.py", line 52, in transaction
    with session.begin():
  File "<string>", line 2, in begin
  File "sqlalchemy/util/deprecations.py", line 375, in warned
    return fn(*args, **kwargs)
  File "sqlalchemy/orm/session.py", line 1347, in begin
    trans = SessionTransaction(self, nested=nested)
  File "sqlalchemy/orm/session.py", line 545, in __init__
    self._take_snapshot(autobegin=autobegin)
  File "sqlalchemy/orm/session.py", line 662, in _take_snapshot
    self.session.flush()
  File "sqlalchemy/orm/session.py", line 3449, in flush
    self._flush(objects)
  File "sqlalchemy/orm/session.py", line 3589, in _flush
    transaction.rollback(_capture_exception=True)
  File "sqlalchemy/util/langhelpers.py", line 70, in __exit__
    compat.raise_(
  File "sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "sqlalchemy/orm/session.py", line 3549, in _flush
    flush_context.execute()
  File "sqlalchemy/orm/unitofwork.py", line 456, in execute
    rec.execute(self)
  File "sqlalchemy/orm/unitofwork.py", line 630, in execute
    util.preloaded.orm_persistence.save_obj(
  File "sqlalchemy/orm/persistence.py", line 245, in save_obj
    _emit_insert_statements(
  File "sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
    result = connection._execute_20(
  File "sqlalchemy/engine/base.py", line 1710, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
    return connection._execute_clauseelement(
  File "sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
    ret = self._execute_context(
  File "sqlalchemy/engine/base.py", line 1948, in _execute_context
    self._handle_dbapi_exception(
  File "sqlalchemy/engine/base.py", line 2129, in _handle_dbapi_exception
    util.raise_(
  File "sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "sqlalchemy/engine/base.py", line 1905, in _execute_context
    self.dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
```

when a user attempts to login after they previously purged their
account.
2023-06-30 12:46:35 +02:00
Marius van den Beek fe21c1af12 Merge pull request #16332 from mvdbeek/fix_for_default_when_statement
[23.0] Fix default when statement evaluation
2023-06-28 15:20:36 +02:00
Marius van den Beek 96bc25edc2 Merge pull request #16330 from mvdbeek/fix_folder_access_anonymous_user
[23.0] Fix folder access for anonymous user
2023-06-28 15:04:20 +02:00
mvdbeek 84a053d9a4 Fix default when statement evaluation
One can either do `${return inputs.when}` or the shorter
`$(inputs.when)`, not `${inputs.when}`. Fixes https://github.com/galaxyproject/galaxy/issues/16331
2023-06-28 13:47:21 +02:00
mvdbeek 6a2b095aba Fix folder access for anonymous user
if trans.user is None the value passed on to the model is None,
and that leads to
```
WouldBlock: null
  File "anyio/streams/memory.py", line 98, in receive
    return self.receive_nowait()
  File "anyio/streams/memory.py", line 93, in receive_nowait
    raise WouldBlock
EndOfStream: null
  File "starlette/middleware/base.py", line 78, in call_next
    message = await recv_stream.receive()
  File "anyio/streams/memory.py", line 118, in receive
    raise EndOfStream
ValidationError: 1 validation error for FileLibraryFolderItem
can_manage
  none is not an allowed value (type=type_error.none.not_allowed)
  File "starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "starlette_context/middleware/raw_middleware.py", line 93, in __call__
    await self.app(scope, receive, send_wrapper)
  File "starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
  File "galaxy/webapps/galaxy/fast_app.py", line 103, in add_x_frame_options
    response = await call_next(request)
  File "starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "starlette/routing.py", line 66, in app
    response = await func(request)
  File "fastapi/routing.py", line 241, in app
    raw_response = await run_endpoint_function(
  File "fastapi/routing.py", line 169, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "starlette/concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "galaxy/webapps/galaxy/api/folder_contents.py", line 114, in index
    return self.service.index(trans, folder_id, payload)
  File "galaxy/webapps/galaxy/services/library_folder_contents.py", line 83, in index
    self._serialize_library_dataset(trans, current_user_roles, tag_manager, content_item)
  File "galaxy/webapps/galaxy/services/library_folder_contents.py", line 168, in _serialize_library_dataset
    dataset_item = FileLibraryFolderItem(
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
    raise validation_error
```
as seen in https://sentry.galaxyproject.org/share/issue/3eec9cfa26744415b0c8d9885de9d273/
2023-06-28 12:24:06 +02:00
Martin Cech 1d4460a03a Merge pull request #15852 from anuprulez/patch-4
[23.0] Fix tool recommendations shown in a d3 js tree
2023-06-27 04:53:22 +02:00
mvdbeek 1ef440e92c Start work on 23.0.4.dev0 2023-06-26 09:32:37 +02:00
mvdbeek b4adbb4ec5 Create version 23.0.3 v23.0.3 2023-06-26 09:32:32 +02:00
Marius van den Beek 5c4fa2ea79 Merge pull request #16308 from nuwang/add_missing_init
[23.0] Missing init prevents models.py being bundled into tool_util
2023-06-26 07:27:29 +02:00
nuwang 4a66240f7b Missing init prevents models.py being bundled into tool_util 2023-06-25 20:46:20 +05:30
Martin Cech 353359fc20 Merge pull request #16271 from nuwang/forward_port_oidc_username_fix
[23.0] Forward port of slugify username received from oidc
2023-06-22 00:16:24 +02:00
Marius van den Beek bd6b2e83c3 Merge pull request #16284 from nsoranzo/release_23.0_fix_python3.11
[23.0] Fix converting Enum value to str for Python 3.11
2023-06-21 19:55:51 +02:00
Nicola Soranzo 25398bf818 Fix converting Enum value to str for Python 3.11
Fix the following traceback:

```
Traceback (most recent call last):
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/util/dictifiable.py", line 57, in to_dict
    visible_keys = self.__getattribute__(f"dict_{view}_visible_keys")
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'WorkflowInvocation' object has no attribute 'dict_InvocationSerializationView.element_visible_keys'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/web/framework/decorators.py", line 337, in decorator
    rval = func(self, trans, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/webapps/galaxy/api/workflows.py", line 891, in show_invocation
    return self.__encode_invocation(workflow_invocation, **kwd)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/webapps/galaxy/api/workflows.py", line 1108, in __encode_invocation
    return self.invocations_service.serialize_workflow_invocation(invocation, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/webapps/galaxy/services/invocations.py", line 202, in serialize_workflow_invocation
    as_dict = invocation.to_dict(view, step_details=step_details, legacy_job_state=legacy_job_state)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/model/__init__.py", line 7814, in to_dict
    rval = super().to_dict(view=view, value_mapper=value_mapper)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmpiustglxt/galaxy-dev/lib/galaxy/util/dictifiable.py", line 59, in to_dict
    raise Exception(f"Unknown Dictifiable view: {view}")
Exception: Unknown Dictifiable view: InvocationSerializationView.element
```

Broken in https://github.com/galaxyproject/galaxy/pull/13875 .
2023-06-21 17:33:45 +01:00
nuwang 6544f0d3f2 Slugify username received from oidc 2023-06-20 10:45:11 +05:30
John Chilton 0f4db94ef0 Merge pull request #16262 from bernt-matthias/xsd/sep
[23.0] xsd: add missing `sep` attribute for `has_n_columns`
2023-06-16 09:18:28 -04:00
Matthias Bernt 0f65705c55 add missing sep attribute for has_n_columns
forgotten here https://github.com/galaxyproject/galaxy/pull/12528
but I guess we do not need to backport further since it only affects
the tool linting xsd check
2023-06-16 11:24:18 +02:00
Nate Coraor 4a4e5abca7 Merge pull request #16244 from mvdbeek/load_data_tables_in_celery_worker
[23.0] Load data tables in Celery worker
2023-06-15 16:46:24 -04:00
Nate Coraor 6cde0781ac Merge pull request #16247 from natefoo/tool-data-bundle-file-path
[23.0] When importing tool data bundles, use the first loc file for the matching table
2023-06-15 10:14:41 -04:00
Dannon Baker 7f4dc26672 Amend schema to fix lint 2023-06-14 14:59:35 -04:00
Nate Coraor 0dfe8b44ad When importing tool data bundles, use the first loc file for the
matching table, unless an explicit file path is passed via the API.
2023-06-14 14:41:43 -04:00
mvdbeek 362bf76a0c Load data tables in Celery worker
This is needed so that data bundle imports will work.
The test case covering this will just receive the full integration app
and thus didn't have this problem.
Should fix:
```
[2023-06-13 21:33:38,899: ERROR/main] Task galaxy.import_data_bundle[0853f72f-c8ce-4e59-b6e8-25e208ce79f7] raised unexpected: AttributeError("'GalaxyManagerApplication' object has no attribute 'tool_data_tables'")
Traceback (most recent call last):
  File "/cvmfs/main.galaxyproject.org/venv/lib/python3.8/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/cvmfs/main.galaxyproject.org/venv/lib/python3.8/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/celery/__init__.py", line 163, in wrapper
    rval = app.magic_partial(func)(*args, **kwds)
  File "/cvmfs/main.galaxyproject.org/venv/lib/python3.8/site-packages/lagom/wrapping.py", line 27, in _bound_func
    bound_args, bound_kwargs = argument_updater(args, kwargs)
  File "lagom/container.py", line 350, in _update_args
  File "lagom/container.py", line 425, in _infer_dependencies
  File "lagom/container.py", line 265, in resolve
  File "lagom/container.py", line 392, in _reflection_build_with_err_handling
  File "lagom/container.py", line 408, in _reflection_build
  File "/cvmfs/main.galaxyproject.org/galaxy/lib/galaxy/managers/tool_data.py", line 114, in __init__
    self.tool_data_tables = app.tool_data_tables
AttributeError: 'GalaxyManagerApplication' object has no attribute 'tool_data_tables'
```
2023-06-14 08:06:24 +02:00
Marius van den Beek 8d1f9f8acf Merge pull request #16241 from mvdbeek/bump_galaxy_release_util_dep
[23.0] Bump galaxy-release-util version to 0.1.2
2023-06-13 19:32:57 +02:00
mvdbeek cf7c012491 Bump galaxy-release-util version to 0.1.2 2023-06-13 18:59:57 +02:00
mvdbeek 414abff510 Start work on 23.0.3.dev0 2023-06-13 17:58:19 +02:00
mvdbeek a7f0a365be Create version 23.0.2 v23.0.2 2023-06-13 17:58:13 +02:00
mvdbeek a08a74f1e3 Merge branch 'release_22.05' into release_23.0 2023-06-13 17:27:59 +02:00
Marius van den Beek 6d9b0fd20e Merge pull request #16239 from nuwang/fix_dropbox_import
[22.05] Fix dropbox import to support newer versions
2023-06-13 17:13:18 +02:00
Marius van den BeekandNicola Soranzo 8675451438 Comment on import fallback
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2023-06-13 17:13:04 +02:00
Marius van den Beek 2622b9dc51 Merge pull request #16240 from mvdbeek/galaxy-release-util-ci
[23.0] Use galaxy-release-util to upload python packages
2023-06-13 16:42:01 +02:00
Marius van den Beek cbc7b91035 Merge pull request #16237 from bernt-matthias/topic/lint-expect-failure-and-numoutputs
[23.0] linter: allow absent output counts in case of expected test failure
2023-06-13 15:58:17 +02:00
mvdbeek 787b8040c8 Use galaxy-release-util to upload python packages 2023-06-13 15:49:37 +02:00
nuwang cb412d1dd3 Fix dropbox import to support newer versions 2023-06-13 13:40:14 +05:30
Matthias Bernt 54e54bbfce linter: allow absent output counts in case of expected test failure
otherwise we contradict that failing tests must nut have outputs

fixes: https://github.com/galaxyproject/galaxy/issues/16235
2023-06-12 23:08:04 +02:00
Martin Cech 84d2ba8ee2 Merge pull request #16230 from mvdbeek/version_py_version_fix
[23.0] Fix next version number
2023-06-12 12:11:09 -04:00
mvdbeek 48573d3a1f Fix next version number 2023-06-10 11:22:42 +02:00
John Chilton f16be61b07 Merge pull request #16219 from mvdbeek/fix_tool_util_dependency_on_cheetah
[23.0] Don't fail CWL tool parsing when Cheetah not installed
2023-06-09 16:06:33 -04:00
mvdbeek ad5b83941b Don't fail cwl tool parsing when Cheetah not installed 2023-06-09 17:07:01 +02:00
Marius van den Beek a5ab850a9f Merge pull request #16215 from mvdbeek/avoid_deprecated_selenium_options
[23.0] Port selenium setup to non-deprecated selenium options
2023-06-09 14:29:55 +02:00
Marius van den Beek b2138ea390 Merge pull request #16216 from mvdbeek/rename_web_apps
[23.0] Rename galaxy-webapps back to galaxy-web-apps
2023-06-09 09:43:08 +02:00
mvdbeek 370a90d0c4 Fix Safari setup 2023-06-08 21:21:51 +02:00
Marius van den BeekandNuwan Goonasekera d210aab27f Make goog:loggingPrefs chrome specific
Co-authored-by: Nuwan Goonasekera <2070605+nuwang@users.noreply.github.com>
2023-06-08 21:08:04 +02:00
mvdbeek 5d517421bd Rename galaxy-webapps back to galaxy-web-apps
PyPI doesn't let us create projects with names that can be confused.
2023-06-08 20:48:03 +02:00
mvdbeek 5b5f224bd1 Port to non-deprecated selenium options
Should be compatible with new and old selenium package.
The last release on PyPI removed some deprecated interfaces,
which fails our packaging tests.
2023-06-08 20:40:50 +02:00
Marius van den Beek 9fcfedac31 Merge pull request #16214 from mvdbeek/point_release_deps_fixes_and_docs
[23.0] Point release deps fixes and docs
2023-06-08 18:34:59 +02:00