Commit Graph
30501 Commits
Author SHA1 Message Date
Nicola Soranzo e66bb0912d Drop last references to uWSGI 2022-03-22 14:31:08 +00:00
Dannon 48579cf698 Merge pull request #13288 from dannon/menu-cleanup
[WIP] Menu cleanup
2022-03-22 08:15:15 -04:00
Assunta DeSanto 499ed041fd removing Search & Mailing List from ToolShed Menu 2022-03-22 00:33:35 -04:00
Assunta DeSanto 30773a478e removing wiki line from toolshed menu 2022-03-21 11:15:15 -04:00
Assunta DeSanto b2278f842f reverting how to cite link to link on HUB 2022-03-21 09:15:41 -04:00
Assunta DeSanto ff8a036fc4 Merge branch 'dev' of https://github.com/galaxyproject/galaxy into menu-cleanup 2022-03-21 09:12:41 -04:00
mvdbeek a9420d24b5 Drop remaining references to uWSGI 2022-03-21 13:44:37 +01:00
Björn Grüning 492e560b65 Merge pull request #13519 from jmchilton/no_implicit_optional
Disable implicit optional types.
2022-03-21 15:11:24 +03:00
Marius van den Beek cbc4bc9f06 Merge pull request #13579 from mvdbeek/legacy_openapi
Add legacy routes to openapi schema
2022-03-21 12:12:31 +01:00
John Chilton 4e01cfe3eb Merge pull request #13562 from ic4f/dev_sa20_3
Misc. SQLAlchemy 2.0 upgrades
2022-03-20 18:37:17 -04:00
mvdbeek ef0b04c345 Add more tags 2022-03-20 14:26:28 +01:00
mvdbeek d8271307d8 Restore tags in customized openapi schema 2022-03-20 14:26:28 +01:00
mvdbeek 102a7eab63 Add apispec dependency 2022-03-20 14:26:28 +01:00
mvdbeek e5eed5cbe0 Include non-FastAPI routes in api/docs 2022-03-19 16:56:46 +01:00
James Taylor 9a34fba455 API Schema Generation.
Traverse all registered API routes, attempt to load schema information
if available in docstring, register with an `APISpec` instance, and
convert to YAML.
2022-03-19 15:08:56 +01:00
Nicola Soranzo 00f7522e7e Merge branch 'release_22.01' into dev 2022-03-19 01:57:33 +00:00
Nicola Soranzo 0075ecb275 Fix model discovery unit tests
that I broke when merging
https://github.com/galaxyproject/galaxy/pull/13552 forward.

Fix:

```
app = <galaxy.model.unittest_utils.data_app.GalaxyDataTestApp object at 0x7f0e00dd3640>, target = '/tmp/tmpyj5d8268', work_directory = '/tmp/tmp_a7e191v'

    def _import_directory_to_history(app, target, work_directory):
        sa_session = app.model.context

        u = model.User(email="collection@example.com", password="password")
        import_history = model.History(name="Test History for Import", user=u)

        sa_session = app.model.context
        sa_session.add_all([u, import_history])
        sa_session.flush()

        assert len(import_history.datasets) == 0

        import_options = store.ImportOptions(allow_dataset_object_edit=True)
>       import_model_store = store.get_import_model_store_for_directory(target, app=app, user=u, import_options=import_options, tag_handler=app.tag_handler.create_tag_handler_session())
E       AttributeError: 'GalaxyDataTestApp' object has no attribute 'tag_handler'

test/unit/data/model/test_model_discovery.py:235: AttributeError
```
2022-03-18 19:12:20 +00:00
Nicola Soranzo 5114b67902 Merge branch 'release_21.09' into release_22.01 2022-03-18 18:28:15 +00:00
Dannon 1946eaf9a9 Merge pull request #13572 from afgane/nonce-mismatch-hint
Add nonce mismatch hint to exception msg
2022-03-18 13:09:36 -04:00
Nicola Soranzo 13d0f92fc0 Merge pull request #13071 from mvdbeek/drop_wsgi_mode 2022-03-18 13:41:33 +00:00
John Chilton 316998dea1 Merge pull request #13573 from mvdbeek/fix_tool_shed_bootstrapping
[22.01] Fix tool shed bootstrapping
2022-03-18 09:11:53 -04:00
John Chilton 954c78f301 Disable implicit optional typing across the project. 2022-03-18 09:06:30 -04:00
mvdbeek 7aae40d1af Include unit test for sessionless export/import with tags
Seems like the only way you could previously include tags
was if you included them in the data fetch tool.
2022-03-18 12:16:17 +01:00
mvdbeek 558cad246e Return tag in get_tag_by_name function 2022-03-18 11:12:06 +01:00
mvdbeek 6454ef1aad Drop converting things for which we have no equivalent 2022-03-17 21:20:59 +01:00
mvdbeek dc75df3b0b Fall back to generic WebApplicationStack
if we're building a web app, but aren't using gunicorn. I don't think
this has a big impact currently, since you can only set base pools via
lib/galaxy/main.py. I think you could set up alternative logging based
on the web stack class though, and this seems cleaner.
2022-03-17 21:20:59 +01:00
mvdbeek 96879486e2 Drop unused application_stack argument 2022-03-17 21:20:59 +01:00
mvdbeek b688aaf545 Drop unused get_stack_facts 2022-03-17 21:20:59 +01:00
Nicola Soranzo fe88225a14 Fix typo 2022-03-17 19:01:44 +00:00
mvdbeek 3d7dde0f9b Fix tool shed /api/authenticate/baseauth/
Fixes:
```
AttributeError: 'UniverseApplication' object has no attribute 'user_manager'
  File "galaxy/web/framework/decorators.py", line 312, in decorator
    rval = func(self, trans, *args, **kwargs)
  File "galaxy/webapps/galaxy/api/authenticate.py", line 63, in get_api_key
    user = self.app.user_manager.get_user_by_identity(identity)
```
2022-03-17 19:33:18 +01:00
Sergey Golitsynskiy f59a1ace97 Wrap sql stms in text() (model/) 2022-03-17 12:41:47 -04:00
Enis Afgan fe0083fb2b Add nonce mismatch hint to exception msg 2022-03-17 12:23:23 -04:00
Simon Bray a362874f07 allow setting a workflow as (un)importable via the API 2022-03-17 17:20:21 +01:00
mvdbeek 92d4ffc77a Fix extended_metadata jobs with dce inputs
This broke when we started re-exporting jobs to update their command
line in https://github.com/galaxyproject/galaxy/pull/12459/commits/65e0ee213ae574bc0a45dfcc60e997863ff96a7c#diff-bfb854f344e56a25a86761f80a639de09add404a7a53ebde45afb908762fc1eaR161:

```
ERROR    galaxy.jobs:__init__.py:1836 problem importing job outputs. stdout [] stderr [
Traceback (most recent call last):
  File "metadata/set.py", line 1, in <module>
    from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 121, in set_metadata
    set_metadata_portable()
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 307, in set_metadata_portable
    export_store.export_job(job, include_job_data=False)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 1320, in export_job
    self.export_jobs([job], include_job_data=include_job_data)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 1336, in export_jobs
    job_attrs = job.serialize(self.security, self.serialization_options)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 339, in serialize
    return self._serialize(id_encoder, serialization_options)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 1447, in _serialize
    params_objects = remap(params_objects, remap_objects)
  File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/boltons/iterutils.py", line 1128, in remap
    visited_item = visit(path, key, value)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 1441, in remap_objects
    new_id = serialization_options.get_identifier_for_id(id_encoder, obj["id"])
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 323, in get_identifier_for_id
    raise NotImplementedError()
NotImplementedError
]
```

To test run `GALAXY_CONFIG_METADATA_STRATEGY=extended ./run_tests.sh
-api
lib/galaxy_test/api/test_jobs.py::JobsApiTestCase::test_dce_submission_security`
2022-03-17 10:28:06 +01:00
Marius van den Beek ce7d376416 Merge pull request #13553 from mvdbeek/fix_extended_metadata_jobs_with_dce_inputs
[22.01] Fix extended_metadata jobs with dce inputs
2022-03-17 08:45:03 +01:00
Dannon bbab98ab46 Merge pull request #13563 from galaxyproject/blankenberg-configfiles-galaxy.xsd
Fix for configfiles examples in galaxy.xsd
2022-03-16 21:04:36 -04:00
Daniel Blankenberg ad32279dbb Fix for configfiles examples in galaxy.xsd 2022-03-16 17:51:02 -04:00
Sergey Golitsynskiy 64655d09b7 Fix mypy error: ignore dynamic attr 2022-03-16 16:53:54 -04:00
Sergey Golitsynskiy 158b9f0a70 s/eagerload/joinedload (SQLAlchemy 2.0)
Ref: https://docs.sqlalchemy.org/en/14/orm/loading_relationships.html?highlight=eagerload#sqlalchemy.orm.eagerload
2022-03-16 16:48:33 -04:00
Sergey Golitsynskiy dbbbc6c498 Use class attr in load options (_contained_id_map)
TODO: database.actions can be fixed only after moving Dataset to
declarative mapping.
2022-03-16 16:32:54 -04:00
Sergey Golitsynskiy 823ba91110 Use class attr in load options (_subcontainer_id_map) 2022-03-16 16:16:39 -04:00
Sergey Golitsynskiy 20f6e8d7a1 Use text() to wrap sql 2022-03-16 15:16:06 -04:00
Sergey Golitsynskiy 0963a02fb9 Use class attr in load options (all_roles()) 2022-03-16 15:05:35 -04:00
Sergey Golitsynskiy c2bee38662 Use text() to wrap sql string
Fixes RemovedIn20Warning
Ref: https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#execute-method-more-strict-execution-options-are-more-prominent
2022-03-16 14:40:24 -04:00
Sergey Golitsynskiy 25985fc991 Use class attributes instead of strings in load options
Fixes multiple instances of RemovedIn20Warning
Joining / loading on relationships uses attributes, not strings
Ref: https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#orm-query-joining-loading-on-relationships-uses-attributes-not-strings
2022-03-16 14:24:58 -04:00
John Chilton 427847bcc4 Fix up some import "issues". 2022-03-16 14:22:50 -04:00
John Chilton d4b1b1625a Merge pull request #13476 from jmchilton/no_implicit_reexport
Disable implicit re-imports across the project.
2022-03-16 14:04:31 -04:00
Sergey Golitsynskiy 63e613436d Fix implicit exection and autocommit (both removed in 2.0)
Ref:
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#library-level-but-not-driver-level-autocommit-removed-from-both-core-and-ormo
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#implicit-and-connectionless-execution-bound-metadata-removed

Exposed by the following tests:

test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_hdas
test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_error_hda
test/unit/data/model/test_model_discovery.py::test_persist_target_library_dataset
test/unit/data/model/test_model_discovery.py::test_persist_target_library_folder
test/unit/data/model/test_model_discovery.py::test_persist_target_hdca
  /home/sergey/0dev/galaxy/_galaxy/dev/lib/galaxy/model/triggers/utils.py:8: RemovedIn20Warning: The DDLElement.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
    cmd.execute(bind=engine)

test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_hdas
test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_error_hda
test/unit/data/model/test_model_discovery.py::test_persist_target_library_dataset
test/unit/data/model/test_model_discovery.py::test_persist_target_library_folder
test/unit/data/model/test_model_discovery.py::test_persist_target_hdca
  /home/sergey/0dev/galaxy/_galaxy/dev/lib/galaxy/model/triggers/utils.py:8: RemovedIn20Warning: The Engine.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
    cmd.execute(bind=engine)
2022-03-16 12:16:43 -04:00
Sergey Golitsynskiy 41d7cf20e4 Merge pull request #13541 from ic4f/dev_13376_migration
Add source_metadata column to workflow table as alembic revision
2022-03-16 11:01:25 -04:00
mvdbeek cf359f0b87 Fix extended_metadata jobs with dce inputs
This broke when we started re-exporting jobs to update their command
line in https://github.com/galaxyproject/galaxy/pull/12459/commits/65e0ee213ae574bc0a45dfcc60e997863ff96a7c#diff-bfb854f344e56a25a86761f80a639de09add404a7a53ebde45afb908762fc1eaR161:

```
ERROR    galaxy.jobs:__init__.py:1836 problem importing job outputs. stdout [] stderr [
Traceback (most recent call last):
  File "metadata/set.py", line 1, in <module>
    from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 121, in set_metadata
    set_metadata_portable()
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 307, in set_metadata_portable
    export_store.export_job(job, include_job_data=False)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 1320, in export_job
    self.export_jobs([job], include_job_data=include_job_data)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 1336, in export_jobs
    job_attrs = job.serialize(self.security, self.serialization_options)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 339, in serialize
    return self._serialize(id_encoder, serialization_options)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 1447, in _serialize
    params_objects = remap(params_objects, remap_objects)
  File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/boltons/iterutils.py", line 1128, in remap
    visited_item = visit(path, key, value)
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 1441, in remap_objects
    new_id = serialization_options.get_identifier_for_id(id_encoder, obj["id"])
  File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 323, in get_identifier_for_id
    raise NotImplementedError()
NotImplementedError
]
```

To test run `GALAXY_CONFIG_METADATA_STRATEGY=extended ./run_tests.sh
-api
lib/galaxy_test/api/test_jobs.py::JobsApiTestCase::test_dce_submission_security`
2022-03-16 13:44:07 +01:00