Ref:
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#library-level-but-not-driver-level-autocommit-removed-from-both-core-and-ormohttps://docs.sqlalchemy.org/en/14/changelog/migration_20.html#implicit-and-connectionless-execution-bound-metadata-removed
Exposed by the following tests:
test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_hdas
test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_error_hda
test/unit/data/model/test_model_discovery.py::test_persist_target_library_dataset
test/unit/data/model/test_model_discovery.py::test_persist_target_library_folder
test/unit/data/model/test_model_discovery.py::test_persist_target_hdca
/home/sergey/0dev/galaxy/_galaxy/dev/lib/galaxy/model/triggers/utils.py:8: RemovedIn20Warning: The DDLElement.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
cmd.execute(bind=engine)
test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_hdas
test/unit/data/model/test_model_discovery.py::test_model_create_context_persist_error_hda
test/unit/data/model/test_model_discovery.py::test_persist_target_library_dataset
test/unit/data/model/test_model_discovery.py::test_persist_target_library_folder
test/unit/data/model/test_model_discovery.py::test_persist_target_hdca
/home/sergey/0dev/galaxy/_galaxy/dev/lib/galaxy/model/triggers/utils.py:8: RemovedIn20Warning: The Engine.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
cmd.execute(bind=engine)
I don't think this tool is really used much, but it is loaded in
`tool_conf.xml`.
https://github.com/galaxyproject/galaxy/pull/11877 added type
annotations, and that's a syntax error on 2.7
The alternative would be to fix the rest of the tool for 3.x,
but I think we're just wasting our time there and we shouldn't
touch this again.
This broke when we started re-exporting jobs to update their command
line in https://github.com/galaxyproject/galaxy/pull/12459/commits/65e0ee213ae574bc0a45dfcc60e997863ff96a7c#diff-bfb854f344e56a25a86761f80a639de09add404a7a53ebde45afb908762fc1eaR161:
```
ERROR galaxy.jobs:__init__.py:1836 problem importing job outputs. stdout [] stderr [
Traceback (most recent call last):
File "metadata/set.py", line 1, in <module>
from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 121, in set_metadata
set_metadata_portable()
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 307, in set_metadata_portable
export_store.export_job(job, include_job_data=False)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 1320, in export_job
self.export_jobs([job], include_job_data=include_job_data)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 1336, in export_jobs
job_attrs = job.serialize(self.security, self.serialization_options)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 339, in serialize
return self._serialize(id_encoder, serialization_options)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 1447, in _serialize
params_objects = remap(params_objects, remap_objects)
File "/home/runner/work/galaxy/galaxy/galaxy root/.venv/lib/python3.7/site-packages/boltons/iterutils.py", line 1128, in remap
visited_item = visit(path, key, value)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 1441, in remap_objects
new_id = serialization_options.get_identifier_for_id(id_encoder, obj["id"])
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/__init__.py", line 323, in get_identifier_for_id
raise NotImplementedError()
NotImplementedError
]
```
To test run `GALAXY_CONFIG_METADATA_STRATEGY=extended ./run_tests.sh
-api
lib/galaxy_test/api/test_jobs.py::JobsApiTestCase::test_dce_submission_security`
xref: https://github.com/galaxyproject/galaxy/issues/13551, fixes
```
galaxy.jobs ERROR 2022-03-15 00:21:29,515 [pN:main,p:2180,tN:LocalRunner.work_thread-3] problem importing job outputs. stdout [] stderr [
Traceback (most recent call last):
File "metadata/set.py", line 1, in <module>
from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata()
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 121, in set_metadata
set_metadata_portable()
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/metadata/set_metadata.py", line 291, in set_metadata_portable
collect_dynamic_outputs(job_context, output_collections)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/job_execution/output_collect.py", line 147, in collect_dynamic_outputs
persist_elements_to_hdca(job_context, elements, hdca, collector=DEFAULT_DATASET_COLLECTOR)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/discover.py", line 719, in persist_elements_to_hdca
filenames,
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/discover.py", line 292, in populate_collection_elements
final_job_state=final_job_state,
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/discover.py", line 354, in _populate_elements
self.add_tags_to_datasets(datasets=element_datasets["datasets"], tag_lists=element_datasets["tag_lists"])
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/discover.py", line 576, in add_tags_to_datasets
self.tag_handler.add_tags_from_list(user, dataset, tags, flush=False)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 61, in add_tags_from_list
return self.set_tags_from_list(user, item, new_tags_set, flush=flush)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 75, in set_tags_from_list
self.apply_item_tags(user, item, unicodify(new_tags_str, "utf-8"), flush=flush)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 207, in apply_item_tags
self.apply_item_tag(user, item, name, value, flush=flush)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 179, in apply_item_tag
tag = self._get_or_create_tag(lc_name)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 281, in _get_or_create_tag
tag = self._create_tag(scrubbed_tag_str)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 246, in _create_tag
tag = self._create_tag_instance(tag_name)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 428, in _create_tag_instance
tag = super()._create_tag_instance(tag_name)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/tags.py", line 259, in _create_tag_instance
Session = sessionmaker(self.sa_session.bind)
AttributeError: 'NoneType' object has no attribute 'bind'
]
Traceback (most recent call last):
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/jobs/__init__.py", line 1828, in finish
import_model_store.perform_import(history=job.history, job=job)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 209, in perform_import
datasets_attrs = self.datasets_properties()
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 930, in datasets_properties
datasets_attrs = load(open(datasets_attrs_file_name))
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp19s6x278/tmpt0rr8vmw/tmp2mjq6ysx/database/job_working_directory1/000/7/metadata/outputs_populated/datasets_attrs.txt'
galaxy.jobs.runners ERROR 2022-03-15 00:21:29,516 [pN:main,p:2180,tN:LocalRunner.work_thread-3] (7/) Job wrapper finish method failed
Traceback (most recent call last):
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/jobs/runners/__init__.py", line 594, in _finish_or_resubmit_job
job_stderr=job_stderr,
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/jobs/__init__.py", line 1828, in finish
import_model_store.perform_import(history=job.history, job=job)
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 209, in perform_import
datasets_attrs = self.datasets_properties()
File "/home/runner/work/galaxy/galaxy/galaxy root/lib/galaxy/model/store/__init__.py", line 930, in datasets_properties
datasets_attrs = load(open(datasets_attrs_file_name))
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp19s6x278/tmpt0rr8vmw/tmp2mjq6ysx/database/job_working_directory1/000/7/metadata/outputs_populated/datasets_attrs.txt'
```
Sergey Golitsynskiy