mvdbeek 90cff585a4 Handle MaxDiscoveredFilesExceededError gracefully for dynamic collections ...

Tools whose only discoverable outputs are dynamic collections (no primary
assign_primary_output dataset) were raising MaxDiscoveredFilesExceededError
from inside collect_dynamic_outputs, where a blanket `except Exception`
swallowed it via `log.exception`, shipping a traceback to Sentry and
leaving the outer handlers in set_metadata_portable / MinimalJobWrapper
without the error to record in job_messages.

Catch MaxDiscoveredFilesExceededError in collect_dynamic_outputs, mark the
collection as population-failed, register it with the job context so the
FAILED state is serialized through the extended-metadata export store, and
re-raise so the outer handlers can record a max_discovered_files entry in
job_messages. Log at warning level instead of error in the outer handlers.

In set_metadata_portable the pre-_finalize export_job was redundant
(immediately overwritten by _finalize) so drop it; add a single
post-_finalize export_job in the error path because _finalize cannot find
the job through included_datasets / included_collections when discovery
failed before anything was added to them.

Fixes #22394

2026-04-07 11:23:23 +02:00

..

chained_dyndest_rules

Add type annotations to job handling code

2025-10-25 13:52:24 +01:00

delay_rules

…

job_resource_rules

Add integration test for ignoring invalid resource request in tool state

2023-08-22 15:29:03 +02:00

mock_biotools_content/data

Parse edam operation and topic data from bio.tools if available.

2021-09-07 14:29:38 -04:00

objectstore

Make objectstore cache downloads atomic to fix flaky composite_output test

2026-03-30 16:16:45 +02:00

objectstore_selection_rules

Add type annotations to job handling code

2025-10-25 13:52:24 +01:00

oidc

Add tests for oidc usernames

2026-01-26 19:43:38 +05:30

panel_views_1

Exclude multiple_versions from test_global_filters_on_integrated_panel

2023-11-16 15:08:21 +01:00

resubmission_rules

Add type annotations to job handling code

2025-10-25 13:52:24 +01:00

test_visualization_plugins/jupyterlite/static

Remove currently unused param option

2026-01-27 19:43:03 +03:00

webdav

…

__init__.py

…

chained_dyndest_job_conf.xml

…

conftest.py

Replace pytest-celery with `celery.contrib.pytest` plugin loading

2024-04-16 17:16:52 +01:00

delay_job_conf.yml

Allow specifying useful classes of tools for mapping in job conf.

2021-07-14 13:46:56 -04:00

dockerized_job_conf.yml

Test that tmp_dir gets mounted correctly

2023-08-09 15:34:39 +02:00

dummy_job_resource_parameters.xml

Add integration test for ignoring invalid resource request in tool state

2023-08-22 15:29:03 +02:00

embedded_pulsar_docker_job_conf.yml

Drop more uWSGI legacy things

2022-03-15 21:32:00 +01:00

embedded_pulsar_job_conf.yml

Add test for proposed prod-like deployment

2025-09-21 17:03:17 +02:00

embedded_pulsar_metadata_extended_job_conf.yml

Disable conda_auto_init in embedded_pulsar_metadata_extended_job_conf.yml

2023-11-21 17:44:37 +01:00

embedded_pulsar_metadata_job_conf.yml

Allow specifying useful classes of tools for mapping in job conf.

2021-07-14 13:46:56 -04:00

embedded_pulsar_none_job_conf.yml

Test Pulsar integration with default_file_action: none

2026-01-13 20:00:56 +01:00

embedded_pulsar_singularity_job_conf.yml

Add `--no-mount tmp` for singularity destination in tests

2023-02-28 18:49:30 +00:00

embedded_pulsar_tpv_job_conf.yml

Match last version of https://github.com/galaxyproject/total-perspective-vortex/pull/166

2025-09-30 21:09:33 +01:00

fallback_container_resolver.yml

add test for loading a per destination container resolver config

2023-04-02 10:39:44 +02:00

file_sources_conf_remote_files.yml

Add url_regex matching for DRS filesources and add integration test

2023-06-14 19:27:31 +05:30

file_sources_conf_vault.yml

Add new secret type to user preferences that is not sent to client

2021-12-01 10:48:36 +05:30

file_sources_conf.yml

Restore original filename for file_source_conf_webdav.yml

2023-06-14 19:27:31 +05:30

handler_template_job_conf.xml

…

io_injection_job_conf.yml

…

job_resource_error_recovery_job_conf.yml

Add integration test for ignoring invalid resource request in tool state

2023-08-22 15:29:03 +02:00

refgenie.yml

add more tests

2021-03-25 14:01:17 +01:00

resubmission_default_job_conf.yml

Allow specifying useful classes of tools for mapping in job conf.

2021-07-14 13:46:56 -04:00

resubmission_dynamic_job_conf.xml

Allow specifying useful classes of tools for mapping in job conf.

2021-07-14 13:46:56 -04:00

resubmission_job_conf.yml

Allow specifying useful classes of tools for mapping in job conf.

2021-07-14 13:46:56 -04:00

resubmission_job_resource_parameters_conf.xml

…

resubmission_pulsar_job_conf.xml

Lower pulsar transport_timeout param instead of extending wait

2025-03-17 10:21:43 +01:00

resubmission_runners.py

Code cleanups from pyupgrade

2025-07-31 11:17:22 +01:00

resubmission_small_memory_job_conf.xml

…

resubmission_small_memory_resubmission_to_large_job_conf.xml

…

resubmission_tool_detected_always_error_job_conf.xml

…

resubmission_tool_detected_resubmit_job_conf.xml

…

sets_tmp_dir_expression_job_conf.xml

Fix another integration test

2021-02-25 13:49:58 +01:00

sets_tmp_dir_to_true_job_conf.xml

…

simple_job_conf.xml

…

singularity_job_conf.yml

require_container in tests

2023-06-20 10:28:21 +02:00

test_agents.py

Move inline imports to module level in agent tests

2026-02-11 14:28:11 -05:00

test_apache_nginx_sendfile.py

Merge remote-tracking branch 'upstream/release_22.05' into dev

2022-11-16 12:07:04 +01:00

test_async_downloads.py

Replace `unittest.TestCase` in API, framework, performance, Selenium and ToolShed tests

2022-11-07 15:44:01 +00:00

test_celery_fetch_quota.py

Enforce storage quota for Celery-based data fetch jobs

2026-02-26 15:29:33 +01:00

test_celery_tasks.py

Merge branch 'release_23.2' into dev

2024-01-18 12:23:00 +01:00

test_celery_user_rate_limit.py

Code cleanups from pyupgrade

2025-07-31 11:17:22 +01:00

test_chained_dynamic_destinations.py

Use test_driver.mkdtemp systematically

2026-01-01 15:08:26 +01:00

test_cli_runners.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_coexecution.py

Move more setup code into handle_galaxy_config_kwds which has test_driver configured

2026-01-01 15:08:26 +01:00

test_config_options_users.py

Fix Integeration typo

2022-11-23 01:20:00 +00:00

test_config_schema.py

Replace `unittest.TestCase` with pytest-based partial re-implementation

2022-09-27 03:34:58 +01:00

test_configuration_decode.py

Replace `unittest.TestCase` with pytest-based partial re-implementation

2022-09-27 03:34:58 +01:00

test_container_resolvers.py

Drop support for Python 3.9

2026-01-19 18:39:16 +00:00

test_containerized_jobs.py

Merge branch 'release_25.1' into release_26.0

2026-02-24 13:12:42 +00:00

test_credentials.py

Refactors credential test suite to use CredentialsPopulator

2026-02-19 14:37:53 +01:00

test_data_manager_refgenie.py

Update to black 2024 stable style

2024-01-30 18:09:51 +00:00

test_data_manager_workflow_bundle.py

Add data manager bundle workflow integration test

2024-02-16 15:53:16 +01:00

test_data_manager.py

Update to black 2024 stable style

2024-01-30 18:09:51 +00:00

test_dataset_copy_metadata_files.py

Fix copying metadata to copied job outputs

2023-11-10 18:03:45 +01:00

test_dataset_hashing.py

Calculate hash for new non-deferred datasets when finishing a job

2024-11-21 15:55:14 +00:00

test_datatype_upload.py

Merge pull request #17106 from tuncK/tarxz

2024-03-01 11:03:36 -05:00

test_default_permissions.py

Use `history_id` fixture instead of instance attribute. More type annotations

2022-11-23 04:04:51 +00:00

test_drs_compact_identifiers.py

Add DRS compact identifier support

2026-01-27 08:46:10 -05:00

test_dynamic_edam_loading.py

Replace `unittest.TestCase` with pytest-based partial re-implementation

2022-09-27 03:34:58 +01:00

test_edam_toolbox.py

change api/tool_panels from dict of views to dict containing

2023-10-26 12:50:31 -05:00

test_error_report.py

tool error integration test

2024-04-11 10:09:05 -04:00

test_extended_metadata_mapping.py

Numerous fixes for tool input format "21.01" parameters.

2024-10-30 10:52:08 -04:00

test_extended_metadata_outputs_to_working_directory.py

Fix and extend extended metadata outputs to working directory tests

2023-09-12 13:07:10 +02:00

test_extended_metadata.py

Move tests into TestExtendedMetadataIntegration

2026-03-18 16:29:41 +01:00

test_fail_job_tool_unavailable.py

YamlToolSource parser fixes

2025-05-12 10:54:24 +02:00

test_flush_per_n_datasets.py

Rename classes derived from `IntegrationInstance` which are not test cases

2024-02-16 10:53:04 +00:00

test_galaxy_interactor.py

Format Python code with black 23.1.0

2023-02-01 23:51:03 +00:00

test_genomes.py

Drop support for Python 3.8

2025-02-25 19:20:46 +00:00

test_handler_assignment_methods.py

Format Python code with black 23.1.0

2023-02-01 23:51:03 +00:00

test_history_archiving.py

Deduplicate history archive test additions

2025-12-18 12:38:55 -05:00

test_history_import_export.py

Type annotation fixes for mypy 1.16.0

2025-06-10 02:22:18 +01:00

test_interactivetools_api.py

Move more setup code into handle_galaxy_config_kwds which has test_driver configured

2026-01-01 15:08:26 +01:00

test_job_cache.py

Refactor to decrease duplication

2026-01-09 11:44:41 +01:00

test_job_environments.py

Use test_driver.mkdtemp systematically

2026-01-01 15:08:26 +01:00

test_job_files_tus.py

Allow job files to consume TUS uploads

2024-01-16 10:03:02 -05:00

test_job_files.py

Code cleanups from pyupgrade

2025-07-31 11:17:22 +01:00

test_job_outputs_to_working_directory.py

Test `from_work_dir="dir"` feature

2025-10-01 09:01:31 +02:00

test_job_recovery.py

Even more typing

2022-11-28 04:50:00 +00:00

test_job_resource_error_recovery.py

Add integration test for ignoring invalid resource request in tool state

2023-08-22 15:29:03 +02:00

test_job_resubmission.py

fix test

2025-03-26 18:07:57 +01:00

test_kubernetes_runner.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_landing_requests.py

Use correct dataset_populator method after rebase

2026-01-21 10:29:50 +01:00

test_legacy_store_by.py

Rename classes derived from `IntegrationInstance` which are not test cases

2024-02-16 10:53:04 +00:00

test_local_job_cancellation.py

Use model classes from `galaxy.model instead of app.model` object

2025-02-26 12:41:40 +00:00

test_materialize_dataset_instance_tasks.py

Calculate hash for new non-deferred datasets when finishing a job

2024-11-21 15:55:14 +00:00

test_max_discovered_files.py

Handle MaxDiscoveredFilesExceededError gracefully for dynamic collections

2026-04-07 11:23:23 +02:00

test_metadata_strategy.py

Test case to demonstrate disk usage update for work dir outputs is broken with celery_extended.

2023-05-18 16:01:14 +02:00

test_model_store_scripts.py

Replace `unittest.TestCase` with pytest-based partial re-implementation

2022-09-27 03:34:58 +01:00

test_notifications.py

Adjust API tests

2025-09-23 22:16:20 +02:00

test_page_revision_json_encoding.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_panel_views.py

Merge branch 'release_23.1' into dev

2023-11-21 09:36:51 +01:00

test_plugins.py

Fix invalid function schema error for tools with None parameters

2026-03-10 14:18:12 +01:00

test_prefix_handling.py

Various fixes to path prefix handling

2023-05-05 16:38:17 +02:00

test_pulsar_embedded_containers.py

Use `history_id` fixture instead of instance attribute. More type annotations

2022-11-23 04:04:51 +00:00

test_pulsar_embedded_copy_working.py

…

test_pulsar_embedded_extended_metadata.py

Test sessionless tag creation

2025-09-19 15:44:48 +02:00

test_pulsar_embedded_mq.py

Fix uploads in pulsar embedded test case

2025-02-26 14:55:08 +01:00

test_pulsar_embedded_none.py

Test Pulsar integration with default_file_action: none

2026-01-13 20:00:56 +01:00

test_pulsar_embedded_relay.py

Pulsar relay implementation

2025-11-28 11:12:12 +01:00

test_pulsar_embedded_remote_metadata.py

Fix tool stdio on pulsar and implement job stdio

2023-04-06 17:07:00 +02:00

test_pulsar_embedded.py

Add test that ensures irrelevant outputs are not copied back

2026-01-12 16:08:18 +01:00

test_purge_datasets.py

Mark dataset collections as deleted when purging a history

2026-03-31 09:45:45 +02:00

test_quota.py

remove assert

2024-09-19 09:39:08 +02:00

test_recalculate_user_disk_usage.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_remote_files_histories.py

Update to black 2024 stable style

2024-01-30 18:09:51 +00:00

test_remote_files_posix.py

Type annotation fixes for mypy 1.16.0

2025-06-10 02:22:18 +01:00

test_remote_files.py

Use test_driver.mkdtemp systematically

2026-01-01 15:08:26 +01:00

test_repository_operations.py

Remove transaction helper from test.integration

2025-01-23 11:39:24 -05:00

test_resolvers.py

Use test_driver.mkdtemp systematically

2026-01-01 15:08:26 +01:00

test_save_job_id_on_datasets.py

Removed deprecated `current` attribute of scoped_session

2025-02-21 09:29:41 +00:00

test_scripts_pgcleanup.py

Code cleanups from pyupgrade

2025-07-31 11:17:22 +01:00

test_scripts.py

Format code with black 25.1.0

2025-02-16 18:07:51 +00:00

test_shed_tool_tests.py

Update to black 2024 stable style

2024-01-30 18:09:51 +00:00

test_storage_cleaner.py

Code cleanups from pyupgrade

2025-07-31 11:17:22 +01:00

test_structured_dataset.py

Fix new UP031 errors from ruff 0.4.2

2024-04-29 00:09:38 +01:00

test_tool_data_bundles.py

Push extra files from object store cache to object store

2023-08-26 08:30:16 +02:00

test_tool_data_delete.py

Use test_driver.mkdtemp systematically

2026-01-01 15:08:26 +01:00

test_tool_submission_errors.py

Serialize message exceptions on execution error

2025-01-30 09:43:55 +01:00

test_upload_configuration_options.py

Tests for preserving column definitions in different metadata collection strategies.

2025-08-14 09:57:45 -04:00

test_user_defined_tool_job_conf.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_user_preferences.py

Add first user data access tests

2024-06-10 15:48:10 -04:00

test_users.py

Code cleanups from pyupgrade

2025-07-31 11:17:22 +01:00

test_vault_extra_prefs.py

rename parse_requirements_and_containers to parse_requirements,

2025-10-07 16:05:13 +02:00

test_vault_file_source.py

Use test_driver.mkdtemp systematically

2026-01-01 15:08:26 +01:00

test_web_framework_config.py

Allow CORS requests to api/workflow_landings

2024-10-10 09:40:28 +02:00

test_webdav.py

Restore original filename for file_source_conf_webdav.yml

2023-06-14 19:27:31 +05:30

test_work_queue_put_failure.py

use history_id fixture instead of instance variable with the same name

2022-11-23 01:19:59 +00:00

test_workflow_completion_hooks.py

Drop redundant completion hook test, improve remaining test

2026-01-21 14:44:55 +01:00

test_workflow_handler_configuration.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_workflow_invocation.py

Add test for workflow scheduling with unresolved 'auto' extension collection

2026-03-30 15:51:59 +02:00

test_workflow_refactoring.py

Switch to only MIT license

2026-03-06 19:05:20 +01:00

test_workflow_scheduling_options.py

Apply 2026 black style

2026-01-20 10:25:45 +00:00

test_workflow_sync.py

Format Python code with black 23.1.0

2023-02-01 23:51:03 +00:00

test_workflow_tasks.py

Implement workflow completion monitoring with extensible hooks

2026-01-21 12:21:08 +01:00

user_preferences_extra_conf.yml

Add integration tests for non-vault user preferences handling

2025-02-13 18:40:55 +01:00

workflow_handler_configuration_job_conf.xml

…