opensource/galaxy
3
0
Fork 0
mirror of https://github.com/galaxyproject/galaxy.git synced 2026-09-24 16:30:27 +08:00
Code Issues Packages Projects Releases Wiki Activity
77,490 Commits 69 Branches 243 Tags
db7395b42378fe8aec00a5d25e452e033bc19164
Commit Graph
1110 Commits
Author SHA1 Message Date
Nicola Soranzo e5f0a8c4eb Merge branch 'release_24.0' into release_24.1 2024-12-10 10:51:03 +00:00
mvdbeek 2d7667d958 Add regression test for #18927 2024-11-27 18:34:41 +01:00
mvdbeek 2bbae08220 Merge branch 'release_24.0' into release_24.1 2024-09-20 08:27:18 +02:00
mvdbeek e1d6c93a21 Fix discovered outputs with directory metadata and distributed object 
stores

The problem is that `object_store.is_private` would check in which
object store the dataset is stored, and that fails if we haven't written
to the object store yet (which is why extended metadata is a
workaround).

Fixes https://github.com/galaxyproject/galaxy/issues/17208
 2024-09-19 19:51:51 +02:00
M BerntandMarius van den Beek fc9d2f1dc7 remove assert 
Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>
 2024-09-19 09:39:08 +02:00
Matthias Bernt 1d9a61132c add test for updating quota description 2024-09-19 09:39:08 +02:00
Marius van den Beek 310cbd976c Merge pull request #18809 from mvdbeek/collect_job_metrics_also_when_job_failed 
[24.1] Collect job metrics also when job failed
 2024-09-13 17:40:18 +02:00
mvdbeek 4a5b3dd0bb Fix job metric collection when job fails 2024-09-13 14:36:12 +02:00
mvdbeek 14909829f7 Test metric collection when runner fails 2024-09-13 12:37:14 +02:00
mvdbeek 1dce8cf75b Add test for keeping picked dataset's extension in place 2024-09-10 18:50:12 +02:00
mvdbeek 302328f05f Merge branch 'release_24.0' into release_24.1 2024-08-09 15:57:24 +02:00
davelopez d9051005bc Remove unnecessary assert for file_size 
The file_size should be preserved in the database for traceability and other uses.
 2024-08-09 14:03:53 +02:00
mvdbeek e395ea7d6d Stabilize test_purge_while_job_running test 
In the extended_metadata + remote tool_evaluation_strategy the
commandline is templated as part of the job, so we need to give the test
more time to start running before we delete outputs (as then we can't
template the command line anymore).
 2024-07-04 17:35:57 +02:00
mvdbeek 33f27d5456 Merge branch 'release_24.0' into release_24.1 2024-07-01 18:23:55 +02:00
Marius van den Beek 444ae58736 Merge pull request #18446 from mvdbeek/when_upgrade_fix 
[24.0] Fix dropped when_expression on step upgrade
 2024-07-01 18:19:18 +02:00
Dannon Baker 6ec7521e1a Merge remote-tracking branch 'upstream/release_24.0' into release_24.1 2024-06-28 09:48:27 +02:00
mvdbeek ed8dc4b8e2 Fix dropped when_expression on workflow/tool_upgrade 
Fixes https://github.com/galaxyproject/galaxy/issues/18441
 2024-06-27 15:31:30 +02:00
mvdbeek 5050ccde3d Add test to make sure when_expression is not lost on refactoring 2024-06-27 15:31:29 +02:00
mvdbeek 7c6fc1800f Replace busybox:ubuntu-14.04 image with busybox:1.36.1-glibc 2024-06-21 16:58:21 +02:00
mvdbeek b8e2bfc9c7 Merge branch 'release_24.0' into release_24.1 2024-06-11 13:42:48 +02:00
mvdbeek f0e09cc8bd Fix up tests 2024-06-11 08:18:14 +02:00
mvdbeek a35033d821 Extend test to make sure disk path remains purged 2024-06-10 11:58:46 +02:00
mvdbeek 435a8cd37f Fix up test cases 
Add a sleep so we can delay output purging until command line is
templated. If we don't sleep we generate a command line where the
output path is just `''`. That needs another fix!
 2024-06-10 10:34:15 +02:00
John Chilton 821b31c3b2 More structured indexing for user data objects. 
I think I clung to the idea of having two ways to do this too long. I did a bunch of testing with the old way (exposing integer references based on numeric database primary keys). Having a config option that if changed would break everything exisiting is also a symptom of maybe me clinging too hard for too long.

The result of dropping this option is a much cleaner API schema, simpler API objects, and better typing throughout. We can also be more certain of how things entering the Vault are stored - I think being more strucutured about this is good.

Ultimately, the future facing stuff (e.g. OAuth 2.0) is going to require UUIDs so that I can store things like refresh tokens in the store before the object has been fully created.
 2024-06-07 19:20:11 -04:00
davelopez 28e0a5b77e Fix linting unnecessary list literal 2024-06-07 11:51:26 +02:00
mvdbeek 49660628bb Merge branch 'release_24.0' into release_24.1 2024-06-06 10:18:25 +02:00
davelopez 62049e3803 Rename integration test base class 
To make the `check_test_class_names` linter happy
 2024-06-05 15:30:26 +02:00
davelopez 57f5247060 Add integration tests for exposing user information 2024-06-05 14:42:07 +02:00
mvdbeek 093dcc8d3d Merge branch 'release_24.0' into release_24.1 2024-05-28 14:34:20 +02:00
mvdbeek cc2f83dac0 Raise appropriate exception if accessing deleted input file 2024-05-24 18:26:18 +02:00
Nicola Soranzo 8cdca0f343 Revert some requests import changes 
introduced in https://github.com/galaxyproject/galaxy/pull/18003 .
In particular, when using requests in the tests to connect to Galaxy
or the ToolShed it's not needed to use the modified headers.

Also:
- Move some imports after conditional imports.
- Fix type annotations in `lib/tool_shed/test/base/populators.py`
- Fix typo bug in `test/unit/util/test_get_url.py`
 2024-05-23 15:55:56 +01:00
John Chilton f95da89b3d Implement file source templates 2024-05-20 12:18:47 -04:00
John Chilton d3991a696b Object Store creation templates. 2024-05-20 12:16:02 -04:00
John Chilton 2f43c32fd9 Merge pull request #18003 from mvdbeek/add_galaxy_to_user_agent 
Add galaxy to user agent
 2024-05-20 12:14:13 -04:00
John Chilton 9d2a45f58d Merge pull request #17540 from bwalkowi/add-onedata-objectstore 
Add onedata objectstore
 2024-05-19 16:41:00 -04:00
John Chilton 3c991d3b20 Fix irods test cases for code refactoring. 2024-05-14 08:45:59 -04:00
Bartosz Walkowicz 2d995bb178 Format new onedata related code 2024-05-06 11:50:00 +02:00
Bartosz Walkowicz 1f6b6a0827 Merge branch 'dev' of https://github.com/galaxyproject/galaxy into add-onedata-objectstore 2024-05-06 06:34:16 +02:00
John Chilton e03d162429 Update Azure infrastructure. 2024-05-03 13:57:58 -04:00
Nicola Soranzo 0629908541 Fix new UP031 errors from ruff 0.4.2 2024-04-29 00:09:38 +01:00
Nicola Soranzo 37a50cc39b Fix integration tests for Celery 5.4.0 
Fix the following error:

```
____ TestCeleryUserRateLimitIntegrationNoLimit.test_mock_pass_user_id_task _____

self = <integration.test_celery_user_rate_limit.TestCeleryUserRateLimitIntegrationNoLimit object at 0x7f978fc0e400>

    def test_mock_pass_user_id_task(self):
>       self._test_mock_pass_user_id_task(2, 3, self._user_rate_limit)

test/integration/test_celery_user_rate_limit.py:161:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <integration.test_celery_user_rate_limit.TestCeleryUserRateLimitIntegrationNoLimit object at 0x7f978fc0e400>
num_users = 2, num_calls = 3, tasks_per_user_per_sec = 0.0

        for user_results in results.values():
            last_task_end_time = start_time
            for result in user_results:
>               if result.date_done > last_task_end_time:
E               TypeError: can't compare offset-naive and offset-aware datetimes

test/integration/test_celery_user_rate_limit.py:110: TypeError
```

Also https://docs.python.org/3/library/datetime.html#datetime.datetime.utcnow
is deprecated since Python 3.12 .
 2024-04-25 18:16:39 +01:00
Nicola Soranzo 1d64120a00 Merge branch 'release_24.0' into dev 2024-04-25 14:32:52 +01:00
mvdbeek 857a14e292 Change wrong quota_source value from KeyError to ValueError 
Which is caught in https://github.com/galaxyproject/galaxy/blob/44782a44c14885df2d9d1bcb91698245bfb3aed8/lib/galaxy/managers/base.py#L1121-L1124

Fixes https://sentry.galaxyproject.org/share/issue/16a3342631224d8292ef792298175dea/:
```
KeyError: "Could not find key nul in object store keys [None, 'scratch']"
  File "starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "starlette_context/middleware/raw_middleware.py", line 92, in __call__
    await self.app(scope, receive, send_wrapper)
  File "starlette/middleware/base.py", line 189, in __call__
    with collapse_excgroups():
  File "contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "starlette/_utils.py", line 93, in collapse_excgroups
    raise exc
  File "starlette/middleware/base.py", line 191, in __call__
    response = await self.dispatch_func(request, call_next)
  File "galaxy/webapps/galaxy/fast_app.py", line 108, in add_x_frame_options
    response = await call_next(request)
  File "starlette/middleware/base.py", line 165, in call_next
    raise app_exc
  File "starlette/middleware/base.py", line 151, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "starlette/routing.py", line 74, in app
    response = await func(request)
  File "fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "fastapi/routing.py", line 193, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "starlette/concurrency.py", line 42, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
  File "anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "galaxy/webapps/galaxy/api/history_contents.py", line 465, in index
    items = self.service.index(
  File "galaxy/webapps/galaxy/services/history_contents.py", line 312, in index
    return self.__index_v2(trans, history_id, params, serialization_params, filter_query_params, accept)
  File "galaxy/webapps/galaxy/services/history_contents.py", line 972, in __index_v2
    filters = self.history_contents_filters.parse_query_filters_with_relations(filter_query_params, history_id)
  File "galaxy/managers/history_contents.py", line 525, in parse_query_filters_with_relations
    return super().parse_query_filters(query_filters)
  File "galaxy/managers/base.py", line 1087, in parse_query_filters
    return self.parse_filters(filter_params)
  File "galaxy/managers/base.py", line 1096, in parse_filters
    filter_ = self.parse_filter(attr, op, val)
  File "galaxy/managers/base.py", line 1116, in parse_filter
    orm_filter = self._parse_orm_filter(attr, op, val)
  File "galaxy/managers/history_contents.py", line 601, in _parse_orm_filter
    if (column_filter := get_filter(attr, op, val)) is not None:
  File "galaxy/managers/history_contents.py", line 595, in get_filter
    raise KeyError(f"Could not find key {val} in object store keys {list(ids.keys())}")
```
 2024-04-22 19:44:31 +02:00
Marius van den Beek bc248efe2a Merge pull request #17982 from galaxybot/dev_auto_update_dependencies 
Update Python dependencies
 2024-04-17 10:34:30 +02:00
Nicola Soranzo 1fef07cb24 Replace pytest-celery with `celery.contrib.pytest` plugin loading 
The new 1.0.0 version of pytest-celery will move towards a Docker-based
approach for smoke and production-like testing, bringing in a number of
dependencies that we don't currently need.
Since the pre-1.0.0 version of pytest-celery was just a shim for
``celery.contrib.pytest``, we can simply load the latter for API,
integration and Selenium tests.
 2024-04-16 17:16:52 +01:00
mvdbeek e8a3b8c0cd Create wrapper for requests with default user agent 2024-04-16 17:54:29 +02:00
davelopez 123904aced Adapt notification integration tests to work w/o tasks 2024-04-16 17:31:53 +02:00
davelopez 349c373a99 Run notification integration tests without celery 2024-04-16 17:06:20 +02:00
Nicola Soranzo 053ca5198e Enable `warn_unused_ignores` mypy option 2024-04-16 01:12:58 +01:00
John Chilton 5edf79ce0c tool error integration test 2024-04-11 10:09:05 -04:00