mvdbeek
b8e2bfc9c7
Merge branch 'release_24.0' into release_24.1
2024-06-11 13:42:48 +02:00
Marius van den Beek
96c9be32ce
Merge pull request #18342 from mvdbeek/fix_copying_purged_files
...
[24.0] Do not copy purged outputs to object store
2024-06-11 13:35:10 +02:00
John Davis
8351709800
Fix mypy after merge
2024-06-11 00:01:27 -04:00
John Davis
b2bbdcbb7f
Merge branch 'release_24.0' into release_24.1
2024-06-10 23:59:39 -04:00
John Davis
d0414c3d95
Fix check for anonymous
2024-06-10 18:45:38 -04:00
davelopez and Nicola Soranzo
9df23a64d3
Require session to list jobs instead of returning empty job list for anonymous
...
Co-authored-by: Nicola Soranzo <nicola.soranzo@earlham.ac.uk >
2024-06-10 16:33:47 +02:00
mvdbeek
3c7730c199
Delete purged outputs from object store when importing dataset for purged output
2024-06-10 16:21:22 +02:00
mvdbeek
50b7f8f8da
Do not overwrite purged or deleted states when importing datasets from job
2024-06-10 16:21:22 +02:00
mvdbeek
5ab3120270
Purge dataset if finishing job for purged output
2024-06-10 16:21:21 +02:00
davelopez and mvdbeek
a6b45e561c
Filter jobs by session instead of using the session current history for anonymous
...
Co-authored-by: mvdbeek <m.vandenbeek@gmail.com >
2024-06-10 13:34:27 +02:00
Marius van den Beek
2585d1e604
Merge pull request #18291 from jmchilton/cleanup_user_data_indexing
...
[24.1] More structured indexing for user data objects.
2024-06-10 13:27:26 +02:00
davelopez
af923d6b8f
Fix anonymous user job retrieval logic
...
For anonymous users, if a history_id is provided, do not override with the current session history and instead rely on the history accessibility
2024-06-10 12:33:11 +02:00
mvdbeek
c489eb20af
Set file_size and total_size to 0 for discovered but purged datasets
2024-06-10 12:10:53 +02:00
mvdbeek
fc42b61d30
Fix metadata setting on purged outputs
2024-06-10 11:55:36 +02:00
mvdbeek
eb2e318b3d
Elevate external metadata setting to warning
...
This is a tricky one. There could be systematic issues here that an
admin would want to fix, but it could also just be the case that a user
forced a wrong datatype. Ideally we'd probably tag this as job execution
issue ? It's not very different from a job error IMO.
2024-06-10 11:53:41 +02:00
mvdbeek
03eca7cc52
Fix handling of collecting discovered but purged outputs
2024-06-10 10:34:15 +02:00
mvdbeek
65e9605f4a
Don't fail _finish_or_resubmit if tool_stdout / tool_stderr not written
...
These are only written if the command actually ran, so we'd fail here
for instance if the outputs are deleted before the job had a chance to
run.
2024-06-08 20:09:18 +02:00
mvdbeek
a2fdcb3c72
Merge branch 'release_24.0' into release_24.1
2024-06-08 15:54:19 +02:00
mvdbeek
c6930fff72
Merge branch 'release_23.2' into release_24.0
2024-06-08 15:53:41 +02:00
Marius van den Beek
18d9bbba45
Merge pull request #18349 from mvdbeek/fix_dce_to_cwl
...
[23.2] Allow DCE as outer input to to_cwl
2024-06-08 15:43:01 +02:00
John Chilton
821b31c3b2
More structured indexing for user data objects.
...
I think I clung to the idea of having two ways to do this too long. I did a bunch of testing with the old way (exposing integer references based on numeric database primary keys). Having a config option that if changed would break everything exisiting is also a symptom of maybe me clinging too hard for too long.
The result of dropping this option is a much cleaner API schema, simpler API objects, and better typing throughout. We can also be more certain of how things entering the Vault are stored - I think being more strucutured about this is good.
Ultimately, the future facing stuff (e.g. OAuth 2.0) is going to require UUIDs so that I can store things like refresh tokens in the store before the object has been fully created.
2024-06-07 19:20:11 -04:00
John Davis
189184e2ae
Fix mypy error (surfaced after merge)
2024-06-07 17:03:50 -04:00
John Davis
bbb10cbf61
Merge branch 'release_24.0' into release_24.1
2024-06-07 14:48:42 -04:00
John Davis
d4a4b91ac4
Merge pull request #18352 from jmchilton/241_file_sources_anon
...
Fix #18316 (anonymous file sources)
2024-06-07 14:43:44 -04:00
John Chilton
7acf35f9a5
Fix #18316... I think.
2024-06-07 14:07:52 -04:00
Marius van den Beek
f38461bd82
Merge pull request #18333 from davelopez/24.0_fix_500_for_anonymous_job_listing
...
[24.0] Fix authentication error for anonymous users querying jobs
2024-06-07 17:45:54 +02:00
mvdbeek
dff6307e91
Allow DCE as outer input to to_cwl
2024-06-07 17:29:07 +02:00
Nicola Soranzo
0fb77c875e
Merge branch 'release_24.0' into release_24.1
2024-06-07 16:28:23 +01:00
Nicola Soranzo
62eadc87c5
Merge branch 'release_23.2' into release_24.0
2024-06-07 16:26:04 +01:00
Nicola Soranzo
3817559039
Merge branch 'release_23.1' into release_23.2
2024-06-07 16:24:43 +01:00
mvdbeek
5339fabbf3
Merge branch 'release_24.0' into release_24.1
2024-06-07 16:52:46 +02:00
mvdbeek
8e6e25d7fb
Don't push purged dataset contents to object store
...
This in particular needs a lot of new tests.
We will also need to actively purge datasets in the model store import
code, since users might have purged datasets while the job ran.
Again, more tests needed.
2024-06-07 15:20:45 +02:00
mvdbeek
a1d41ab0d6
Raise appropriate exception if require path parameter is missing
2024-06-07 15:20:45 +02:00
mvdbeek
c3a5d61d61
Don't include purged outputs in from_work_dir outputs
2024-06-07 15:20:44 +02:00
mvdbeek
87124a396a
Don't copy from_work_dir outputs to purged path in shell fragment
2024-06-07 15:20:44 +02:00
davelopez
67fd14046b
Fix typing issue in reused variable
2024-06-07 11:06:06 +02:00
davelopez and Nicola Soranzo
7caee0b751
Import from typing in preparation for more type annotations
...
Co-authored-by: Nicola Soranzo <nicola.soranzo@earlham.ac.uk >
2024-06-06 16:03:35 +02:00
davelopez
497ca2fc55
Fix mypy: now it catches this error thanks to the ProvidesUserContext type hint
2024-06-06 15:04:30 +02:00
davelopez
2e912729e7
Fix error for anonymous user listing jobs
2024-06-06 14:59:36 +02:00
Marius van den Beek and Nicola Soranzo
cf322eb531
restore line.strip()
...
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com >
2024-06-06 12:47:34 +02:00
Marius van den Beek
8df21c6ae0
Merge pull request #18325 from bgruening/update_config_24.1
...
[24.1] rebuild Galaxy config
2024-06-06 11:30:21 +02:00
mvdbeek
c49a9ef35f
Fix seek in slurm memory check
...
Fixes
https://sentry.galaxyproject.org/share/issue/e196a12836fe40da83c5e9e24c4c8329/ :
```
UnsupportedOperation: can't do nonzero end-relative seeks
File "galaxy/jobs/runners/slurm.py", line 217, in __check_memory_limit
f.seek(-2048, os.SEEK_END)
```
which has probably been broken since we moved to python 3.
2024-06-06 11:20:05 +02:00
mvdbeek
49660628bb
Merge branch 'release_24.0' into release_24.1
2024-06-06 10:18:25 +02:00
mvdbeek
dc145ed404
Merge branch 'release_23.2' into release_24.0
2024-06-06 10:17:56 +02:00
Marius van den Beek
bd4a5293aa
Merge pull request #18319 from mvdbeek/skip_history_creation_with_require_login_true
...
[23.2] Skip new history creation if user is anonymous and login is required
2024-06-06 10:14:07 +02:00
Marius van den Beek
7867b69f01
Merge pull request #18329 from davelopez/24.0_fix_users_api_serialization
...
[24.0] Fix users API serialization when listing users
2024-06-06 10:12:27 +02:00
mvdbeek
8ee3b902b3
Make repo permissions world-readable
2024-06-05 17:59:13 +02:00
mvdbeek
0122cdb16e
Merge branch 'release_24.0' into release_24.1
2024-06-05 17:43:18 +02:00
mvdbeek
7669eff263
Downgrade doi fetch error to debug
...
There's nothing an admin can do about, and it doesn't impact tool
functionality.
2024-06-05 16:23:38 +02:00
mvdbeek
7228c5aad1
Merge branch 'release_23.2' into release_24.0
2024-06-05 15:25:29 +02:00
mvdbeek