Commit Graph
72633 Commits
Author SHA1 Message Date
mvdbeek 8d52714199 Add test for implicit map over conversion in workflow
This test fails because cut fails on the compressed input with
`cut: /private/var/folders/df/6xqpqpcd7h73b6jpx9t6cwhw0000gn/T/tmpu8gqd71c/tmpmae9egoz/tmpz8sonmgx/database/objects/6/9/8/dataset_69847aef-7b3f-4915-a8ae-394921dbe388.dat: Illegal byte sequence`
2024-04-09 19:44:36 +02:00
mvdbeek e35ea08c9b Merge branch 'release_23.1' into release_23.2 2024-04-09 15:57:34 +02:00
mvdbeek 796ed40335 Merge branch 'release_23.0' into release_23.1 2024-04-09 15:57:03 +02:00
Marius van den Beek 3abb181423 Merge pull request #17944 from nsoranzo/release_23.0_fix_17938
[23.] Fix output datatype when uncompressing a dataset with incorrect datatype
2024-04-09 14:49:04 +02:00
Nicola Soranzo 0611465e3b Fix output datatype when uncompressing a dataset with incorrect datatype
with the ``CONVERTER_gz_to_uncompressed`` tool.

Previously, if a user forced an incorrect datatype to a compressed input
dataset (e.g. `fastqsanger` instead of `fastqsanger.gz`), then the output
datatype assigned by this tool would be an invalid one (`fastqsan`).

Fix #17938.
2024-04-09 11:39:39 +01:00
Nicola Soranzo ad45629885 Merge pull request #17916 from mvdbeek/fix_cwl_test_update_script
[23.2] Adjust update_cwl_conformance_tests.sh for removed branch
2024-04-05 14:45:20 +01:00
mvdbeek bd846d9b2e Adjust update_cwl_conformance_tests.sh for removed branch 2024-04-05 13:49:31 +02:00
Marius van den Beek 173b326a76 Merge pull request #17901 from mvdbeek/fix_workflow_saving
[23.2] Fix saving workflows with freehand_comments only
2024-04-04 18:45:40 +02:00
mvdbeek b825f798ca Reduce for loop nesting 2024-04-04 16:14:55 +02:00
mvdbeek 023178db2f Fix saving workflows with freehand_comments only 2024-04-04 15:53:33 +02:00
Marius van den Beek a7398dfb66 Merge pull request #17875 from SaimMomin12/release_23.2
Added 4dn_pairs and 4dn_pairsam datatypes
2024-04-03 11:20:36 +02:00
Saim Momin 2b42faf2f2 Added 4dn_pairs and 4dn_pairsam datatypes 2024-04-02 11:07:44 +02:00
mvdbeek c88d28574a Merge branch 'release_23.1' into release_23.2 2024-03-30 00:22:24 +01:00
Marius van den Beek 95a3797eff Merge pull request #17868 from mvdbeek/parse_extensions_as_lowercase_and_stripped
[23.1] Normalize extensions when loading tool
2024-03-30 00:21:58 +01:00
mvdbeek 45c4675a47 Normalize extensions when loading tool
Fixes https://github.com/galaxyproject/galaxy/issues/17864
2024-03-29 22:15:34 +01:00
Marius van den Beek ea3025ddd1 Merge pull request #17854 from mvdbeek/fix_role_query
[23.2] Fix user login when duplicate UserRoleAssociation exists
2024-03-28 09:29:07 +01:00
mvdbeek 50d9d1f440 Fix user login when duplicate UserRoleAssociation exists
In https://github.com/galaxyproject/galaxy/commit/3fd5b0252f840dbb92aaebf31b9d21034b21c578 we changed the sqlalchemy query construct `one_or_none`
method to use the core statement `.scalar_one_or_none`, which bypasses
the identity map. Since there are a handful of legacy accounts with
duplicate UserRoleAssociation rows on usegalaxy.{org,eu} the
deduplication that occured via the entity map meant that no exception
was raised, while for the core level construct this happened.

Fortunately it's easy enough to simply add a distinct
statement to return to the old behavior.

Here's an ipython session to prove this:

"""
In [4]: ura = sa_session.query(UserRoleAssociation).filter_by(user_id=1, role_id=1).all()

In [5]: ura
Out[5]: [<galaxy.model.UserRoleAssociation(1) at 0x165809190>]

In [6]: new_ura = UserRoleAssociation(sa_session.get(User, 1), sa_session.get(Role, 1))

In [7]: sa_session.add(new_ura)

In [8]: sa_session.flush()

In [9]:         role = (
   ...:             sa_session.query(Role)
   ...:             .filter(
   ...:                 and_(
   ...:                     UserRoleAssociation.table.c.user_id == 1,
   ...:                     Role.id == UserRoleAssociation.table.c.role_id,
   ...:                     Role.type == Role.types.PRIVATE,
   ...:                 )
   ...:             )
   ...:             .one_or_none()
   ...:         )

In [10]:         stmt = select(Role).where(
    ...:             and_(
    ...:                 UserRoleAssociation.user_id == 1,
    ...:                 Role.id == UserRoleAssociation.role_id,
    ...:                 Role.type == Role.types.PRIVATE,
    ...:             )
    ...:         )
    ...:         role = sa_session.execute(stmt).scalar_one_or_none()
---------------------------------------------------------------------------
MultipleResultsFound                      Traceback (most recent call last)
Cell In[10], line 8
      1 stmt = select(Role).where(
      2     and_(
      3         UserRoleAssociation.user_id == 1,
   (...)
      6     )
      7 )
----> 8 role = sa_session.execute(stmt).scalar_one_or_none()

File ~/src/galaxy/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py:1225, in Result.scalar_one_or_none(self)
   1212 def scalar_one_or_none(self):
   1213     """Return exactly one scalar result or ``None``.
   1214
   1215     This is equivalent to calling :meth:`_engine.Result.scalars` and
   (...)
   1223
   1224     """
-> 1225     return self._only_one_row(
   1226         raise_for_second_row=True, raise_for_none=False, scalar=True
   1227     )

File ~/src/galaxy/.venv/lib/python3.11/site-packages/sqlalchemy/engine/result.py:614, in ResultInternal._only_one_row(self, raise_for_second_row, raise_for_none, scalar)
    612     if next_row is not _NO_ROW:
    613         self._soft_close(hard=True)
--> 614         raise exc.MultipleResultsFound(
    615             "Multiple rows were found when exactly one was required"
    616             if raise_for_none
    617             else "Multiple rows were found when one or none "
    618             "was required"
    619         )
    620 else:
    621     next_row = _NO_ROW

MultipleResultsFound: Multiple rows were found when one or none was required

In [11]:         stmt = select(Role).where(
    ...:             and_(
    ...:                 UserRoleAssociation.user_id == 1,
    ...:                 Role.id == UserRoleAssociation.role_id,
    ...:                 Role.type == Role.types.PRIVATE,
    ...:             )
    ...:         ).distinct()
    ...:         role = sa_session.execute(stmt).scalar_one_or_none()
"""

I think that's what the sqlalchemy docs for [Query.one_or_none](https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.one_or_none) mean to
communicate with

> Returns None if the query selects no rows. Raises sqlalchemy.orm.exc.MultipleResultsFound if multiple object identities are returned, or if multiple rows are returned for a query that returns only scalar values as opposed to full identity-mapped entities.

Fixes https://github.com/galaxyproject/galaxy/issues/17848
2024-03-27 19:22:38 +01:00
Martin Cech a8727a32db Merge pull request #17842 from mvdbeek/docs_x_accel_redirect
[23.2] Proxy Access-Control-* headers when using x-accel-redirect
2024-03-26 19:32:26 +01:00
mvdbeek 2ce2686205 Proxy Access-Control-* headers when using x-accel-redirect 2024-03-26 10:54:29 +01:00
John Davis 0d70005d32 Merge pull request #17837 from jdavcs/23.2_releasenotesfix
[23.2] Fix error in release notes
2024-03-25 18:51:53 -04:00
John Davis 93ff028028 Fix error in notes 2024-03-25 17:38:27 -04:00
Nicola Soranzo 2319271662 Merge branch 'release_23.1' into release_23.2 2024-03-25 11:54:31 +00:00
mvdbeek 67ed6a7732 Merge branch 'release_23.0' into release_23.1 2024-03-25 10:11:20 +01:00
Marius van den Beek 77fc8494de Merge pull request #17828 from mvdbeek/fix_large_memory_usage_qza
[23.0] Fix excessive memory usage for npz files
2024-03-25 10:10:50 +01:00
mvdbeek 28c19ebadf Assert that at least one file in npz zipfile ends with .npy
from https://pydoc.dev/numpy/latest/numpy.lib.npyio.NpzFile.html:

> NpzFile is used to load files in the NumPy .npz data archive format. It assumes that files in the archive have a .npy extension, other files are ignored.
2024-03-25 09:20:50 +01:00
Matthias Bernt ed69e0f821 Npz sniffing: do not read the whole file
takes to much memory
2024-03-25 09:20:27 +01:00
Marius van den Beek 2ce39de081 Merge pull request #17824 from bernt-matthias/test-fix 2024-03-23 19:17:40 +01:00
Dannon Baker e3805c7bfb Refactoring for clarity. 2024-03-23 16:40:24 +01:00
Marius van den Beek 1fdbb121dc Create new history if user has no history AND for anonymous users 2024-03-23 16:40:13 +01:00
Dannon Baker c290412124 Fix creation and association of new histories w/ brand new single/remote user 2024-03-23 16:39:43 +01:00
mvdbeek 5d7a18e964 Merge branch 'release_23.1' into release_23.2 2024-03-18 11:31:35 +01:00
Marius van den Beek 4b3bc55067 Merge pull request #17736 from ahmedhamidawan/dataset_handle_missing_indexer
[23.1] Handle missing indexer for a dataset
2024-03-18 08:26:55 +01:00
Ahmed Awan d6914b7387 exception without the try/catch block
Also, improve clarity a bit...
2024-03-17 13:22:58 -05:00
Ahmed Awan b84f945909 [23.1] Handle missing indexer for a dataset
In case a dataset does not have an indexer, raise an exception.
Fixes the bug mentioned in https://github.com/galaxyproject/galaxy/pull/17639#issuecomment-1995990448
2024-03-14 15:19:24 -05:00
Marius van den Beek 6f0711adc9 Merge pull request #17657 from mvdbeek/limit_new_anon_histories
[23.2] Limit new anon histories
2024-03-13 11:50:10 +01:00
Marius van den Beek 27feb74017 Merge pull request #17678 from mvdbeek/no_runtime_value_connected_value_mixup
[23.2] Separate `ConnectedValue` from `RuntimeValue`
2024-03-13 09:47:56 +01:00
Marius van den Beek 52ca282eef Merge pull request #17679 from mvdbeek/fix_npz_sniffer
[23.2] Assert that at least one file in npz zipfile ends with .npy
2024-03-13 09:46:58 +01:00
mvdbeek 13a691c714 Explicitly exclude ConnectedValue from runtime input check 2024-03-12 19:23:23 +01:00
mvdbeek fd5fd152f3 Assert that at least one file in npz zipfile ends with .npy
from https://pydoc.dev/numpy/latest/numpy.lib.npyio.NpzFile.html:

> NpzFile is used to load files in the NumPy .npz data archive format. It assumes that files in the archive have a .npy extension, other files are ignored.
2024-03-12 18:26:12 +01:00
mvdbeek c5839eec37 Add test for history creation logic 2024-03-12 15:50:21 +01:00
Marius van den Beek 93eb541e21 Merge pull request #17674 from bernt-matthias/topic/npz-sniff
[23.2] Npz sniffing: do not read the whole file
2024-03-12 15:10:06 +01:00
Matthias Bernt 117120c0a2 Npz sniffing: do not read the whole file
takes to much memory
2024-03-12 14:07:47 +01:00
mvdbeek 5e080223a5 Only create histories if client includes session cookie 2024-03-11 15:45:54 +01:00
mvdbeek 781476858e Don't create unnecessary histories for anon users
The previous galaxy_session.user guard meant that anon
users would always get a new history.
2024-03-11 13:26:27 +01:00
Marius van den Beek b73d9e29fb Merge pull request #17641 from bernt-matthias/yaml-nested-assertions
[23.2] Yaml nested assertions: fix parsing
2024-03-11 11:49:59 +01:00
Marius van den Beek 8697be0cf5 Merge pull request #17634 from ahmedhamidawan/fix_tool_panel_workflow_bugs
[23.2] Fix tool panel workflow and favorites button bugs
2024-03-11 11:39:20 +01:00
Matthias Bernt 5a48f66c46 use asserts also in test 2024-03-09 15:10:52 +01:00
Matthias Bernt 9a17a04785 fix: nested assertions must be list 2024-03-08 19:32:04 +01:00
mvdbeek 71741fc8ab Merge branch 'release_23.1' into release_23.2 2024-03-08 15:09:13 +01:00
Marius van den Beek a3c14eaa61 Merge pull request #17639 from mvdbeek/fix_data_reault
[23.1] Fix DataResult type
2024-03-08 15:01:15 +01:00