Commit Graph
53366 Commits
Author SHA1 Message Date
Marius van den Beek c5e00e49dd Merge pull request #11914 from Nerdinacan/history-audit
[21.05] Create history_audit table to avoid write deadlocks during history.update_time updates
2021-05-13 16:26:59 +02:00
Sergey Golitsynskiy 5db752a606 Merge pull request #11947 from almahmoud/k8s_runner_its
[21.05] ITs on K8S tweaks and SSL
2021-05-12 19:23:00 -04:00
almahmoud b0a7c770c8 Merge branch 'release_21.05' of https://github.com/galaxyproject/galaxy into alex_branch 2021-05-12 17:41:37 -04:00
almahmoud e1f517cf4b Remove separate conf files 2021-05-12 13:37:21 -04:00
Sergey Golitsynskiy 93af8ec780 Modify test to ensure correct rows are pruned 2021-05-12 12:14:27 -04:00
Marius van den Beek 25c04a4d30 Merge pull request #11963 from mvdbeek/switch_to_dateutil
[21.05] Use dateutil for python 3.6 compatibility
2021-05-12 17:24:20 +02:00
mvdbeek 56148b972c Unit test for history audit and pruning 2021-05-12 14:50:45 +02:00
mvdbeek e80e1a80d6 Convert timezone aware datetime to naive datetime
sqlalchemy DateTime columns are not timezone aware, so parse `since`
into timezone-aware datetime and then convert to naive datetime object
representing UTC, assuming history.update_time represents UTC time.
2021-05-12 11:57:45 +02:00
almahmoud be2967b24d Fix for subdomain ITs 2021-05-11 20:20:05 -04:00
almahmoud d0971cc1b5 Move configs to schema 2021-05-11 16:43:46 -04:00
mvdbeek 0a088624a6 Use dateutil for python 3.6 compatibility
datetime.fromisoformat was added in python 3.6
2021-05-11 21:46:41 +02:00
almahmoud 3fcceedaa0 Merge branch 'release_21.05' of https://github.com/galaxyproject/galaxy into alex_branch 2021-05-11 13:46:15 -04:00
mvdbeek 9e9343ccd2 Fix cleanup_datasets.py script 2021-05-11 18:34:30 +02:00
mvdbeek e0f0fa338e Add periodic history audit prune task
that runs when not using celery.
2021-05-11 18:34:30 +02:00
mvdbeek 4265dc4b62 Move history audit pruning to HistroyAudit class 2021-05-11 18:34:30 +02:00
mvdbeek 3d59717142 Add celery setup that regularly cleans audit_table 2021-05-11 18:34:30 +02:00
Mason Houtz 014a5b5241 added row triggers for older postgres versions 2021-05-11 18:34:29 +02:00
Mason Houtz deadd268c5 added sqlite-specific conflict clause to history audit table 2021-05-11 18:34:29 +02:00
Mason Houtz 5206b06d6c added compound primary key on audit table and conflict ignore clause on triggers 2021-05-11 18:34:29 +02:00
Mason Houtz 04889e4b40 Added distinct for statement triggers to account for large multi-row collection or dataset updates 2021-05-11 18:34:29 +02:00
mvdbeek e253e49a2f Drop comments 2021-05-11 18:34:29 +02:00
mvdbeek f9fb1085bc Allow grid sorting on class attributes if no mapped column 2021-05-11 18:34:29 +02:00
mvdbeek fc42afd5f1 Add microseconds to sqlite timestamp 2021-05-11 18:34:29 +02:00
mvdbeek a2e923fbb3 Add update_time to History class for mypy 2021-05-11 18:34:29 +02:00
Mason Houtz aa60fbd0b7 removed references to old update_time field 2021-05-11 18:34:29 +02:00
Mason Houtz 7c70eb53b9 deleted sqlite triggers first since sqlite does not support create or replace trigger syntax 2021-05-11 18:34:29 +02:00
Mason Houtz 7bdf0f8dd0 amended triggers to filter out updates with a NULL history_id 2021-05-11 18:34:29 +02:00
Mason Houtz 71656c0372 Implemented new trigger scheme that updates rows in an audit table to avoid deadlocking writes to history 2021-05-11 18:34:29 +02:00
mvdbeek 191a4b848e Merge branch 'release_21.01' into release_21.05 2021-05-11 18:28:04 +02:00
mvdbeek 2552ab47f1 Use f-strings in forward-merged code 2021-05-11 18:09:53 +02:00
mvdbeek 41aca51d2b Merge branch 'release_20.09' into release_21.01 2021-05-11 18:01:46 +02:00
Marius van den Beek f963b67a6e Merge pull request #11955 from mvdbeek/fix_ipv6_port_parsing
[20.09] Fix container port parsing if IPv6 hostname is ``::``
2021-05-11 17:59:47 +02:00
mvdbeek 01cdbd3ebd Fix port_parse rsplit usage 2021-05-11 16:40:57 +02:00
Marius van den BeekandNicola Soranzo 8acf0d16a8 Use different string formatting option
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2021-05-11 14:57:21 +02:00
Marius van den BeekandNicola Soranzo d69a79f29e Use generator expression instead of list comprehension
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2021-05-11 14:56:50 +02:00
Marius van den BeekandNicola Soranzo 4db3af8e5d Open container_monitor_exception.txt in write mode
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2021-05-11 14:54:56 +02:00
Marius van den BeekandNicola Soranzo 7b9d2dbeb9 Skip :: hostname only
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2021-05-11 14:54:28 +02:00
mvdbeek 75f5877951 Log exceptions IT port monitor fails and stop job
That should make it significantly easier to debug issues when the
container monitor dies unexpectedly (which I sort of expect in the
future given that we parse output from the docker CLI).
2021-05-10 17:43:46 +02:00
mvdbeek a1ececdcf8 Fix contianer port parsing if ::: in port line
:: is an unspecified ipv6 address and just recently appeared in the
docker port output.
2021-05-10 17:37:29 +02:00
John Chilton 5c17d7ce57 Merge pull request #11953 from mvdbeek/include_more_info_on_staging_exception
[21.05] Include more info on test input staging exception
2021-05-10 09:41:50 -04:00
Nicola Soranzo 306e774a86 Merge pull request #11951 from mvdbeek/fix_anonymous_user_job_walltime_query 2021-05-10 10:57:10 +01:00
mvdbeek c19a98b8e3 Include tool_id, exit_code and stderr in Exception
That'll be useful to figure out what's going on when data staging fails.
2021-05-07 21:44:25 +02:00
Marius van den Beek 897fde2171 Merge pull request #11950 from mvdbeek/revert_attach_to_pool_default
[21.05] Don't let webless handlers join job-handlers pool automatically
2021-05-07 14:39:50 +02:00
mvdbeek bdc366c64f Fix pause message when over walltime 2021-05-07 13:20:59 +02:00
Maiken Pedersen 958e52642b Fix job query filter in walltime check 2021-05-07 13:19:29 +02:00
mvdbeek bb3e9bffab Don't set default pool 2021-05-07 11:17:02 +02:00
Alexandru Mahmoud 70215cdede Remove duplicate 2021-05-06 16:16:38 -04:00
almahmoud 18a9dd42b9 Linting 2021-05-06 13:35:11 -04:00
Alexandru Mahmoud 1dae08a686 Remove CVMFS-dependent portion 2021-05-06 13:29:50 -04:00
almahmoud c50171dad7 Adding example conf files 2021-05-06 13:19:06 -04:00