This revealed a wrong query filter (comparing
JobToInputDatasetAssociation.dataset_id == HistoryDatasetAssociation.id,
instead of JobToInputDatasetAssociation.dataset_id ==
HistoryDatasetAssociation.dataset_id).
Matching the JobParameters will prevent returning jobs where a job uses the
same input datasets, but where the inputs are used in a different order (and
hence the result may differ)
A precondition to the real meat of the test is failing to be met sometimes in production - this failure is indicating a problem with the test and not with Galaxy. I previously tried to address this with https://github.com/galaxyproject/galaxy/pull/3988 but that didn't work. Now if the trainsient ok occurs the test will be skipped. Most of the time it won't skip and the rest of the test will execute and ensure there aren't regressions in behaviors related to cleaning up datasets after job completion. I'm placing the skips a couple different places so hopefully we can get a stack trace at somepoint - knowing where the code is when the job state is changing from running to ok will help puzzle out what is handing for the two minutes the job is running in the test framework.
A couple things need to occur while the jobs are "running" and it seems that 60 seconds wasn't enough time to sleep for all that to happen on Jenkins.
I checked the timing in the raw logs (they are for some reasons being filtered out in the Jenkins display of the XUnit) - and indeed the jobs are properly sleeping the correct amount of time but the client API calls need some more time it seems. My initial thought was maybe the jobs were being interrupted or something - and that does not appear to be the case.
There are two basic behavior changes here and bunch of testing to verify the new behaviors.
1) Previously purging a dataset would cause a creating job to be cancelled but all outputs would need to be "deleted" in order to cause the creating job to be cancelled. After this change the behavior is the same for both deletion and purging - the job will run until all outputs are deleted or purged.
2) Previously Galaxy jobs wouldn't attempt to cleanup purged datasets that were created by a running tool - this cleanup now occurs.
These are tackled as a pair because I assume the difference in the behaviors between deletion and purging was a hack around tools populating datasets that had been purged while running. Cancelling the job wasn't a sure fix - but it would reduce the likelihood of such problems. I think the new approach is a bit more robust and explicit.
- Move api.helpers to base.populators.
- Refactor helper subclasses adapted to use bioblend instead of test framework stuff into base.populators for greater visibility.
- Refactor workflow format 2 testing stuff for reuse by Selenium.
This strategy proved to work around certain race conditions in tool testing so hopefully it will solve the transiently failing job searching and filtering test cases.
Add tests for new date range and history filtering as well as to ensure only admins get to see external_id and command_line and that users cannot see each other's jobs.
Always allow admins to views all jobs on index (instead of only when user_details is specified) and show. Add user_email and external_id to show (for admins) to bring it inline with index and add command_line to index (for admins) to bring it in line with show.
Show still allow more details including job standard error and output as well as job metrics.
mvdbeek