The the [latest Selenium test execution](https://jenkins.galaxyproject.org/job/selenium/212/testReport/) most of the published histories tests failed. It looks like this is because of the three histories that were supposed to be published, only 2 were actually published. My best guess for what happeend was that the click of the button to publish the history occurred but the next Selenium command happened so quickly that the page had not gotten a chance to respond to the click. This tweaked approach to publishing histories ensures we stay on the page until the "unpublish" link appears - this will either fix the problem or rule this theory out as the explanation.
- Fix rename test for recent change to using alert.
- Test basic adding a tag.
- Outline of a test for downloading workflows.
- Add test for publishing display on index.
- Add test for using the search box on the workflow index page.
This test went from transiently broken to always broken during this release cycle. This PR makes two changes to try to fix both of those conditions. It correct the frame and button Selector for the new workflow run UX for the newer error and this allows explicitly refreshing the browser when waiting for datasets to appear to deal with the transient problems.
This time for the workflow upgrade message test, mirrors the fix in #4524 - the content we are searching for in on this page is no longer in the iframe.
Something releated to AMD broke our way of injecting Sizzle into Galaxy for Selenium tests. This is a much better way that seems to work - just use jQuery ($) as Sizzle if it is available. Avoids an external dependency and another page fetch per test as well as fixing the problem.
Add a test case that just tests the sizzle stuff works on its own. It will be a clear indiciation what is broken if there are related regressions in the future.
- Wait for elements to be clickable before clicking them.
- Allow individual tests to be run on their own and out of order.
- Use randomly generated emails - the tests still can't really be applied to an external Galaxy test server but this gets us closer.
PR #3992 fixed "with self.main_panel()" so that leaving the context actually restore the Selenium context back to the top level page. This surfaced a bunch of bugs related to assertions that were operating in the main panel without declaring it. This fixes those and makes the assertions a bit stronger - such as verifying error messages being asserted about are visible.
- By default this won't occur locally, but you can set GALAXY_TEST_SELENIUM_RETRIES to a non-zero number to enable auto retrying tests that many times.
- Capture the stack trace in the Selenium test error report directory - this will be useful for debugging problems that may fail once but pass on a subsequence attempt. Jenkins now captures these directories and includes their content in the test reports.
- Document the Selenium test error report directory in run_tests.sh as well as this new retry variable.
- Update the Jenkins test script to set this new variable to 1 so transient failures break the build much less frequently.
The ``with self.main_panel():`` construct wasn't cleaning up properly - I've fixed this and eliminated the terrible workarounds in the execution test case that I guess I had added trying to figure out how to make things work. It should be more stable now. (This test case has been failing 1 out 9 times I'd say.)
- New upload tab for "Collections".
- This has a new option to select "Collection Type" in footer.
- Collection upload geared toward homogeneous datasets - must specify genome and file format for all files.
- New button "Build" after all individual datasets have finished their upload to the server.
- Clicking "Build" launches the appropriate collection builder for selected collection type.
- By default, uploading a collection hides the original uploaded datasets (can be unchecked).
Implement tests for all uploaded collection types.
This includes two parallel changes to the Selenium tests.
- It adds a decorator to allow retrying certain atomic actions that may occur during jQuery transitions automatically.
- It refactors things to allow waiting on HIDs to become visible instead of just green.
- Use newer abstraction in the simplest upload test - verify the default extension (auto -> sam) and dbkey (?) also.
- Add a test case that explicitly sets the extension of an upload and verifies.
- Add a test case that explicitly sets a genome and verifies.
- Add a test case that tests setting ext for all uploads and verifies.
- Add a test case that tests setting genome for all uploads and verifies.
See code comment for more information - I am tracking the issue here https://github.com/galaxyproject/galaxy/issues/3598. I think it is important to resolve that test and fix the usability problems with current tours but I'd like to get the Selenium tests stable and testing against new PRs first.
I'm pretty sure the problem is that I was waiting on conditions (e.g. datasets to become "ok") using the API and then assuming the state of the GUI elements was the same. Obviously though the GUI is polling the API and there is some lag there. This commit changes things to specifically wait on history panel GUI changes - obviously the better thing to wait on in a GUI test.
John Chilton