Previously we only detected transitions for retrying actions based on stale element exceptions, this expands that to include popups that may be fading out for instance.
I think this is what is happening with the transiently failing test here https://jenkins.galaxyproject.org/job/selenium/528/artifact/528-test-errors/test_save_as2017092203291506065342/stacktrace.txt. The exception indicates that a click was not clickable because a modal element that was fading out - though it had been previously clickable. I think this should fix that.
I don't get why when we click submit the user does not actually get logged in, but based on the last round of improved error messages this seems to be the case. You might think there is some callback in the login form that doesn't get registered by the time Selenium clicks the submit button - but this doesn't seem to be the case - I don't see any jquery magic happening in login.mako.
Should fix failures like this:
https://jenkins.galaxyproject.org/job/selenium/482/testReport/junit/selenium_tests.test_saved_histories/SavedHistoriesTestCase/test_history_publish/
I'd say at this point this is the most common problem in the Selenium tests.
This also introduces a framework for taking state snapshots of the Galaxy interface during tests that will only get written out if the tests fail. We now take screenshots before and after submitting the login form but this is a general purpose debugging mechanism that could be used other places.
These tests aren't idealized unittest.TestCase because they initialize class level data in instance level methods. While this isn't ideal, it is seems an entirely fair workaround given that SeleniumTestCase setups up the Selenium connection itself in an instance method - so class-level initializers would not be able to setup Galaxy data. Since I think we will continue using this pattern then, probably best to formalize it a bit and improve error handling.
This provides a formal super class for these test cases that provides a uniform method for setting up the class level data and tracks whether this is successful or not. This serves a couple purposes beyond simple uniformity. First, it tracks if the state has actually been setup or not and will skip subsequent tests if it hasn't. Some of these tests aren't passing very consistently on Jenkins and so we get a bunch of extra noise for tests that are attempting to run without their preconditions met - this will fix that and make the original errors much more clear. Moving the "hacky" part of this into the framework itself also means the tests themselves don't have to repeat hacks like seeing if variables are set with ``getattr`` and such - I always prefer one framework hack to a dozen application hacks.
We do this by recording any macros that are required to load a tool. When
loading a tool we register the macro to be watched, and if the macro changes we
reload all corresponding tools.
At the insistence of @nsoranzo, be more explicit about waiting for workflow test in API tests. While I was in there I also setup new abstractions for waiting on state and updated various workflow cancelling tests to use test_history context for better error reporting.
I saw a failed test earlier that would have been fixed by a switch to wait_for_and_click in workflow_editor_click_options. This makes that switch and moves some of these workflow editor helper functions into navigates_galaxy.py for reuse by other test cases and by library users.
Previously just a notification about waiting for ``a.loggedin-only`` failed would show up. This doesn't indicate if the login failed completely or the masthead just failed to update. I suspect the login failed completely, but just in case I've updated the error handling to actually ping the API and check and print a better error message with this information.
New messages look like:
```
TimeoutException: Message: Failed waiting for masthead to update for login, but user API response indicates [test0ol61vs8fo] is logged in. This seems to be a bug in Galaxy. API response was [{u'username': u'test0ol61vs8fo', u'quota_percent': None, u'preferences': {}, u'total_disk_usage': 0.0, u'deleted': False, u'id': u'adb5f5c93f827949', u'nice_total_disk_usage': u'0 bytes', u'quota': None, u'email': u'test0ol61vs8fo@test.test', u'is_admin': False, u'tags_used': [], u'purged': False}].Timeout waiting on CSS selector [a.loggedin-only-x] to become visible.
```
and
```
TimeoutException: Message: Failed waiting for masthead to update for login, API indicates no user is logged in - there is a problem with this test. API response was [{u'quota_percent': None, u'nice_total_disk_usage': u'0 bytes', u'total_disk_usage': 0}]. Timeout waiting on CSS selector [a.loggedin-only] to become visible.
```
- set_tags() appeared in both published and saved history tests, refactored into navigates_galaxy with function name history_panel_add_tags
- is_displayed() appeared in both published and saved history tests, refactored into has_driver as selector_is_displayed
- Introduced history_panel_rename to reduce duplication across history panel tests and newer published and saved history tests.
- Removed custom history click option helper in saved and published history tests and just used the variant in navigates_galaxy.
- use wait_for_and_click a couple more places...
- Be sure when sharing a history with a user we wait to actually see that user's e-mail address appear in the history's sharing list after submission.
- Print something for the failed history sharing assertions (I'm pretty sure they are 403 errors because some aspect of the sharing or login didn't work - but might as well verify).
Locally this never happens, but remotely sometimes when sorting by owner the histories are sorted by owner but not in the same order as the test was asserting. I would assume this is likely a postgres vs. sqlite difference - perhaps the former doesn't sort the rows by the order they were created within equal sort classifications (i.e. same owner) but the latter does. I've fixed this - the test should not have been asserting such a strong condition I don't think.
John Chilton