Commit Graph
3585 Commits
Author SHA1 Message Date
Nicola Soranzo ad02c53154 Fix many links 2018-09-03 14:34:13 +01:00
John Chilton 410ec1927c Merge pull request #6656 from jmchilton/collect_builder_fixes
Assorted fixes for #6572
2018-08-31 10:43:10 -04:00
John Chilton dd0a6884ba Fixes... 2018-08-29 15:40:39 -04:00
Dannon Baker 223c398808 drop admin api keys navigator from selenium configs (which may break other tests, we should wait to see) 2018-08-29 09:05:14 -04:00
mvdbeek 366701cfb2 Strip unicode control characters from tags
Leaving them in can trip up postgresql:
```
Traceback (most recent call last):
  File "lib/galaxy/web/framework/decorators.py", line 283, in decorator
    rval = func(self, trans, *args, **kwargs)
  File "lib/galaxy/webapps/galaxy/api/history_contents.py", line 396, in create
    return self.__create_dataset_collection(trans, history, payload, **kwd)
  File "lib/galaxy/webapps/galaxy/api/history_contents.py", line 525, in __create_dataset_collection
    **create_params
  File "lib/galaxy/managers/collections.py", line 108, in create
    copy_elements=copy_elements,
  File "lib/galaxy/managers/collections.py", line 181, in create_dataset_collection
    copy_elements=copy_elements)
  File "lib/galaxy/managers/collections.py", line 217, in _element_identifiers_to_elements
    copy_elements=copy_elements)
  File "lib/galaxy/managers/collections.py", line 389, in __load_elements
    copy_elements=copy_elements)
  File "lib/galaxy/managers/collections.py", line 434, in __load_element
    self.tag_manager.apply_item_tags(user=trans.user, item=element, tags_str=tag_str)
  File "lib/galaxy/managers/tags.py", line 182, in apply_item_tags
    self.apply_item_tag(user, item, name, value)
  File "lib/galaxy/managers/tags.py", line 154, in apply_item_tag
    tag = self._get_or_create_tag(lc_name)
  File "lib/galaxy/managers/tags.py", line 233, in _get_or_create_tag
    tag = self.get_tag_by_name(scrubbed_tag_str)
  File "lib/galaxy/managers/tags.py", line 205, in get_tag_by_name
    return self.sa_session.query(galaxy.model.Tag).filter_by(name=tag_name.lower()).first()
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2789, in first
    ret = list(self[0:1])
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2581, in __getitem__
    return list(res)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2889, in __iter__
    return self._execute_and_instances(context)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2912, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1416, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
ValueError: A string literal cannot contain NUL (0x00) characters.
```
2018-08-26 22:29:23 +02:00
mvdbeek 50a524496a Add unit test for copy_element option 2018-08-26 22:13:50 +02:00
mvdbeek cfd8379917 Send copy_elements=True in new create collection test 2018-08-26 22:13:50 +02:00
mvdbeek b13cce0a13 Revert "Adjust API tests to copying HDAs when creating collection"
This reverts commit f120390a63700ff0ec6c09185aad26b91ad8f842.
2018-08-26 22:13:50 +02:00
mvdbeek 9409322a73 Revert "Adapt unit tests to copying HDAs when creating collections"
This reverts commit 408a003f171be73e4b347579ee6f662a18fc1990.
2018-08-26 22:13:50 +02:00
mvdbeek 34cfdcac6a Test for proper 403 response
Coincidentally fixed by e013f2fbd1596b103ef195afcba8cbd787c49b33.
2018-08-26 22:13:50 +02:00
mvdbeek a9cb290754 Adjust API tests to copying HDAs when creating collection 2018-08-26 22:13:50 +02:00
mvdbeek d4a80e61e1 Adapt unit tests to copying HDAs when creating collections 2018-08-26 22:13:50 +02:00
mvdbeek 91438996df Test creation of collection from existing dataset, with a new tag 2018-08-26 22:13:50 +02:00
Marius van den Beek 0e69ca8bfb Merge pull request #6545 from jmchilton/rule_builder_use_group_tags
Allow Consuming Tags in the Apply Rules Tool
2018-08-26 11:17:14 +02:00
Marius van den Beek a3fc0b9f32 Merge pull request #6312 from jmchilton/tuple_operations
Lists-as-tuples collection operations
2018-08-26 09:19:49 +02:00
Marius van den Beek 2560919bd0 Merge pull request #6620 from jmchilton/job_recovery_test
Integration test for job recovery.
2018-08-25 16:51:16 +02:00
Dannon 107c89a0a9 Merge pull request #6580 from jmchilton/wf_invocation_update
Add API endpoint to fetch a history's workflow invocations.
2018-08-24 11:41:25 -04:00
John Chilton 3879a01949 Add API endpoint to index all invocations for a user or history.
Refactor toward managers to allow unified serialization of invocations and such. Add test case.
2018-08-24 10:00:47 -04:00
mvdbeek f6a55ee2ee Tweak conda_* setting in test driver
Thanks @nsoranzo!
2018-08-23 20:45:24 +02:00
Dannon 43dbfdc7b1 Merge pull request #6632 from mvdbeek/testing_framework_improvements
Testing framework improvements
2018-08-23 14:24:43 -04:00
mvdbeek 23cfb46001 Allow setting CONDA_PREFIX using GALAXY_TEST_CONDA_PREFIX in testing 2018-08-23 14:57:31 +02:00
Dannon 498e7d576b Merge pull request #6552 from jmchilton/per_destination_object_store
Allow configuring object stores on a per-job-destination basis.
2018-08-22 11:41:16 -04:00
Helena Rasche 53b8f08752 Merge pull request #6568 from jmchilton/dataset-permissions-api-2
Unified dataset permission API.
2018-08-22 15:04:28 +00:00
John Chilton 35ebdc5b10 Add a legacy_job_state mode to restore 17.09 workflow invocation behavior. 2018-08-22 10:45:07 -04:00
John Chilton 2d4b717aaf Test case to clarify workflow API a bit in simple cases. 2018-08-22 10:45:06 -04:00
John Chilton 626195335e Integration test for job recovery (before and after handler has picked it up). 2018-08-21 15:17:12 -04:00
Nicola Soranzo ff0d28c35c Fix unicode_stream framework tests on Docker
This fixes the following traceback:
```
( unicode_stream ) > Test-1 ... galaxy.tools DEBUG 2018-08-21 09:40:59,052 Validated and populated state for tool request (0.098 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,076 Handled output named out_file1 for tool unicode_stream (1.337 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,080 Added output datasets to history (4.399 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,084 Verified access to datasets for Job[unflushed,tool_id=unicode_stream] (0.058 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,086 Setup for job Job[unflushed,tool_id=unicode_stream] complete, ready to flush (5.220 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,127 Flushed transaction for job Job[id=354,tool_id=unicode_stream] (40.627 ms)
galaxy.tools.execute DEBUG 2018-08-21 09:40:59,127 Tool [unicode_stream] created job [354] (56.506 ms)
galaxy.tools.execute DEBUG 2018-08-21 09:40:59,132 Executed 1 job(s) for tool unicode_stream request: (79.434 ms)
galaxy.jobs.mapper DEBUG 2018-08-21 09:40:59,556 (354) Mapped job to destination id: local
galaxy.jobs.handler DEBUG 2018-08-21 09:40:59,561 (354) Dispatching to local runner
galaxy.jobs DEBUG 2018-08-21 09:40:59,571 (354) Working directory for job is: /tmp/tmp3drfmr6x/tmp_ywd33ar/tmpbzo4a1d9/database/job_working_directory_azry9a1y/000/354
galaxy.jobs DEBUG 2018-08-21 09:40:59,574 (354) Persisting job destination (destination id: local)
galaxy.jobs.runners DEBUG 2018-08-21 09:40:59,589 Job [354] queued (27.361 ms)
galaxy.jobs.handler INFO 2018-08-21 09:40:59,604 (354) Job dispatched
galaxy.jobs.runners ERROR 2018-08-21 09:40:59,715 (354) Failure preparing job
Traceback (most recent call last):
  File "/galaxy/lib/galaxy/jobs/runners/__init__.py", line 187, in prepare_job
    job_wrapper.prepare()
  File "/galaxy/lib/galaxy/jobs/__init__.py", line 874, in prepare
    self.command_line, self.extra_filenames, self.environment_variables = tool_evaluator.build()
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 435, in build
    raise e
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 431, in build
    self.__build_config_files()
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 504, in __build_config_files
    self.__write_workdir_file(config_filename, config_text, param_dict, is_template=is_template)
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 578, in __write_workdir_file
    f.write(value)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)
galaxy.tools.error_reports DEBUG 2018-08-21 09:40:59,869 Bug report plugin <galaxy.tools.error_reports.plugins.sentry.SentryPlugin object at 0x7f92efb81668> generated response None
FAIL
```

which is due to the fact that the default encoding in the Docker image
is `ANSI_X3.4-1968`.

Also update `unicode_stream` test tool to IUC best practices.
2018-08-21 16:04:27 +01:00
Nicola Soranzo 670d8b1623 Merge TwillTestCase in ShedTwillTestCase 2018-08-21 12:35:06 +01:00
Nicola Soranzo f54246e1a1 Use encode() method instead of bytearray()
to encode a Unicode string to a binary one.
2018-08-21 12:35:05 +01:00
Nicola Soranzo df8abbe3ef Prefer unicodify() for binary-to-text conversion
which supports non-utf-8 encodings.
2018-08-21 12:35:05 +01:00
Nicola Soranzo 5bddd24d8b Remove unreachable return statement 2018-08-20 23:52:50 +01:00
John Chilton e07cc9a960 Merge pull request #6608 from mvdbeek/more_pytest_improvements
Handle driver shutdown, show all tests in pytest
2018-08-20 13:05:00 -04:00
mvdbeek 2d60d9af01 Wait for first workflow when testing cache
That should make the test less flakey. While in theory we should be able
to replace scheduled workflow jobs with cached jobs, this is not what
we're testing here, and I assume it doesn't work for now.
2018-08-20 11:29:02 +02:00
mvdbeek 2b275b7df0 Start counting test numbers at 1 and use _test_n
That allows for nicer select syntax like
````
pytest test/functional/test_toolbox_pytest.py -k '(random_lines1 or Cut1) and _test_1'
```
2018-08-16 19:14:33 +02:00
mvdbeek 9e07d9cc06 Handle driver shutdown, show all test in pytest
This is pretty hackish as fixtures can't be used during test discovery.
To properly shut down the driver I've added a conftest.py file that will
be called when quitting pytest. We might be able to handle this a
little better if we wrote a full plugin, but for now this works.

Also now reports all test indexes, so you can select e.g test 2 of a
tool.
2018-08-16 18:44:05 +02:00
Dannon d9f3947938 Merge pull request #6605 from mvdbeek/enable_and_fix_framework_py3_test
Enable and fix framework py3 tests
2018-08-16 10:39:14 -04:00
Dannon b9ca3b7c33 Merge pull request #6593 from jmchilton/toward_test_history_cleaning
Migrate API tests in test_jobs.py to use test_history context.
2018-08-16 09:46:39 -04:00
mvdbeek 7559b90a69 Setup a simple pytest tool test 2018-08-16 14:08:45 +02:00
mvdbeek c430d4a650 Fix selenium test setup 2018-08-16 11:16:57 +02:00
mvdbeek a927413df2 Use exit code and quote file parameters for paralellism test tools 2018-08-15 22:30:31 +02:00
mvdbeek fc464f9758 Allow 4 lines diff to acount for the prefix of python2 unicode strings 2018-08-15 22:30:31 +02:00
John Chilton ef0a51537e Merge pull request #6595 from mvdbeek/modernize_data_manager_testing_strip_twill
Modernize data manager testing and strip twill
2018-08-15 10:02:57 -04:00
Björn Grüning 55756ba12d Merge pull request #6582 from jmchilton/selenium_fixes_45
Improve robustness of tour Selenium test.
2018-08-14 20:31:44 +02:00
mvdbeek dd7e4c601a Deal with docker localhost quirks 2018-08-14 14:56:07 +02:00
mvdbeek 620375c9d0 Do test mules on Jenkins, I don't see why this shouldn't work 2018-08-14 14:56:07 +02:00
mvdbeek cfce246845 Fix GRT export test 2018-08-14 14:56:07 +02:00
mvdbeek 293ce5a472 py3: Fix mules test cases
We're now just copying the current python path. Also the handler names
have actually changed, I think this hasn't been updated after a late
change (it is now as it should be, based on the documentation).
2018-08-14 14:56:07 +02:00
mvdbeek 37b5120197 py3: fix installing and running data managers 2018-08-14 14:56:06 +02:00
mvdbeek 3f032f6c38 Rework and simplify data manager testing
We don't rely on TwillTestCase anymore, that is now exclusively used by
ShedTwillTestCase. Then I've just re-used the infrastructure of
ToolsTestCase for discovering tests, while making sure that we do use an
admin API key during testing.
2018-08-13 17:19:25 +02:00
John Chilton c6b593b1ca Migrate API tests in test_jobs.py to use test_history context.
More configurable, better error reporting, cleans up jobs, etc..
2018-08-10 19:43:24 -04:00