Commit Graph
892 Commits
Author SHA1 Message Date
John Chilton f035d642ce Merge pull request #6844 from jmchilton/compressed_types_and_extra_files
Infrastructure/API Support for Composite Uploads/Downloads
2018-11-01 10:57:09 -04:00
M Bernt 1ec18484df added test 2018-10-29 14:47:57 +01:00
Nicola Soranzo f4203ca051 Fix errors reported by flake8 3.6.0
Ignore W504.
2018-10-24 23:22:19 +01:00
John Chilton c2a49de6ce Directory datatype for Galaxy. 2018-10-24 09:39:06 -04:00
mvdbeek 19c98b2745 Fix tests and name tool more appropriately 2018-10-24 12:11:52 +02:00
mvdbeek 26680e0ca1 Add test for remove_value form dynamic options
Unfortunately you can select an invalid value (in the API, not the UI)
and Galaxy will replace the invalid value with the first valid value.
2018-10-24 11:34:09 +02:00
Nate Coraor ca95d7a26e Merge remote-tracking branch 'upstream/release_18.09' into dev 2018-10-12 10:32:43 -04:00
Marius van den Beek 01f1f1b95f Merge pull request #6858 from jmchilton/fix_metadata_revert
[18.09] Revert selective metadata flushing - breaks multiple uploads/outputs.
2018-10-12 15:44:01 +02:00
John Chilton f3da0864b4 Test case for regression #6856. 2018-10-12 09:03:12 -04:00
Nicola Soranzo 5e71401a7e Merge branch 'release_18.09' into dev 2018-10-12 09:47:34 +01:00
Nicola Soranzo d22f652f91 Merge branch 'release_18.05' into release_18.09 2018-10-12 09:44:18 +01:00
mvdbeek 5844bdc7aa Add integration test for remote shell CLI runners
This tests both the SecureShell and Paramiko shell plugins and the
Slurm and Torque CLI runner.
2018-10-10 10:10:51 +02:00
Marius van den Beek 10e984ce85 Merge pull request #6746 from jmchilton/workflow_state_touch_1
Fill in needed state defaults when importing manually crafted workflows.
2018-10-06 10:17:41 +02:00
mvdbeek ab660686c2 Add test for single and multi-input identifiers
test_identifier_map_over_input_in_conditional fails at the
moment (same issue as in https://github.com/galaxyproject/galaxy/issues/6182)
2018-09-28 19:01:28 +02:00
John Chilton 3e774713a9 Fill in needed state defaults when importing manually curated workflows. 2018-09-24 09:38:34 -04:00
mvdbeek 8b757f22d1 Exclude bed_to_bigbed and wig_to_bigwig from tests
They work, but required hg17.len to be present.
2018-09-05 15:25:33 +02:00
mvdbeek da4906dc56 Add GeneBed_Maf_Fasta2 to no test list 2018-09-05 14:22:01 +02:00
John Chilton 8148bd91de Merge pull request #6638 from mvdbeek/move_tests_to_pytest_setup
Move tests to pytest setup
2018-09-05 06:50:34 -04:00
Marius van den Beek a3fc0b9f32 Merge pull request #6312 from jmchilton/tuple_operations
Lists-as-tuples collection operations
2018-08-26 09:19:49 +02:00
mvdbeek e73257f8ed Set conda_auto_init and conda_auto_install to true for pytest driver 2018-08-25 14:17:22 +02:00
mvdbeek 300ab73c10 Mark skipped tests instead of not collecting them
Also marks tests as tool or data_manager, so `pytest -m
<tool|data_manager>` can be used to select a subset of tests.
2018-08-25 14:17:00 +02:00
mvdbeek 0b64442ed3 Skip tool tests in test/functional/known_broken_tools.txt 2018-08-25 14:16:22 +02:00
Nicola Soranzo ff0d28c35c Fix unicode_stream framework tests on Docker
This fixes the following traceback:
```
( unicode_stream ) > Test-1 ... galaxy.tools DEBUG 2018-08-21 09:40:59,052 Validated and populated state for tool request (0.098 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,076 Handled output named out_file1 for tool unicode_stream (1.337 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,080 Added output datasets to history (4.399 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,084 Verified access to datasets for Job[unflushed,tool_id=unicode_stream] (0.058 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,086 Setup for job Job[unflushed,tool_id=unicode_stream] complete, ready to flush (5.220 ms)
galaxy.tools.actions INFO 2018-08-21 09:40:59,127 Flushed transaction for job Job[id=354,tool_id=unicode_stream] (40.627 ms)
galaxy.tools.execute DEBUG 2018-08-21 09:40:59,127 Tool [unicode_stream] created job [354] (56.506 ms)
galaxy.tools.execute DEBUG 2018-08-21 09:40:59,132 Executed 1 job(s) for tool unicode_stream request: (79.434 ms)
galaxy.jobs.mapper DEBUG 2018-08-21 09:40:59,556 (354) Mapped job to destination id: local
galaxy.jobs.handler DEBUG 2018-08-21 09:40:59,561 (354) Dispatching to local runner
galaxy.jobs DEBUG 2018-08-21 09:40:59,571 (354) Working directory for job is: /tmp/tmp3drfmr6x/tmp_ywd33ar/tmpbzo4a1d9/database/job_working_directory_azry9a1y/000/354
galaxy.jobs DEBUG 2018-08-21 09:40:59,574 (354) Persisting job destination (destination id: local)
galaxy.jobs.runners DEBUG 2018-08-21 09:40:59,589 Job [354] queued (27.361 ms)
galaxy.jobs.handler INFO 2018-08-21 09:40:59,604 (354) Job dispatched
galaxy.jobs.runners ERROR 2018-08-21 09:40:59,715 (354) Failure preparing job
Traceback (most recent call last):
  File "/galaxy/lib/galaxy/jobs/runners/__init__.py", line 187, in prepare_job
    job_wrapper.prepare()
  File "/galaxy/lib/galaxy/jobs/__init__.py", line 874, in prepare
    self.command_line, self.extra_filenames, self.environment_variables = tool_evaluator.build()
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 435, in build
    raise e
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 431, in build
    self.__build_config_files()
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 504, in __build_config_files
    self.__write_workdir_file(config_filename, config_text, param_dict, is_template=is_template)
  File "/galaxy/lib/galaxy/tools/evaluation.py", line 578, in __write_workdir_file
    f.write(value)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)
galaxy.tools.error_reports DEBUG 2018-08-21 09:40:59,869 Bug report plugin <galaxy.tools.error_reports.plugins.sentry.SentryPlugin object at 0x7f92efb81668> generated response None
FAIL
```

which is due to the fact that the default encoding in the Docker image
is `ANSI_X3.4-1968`.

Also update `unicode_stream` test tool to IUC best practices.
2018-08-21 16:04:27 +01:00
Nicola Soranzo 670d8b1623 Merge TwillTestCase in ShedTwillTestCase 2018-08-21 12:35:06 +01:00
mvdbeek 2b275b7df0 Start counting test numbers at 1 and use _test_n
That allows for nicer select syntax like
````
pytest test/functional/test_toolbox_pytest.py -k '(random_lines1 or Cut1) and _test_1'
```
2018-08-16 19:14:33 +02:00
mvdbeek 9e07d9cc06 Handle driver shutdown, show all test in pytest
This is pretty hackish as fixtures can't be used during test discovery.
To properly shut down the driver I've added a conftest.py file that will
be called when quitting pytest. We might be able to handle this a
little better if we wrote a full plugin, but for now this works.

Also now reports all test indexes, so you can select e.g test 2 of a
tool.
2018-08-16 18:44:05 +02:00
mvdbeek 7559b90a69 Setup a simple pytest tool test 2018-08-16 14:08:45 +02:00
mvdbeek a927413df2 Use exit code and quote file parameters for paralellism test tools 2018-08-15 22:30:31 +02:00
mvdbeek fc464f9758 Allow 4 lines diff to acount for the prefix of python2 unicode strings 2018-08-15 22:30:31 +02:00
mvdbeek 3f032f6c38 Rework and simplify data manager testing
We don't rely on TwillTestCase anymore, that is now exclusively used by
ShedTwillTestCase. Then I've just re-used the infrastructure of
ToolsTestCase for discovering tests, while making sure that we do use an
admin API key during testing.
2018-08-13 17:19:25 +02:00
John Chilton 84597a1235 Lists-as-tuples collection operations. 2018-08-08 10:28:56 -04:00
John Chilton d417e9b239 Flush out collection tag parameters a bit.
- Switch select_tag to group_tag as a parameter name - since it is restricted to group_tags and we don't call data parameters select_data.
- Add minimal XSD documentation for this new parameter type.
- Add example tools - for single and multiple tags.
- Add API tests for these tools and the tag handling.
2018-07-24 14:31:56 -04:00
Dannon 20f10fe6db Merge pull request #6389 from qiagu/data_column_1
fix data_column cast error when multiple is false
2018-07-18 19:27:57 -04:00
John Chilton 019f088543 Allow including paths in inputs json files. 2018-07-05 11:05:15 -04:00
mvdbeek ae9b6057da Fix test_data_managers for python 3 2018-07-04 16:53:50 +02:00
mvdbeek 7f0684e3d5 Fix test_toolbox for python 3 2018-07-04 15:41:47 +02:00
mvdbeek 88132f9d16 Test dumping multiple="false" data_column param as json
Tests https://github.com/galaxyproject/galaxy/pull/6389
2018-06-30 09:11:42 -07:00
mvdbeek 647ba8531a Add test tool and workflow for mapping over with mapping output
This doesn't quite work because of the dynamic output collection,
which fails with:

```
galaxy.tools.parameters.output_collect DEBUG 2018-06-06 18:40:38,784 (3) Add dynamic collection datasets to history for output [reverse] (171.970 ms)
galaxy.tools.parameters.output_collect ERROR 2018-06-06 18:40:38,839 Problem gathering output collection.
Traceback (most recent call last):
  File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 340, in collect_dynamic_outputs
    collection_builder.populate()
  File "/Users/mvandenb/src/galaxy/lib/galaxy/dataset_collections/builder.py", line 88, in populate
    elements = self.build_elements()
  File "/Users/mvandenb/src/galaxy/lib/galaxy/dataset_collections/builder.py", line 59, in build_elements
    new_elements[identifier] = element.build()
AttributeError: 'HistoryDatasetAssociation' object has no attribute 'build'
```

This seems to happen because the inner collection is wrongly detected as
nested. In general I doubt that mapping over colleciton output works
with dynamically discovered output collections.
2018-06-20 09:54:40 -04:00
mvdbeek 7cfbb032f6 Break test by adding "list" to collection_type
This will break tests trying to map over list:pair collections,
for instance test_extract_workflows_with_dataset_collections.
2018-06-20 09:54:36 -04:00
Marius van den Beek 02974b79c7 Merge pull request #5640 from Mataivic/patch-1
Add a FilterEmptyDatasetsTool and allow filtering tools to apply to arbitrary collection structures
2018-06-17 12:25:03 +02:00
mvdbeek 8a4d18fba1 Add filter empty collection tool
I think having a separate tool for this task makes sense and makes the
distinction between empty and fialed datasets clearer.
2018-06-13 11:35:21 +02:00
Helena Rasche 9b9ab6a1cf Merge pull request #6317 from mvdbeek/fix_6314
Allow #if $datasets #end if pattern for file lists
2018-06-12 17:36:50 +00:00
mvdbeek 550eb88210 Add a test for not providing an optional input 2018-06-12 18:18:13 +02:00
mvdbeek 0a29e533d3 Use #if $datasets #end if pattern in test tool 2018-06-12 13:55:36 +02:00
mvdbeek fd16302d65 Add test tool and workflow for mapping over with mapping output
This doesn't quite work because of the dynamic output collection,
which fails with:

```
galaxy.tools.parameters.output_collect DEBUG 2018-06-06 18:40:38,784 (3) Add dynamic collection datasets to history for output [reverse] (171.970 ms)
galaxy.tools.parameters.output_collect ERROR 2018-06-06 18:40:38,839 Problem gathering output collection.
Traceback (most recent call last):
  File "/Users/mvandenb/src/galaxy/lib/galaxy/tools/parameters/output_collect.py", line 340, in collect_dynamic_outputs
    collection_builder.populate()
  File "/Users/mvandenb/src/galaxy/lib/galaxy/dataset_collections/builder.py", line 88, in populate
    elements = self.build_elements()
  File "/Users/mvandenb/src/galaxy/lib/galaxy/dataset_collections/builder.py", line 59, in build_elements
    new_elements[identifier] = element.build()
AttributeError: 'HistoryDatasetAssociation' object has no attribute 'build'
```

This seems to happen because the inner collection is wrongly detected as
nested. In general I doubt that mapping over colleciton output works
with dynamically discovered output collections.
2018-06-11 12:20:25 +02:00
mvdbeek 940ed5f5dd Break test by adding "list" to collection_type
This will break tests trying to map over list:pair collections,
for instance test_extract_workflows_with_dataset_collections.
2018-06-08 10:41:07 +02:00
Nicola Soranzo 711070dc4c Merge branch 'release_18.01' into release_18.05 2018-06-07 10:28:42 +01:00
Marius van den Beek 6911153c30 Merge pull request #6213 from jmchilton/one_more_identifier_test
Test element_identifier usage with single data parameters, in repeat/cond.
2018-06-04 16:19:01 +02:00
mvdbeek 3e22b0b11b Test case for mapping over inputs with output ftype determined via input extension 2018-06-04 07:00:41 -04:00
mvdbeek 6ace37e910 Use 4-space indent in identifier_single_in_repeat test tool 2018-06-01 10:56:38 +02:00