Commit Graph
18689 Commits
Author SHA1 Message Date
Martin Cech 46ad1992bd Merge pull request #7071 from mvdbeek/fix_process_key
[18.09] Fix process key
2018-12-04 10:26:49 -05:00
mvdbeek cf25f7dcec Fix nesting of repeats and error when repeat is provided literally.
This fixex #7048, where in addition to repeat values
('repeat_0|param: some_value') there are also values
for the repeat param itself ('repeat': []).
2018-12-04 13:41:32 +01:00
Martin Cech 247fd01810 make the transition non-instrusive 2018-11-29 11:28:15 -05:00
Martin Cech 6fa0c8f932 add new item to the help menu 2018-11-28 12:50:18 -05:00
Dannon Baker 2620367a5a Merge remote-tracking branch 'upstream/release_18.05' into release_18.09 2018-11-27 08:34:42 -05:00
Dannon Baker 9a6fddcdca Merge remote-tracking branch 'upstream/release_18.01' into release_18.05 2018-11-27 08:28:21 -05:00
Nicola Soranzo 5255a0bf20 Fix `involucro_auto_init` default value 2018-11-22 12:50:23 +00:00
Jelle Scholtalbers 59afe04523 fix error when building images where conda channels is passed as a string instead of a list 2018-11-22 12:49:33 +00:00
mvdbeek 1e4821c644 Provide element_identifier attribute in DatsetCollectionWrapper
This allows accessing `$input1.element_identifier` for an `input1` of
collection_type `list:pair`.
2018-11-20 16:13:29 +01:00
mvdbeek 2299e06a69 Add StoredWorkflow.show_in_tool_panel function
This is more DRY and fixes
https://github.com/galaxyproject/galaxy/issues/6889#issuecomment-438991091.

The bug itself was due to comparing
the id of a StoredWorkflowUserShareAssociation instead of StoredWorkflow
when checking if a stored workflow should be in the tool panel.
2018-11-15 14:20:08 +01:00
John Chilton 5e1fb9a7a2 Merge pull request #6983 from mvdbeek/schema_amamdment_tags_in_collection
[18.09] List tags as attribute of TestCollection
2018-11-07 14:37:56 -05:00
mvdbeek 4760a0ac0f Fix recursion issue in gff_filter_by_feature_count 2018-11-07 19:26:52 +00:00
mvdbeek bff5d51245 Use comma-separated instead of comma separated 2018-11-07 19:34:40 +01:00
mvdbeek 0ecb4e7c59 List tags as attribute of TestCollection 2018-11-07 11:13:02 +01:00
Dannon 6487d52d51 Merge pull request #6978 from nsoranzo/release_18.09_paste
[18.09] Fix traceback when starting under Python3 with ``galaxy.ini``
2018-11-06 10:16:41 -05:00
Nicola Soranzo dc9128213f Fix traceback when starting under Python3 with `galaxy.ini`
Update `paste` requirement to 3.0.4 to include:

https://github.com/cdent/paste/pull/14

to fix the following traceback:

```
Starting server in PID 6792.
Traceback (most recent call last):
  File "./scripts/paster.py", line 27, in <module>
    serve.run()
  File "/opt/galaxy/lib/galaxy/util/pastescript/serve.py", line 1055, in run
    invoke(command, command_name, options, args[1:])
  File "/opt/galaxy/lib/galaxy/util/pastescript/serve.py", line 1061, in invoke
    exit_code = runner.run(args)
  File "/opt/galaxy/lib/galaxy/util/pastescript/serve.py", line 226, in run
    result = self.command()
  File "/opt/galaxy/lib/galaxy/util/pastescript/serve.py", line 676, in command
    serve()
  File "/opt/galaxy/lib/galaxy/util/pastescript/serve.py", line 653, in serve
    server(app)
  File "/opt/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 232, in server_wrapper
    **context.local_conf)
  File "/opt/galaxy/lib/galaxy/util/pastescript/loadwsgi.py", line 90, in fix_call
    val = callable(*args, **kw)
  File "/opt/galaxy/.venv3/lib/python3.5/site-packages/paste/httpserver.py", line 1358, in server_runner
    for name, value in kwargs.items():
RuntimeError: dictionary changed size during iteration
galaxy.app DEBUG 2018-11-05 13:38:00,059 Shutting down
```
2018-11-06 12:49:44 +00:00
John Chilton ccb1e08000 In enqueue, always set an object_store_id if one isn't set.
Tool actions other than the regular tool action (such as the upload ones) might not set an object store id even in the legacy mode.
2018-11-05 18:50:51 -05:00
Martin Cech 61bda3d11a Merge pull request #6960 from dannon/collection_ext_dl
[18.09] Allow deferring 'to_ext' extension to the server
2018-11-05 10:11:11 -05:00
Nicola Soranzo 2deb302ae7 Merge pull request #6951 from bernt-matthias/topic/test-colouts-backport
[18.09] Backport: tests need to count data sets and collections
2018-11-05 13:43:36 +00:00
mvdbeek f8b160535c Some minor fixes for the blast datatype
The signature for Data.display_data wasn't quite correct,
and the description of display_data was also a little misleading.
2018-11-05 11:28:45 +01:00
mvdbeek 3be8d66d71 Remove unncecessary implementations of display_data
These were not updated to use `_download_filename`,
and so this fixes the download name for collection elements
to include the element identifier.
Note that the Data class should handle composite data just
fine.
2018-11-05 10:04:52 +01:00
Dannon Baker 69b447169f Allow deferring 'to_ext' extension to the server, implement for collections. 2018-11-01 15:22:52 -04:00
mvdbeek 30aaa296dd Fix sniffing of readname sorted BAM files
This fixes https://github.com/galaxyproject/galaxy/issues/6949.
The problem is that `super` would fall back to the BamInputSorted
sniffer, which prevents sniffing. Includes a test file for the new
datatype upload testing framework.
2018-11-01 12:00:38 -04:00
M Bernt 20cecd2a68 galaxy test needs to count data sets and collection outputs 2018-10-31 12:52:25 +01:00
mvdbeek 49d9409f00 Bam index is only available for coordinate sorted bam files 2018-10-30 18:09:37 +01:00
Marius van den Beek 1482cacee5 Merge pull request #6886 from ilveroluca/fix/k8s_error_handling
Improve k8s job error handling
2018-10-30 12:01:18 +01:00
mvdbeek cf2645f9a6 Fix toolshed error in display_tool if tool is invalid
Fixes the following exception:

```
AttributeError: 'NoneType' object has no attribute 'id'
  File "galaxy/web/framework/middleware/sentry.py", line 43, in __call__
    iterable = self.application(environ, start_response)
  File "/srv/toolshed/main/venv/local/lib/python2.7/site-packages/paste/translogger.py", line 69, in __call__
    return self.application(environ, replacement_start_response)
  File "/srv/toolshed/main/venv/local/lib/python2.7/site-packages/paste/recursive.py", line 85, in __call__
    return self.application(environ, start_response)
  File "routes/middleware.py", line 141, in __call__
    response = self.app(environ, start_response)
  File "/srv/toolshed/main/venv/local/lib/python2.7/site-packages/paste/httpexceptions.py", line 640, in __call__
    return self.application(environ, start_response)
  File "galaxy/web/framework/base.py", line 143, in __call__
    return self.handle_request(environ, start_response)
  File "galaxy/web/framework/base.py", line 222, in handle_request
    body = method(trans, **kwargs)
  File "galaxy/webapps/tool_shed/controllers/repository.py", line 888, in display_tool
    tool_state = tool_util.new_state(trans, tool, invalid=False)
  File "tool_shed/util/tool_util.py", line 211, in new_state
    log.debug('Failed to build tool state for tool "%s" using standard method, will try to fall back on custom method: %s', tool.id, e)
```

(from https://sentry.galaxyproject.org/sentry/toolshed/issues/279820/).

Might also fix some other issues when redering workflow svg for
workflows with invalid tools.
2018-10-29 10:34:37 +01:00
Luca Pireddu c13da94fd7 Fix inverted log message arguments
Thanks @pcm32!
2018-10-26 16:10:29 +02:00
Luca Pireddu cf02c98127 Prefer with open syntax within kubernetes.py 2018-10-26 12:04:43 +02:00
Luca Pireddu e235343f76 Push k8s log IOError handling into __produce_log_file
Handling the IOError within the function should result in other call
sites within the kubernets.py module properly setting the job state even
if the job directory has been removed.
2018-10-26 11:58:44 +02:00
Luca Pireddu c434c27159 Improve k8s runner debug log message 2018-10-26 10:42:23 +02:00
Dannon Baker 9b24afc592 Update version to 18.09 2018-10-25 14:57:25 -04:00
mvdbeek 91a9dbaf5c Revert unncessary double logging of exception 2018-10-24 09:50:56 +02:00
mvdbeek c862d55ec8 Use with statement when handling NamedTemporaryFile 2018-10-24 09:50:33 +02:00
John Chilton 328c410fdc Merge pull request #6904 from mvdbeek/fix_weird_xml
[18.09] Fix pretty printing of shed_tool_conf.xml file
2018-10-23 18:51:15 -04:00
Nate Coraor 3f172cc988 Fix job stopping from DRMAA jobs broken in
a1c58a7088 and improve logging around it.
2018-10-23 14:32:26 -04:00
Nicola Soranzo ecb56d80a1 Merge branch 'release_18.05' into release_18.09 2018-10-23 14:28:54 +01:00
mvdbeek 57a4e50d8c Fix capsule export in docker testing
Found this by removing the blank try/except in the previous commit.
This only affects testing when python can't determine the correct
locale and falls back to ASCII.
2018-10-23 15:23:54 +02:00
mvdbeek a465543260 Drop threading lock and global try/except
The Lock wasn't actually preventing anything, and I don't think
anything would need ot be protected here. mkdtemp should be thread-safe.

We also remove the try/except Statement. I think it'd be better to
expose any exceptions here instead of producing broken capsules and
logging an exception.
2018-10-23 15:19:09 +02:00
mvdbeek 8a39f27b3c Fix toolshed tests 2018-10-23 14:51:14 +02:00
Nicola Soranzo 4dd2ecb13f Fix `make config-validate` 2018-10-23 12:28:47 +01:00
mvdbeek 794d1e03be Fix messed up printing of shed_tool_conf.xml file
This drops xml_to_string from lib/tool_shed/util/xml_util.py
and uses xml_to_string form lib/galaxy/utils/__init__.py.
Also reduces plain string manipulation of xml structures.
Uses xml.dom.minidom to make xml pretty if necessary.
2018-10-22 20:37:30 +02:00
Luca Pireddu cd6c4d4db8 Add job id to missing k8s log text 2018-10-22 14:06:45 +02:00
guerler 46e56417dd Use hasattr instead of relying on value 2018-10-17 15:08:47 -04:00
mvdbeek 126ee5fe2b Create composite file archives in new_file_path instead of TMP 2018-10-17 11:06:55 +02:00
Luca Pireddu ac7c2d0473 Print full job object to the DEBUG log if k8s runner finds it in an unrecognized state 2018-10-17 11:03:10 +02:00
Luca Pireddu 9c81089ed1 Catch IOErrors when the k8s runner tries to collect job logs
This commit should fix cases where successful job can't be marked as
done by Galaxy because the logs have gone missing.
2018-10-17 11:03:05 +02:00
Nicola Soranzo b9a01761b0 Do not traceback when uploading a TS repo with no changes
Fix the following traceback when executing `planemo shed_update`:

```
galaxy.web.framework.decorators ERROR 2018-10-15 18:07:38,736 [p:30249,w:1,m:0] [uWSGIWorker1Core1] Uncaught exception in exposed API method:
Traceback (most recent call last):
  File "lib/galaxy/web/framework/decorators.py", line 154, in decorator
    rval = func(self, trans, *args, **kwargs)
  File "lib/galaxy/webapps/tool_shed/api/repositories.py", line 1116, in create_changeset_revision
    new_repo_alert
  File "lib/tool_shed/util/repository_content_util.py", line 64, in upload_tar
    undesirable_files_removed)
  File "lib/tool_shed/util/commit_util.py", line 225, in handle_directory_changes
    message=commit_message)
  File "lib/tool_shed/util/hg_util.py", line 65, in commit_changeset
    raise Exception(error_message)
Exception: Error committing '/opt/galaxy/database/community_files/000/repo_1' to repository: Command '['hg', 'commit', '-u', u'earlhaminst', '-m', u'planemo upload for repository https://github.com/TGAC/earlham-galaxytools/tree/master/tools/TreeBest commit 22231ebc4e5e990c61243a3609b3347cf63fa024', '/opt/galaxy/database/community_files/000/repo_1']' returned non-zero exit status 1
Output was:
nothing changed

127.0.0.1 - - [15/Oct/2018:18:07:38 +0100] "POST /api/repositories/adb5f5c93f827949/changeset_revision HTTP/1.1" 500 - "-" "python-requests/2.18.4"
```
2018-10-15 20:35:04 +01:00
mvdbeek 0c7cc50e73 Only warn about multi-file zips if we don't keep the compressed archive 2018-10-13 17:49:19 +02:00
mvdbeek 8edb7499c8 Keep xlsx files compressed
Fixes https://github.com/galaxyproject/galaxy/issues/6849
2018-10-13 17:37:41 +02:00