Commit Graph
1054 Commits
Author SHA1 Message Date
Sergey Golitsynskiy 6e78cc6847 Fix bug in a unit test
There was a bug in a context manager inside a unit test. This caused
nondeterministic behavior of some unit tests after a refactoring.

Bug fixed; added 3 unit tests to test the context manager.
2019-07-08 17:59:25 +02:00
Nate Coraor 058fabc121 Merge branch 'dev' of https://github.com/galaxyproject/galaxy into installable 2019-07-08 12:27:21 +02:00
John Chilton 1815a6547a Bring top-level galaxy Python files into galaxy-app. 2019-07-07 19:45:48 +02:00
Nate Coraor a0b1d0066f Unit test fixes 2019-07-07 16:29:58 +02:00
Nate Coraor 2034980b77 Merge branch 'dev' of https://github.com/galaxyproject/galaxy into installable 2019-07-07 13:50:41 +02:00
John Chilton d194520fdf Merge pull request #8152 from jmchilton/tool_deps_serialize
Re-try dependency manager serialization and embedded config.
2019-07-07 12:44:35 +02:00
Nate Coraor 37522f5e8e Merge branch 'dev' of https://github.com/galaxyproject/galaxy into installable 2019-07-05 16:05:17 +02:00
Dannon 72cc063451 Merge pull request #7905 from ValentinMarcon/toolrefs
Implement "xref" tag on the tool structure (with visualisation)
2019-07-05 09:54:24 -04:00
Nate Coraor cdb9040e26 Merge remote-tracking branch 'origin/dev' into installable 2019-07-05 15:51:15 +02:00
Marius van den Beek 71d315cf36 Merge pull request #8246 from mvdbeek/merge_8221
Merge 19.05 into dev
2019-06-29 09:24:02 +02:00
Marius van den Beek 49d09e2c45 Merge pull request #8090 from guerler/new_toolshed
Convert repository client to vue
2019-06-29 09:17:36 +02:00
mvdbeek 928f9453e5 Merge branch 'release_19.05' into dev 2019-06-29 00:24:51 +01:00
ValentinMarcon 03ba7a3c01 standardization of 'reftype' & 'value' 2019-06-28 11:02:57 +02:00
ValentinMarcon 6a6ade54fa xrefs yaml parser 2019-06-27 15:39:07 +02:00
ValentinMarcon 7347e57509 add parsing xml and yaml test 2019-06-27 11:41:31 +02:00
John Chilton 584c117df3 Refactor to cleanup dependencies around galaxy.web.stack.
Move galaxy.web.stack.database_heartbeat into galaxy.model - since it was the only thing in galaxy.web.stack that used galaxy.model at all and it didn't import anything else from galaxy.web.stack.

Once that was done, I refactored this whole package (galaxy.web.stack) into galaxy.web_stack. It doesn't use the framework setup or exported by the base galaxy.web package so it wouldn't seem to belong there. It also has very different dependencies than the rest of that. And finally and most importantly, this elimiantes almost all dependency from galaxy.jobs and galaxy.tools and galaxy.workflows to anything in galaxy.web.
2019-06-26 18:54:52 -04:00
mvdbeek 05a5628df0 Add test for fetching section by label 2019-06-25 18:29:52 +02:00
John Chilton 719eca2d36 Fix portable metadata datatypes.
Absolute paths were preventing this from being re-locatable.
2019-06-20 15:18:22 -04:00
John Chilton 450ae76ddb Re-try dependency manager serialization and embedded config.
PR #8118 - adding to_dict to the dependency manager and an embedded loading of dependency resolution options from galaxy.yml - was really quite poorly thought through.

Certain options could be embedded - but only the resolvers and the actual dictified representation of the DependencyManager couldn't be actually embedded properly in the app config - but this is exactly what I wanted to do on the Pulsar side and it would make no sense to have the exact same options in both apps and configured differently.

So now the actual dictified representation of the whole DependencyManager can be directly embedded in galaxy.yml as the ``dependency_resolution:`` parameter. All the parameters can still be configured at the top level of the YAML file but with 'dependency' in the config option name (some prefixes, some suffixes, etc... a hodgepodge that has grown organically). Give the messiness of the existing options might be better to encourage using ``dependency_resolution:`` in the future.
2019-06-20 14:36:40 -04:00
M BerntandMarius van den Beek dcec15a852 include parameter name in exceptions
- to simplify tool testing its of advantage to know
  which parameter caused a problem
- for select parameters the illegal value and the
  list of legal values is added
- also unified the text of the messages

Co-Authored-By: Marius van den Beek <m.vandenbeek@gmail.com>
2019-06-20 09:49:35 -04:00
Martin Cech 1b4ff60f06 Merge pull request #8163 from bernt-matthias/topic/raise-parameter-name
Improved tool parameter error reporting
2019-06-20 09:39:28 -04:00
Martin Cech 463a1a19e5 Merge pull request #7835 from jmchilton/k8s_staging
Kubernetes 2-container pod staging.
2019-06-19 10:59:57 -04:00
M BerntandMarius van den Beek 27f33249be include parameter name in exceptions
- to simplify tool testing its of advantage to know
  which parameter caused a problem
- for select parameters the illegal value and the
  list of legal values is added
- also unified the text of the messages

Co-Authored-By: Marius van den Beek <m.vandenbeek@gmail.com>
2019-06-18 16:18:58 +02:00
John Chilton 1753e7752e Improvements to Kubernetes Pod Staging runner approach.
- Rev pulsar-galaxy-lib for improved error messages.
- Rev to a newer container - one that actually logs.
- More use of host.docker.internal to improve testing robustness.
- Add documentation for overriding container to use.
2019-06-17 11:47:27 -04:00
John Chilton ac1b25ed4c Kubernetes 2-container pod staging.
The tests included here are really very manual and brittle tests, but I'm not sure how to do much better. See documentation in the file docstring for more information.
2019-06-17 11:47:27 -04:00
mvdbeek 1d7deba912 Remove handle_panel_update
We can just call AbstractToolBox.create_section directly as the new
install operation is atomic (thank you so much @guerler!).

We don't need to have other processes create a new section, that'll
happen when we reload the toolbox anyway. Avoids a threading issue in
some circumstances that leads to tools appearing outside of sections.
Originally I added the task message in
https://github.com/galaxyproject/galaxy/commit/51c586c264f2acd912060297804d61c2539f2483
because the install process was a two step process, where we'd first
create the tool panel section (maybe in thread1) and then place the tool
into the not-yet-existing panel section in another thread.

I tried fixing that up again in
https://github.com/galaxyproject/galaxy/commit/fb69afae28c434f35860ccaa4591e95fe3acfe91,
which I think was almost correct, but send_control_task should have been
run with `noop_self` to avoid confliciting threads updating the
sections. Anyway, this is much better now, I think.
2019-06-13 23:00:29 +02:00
John Chilton d4e1a488bf Allow using galaxy-set-metadata to set metadata.
This option, combined with new "portable" metadata generation, greatly simplifies remote metadata collection with Pulsar.
2019-06-12 14:43:45 -04:00
John Chilton 7b6d06bd4e Refactor get_metadata_compute_strategy to not depend on `app`.
This allows me to claim galaxy-job-execution doesn't depend on ``app``.
2019-06-12 14:43:45 -04:00
John Chilton 83ba4517c9 Refactor galaxy.jobs.datasets into galaxy.job_execution.datasets.
Needed for metadata evaluation stuff (at least in testing) and removes some circular dependencies in the form of stuff in galaxy.tools depending on stuff in galaxy.jobs.
2019-06-12 14:43:45 -04:00
John Chilton de30bde5e4 Refactor galaxy.tools.provided_metadata -> galaxy.tool_util.
Needed for remote job output metadata collection.
2019-06-12 14:43:44 -04:00
John Chilton 33fd1b1a8a Refactor output_checker.py into galaxy-tool-util.
Needed for remote job completion.
2019-06-12 14:43:44 -04:00
John Chilton 70730f673d Allow declaring depedency resolvers right in Galaxy config, serialization. 2019-06-07 15:17:21 -04:00
Nicola Soranzo a69e03c9c7 Add missing files from galaxy-lib 2019-06-07 01:31:56 +01:00
mvdbeek bf31260192 Call tool_script.sh with shell 2019-06-01 13:24:51 +02:00
Nicola Soranzo 0e14a47e6c Merge branch 'release_19.05' into dev 2019-05-27 22:50:04 +01:00
John Chilton 0e33056989 Merge pull request #8039 from mvdbeek/non_legacy_object_store_fix
[19.05] Backport 7939: Fix non-eager object store with nested object stores
2019-05-27 12:09:34 -04:00
mvdbeek 5b05866545 Explicitly create extra_files_path when discovering outputs with extra files 2019-05-27 14:37:53 +02:00
mvdbeek 563ae4d7d3 Raise ObjectNotFound if file does not exist in object store
And return an empty string if a path does not exist when calling
`get_filename` or` get_extra_files_path` on a Dataset.
2019-05-27 14:37:53 +02:00
mvdbeek 62f5572a43 Add carriage return unit test 2019-05-27 11:35:20 +02:00
Nicola Soranzo d9b0f54906 Try to verify tool test output first as Unicode 2019-05-26 15:07:36 +01:00
mvdbeek 2d71b0bdfe Use io.open with errors="repalce" instead of comparing byte strings 2019-05-23 11:07:06 +02:00
mvdbeek ab7a38631e Unit test verify module 2019-05-22 17:45:00 +02:00
Dannon f6011bab5b Merge pull request #7992 from jmchilton/more_packages
Add galaxy-tool-util and galaxy-job-metrics packages.
2019-05-20 16:36:11 -04:00
mvdbeek 9c06527bbb Simplify convert_newlines_sep2tabs
Just call convert_newlines with a compiled regex
2019-05-19 09:30:26 +02:00
mvdbeek b67ddbf036 Drop (seemingly?) unused sep2tabs function 2019-05-19 09:30:26 +02:00
mvdbeek 3c5c7a0da2 Parametrize sniff_test 2019-05-17 15:41:14 -04:00
mvdbeek 6878e3973a Fix test_sniff unittest 2019-05-17 15:41:14 -04:00
John Chilton 0e707f2901 Improve sniffing test cases. 2019-05-17 15:41:13 -04:00
John Chilton d3f2ce009e Attempt binary conversion of newlines if UTF conversion fails. 2019-05-17 15:41:13 -04:00
John Chilton 85f9ead2a2 Improved newline conversion handling.
- Fix bug where sep2tabs didn't work with '\r\n'
- Fix bug where sep2tabs didn't work with '\r' - we were opening it with universal newlines.
- Fix memory bug where convert_newlines would read unbounded buffers - other methods still do though :(.
- Simplify convert_newlines to just let universal newline handling handle the conversion.
2019-05-17 15:41:13 -04:00