opensource/galaxy
3
0
Fork 0
mirror of https://github.com/galaxyproject/galaxy.git synced 2026-10-25 00:12:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
33,011 Commits 62 Branches 243 Tags
8bee7b80cbe7c898da0c9968d1aae291da807b6d
Commit Graph
33011 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
mvdbeek 8bee7b80cb Improve error message for mismatched data inputs 2018-05-04 17:36:51 +02:00
Dannon b35340c77f Merge pull request #5997 from jmchilton/1801_tool_state_opt 
[18.01] Fix tool state performance for large collections.
 2018-05-02 08:54:55 -04:00
Dannon aebc0da891 Merge pull request #6020 from natefoo/lastz-1.3 
[18.01] lastz_wrapper_2 no longer requires Galaxy's python env as of version 1.3
 2018-04-30 16:12:43 -04:00
Nate Coraor 790ec387d7 lastz_wrapper_2 no longer requires Galaxy's python env as of version 1.3 2018-04-30 14:42:04 -04:00
Martin Cech b477329842 Merge pull request #6015 from jmchilton/fix_library_paired 
[18.01] Fix library to list of pairs collection builder.
 2018-04-30 14:15:53 -04:00
John Chilton f50fe5d455 Rebuild client. 2018-04-30 14:12:59 -04:00
John Chilton fdfe79f67e Fix library to list of pairs collection builder. 2018-04-30 11:52:42 -04:00
John Chilton fbc7d49de1 Fix for data collection parameters in tool state optimization branch. 2018-04-30 08:18:34 -04:00
Marius van den Beek abf23cdc0b Merge pull request #6010 from bgruening/test_bam_subtypes 
[18.01] enable testing for more BAM types
 2018-04-30 09:30:38 +02:00
Bjoern Gruening 717e1104c1 enable testing for more BAM types 2018-04-28 22:16:34 +02:00
Martin Cech 103cb51ec3 Merge pull request #6003 from natefoo/uwsgi-socket-docs 
[18.01] Correct uWSGI UNIX domain socket syntax in docs
 2018-04-27 13:32:35 -04:00
Martin Cech 26ab5388ba Merge pull request #5930 from natefoo/gie-late-ports 
[18.01] Only try to determine GIE container ports once at launch, then hand off to readiness check
 2018-04-27 11:37:38 -04:00
Marius van den Beek 9da13d2d3d Merge pull request #5990 from jmchilton/collection_count_fix 
[18.01] Fix collection element count getting lost during collection copies.
 2018-04-27 17:34:21 +02:00
Nate Coraor cbaff1dca2 Correct uWSGI UNIX domain socket syntax. 2018-04-27 10:49:07 -04:00
John Chilton afe938fe04 ToolBuildOptimize - fetch fewer collections, prefetch more of HDCA. 
Low hanging fruit is to exclude hidden collections, that probably won't buy much for typical uses.

This eliminates one extra tag query per dataset collection that appears in the rendered result, this probably buys us a bit more than the hidden collection thing but is still probably a good choice (unless there collections with a large number of tags :(...).

This also eliminates the extra fetch of the first, outer-est collection associated the history dataset collection - not its elements just the collection. This saves a number of queries roughly equal to the number of HDCAs in the history and unlike the tag thing there is no downside really here - there will always be one collection.

Before and after profiling of a tool form build:

https://gist.github.com/jmchilton/d68565662f7f4b7ee2640f09fbb92962
 2018-04-27 10:16:21 -04:00
John Chilton 740c4c9344 ToolBuildOptimize - do not fetch hidden datasets for inclusion. 
Only fetch visible datasets into big, cached list of history datasets under consideration. Hidden datasets don't seem to be used by the fetcher or initial value stuff so it seems fine to exclude them. The advantage should be clear for histories with a large number of datasets hidden below a signficantly smaller number collections.
 2018-04-27 10:16:21 -04:00
John Chilton 2eea566c10 ToolBuildOptimize - Add SummaryDatasetCollectionMatcher. 
Use summary data pulled from the database for collections when possible instead of loading potentially hundreds of thousands of individual datasets.
 2018-04-27 10:16:21 -04:00
John Chilton efe5d8b973 DatasetMatcherClean - Implement DatasetMatcherFactory to reason for whole tool. 
In subsequent commit I'll use this central store of all the inputs for a tool to determine if summary data about collections can be used instead of processing individual datasets one at a time.

Even this commit though uses the abstraction to optimize datatype checking and cache commons checks when possible - should lead to a lot fewer objects being created when processing a large history.
 2018-04-27 10:16:21 -04:00
John Chilton ca80232db3 DatasetMatcherClean - Remove unused method. 
It calls things that I'm changing in subsequent commits so I thought I'd just axe it now to clear things up.
 2018-04-27 10:16:21 -04:00
John Chilton 412f03d844 DatasetMatcherClean - Eliminate DatasetMatcher.value - it is never set. 
Also eliminate any logic related to it having a value. This had a purpose originally, but is no longer set. Cleaning this up makes subsequent commits a bit cleaner also.
 2018-04-27 10:16:21 -04:00
John Chilton 7ca2ef587c DatasetMatcherClean - Skip second, unneeded call to filter(). 
xref 8f813712f5 to some degree.
 2018-04-27 10:16:21 -04:00
John Chilton c3303939c0 ToolBuildOptimize - Optimize state checking in dataset matcher. 
Pre-calculate valid states, skip now unneeded helper method.
 2018-04-27 10:16:21 -04:00
John Chilton 0ca1401bd8 ToolBuildOptimize - Eliminate check_security on dataset matcher. 
It was only used for collections and we need to drop it as prohibitively expensive to calculate. No need to filter collections ahead of time that way anyhow - it is the tool action's job to block the execution of datasets without permission so hopefully we aren't deriving any security value from this filter.
 2018-04-27 10:16:21 -04:00
Martin Cech c7b426089a Merge pull request #6000 from jmchilton/fix_build_maybe 
[18.01] Backport #5972
 2018-04-27 10:11:24 -04:00
John Chilton a29770ca55 Pin the version of node we use in CI, etc.... 
Our dependencies won't resolve under node 10 currently (because of upath at least https://github.com/anodynos/upath/blob/master/package.json#L43 but maybe other packages as well).
 2018-04-27 08:09:51 -04:00
Nicola Soranzo a3e6963666 Merge branch 'release_17.09' into release_18.01 2018-04-26 19:01:48 +01:00
Marius van den Beek d5ec01d5da Merge pull request #5986 from nsoranzo/release_17.09_doc_fixes 
[17.09] Fix 2 typos in the docs
 2018-04-26 19:26:14 +02:00
John Chilton a7a4e12e79 Fix collection element count getting lost during collection copies. 2018-04-26 12:56:05 -04:00
Nicola Soranzo 7d81d2cfcb Fix 2 typos in the docs 2018-04-26 10:23:14 +01:00
Björn Grüning 1b1962ea49 Merge pull request #5924 from mvdbeek/fix_data_managers 
[18.01] Provide ToolDataTableManager for backwards compatibility
 2018-04-25 21:47:41 +02:00
Dannon e68eaeecbf Merge pull request #5981 from natefoo/job-doc-fixes 
[18.01] Minor Job documentation corrections
 2018-04-25 13:29:41 -04:00
Nate Coraor e58034b987 Correct examples in dynamic job conf doc, remove handlers section from 
basic sample job config (defining a single default handler is no longer
needed). Fixes #5976.
 2018-04-25 13:10:04 -04:00
John Chilton 20e9934bc5 Merge pull request #5958 from abretaud/movemerge1801 
[18.01] Backport to 18.01 the move_merge fix from #5922
 2018-04-24 16:37:08 -04:00
Martin Cech 88bf6bbdaa Merge pull request #5943 from erasche/fix-ftp-composite-upload 
[18.01] Correct access for FTP files in composite datatype
 2018-04-24 11:31:40 -04:00
Dannon Baker 9f8021f1b4 Merge remote-tracking branch 'upstream/release_17.09' into release_18.01 2018-04-23 16:02:15 -04:00
Dannon Baker 8e6cdd73aa Merge remote-tracking branch 'upstream/release_17.05' into release_17.09 2018-04-23 16:01:42 -04:00
Dannon d80e60ce74 Merge pull request #5947 from nsoranzo/release_17.05_fix_gatk2_install 
[17.05] Resort to os.path.exists() also for paths not ending in .loc
 2018-04-23 15:59:32 -04:00
Anthony Bretaudeau 3ab2afb5e5 backport move_merge fix from #5922 2018-04-23 17:19:17 +02:00
Dannon 2d772d74b9 Merge pull request #5956 from bernt-matthias/topic/cleanup_for18.01 
removed config.ini parameter from cleanup scripts
 2018-04-23 09:47:41 -04:00
Matthias Bernt 0c321cb86e removed config.ini parameter from cleanup scripts 
fixes https://github.com/galaxyproject/galaxy/issues/5941
 2018-04-23 14:58:48 +02:00
Nicola Soranzo 36a182a6c8 Resort to os.path.exists() also for paths not ending in .loc 
Fix installation of gatk2 iuc's repo, which otherwise does not
add the data tables to `config/shed_tool_data_table_conf.xml` .

Introduced in commit 43c9c287a9 .
 2018-04-21 01:12:22 +01:00
Bjoern Gruening a5c75d2f95 Correct access for FTP files in composite datatype 2018-04-20 17:12:12 +02:00
Nate Coraor 50f9ca82ba Don't attempt to determine ports at GIE container launch time if they 
fail to be determined on the first attempt. They'll be determined by the
readiness check instead.
 2018-04-19 12:10:34 -04:00
John Chilton 393f055a23 Merge pull request #5928 from erasche/document-gh-reporter 
[18.01] forgot to document github bug reporter
 2018-04-19 11:57:53 -04:00
E Rasche fe0a4853b9 forgot to document github bug reporter 2018-04-19 17:13:32 +02:00
Martin Cech c85f07cf5f Merge pull request #5921 from erasche/fix-docs 
[18.01] correct signal num in docs
 2018-04-19 10:54:58 -04:00
mvdbeek cd9db0bffa Provide ToolDataTableManager for backwards compatibility 
Broken in 7df9dce63d
 2018-04-19 15:22:06 +02:00
E Rasche 3182dfa70e correct signal num 2018-04-19 13:20:17 +02:00
Dannon c4e1242312 Merge pull request #5893 from martenson/backport-skip 
[18.01] Skip new script tests in dev branch, breaking integration suite
 2018-04-16 16:22:37 -04:00
Nicola Soranzo 1ff16ddfc4 Exclude only test_grt_export integration test 
which takes 35' by itself.
 2018-04-16 14:18:46 -04:00