Commit Graph
1058 Commits
Author SHA1 Message Date
John Chilton d69962e7f4 Implement model abstraction for `ImplicitCollectionJobs`.
These aggregate all the jobs contributing to an implicitly created HistoryDatasetCollectionAssociation.

- Model enhancements that allow this.
- API endpoint for jobs summary of history content (somewhat uniform across HDAs and HDCAs just for consistency).
- Test cases.
- Rework history contents API a bit to eliminate more costly state summary checks by default.
2017-11-30 10:02:02 -05:00
Nate Coraor 49158ba38b Add multi-farm support to the main script 2017-11-17 17:09:36 -05:00
Nate Coraor e858ceb84a Do not autostart uWSGI mules 2017-11-17 17:05:43 -05:00
Nate Coraor b4863b2df8 Fix lint errors 2017-11-17 13:59:25 -05:00
Nate Coraor 1ec960bcf6 Improve shutdown cleanliness under uWSGI. 2017-11-17 13:52:24 -05:00
Nate Coraor c18521d7f7 Move signal handling from uWSGI stack to main 2017-11-17 13:51:13 -05:00
Nate Coraor 418fd53739 Merge remote-tracking branch 'origin/dev' into job-handler-mules 2017-11-15 12:42:20 -05:00
Nate Coraor c340d4ebec Fix tool migrations with config file changes, do everything from
manage_db.sh.
2017-11-15 12:37:22 -05:00
Nicola Soranzo bbd96c435b Merge pull request #4903 from jmchilton/dev_at_same_time
Install dev and regular dependencies at the same time.
2017-11-15 11:47:40 +00:00
Nicola Soranzo 2cd95c48f6 Fix import order everywhere
- Add flake8-import-order to flake8 Pipfile and remove py27-lint-imports
  and py27-lint-imports-include-list tox envs
- Fix most E201 and E202 errors reported by flake8-import-order v0.15,
  but pin flake8-import-order to v0.14.3 until
  https://github.com/PyCQA/flake8-import-order/issues/123
  is fixed

This let us drop 2 jobs on Travis per each job.
2017-11-14 19:42:39 +00:00
Nate Coraor f5ee540df1 To avoid duplication, move common utilities used by external scripts to
galaxy.util.script. Update one such script to use the new module.
2017-11-13 12:18:09 -05:00
Nate Coraor 806438edc9 Revert script changes unrelated to uWSGI mule work. Includes
galaxy.util.script so that get_uwsgi_args.py still works, but that will
be PR'd separately and merged later.
2017-11-13 12:08:28 -05:00
Nate Coraor b75583ea26 Fix post-merge lint issues 2017-11-09 14:00:46 -05:00
Nate Coraor 30266c5221 Merge remote-tracking branch 'origin/dev' into job-handler-mules 2017-11-09 13:11:46 -05:00
John Chilton a1e91231aa If installing dev dependencies, do so along side regular dependencies.
Catch conflicts and works faster.
2017-11-02 09:49:08 -04:00
Nicola Soranzo 07d11c6d0b Merge pull request #4876 from jmchilton/update_linting_deps
Absolute pins on linting dependencies.
2017-10-31 21:12:58 +00:00
Nicola Soranzo aacdf8d939 Merge branch 'release_17.09' into dev
Conflicts:
	static/style/blue/base.css
2017-10-30 19:15:50 +00:00
Nicola Soranzo 45d4fefe63 Merge pull request #4898 from jmchilton/fix_extra_index
[17.09] Fix extra index URL call in common startup.
2017-10-30 15:52:41 +00:00
John Chilton 09fdac513d Fix extra index URL call in common startup.
- Use it for dev requirements also.
- Use more correct API target that supports older versions of dependencies on PyPI.

xref https://gist.github.com/jmchilton/ad8f180f5ac1eb5f689f815b04f603fa
2017-10-30 10:49:29 -04:00
John Chilton ccb5f61845 Absolute pins on linting dependencies.
- Use completely pinned and hashed dependencies for more consistent, reproducible linting.
- Leverage the awesomeness of pipenv to automate updating these.
- These will always be locked now so builds don't break randomly on other people updating their software. When we finally have the bot up and running that forward merges fixes for Galaxy we can add another if statement that runs this check against dev once a day and opens a PR if needed.
2017-10-26 15:53:25 -04:00
John Chilton 62826a1160 Merge pull request #4847 from nsoranzo/flake8_3.5.0
Fix all E722 errors and ignore E741
2017-10-24 13:04:22 -04:00
Martin Cech f08700b208 rename wiki to hub 2017-10-24 10:48:16 -04:00
Nicola Soranzo 9b4de72ca6 Fix all E722 errors and ignore E741
Introduced in flake8 3.5.0
Fix import order.
2017-10-24 11:29:09 +01:00
Nate Coraor b8dfceb72a Merge branch 'release_17.09' into dev 2017-10-23 10:38:14 -04:00
Nate Coraor 646a8d1092 Support a log directory argument for pgcleanup.py 2017-10-20 12:43:09 -04:00
E Rasche 0f311470d9 Backport of hardening fixes to 17.09 2017-10-18 13:03:49 -04:00
Nicola Soranzo b956406c4d Fix Python3 compatibility and import order for 21 files 2017-10-16 19:43:01 +01:00
Nicola Soranzo d1bc98dfcb Python3: fix remaining print statements
Remove some debugging prints.
Fix import order.
Consolidate exception logging in ToolShed database migrations.
2017-10-12 18:34:29 +01:00
Nicola Soranzo 9b1832abb1 Fix import order for cron/, scripts/ and test/ 2017-09-25 23:44:39 +01:00
Nicola Soranzo 8c984c9b3e Remove unused binary_compatibility.py script
Follow-up on commit 39d78c7226 from
PR https://github.com/galaxyproject/galaxy/pull/2383
2017-09-25 23:37:23 +01:00
John Chilton 0c6dfd7a41 Merge pull request #4604 from erasche/hardening
Various bits of hardening
2017-09-25 15:15:48 -04:00
Nicola Soranzo 0c23d40b22 Merge branch 'release_17.09' into dev 2017-09-25 10:56:37 +01:00
E Rasche 15c73233c5 fix lint 2017-09-25 11:39:15 +02:00
E Rasche 290971ca7a more review comments 2017-09-25 11:39:15 +02:00
E Rasche 9b8e76f9d9 Convert urllib to requests
Which has built-in protection against urls like "file:///tmp/a"
2017-09-25 11:39:13 +02:00
E Rasche 32b85ecb59 Only permit yaml.safe_loading of data
Event trusted data, belt + suspenders method.
2017-09-25 11:39:12 +02:00
John Chilton 8fa1f9a2fe Small tweaks to fixup #2866. 2017-09-22 14:47:15 -04:00
Nate Coraor 244fc834cd Explicitly include PyPI as an --extra-index-url to pip so we can install
from PyPI even when a package exists (but the correct version or
architecture does not exist) on wheels.galaxyproject.org. Previously,
PyPI was only checked after wheels.galaxyproject.org because pypiserver
performs an automatic redirect to PyPI for any packages that do not
exist in its index.
2017-09-21 16:33:40 -04:00
Martin Cech 095c1045b6 add logging to ts index building 2017-09-06 14:05:05 -04:00
Nate Coraor 09feaf6f4d - Do all logging configuration other than paste fileConfig logging from
dictConfig. Default log message format varies by stack.
- Refactor a lot of file extension handling in to galaxy.util.path.
- Simplify and improve config file finding.
- Simplify and improve native app creation.
- Integrate uWSGI argument generation with @jmchilton's improvements to
  the startup scripts.
- Fix starting without a config file under uWSGI
2017-09-01 01:08:16 -04:00
Nate Coraor bcfa874158 Fix for job_handler_count = 0 2017-08-28 16:52:06 -04:00
Nate Coraor 4d962d4ad2 Only include --paste-logger in uWSGI args if logging is configured in
the PasteDeploy config
2017-08-28 16:34:02 -04:00
Nate Coraor 1326a4d671 Cleaning, lint and test fixes. 2017-08-28 16:22:11 -04:00
Nate Coraor 17ea3d671f Improve automatic uWSGI startup options interaction with [uwsgi] section
in config file
2017-08-28 15:18:09 -04:00
Nate Coraor aff0a7460e Finish dictConfig logging feature 2017-08-28 14:13:31 -04:00
Nate Coraor 14e9dddade WIP: support log configuration from yaml/json app config with
logging.config.dictConfig. Still working on yaml/json config loading in
the webless app.
2017-08-28 09:54:21 -04:00
Nate Coraor 68542b615c Remove unneeded server_name manipulation from galaxy-main script 2017-08-21 21:57:56 -04:00
Nate Coraor d7dd9b2b65 WIP: drop stack configurability, this will come later. Cleanup for WIP
PR
2017-08-21 21:50:46 -04:00
Nate Coraor 27a0b1ba40 WIP: attempted to support workers shared between pools, but locking
won't allow for this, since you can't wait on messages from a specific
farm.
2017-08-21 21:48:01 -04:00
Nate Coraor ece3f242fb Abandoning unprogrammed mules (for now), fixes for programmed mules 2017-08-21 21:46:39 -04:00