Commit Graph
1308 Commits
Author SHA1 Message Date
mvdbeek c8392299c8 Make processes and mules report heartbeat in database
It's important to not set the DatabaseHeartbeat `server_name` prior to forking,
which we avoid be getting `server_name` from
`application_stack.app.config.server_name`.
`WorkerProcess.server_name` for uwsgi web processes will look like:
`main.web.1` and mules like `main.job-handlers.1` while webless workers
remain named `handler1`.
2019-03-29 21:46:18 +01:00
Dannon Baker 485eece7f3 Grammatical tweaks. 2019-03-26 09:31:33 -04:00
Dannon Baker 8bb2b6bfbc Add extra verbosity covering non-client-building branches in startup. 2019-03-26 09:04:45 -04:00
Dannon Baker 2bb159ef52 Merge remote-tracking branch 'upstream/release_19.01' into dev 2019-03-22 10:49:37 -04:00
Martin Cech e65604bc06 Merge pull request #7556 from jmchilton/expression_tools
Implement expression tools and non-data tool outputs.
2019-03-20 18:52:59 -04:00
John Chilton 8d982b6410 Merge pull request #7532 from dannon/plugin-staging
Static plugin staging as a part of client build instead of startup
2019-03-20 13:47:17 -04:00
mvdbeek 5e778f4812 Install node as part of common_startup.sh 2019-03-20 10:02:56 +01:00
John Chilton 528497f7c4 Refactor galaxy.web.security into galaxy.security.idencoding.
I want to be able to use this class in galaxy.model and it doesn't have any dependnecies on "web" stuff so I'd like to move it out of galaxy.model and into galaxy.security. galaxy.model shouldn't have dependencies on galaxy.web (and doesn't after the recent 976f5ad367), so this pre-emptively ensures it doesn't require these dependencies for https://github.com/galaxyproject/galaxy/pull/7367.

This has the benefit of also eliminating any galaxy.web dependencies from the generic test base file (test/base/testcase.py) which has long been a goal of mine as well.

There are places we legimately use ID encoding and decoding for security (i.e. job files API) but for the most part it doesn't provide security for API endpoints and this causes confusion repeatedly, so I've started the process of renaming the contained class SecurityHelper to something I feel makes it clearer this is just about encoding and decoding IDs.
2019-03-19 14:28:31 -04:00
Dannon Baker 085d83d34c Fix template name substitution var 2019-03-19 10:45:20 -04:00
Dannon Baker 5d771424bf Drop debug github object print 2019-03-19 10:24:55 -04:00
Dannon Baker 29eca308f4 Drop static-safe navigation to plugin dirs. These are no longer symlinks, they are staged files. 2019-03-14 13:23:45 -04:00
Dannon Baker d72643fc2a Update plugin_staging to actually stage instead of symlink, since it no longer needs to be quite as fast (on startup). This will allow us to drop unnecessary trust rules from uwsgi static. 2019-03-14 13:18:43 -04:00
Dannon Baker 99082ff486 Drop plugin staging from common_startup 2019-03-14 11:40:52 -04:00
Helena Rasche ea0700a032 fix some small bugs 2019-03-14 10:09:36 +01:00
Helena Rasche ec086c1e8b Attempt to automate this? 2019-03-14 10:02:23 +01:00
Hervé Ménager 557407c384 correct virtualenv download with no wget/curl
the embedded python code needs a small correction to work in the no curl/wget available case.
2019-03-06 16:24:07 +01:00
Nicola Soranzo 84648f4986 Remove or replace references to Biostars 2019-03-01 22:59:16 +00:00
Nate Coraor ef8f208002 Merge remote-tracking branch 'upstream/release_19.01' into dev 2019-02-22 13:24:15 -05:00
Dannon Baker 726ad9a0f6 Bump yarn's network timeout from 2 minutes to 5 minutes. 2019-02-21 11:48:11 -05:00
Nicola Soranzo c6f1183939 Update documentation about dependencies 2019-02-18 22:36:32 +00:00
Sergey Golitsynskiy c21e07e1a0 Do not default to DEV_WHEELS=1 if no Git is found
If there is no Git repo in a Galaxy install, DEV_WHEELS will not be
turned on by default. If they are needed, use the --dev-wheels flag.

(Current behavior: common_startup.sh will turn on the DEV_WHEELS option
if no git repo is found, making the assumption that it is dealing with a
dev install. Thus, dev dependencies are installed whenever there's no
Git repo, including the use case when Galaxy was downloaded as a tarball
- when they may or may not be desired. There is no option to override
this behavior without editing the script.)
2019-02-15 17:05:48 -05:00
Dannon Baker 4427d3a982 Separate client node version requirement into individual dotfile for easier reading/use in ansible playbooks 2019-02-13 14:52:08 -05:00
Nicola Soranzo 21ce99e231 Merge branch 'release_19.01' into dev 2019-02-12 12:03:03 +00:00
Nicola Soranzo b33b06440f Merge branch 'release_18.09' into release_19.01 2019-02-12 11:51:07 +00:00
Nicola Soranzo 6589bf3314 Drop support for Python 3.4. Officially support Python >=3.5
xref. https://github.com/galaxyproject/galaxy/issues/7260
2019-01-31 23:31:44 +00:00
Dannon Baker a10d2c51d3 Add/pin configurable node version to 18.09. Backport/tweak of https://github.com/galaxyproject/galaxy/pull/7220) 2019-01-18 11:54:59 -05:00
Dannon Baker bad8cffdbc Verify .venv's node version is in line with what we've specified as being locked, rather than just checking if node exists. 2019-01-15 14:24:12 -05:00
Nate Coraor 473210714e Add support for dynamic handlers when using DB serialization handler assignment 2018-12-18 13:34:31 -05:00
Nicola Soranzo b22b7e5868 Merge branch 'release_18.09' into dev 2018-12-16 23:02:41 +00:00
Nicola Soranzo 975c8b605f Fix activation of `_galaxy_` conda env
- Use `conda_activate`, which works also under dash
- Run `conda_activate` also when `$GALAXY_CONDA_ENV` already exists
- Run `virtualenv` when the conda env is active but `$GALAXY_VIRTUAL_ENV`
  does not exist
- Update the default for `GALAXY_CONDA_ENV` in
  scripts/common_startup_functions.sh

Follow-up on https://github.com/galaxyproject/galaxy/pull/6975 .
2018-12-14 00:22:12 +00:00
Björn Grüning 6db721915f Merge pull request #7041 from erasche/fix-grt
[18.09] GRT: Rip out all parameters, round file sizes
2018-12-12 19:22:32 +01:00
Nicola Soranzo 16f3be66be Merge branch 'release_18.09' into dev 2018-12-06 14:27:16 +00:00
Dannon Baker 66fae561b2 Don't use pushd/popd 2018-12-02 22:04:45 -05:00
Dannon Baker fa408f4850 exit with error code (and appropriate message) on either build failure 2018-11-29 11:19:49 -05:00
Dannon Baker 77aec5a79d Standardize on spaces in common_startup.sh (tabs were mixed in here) 2018-11-29 11:11:37 -05:00
Dannon Baker be43693ef0 Avoid using make for autobuild; reducing necessary dependencies out of the box. 2018-11-29 11:11:11 -05:00
Nicola Soranzo 9007661765 Merge branch 'release_18.09' into dev 2018-11-23 17:10:04 +00:00
Nicola Soranzo 992ea5e4b9 Pass through `--daemon` option to gunicorn
Documented at
http://docs.gunicorn.org/en/stable/settings.html#daemon
2018-11-23 14:56:15 +00:00
Helena Rasche 855ccb1279 GRT: Rip out all parameters, round file sizes 2018-11-23 12:10:14 +01:00
Nicola Soranzo eaacc5bc9f Merge branch 'release_18.09' into dev 2018-11-15 14:06:09 +01:00
Nicola Soranzo fbd2284e5d Fix PID file handling for gunicorn
gunicorn uses `--pid`, not `--pid-file`.
2018-11-15 11:27:14 +01:00
Nicola Soranzo a53629ebae Update virtualenv installation to v16.1.0
which bundles `wheel` 0.32.2.

The old virtualenv 13.1.2 included wheel 0.24.0, where tagging is broken
for Python >=3.5, producing tracebacks like:

```
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-c53eutpg/uwsgi/setup.py", line 138, in <module>
    'Programming Language :: Python :: 3.6',
  File "/tmp/wheel_venv/lib/python3.7/site-packages/setuptools/__init__.py", line 143, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/tmp/wheel_venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 213, in run
    archive_basename = self.get_archive_basename()
  File "/tmp/wheel_venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 161, in get_archive_basename
    impl_tag, abi_tag, plat_tag = self.get_tag()
  File "/tmp/wheel_venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 155, in get_tag
    assert tag == supported_tags[0]
AssertionError
```

This was fixed in wheel 0.25.0. Reported by @wm75.

Also update `urlretrieve` import for Python 3.
2018-11-13 17:15:54 +01:00
Marius van den Beek be76d9f801 Merge pull request #6995 from bernt-matthias/topic/external-runner-json-load-order
external_runner: load json template first
2018-11-11 11:51:07 +01:00
Dannon 3ed0ce862b Merge pull request #6975 from mvdbeek/conda_run_sh
Use virtualenv when conda is on PATH
2018-11-09 15:45:54 -05:00
M Bernt 97ba92f3cf external_runner: load json template first
if the JSON is loaded first the cluster_files_directory
does not need to be accessible by the real user (ie all users)
2018-11-09 17:48:36 +01:00
Dannon Baker e8dbb8c58a Upgrade nodeenv's nodejs to 10.13.0 LTS, now that the client build works fine with v10 2018-11-08 10:02:16 -05:00
mvdbeek 9f8c29520d Use virtualenv when conda is on PATH
This uses our proper pinned requirements and is sginificantly easier
to set up and maintain.
2018-11-07 14:20:05 +01:00
Dannon Baker 7dbc81ca82 Add images to default static-safe location 2018-10-30 17:41:14 -04:00
Nicola Soranzo c591691209 Fix typos in `scripts/bootstrap_history.py` 2018-10-29 21:08:41 +00:00
Dannon Baker 5d51492a53 Getgalaxy links 2018-10-29 11:54:05 -04:00