Commit Graph
4493 Commits
Author SHA1 Message Date
Sergey Golitsynskiy 502418890a Break config/ dependency on config_manage script
- Change __init__ method signature in schema.py to break its dependency
  on namedtuple defined in config_manage.py
- Adjust relevant unit test + config_manage.py
- Move config_schema.yml path to config/__init__.py
- Reverse dependency: config_manage.py now imports from
  config/__init__.py
2019-08-27 22:32:43 -04:00
Martin Cech 94e08ebb05 Merge pull request #8309 from cganote/purgegoodies
Implement undelete and purge API for users
2019-08-27 13:05:58 -04:00
Martin Cech bada84445f incorporate @nsoranzo's abstraction suggestions 2019-08-27 10:56:35 -04:00
John Chilton bd21368b27 Merge pull request #7913 from jmchilton/validation
Datatype Validation and Tool-based QC
2019-08-27 09:26:13 -04:00
Martin Cech addfeffb9e avoid calling superclass method for purging
Because multiple inheritance will result in calling this class 'delete' method
which is not the correct target. This also avoid a database operation
I believe, since the PurgableManagerMixin.purge also calls delete before purging.
2019-08-26 15:27:36 -04:00
mvdbeek bb832fbeaa Adjust testcase for new delete job behavior 2019-08-26 18:45:43 +05:30
John Chilton a47742feb8 Merge pull request #8509 from ic4f/dev_config_refactor
Factor out yaml read/write from config_manage.py
2019-08-24 11:20:40 -04:00
Nicola Soranzo 28096601b2 Merge pull request #8497 from mvdbeek/fix_external_metadata
Fix setting metadata externally
2019-08-24 09:07:58 +01:00
Sergey Golitsynskiy 9f999ea166 Factor out yaml processing out of config_manage
- Create new utility module for processing yaml
- Move yaml processing into new utility module
- Factor out Schema classes out of config_manage into schema module
- Refactor yaml reading to accommodate a unit test (minor change)
- Add unit test for new schema module

Overall: this is first step towards refactoring config.
2019-08-23 16:13:16 -04:00
John Chilton e9ee529aca Merge pull request #8502 from ic4f/dev_move_config_manage
Move config_manage out of webapps/ into config/
2019-08-23 12:02:28 -04:00
John Chilton 54c77a22c9 Cleanup of validation PR. 2019-08-23 09:43:03 -04:00
Marius van den Beek 4b93b9fc0a Merge pull request #8440 from bgruening/n_lines_validation
add assert for n_lines in tool output verifications
2019-08-22 22:13:04 +02:00
Sergey Golitsynskiy fb98cf3585 Move config_manage out of webapps/ into config/
First pass.
2019-08-22 15:46:08 -04:00
John Chilton 55b384cc50 Add ability to collect QC from jobs. 2019-08-22 15:45:25 -04:00
John Chilton f5d436f13e Datatype-based validation framework.
There was an attempt to do this over a decade ago that didn't take off:
- xref 925aec1bd0
- xref f4f9e1d7b2
2019-08-22 15:45:25 -04:00
John Chilton 6a1279e702 Merge pull request #8475 from ic4f/dev_config_reload
Enable dynamic reloading of select config options
2019-08-22 15:32:57 -04:00
Martin Cech 2fa7eba30c fix tests 2019-08-22 15:04:00 -04:00
Juleen Graham 4e7371f0e8 Addressed lint issues 2019-08-22 15:04:00 -04:00
Juleen Graham b76242c586 add api test 2019-08-22 15:04:00 -04:00
mvdbeek cd9af5b0e7 Add an API test for deleting users 2019-08-22 15:04:00 -04:00
John Chilton 5550b6b195 Merge pull request #8250 from selten/data-managers-update
Data managers - allow removal of entries
2019-08-22 11:47:57 -04:00
John Chilton 3d897b352f Merge pull request #8211 from bernt-matthias/topic/filter-static-regex
new regular expression filter, and efficient sort filter
2019-08-22 11:39:20 -04:00
mvdbeek 7266e3cd2e Don't retry metadata setting internally 2019-08-22 14:05:50 +02:00
Sergey Golitsynskiy 7e7eee3c1a Move relodable options into a function
The previous version loaded these into a constant. As a result,
importing the module implied reading a file specified by a path (in the
GALAXY_APP App named_tuple) which broke building the docs (relative
path became incorrect when module executed as script called from the
Makefile that is inside doc/

Also, modified unit tests accordingly. (assertions are grouped for
performance)
2019-08-21 15:46:08 -04:00
Sergey Golitsynskiy e73e024ade Merge remote-tracking branch 'upstream/dev' into dev_config_reload 2019-08-20 17:00:08 -04:00
Daniel Blankenberg 34ec3bfd3a FIXME note on user's API key in tool. 2019-08-20 16:38:20 -04:00
Daniel Blankenberg 725016230e Add warning on backwards compatibility of __app__ in future. 2019-08-20 16:36:29 -04:00
Daniel Blankenberg f16da013ad Set requires_domain="True" for all examples. 2019-08-20 16:30:41 -04:00
Sergey Golitsynskiy b946e531fa Load reloadable options from config_schema.yml 2019-08-20 16:26:34 -04:00
Daniel Blankenberg 00e575fa1d interactive_tool_juypter_notebook.xml -> interactive_tool_jupyter_notebook.xml 2019-08-20 16:13:30 -04:00
Peter Selten 6f9a1528f0 Merge 2019-08-20 21:22:49 +02:00
Daniel Blankenberg c3c4aefcbd Merge branch 'dev' of https://github.com/galaxyproject/galaxy into realtimetools 2019-08-20 13:44:44 -04:00
John Chilton b179c72412 Merge pull request #8078 from mvdbeek/single_task_scheduler
Delegate config file watching to single handler
2019-08-20 10:35:13 -04:00
M BerntandNicola Soranzo 779fabeadf argument tag: move name+argument parsing
- to tool_util.parser.util
- make ToolParameter.build work with InputSource
  instead of plain XML as commonly done in tests
- refactor stdio related function to tool_util.parser.stdio
  to avoid circular import

Co-Authored-By: Nicola Soranzo <nicola.soranzo@gmail.com>
2019-08-19 10:52:36 +02:00
Dannon Baker b191003626 Add suggestions/fixes from @nsoranzo -- thanks! 2019-08-18 19:25:44 +01:00
Nicola Soranzo e007cc928f Fix `items() and keys()` not supporting indexing on Py3 2019-08-18 19:25:44 +01:00
Dannon Baker d6cf262ffe Swap all odicts for OrderedDicts, remove galaxy.util.odict 2019-08-18 19:25:44 +01:00
M Bernt 96db74d0f4 add tests for static_value and regexp filter 2019-08-18 16:22:06 +02:00
mvdbeek fa9d1ef29b Fix database_heartbeat unit test 2019-08-18 16:01:32 +02:00
mvdbeek 30027a094b Rename instances of GalaxyQueueWorker from control_worker to queue_worker
It just seems more logical, and I repeatedly was using the queue_worker
instead of control_worker and wondering why things didn't work.
2019-08-18 16:01:32 +02:00
mvdbeek 6b52f62fe4 Adapt unit test to new DatabaseHeartbeat import 2019-08-18 16:01:32 +02:00
mvdbeek 79df4cb4f5 Fix import order 2019-08-18 16:01:32 +02:00
mvdbeek e73a20db02 Require explicit thread start 2019-08-18 16:01:32 +02:00
mvdbeek 2598fd3115 Fix test by monkey patching send_control_task 2019-08-18 16:01:32 +02:00
mvdbeek 3abaf24ba3 Share start/stop logic in Watcher classes 2019-08-18 16:01:32 +02:00
mvdbeek f65ed9b43d Delegate config file watching to single handler
This should reduce IO especially on instances that use a lot of
handlers and/or on a remote file system.

The process that watches the tool conf files will now
be the active process that alphabetially compares highest.
If that process goes away or is superseded by a process that
compares higher now that process will take over.
If a change has been found a control task will be sent via
the kombu messaging system.

Overall this works nicely, but while stopping the current
config watcher there is a small break during which changes
to the config don't trigger a reload, this could be handled
more gracefully.
2019-08-18 16:01:32 +02:00
Nicola Soranzo ce4b69c692 Merge pull request #8469 from jmchilton/amqp_sep
Try separate amqp connection from database for integration testing.
2019-08-17 10:39:48 +01:00
Sergey Golitsynskiy d02d627aa7 Add missing property to correct mock object 2019-08-17 02:59:03 -04:00
Sergey Golitsynskiy 81937256e5 Add missing property to mock object 2019-08-17 02:44:57 -04:00
Sergey Golitsynskiy 5188a42336 Enable dynamic reloading of select config options 2019-08-17 02:09:32 -04:00