In unindeted one level too much in the previous commit, which would only reload
the last table (thanks @dannon). The alternative would be to reload all tables,
but I think this is more efficient.
In order to get rid of the conda error described in #5187 I raised the version number beginning from which conda is allowed to use the '--offline' parameter again to "4.4"
I like this better for three reasons:
- Since usually it is scripts producing this JSON - we have the most control at that point for determining the failure and we don't have to deal with an artificial dependency between the tool's stdio and the output.
- At some point we could potentially allow some datasets to be ok now even though the job fails.
- It is a cleaner interface at the Python level between job finish and output collection IMO (no need for isinstance checking).
Without this updates of data managers fails with
```
AttributeError: 'NoneType' object has no attribute 'get'
File "galaxy/web/framework/middleware/sentry.py", line 40, in __call__
iterable = self.application(environ, start_response)
File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/paste/recursive.py", line 85, in __call__
return self.application(environ, start_response)
File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/paste/httpexceptions.py", line 640, in __call__
return self.application(environ, start_response)
File "galaxy/web/framework/base.py", line 136, in __call__
return self.handle_request(environ, start_response)
File "galaxy/web/framework/base.py", line 215, in handle_request
body = method(trans, **kwargs)
File "galaxy/web/framework/decorators.py", line 98, in decorator
return func(self, trans, *args, **kwargs)
File "galaxy/webapps/galaxy/controllers/admin_toolshed.py", line 1849, in update_to_changeset_revision
persist=True)
File "tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py", line 30, in __init__
metadata_dict=metadata_dict, user=None)
File "tool_shed/metadata/metadata_generator.py", line 63, in __init__
self.metadata_dict = {'shed_config_filename': self.shed_config_dict.get('config_filename', None)}
```
This should prevent errors such as:
```
galaxy.util ERROR 2017-11-17 09:20:12,842 Error parsing file /data/users/mvandenb/gx/config/shed_tool_data_table_conf.xml
Traceback (most recent call last):
File "lib/galaxy/util/__init__.py", line 217, in parse_xml
root = tree.parse(fname, parser=ElementTree.XMLParser(target=DoctypeSafeCallbackTarget()))
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 657, in parse
self._root = parser.close()
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
ParseError: no element found: line 1, column 0
```
reported in https://github.com/galaxyproject/galaxy/issues/5031.
TypeError: Non-hexadecimal digit found
File "galaxy/web/framework/middleware/sentry.py", line 40, in __call__
iterable = self.application(environ, start_response)
File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/paste/recursive.py", line 85, in __call__
return self.application(environ, start_response)
File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/paste/httpexceptions.py", line 640, in __call__
return self.application(environ, start_response)
File "galaxy/web/framework/base.py", line 136, in __call__
return self.handle_request(environ, start_response)
File "galaxy/web/framework/base.py", line 215, in handle_request
body = method(trans, **kwargs)
File "galaxy/web/framework/decorators.py", line 98, in decorator
return func(self, trans, *args, **kwargs)
File "galaxy/webapps/galaxy/controllers/admin_toolshed.py", line 660, in manage_repositories
return self.repository_installation_grid(trans, **kwd)
File "galaxy/web/framework/helpers/grids.py", line 85, in __call__
query = self.build_initial_query(trans, **kwargs)
File "tool_shed/galaxy_install/grids/admin_toolshed_grids.py", line 334, in build_initial_query
clause_list.append(self.model_class.table.c.id == trans.security.decode_id(tool_shed_repository_id))
File "galaxy/web/security/__init__.py", line 107, in decode_id
return int(id_cipher.decrypt(obj_id.decode('hex')).lstrip("!"))
File "encodings/hex_codec.py", line 42, in hex_decode
output = binascii.a2b_hex(input)
This is due to `tool_shed_repository_ids` being a json dump of a list
in certain consitions (noticed this after installing a new repository).
This would seem to be a fairly serious memory leak in the abstract but I don't have data that it fixes anything. None the less if it gets into dev and the tests seem to pass I'll open a PR to backport it to at least 17.05 and maybe back even more.
I broke this in https://github.com/galaxyproject/galaxy/pull/4388.
I went through the other changes in that PR, and I believe this should be the
only error.
Figured out through the great detective work of @abernard.
This fixes the first error in https://github.com/galaxyproject/galaxy/issues/5031:
```
Exception in thread ToolConfWatcher.thread:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "lib/galaxy/tools/toolbox/watcher.py", line 138, in check
self.reload_callback()
File "lib/galaxy/webapps/galaxy/config_watchers.py", line 24, in <lambda>
self.tool_config_watcher = get_tool_conf_watcher(reload_callback=lambda: reload_toolbox(self.app), tool_cache=self.app.tool_cache)
File "lib/galaxy/queue_worker.py", line 92, in reload_toolbox
_get_new_toolbox(app)
File "lib/galaxy/queue_worker.py", line 111, in _get_new_toolbox
new_toolbox = tools.ToolBox(tool_configs, app.config.tool_path, app)
File "lib/galaxy/tools/__init__.py", line 226, in __init__
app=app,
File "lib/galaxy/tools/toolbox/base.py", line 1061, in __init__
super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, app)
File "lib/galaxy/tools/toolbox/base.py", line 87, in __init__
self._save_integrated_tool_panel()
File "lib/galaxy/tools/toolbox/integrated_panel.py", line 46, in _save_integrated_tool_panel
self._write_integrated_tool_panel_config_file()
File "lib/galaxy/tools/toolbox/integrated_panel.py", line 106, in _write_integrated_tool_panel_config_file
os.chmod(self._integrated_tool_panel_config, 0o644)
OSError: [Errno 2] No such file or directory: '/data/users/mvandenb/gx/config/integrated_tool_panel.xml'
```
Nicola Soranzo