hid-less items in a history are bad, and this is potentially one way to
produce them.
We've got a bunch of them in the history and this has been showing up in
as https://sentry.galaxyproject.org/share/issue/3cc69eedc9f841b6ae8ac02e3f88e8fe/:
```
ValidationError: 210 validation errors for HistoryContentsWithStatsResult
contents.0.HDACustom.type
Input should be 'file' [type=literal_error, input_value='collection', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/literal_error
contents.0.HDACustom.history_content_type
Input should be 'dataset' [type=literal_error, input_value='dataset_collection', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/literal_error
contents.0.HDADetailed.hid
Input should be a valid integer [type=int_type, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.5/v/int_type
contents.0.HDADetailed.type
Input should be 'file' [type=literal_error, input_value='collection', input_type=str]
For further information visit https://errors.pydantic.dev/2.5/v/literal_error
contents.0.HDADetailed.history_content_type
Input should be 'dataset' [type=literal_error, input_value='dataset_collection', input_type=str]
For fur...
File "starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "starlette/middleware/errors.py", line 186, in __call__
raise exc
File "starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "starlette_context/middleware/raw_middleware.py", line 92, in __call__
await self.app(scope, receive, send_wrapper)
File "starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File "starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
File "galaxy/webapps/galaxy/fast_app.py", line 108, in add_x_frame_options
response = await call_next(request)
File "starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "starlette/routing.py", line 758, in __call__
await self.middleware_stack(scope, receive, send)
File "starlette/routing.py", line 778, in app
await route.handle(scope, receive, send)
File "starlette/routing.py", line 299, in handle
await self.app(scope, receive, send)
File "starlette/routing.py", line 79, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "starlette/routing.py", line 74, in app
response = await func(request)
File "fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
File "fastapi/routing.py", line 193, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "starlette/concurrency.py", line 42, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
File "anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
return await future
File "anyio/_backends/_asyncio.py", line 851, in run
result = context.run(func, *args)
File "galaxy/webapps/galaxy/api/history_contents.py", line 465, in index
items = self.service.index(
File "galaxy/webapps/galaxy/services/history_contents.py", line 312, in index
return self.__index_v2(trans, history_id, params, serialization_params, filter_query_params, accept)
File "galaxy/webapps/galaxy/services/history_contents.py", line 1006, in __index_v2
return HistoryContentsWithStatsResult(contents=items, stats={"total_matches": total_matches})
```
I'm hesitatant to relax the validation here, these are really pretty bad
bugs. We do appear to have test coverage here via https://app.codecov.io/gh/galaxyproject/galaxy/commit/4a4ead2b5ee913b61a97b84b60e77f28415d5ad9/blob/lib/galaxy/tools/actions/upload_common.py#L144
Fixes https://sentry.galaxyproject.org/share/issue/97b5b9cfcef14ca0b59b851684170698/
```
AttributeError: 'NoneType' object has no attribute 'id'
File "galaxy/tools/__init__.py", line 2601, in populate_model
tool_dict = input.to_dict(request_context, other_values=other_values)
File "galaxy/tools/parameters/basic.py", line 2648, in to_dict
"id": trans.app.security.encode_id(target.id),
```
Which is in a try/except anyway, so fairly inconsequential, but results
in cleaner logs / one less sentry error.
Marginally better than an uncaught exception IMO.
Fixes
https://sentry.galaxyproject.org/share/issue/45391593603042799ef5ac5b18d22440/:
```
Exception: Failed to find workflow output named [Fasta Statistics on input dataset(s): Fasta summary stats], workflow doesn't define output by that name - valid names are ['Flye assembly on input dataset(s) (consensus)', 'Flye assembly on input dataset(s) (assembly_graph)', 'Flye assembly on input dataset(s) (Graphical Fragment Assembly)', 'Flye assembly on input dataset(s) (assembly_info)', 'Flye assembly on input dataset(s) (log)', None, 'Bandage Image on input dataset(s): Assembly Graph Image', 'Quast on input dataset(s): PDF report', 'Quast on input dataset(s): HTML report', 'Quast on input dataset(s): tabular report', 'Quast on input dataset(s): Log', 'Bar chart showing contig sizes'].
File "starlette/applications.py", line 123, in __call__
await self.middleware_stack(scope, receive, send)
File "starlette/middleware/errors.py", line 186, in __call__
raise exc
File "starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "starlette_context/middleware/raw_middleware.py", line 92, in __call__
await self.app(scope, receive, send_wrapper)
File "starlette/middleware/base.py", line 189, in __call__
with collapse_excgroups():
File "contextlib.py", line 155, in __exit__
self.gen.throw(typ, value, traceback)
File "starlette/_utils.py", line 93, in collapse_excgroups
raise exc
File "starlette/middleware/base.py", line 191, in __call__
response = await self.dispatch_func(request, call_next)
File "galaxy/webapps/galaxy/fast_app.py", line 108, in add_x_frame_options
response = await call_next(request)
File "starlette/middleware/base.py", line 165, in call_next
raise app_exc
File "starlette/middleware/base.py", line 151, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "starlette/routing.py", line 758, in __call__
await self.middleware_stack(scope, receive, send)
File "starlette/routing.py", line 778, in app
await route.handle(scope, receive, send)
File "starlette/routing.py", line 299, in handle
await self.app(scope, receive, send)
File "starlette/routing.py", line 79, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "starlette/routing.py", line 74, in app
response = await func(request)
File "fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
File "fastapi/routing.py", line 193, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
File "starlette/concurrency.py", line 42, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
File "anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "anyio/_backends/_asyncio.py", line 2144, in run_sync_in_worker_thread
return await future
File "anyio/_backends/_asyncio.py", line 851, in run
result = context.run(func, *args)
File "galaxy/webapps/galaxy/api/workflows.py", line 1485, in show_invocation_report
return self.invocations_service.show_invocation_report(trans, invocation_id)
File "galaxy/webapps/galaxy/services/invocations.py", line 143, in show_invocation_report
wfi_report = self._workflows_manager.get_invocation_report(trans, invocation_id, format=format)
File "galaxy/managers/workflows.py", line 428, in get_invocation_report
return generate_report(
File "galaxy/workflow/reports/__init__.py", line 14, in generate_report
return plugin.generate_report_json(trans, invocation, runtime_report_config_json=runtime_report_config_json)
File "galaxy/workflow/reports/generators/__init__.py", line 43, in generate_report_json
internal_markdown = self._generate_internal_markdown(
File "galaxy/workflow/reports/generators/__init__.py", line 73, in _generate_internal_markdown
internal_markdown = resolve_invocation_markdown(trans, invocation, workflow_markdown)
File "galaxy/managers/markdown_util.py", line 958, in resolve_invocation_markdown
galaxy_markdown = _remap_galaxy_markdown_calls(_remap, workflow_markdown)
File "galaxy/managers/markdown_util.py", line 1012, in _remap_galaxy_markdown_calls
return _remap_galaxy_markdown_containers(_remap_container, markdown)
File "galaxy/managers/markdown_util.py", line 971, in _remap_galaxy_markdown_containers
(replacement, whole_block) = func(replace)
File "galaxy/managers/markdown_util.py", line 1008, in _remap_container
return func(match.group(1), f"{matching_line}\n")
File "galaxy/managers/markdown_util.py", line 926, in _remap
ref_object = invocation.get_output_object(name)
File "galaxy/model/__init__.py", line 8593, in get_output_object
raise Exception(
```
```
An error occurred when refreshing user token: (invalid_request) Missing Code
```
is the message we're currently logging, that's not enough to do anything
with.
The new 1.0.0 version of pytest-celery will move towards a Docker-based
approach for smoke and production-like testing, bringing in a number of
dependencies that we don't currently need.
Since the pre-1.0.0 version of pytest-celery was just a shim for
``celery.contrib.pytest``, we can simply load the latter for API,
integration and Selenium tests.
which can contain None values.
Fixes the following traceback:
```
TypeError: sequence item 0: expected str instance, NoneType found
File "galaxy/web/framework/decorators.py", line 346, in decorator
rval = func(self, trans, *args, **kwargs)
File "galaxy/webapps/galaxy/api/tools.py", line 247, in build
return tool.to_json(trans, kwd.get("inputs", kwd), history=history)
File "galaxy/tools/__init__.py", line 2509, in to_json
populate_state(request_context, self.inputs, params.__dict__, state_inputs, state_errors)
File "galaxy/tools/parameters/__init__.py", line 412, in populate_state
_populate_state_legacy(
File "galaxy/tools/parameters/__init__.py", line 625, in _populate_state_legacy
check_param(request_context, input, param_value, context, simple_errors=simple_errors)
File "galaxy/tools/parameters/__init__.py", line 246, in check_param
value = param.from_json(value, trans, param_values)
File "galaxy/tools/parameters/basic.py", line 1045, in from_json
f"invalid options ({','.join(set(value) - set(legal_values))!r}) were selected (valid options: {','.join(legal_values)})",
```
Where `None` was in values. Note also that pyright underlined these
lines, not sure why mypy has ignored this error.
This commit updates the notification creation and sharing logic to include the `galaxy_url` field in the `NotificationCreateRequest` and `SharedItemNotificationFactory` classes. This field is used to generate links in the notification content and is passed as an argument when sending notifications to recipients.
This is useful when we know in advance the list of recipient users and that no group or roles need to be resolved. Like when we share an item with a list of users.
External notification channels (like email) are processed in Celery tasks. When Celery is not enabled in the server, we cannot show those channels to the user and we have to indicate in the API that those will be ignored.
Martin Cech