Merge pull request #21922 from jmchilton/toolshed_2_cleanup_v2

Make Shed 2 the Default - Drop Legacy Shed Tests in CI
This commit is contained in:
Marius van den Beek
2026-02-25 11:47:40 +01:00
committed by GitHub
54 changed files with 293 additions and 2566 deletions
+1 -4
View File
@@ -24,7 +24,6 @@ jobs:
fail-fast: false
matrix:
python-version: ['3.10', '3.14']
shed-api: ['v1', 'v2']
test-install-client: ['galaxy_api', 'standalone']
services:
postgres:
@@ -77,11 +76,9 @@ jobs:
run: ./run_tests.sh -toolshed
env:
TOOL_SHED_TEST_INSTALL_CLIENT: ${{ matrix.test-install-client }}
TOOL_SHED_API_VERSION: ${{ matrix.shed-api }}
TOOL_SHED_TEST_BROWSER: ${{ matrix.shed-api == 'v1' && 'twill' || 'playwright' }}
working-directory: 'galaxy root'
- uses: actions/upload-artifact@v6
if: failure()
with:
name: Toolshed test results (${{ matrix.python-version }}, ${{ matrix.shed-api }}, ${{ matrix.test-install-client }})
name: Toolshed test results (${{ matrix.python-version }}, ${{ matrix.test-install-client }})
path: 'galaxy root/run_toolshed_tests.html'
+14 -15
View File
@@ -8,10 +8,10 @@ Integration and browser tests for the Tool Shed. Requires a running server.
# From packages/tool_shed directory
# Start shed in another terminal first
TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
./run_tool_shed.sh
# Run all functional tests
TOOL_SHED_API_VERSION=v2 uv run pytest tool_shed/test/functional/ -v
uv run pytest tool_shed/test/functional/ -v
# Run specific test
uv run pytest tool_shed/test/functional/test_frontend_login.py -v
@@ -22,6 +22,7 @@ uv run pytest tool_shed/test/functional/test_frontend_login.py -v
### Numbered Tests (`test_0xxx`, `test_1xxx`)
Legacy comprehensive tests using Twill/API:
- `test_0xxx` - Tool Shed functionality (repos, dependencies, metadata)
- `test_1xxx` - Galaxy installation scenarios
@@ -31,22 +32,20 @@ These test complex multi-step workflows and dependency chains.
Modern Playwright/API tests:
| File | Description |
|------|-------------|
| `test_frontend_*.py` | Browser UI tests (Playwright) |
| `test_shed_*.py` | API-level tests |
| `test_repositories_integration.py` | Repository operations |
| `test_component_showcase.py` | Component visual tests |
| File | Description |
| ---------------------------------- | ----------------------------- |
| `test_frontend_*.py` | Browser UI tests (Playwright) |
| `test_shed_*.py` | API-level tests |
| `test_repositories_integration.py` | Repository operations |
| `test_component_showcase.py` | Component visual tests |
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `TOOL_SHED_API_VERSION` | v1 | Set to `v2` for modern API |
| `TOOL_SHED_TEST_BROWSER` | playwright | Browser: `playwright` or `twill` |
| `TOOL_SHED_TEST_EXTERNAL` | - | Use external running shed |
| `TOOL_SHED_TEST_HOST` | localhost | Host for external shed |
| `TOOL_SHED_TEST_SCREENSHOTS` | - | Directory for test screenshots |
| Variable | Default | Description |
| ---------------------------- | --------- | ------------------------------ |
| `TOOL_SHED_TEST_EXTERNAL` | - | Use external running shed |
| `TOOL_SHED_TEST_HOST` | localhost | Host for external shed |
| `TOOL_SHED_TEST_SCREENSHOTS` | - | Directory for test screenshots |
## Test Patterns
-20
View File
@@ -1,5 +1,4 @@
import os
from functools import wraps
from typing import (
Any,
Optional,
@@ -102,25 +101,6 @@ class ShedGalaxyInteractorApi(GalaxyInteractorApi):
super().__init__(**interactor_kwds)
def make_skip_if_api_version_wrapper(version):
def wrapper(method):
@wraps(method)
def wrapped_method(api_test_case, *args, **kwd):
interactor: ShedApiInteractor = api_test_case.api_interactor
api_version = interactor.api_version
if api_version == version:
raise pytest.skip(f"{version} tool shed API found, skipping test")
return method(api_test_case, *args, **kwd)
return wrapped_method
return wrapper
skip_if_api_v1 = make_skip_if_api_version_wrapper("v1")
skip_if_api_v2 = make_skip_if_api_version_wrapper("v2")
class ShedApiTestCase(ShedBaseTestCase, UsesShedApi):
_galaxy_interactor: Optional[GalaxyInteractorApi] = None
+64 -109
View File
@@ -692,7 +692,7 @@ class ShedTwillTestCase(ShedApiTestCase):
@property
def invalid_tools_labels(self) -> str:
return "Invalid Tools" if self.is_v2 else "Invalid tools"
return "Invalid Tools"
def create(
self,
@@ -702,39 +702,22 @@ class ShedTwillTestCase(ShedApiTestCase):
username: str = "admin-user",
redirect: Optional[str] = None,
) -> tuple[bool, bool, bool]:
# HACK: don't use panels because late_javascripts() messes up the twill browser and it
# can't find form fields (and hence user can't be logged in).
params = dict(cntrller=cntrller, use_panels=False)
self.visit_url("/user/create", params)
self._submit_register_form(
email,
password,
username,
redirect,
)
previously_created = False
username_taken = False
invalid_username = False
if not self.is_v2:
try:
self.check_page_for_string("Created new user account")
except AssertionError:
try:
# May have created the account in a previous test run...
self.check_page_for_string(f"User with email '{email}' already exists.")
previously_created = True
except AssertionError:
try:
self.check_page_for_string("Public name is taken; please choose another")
username_taken = True
except AssertionError:
# Note that we're only checking if the usr name is >< 4 chars here...
try:
self.check_page_for_string("Public name must be at least 4 characters in length")
invalid_username = True
except AssertionError:
pass
return previously_created, username_taken, invalid_username
return self._ensure_user_via_api(email, password, username)
def _ensure_user_via_api(self, email: str, password: str, username: str) -> tuple[bool, bool, bool]:
"""Create user via admin API if not already present.
Returns (previously_created, username_taken, invalid_username).
"""
admin = self.admin_api_interactor
all_users = admin.get("users").json()
if any(u["username"] == username for u in all_users):
return (True, False, False)
response = admin.post("users", json={"email": email, "username": username, "password": password})
if response.status_code != 200:
# User creation failed (e.g. reserved username)
return (False, False, True)
return (False, False, False)
def last_page(self):
"""
@@ -758,34 +741,48 @@ class ShedTwillTestCase(ShedApiTestCase):
logout_first: bool = True,
explicit_logout: bool = False,
):
if self.is_v2:
# old version had a logout URL, this one needs to check
# page if logged in
self.visit_url("/")
# Clear cookies.
self.visit_url("/")
if logout_first:
self.logout(explicit=explicit_logout)
# test@bx.psu.edu is configured as an admin user
# Ensure user exists via API
previously_created, username_taken, invalid_username = self.create(
email=email, password=password, username=username, redirect=redirect
)
# v2 doesn't log you in on account creation... so force a login here
if previously_created or self.is_v2:
# The account has previously been created, so just login.
# HACK: don't use panels because late_javascripts() messes up the twill browser and it
# can't find form fields (and hence user can't be logged in).
params = {"use_panels": False}
self.visit_url("/user/login", params=params)
self.submit_form(button="login_button", login=email, redirect=redirect, password=password)
@property
def is_v2(self) -> bool:
return self.api_interactor.api_version == "v2"
if invalid_username:
return
# Re-visit root after logout to get fresh session + CSRF token
self.visit_url("/")
cookies = self._page.context.cookies()
csrf_token = ""
for cookie in cookies:
if cookie["name"] == "session_csrf_token":
csrf_token = cookie["value"]
break
# Establish browser session via internal login API
self._page.evaluate(
"""async ([login, password, csrfToken]) => {
const response = await fetch('/api_internal/login', {
method: 'PUT',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
login: login,
password: password,
session_csrf_token: csrfToken,
})
});
if (!response.ok) {
const text = await response.text();
throw new Error('Login failed: ' + response.status + ' ' + text);
}
return await response.json();
}""",
[email, password, csrf_token],
)
# Reload to pick up the new session
self.visit_url("/")
@property
def _playwright_browser(self) -> PlaywrightShedBrowser:
# make sure self.is_v2
browser = self._browser
assert isinstance(browser, PlaywrightShedBrowser)
return browser
@@ -802,14 +799,10 @@ class ShedTwillTestCase(ShedApiTestCase):
and be explicit in logging out to provide extract test
structure.
"""
if self.is_v2:
if explicit:
self._playwright_browser.explicit_logout()
else:
self._playwright_browser.logout_if_logged_in()
if explicit:
self._playwright_browser.explicit_logout()
else:
self.visit_url("/user/logout")
self.check_page_for_string("You have been logged out")
self._playwright_browser.logout_if_logged_in()
def submit_form(self, form_no=-1, button="runtool_btn", form=None, **kwd):
"""Populates and submits a form from the keyword arguments."""
@@ -860,15 +853,7 @@ class ShedTwillTestCase(ShedApiTestCase):
self.check_for_strings(strings_displayed=["Role", "has been associated"])
def browse_category(self, category: Category, strings_displayed=None, strings_not_displayed=None):
if self.is_v2:
self.visit_url(f"/repositories_by_category/{category.id}")
else:
params = {
"sort": "name",
"operation": "valid_repositories_by_category",
"id": category.id,
}
self.visit_url("/repository/browse_valid_categories", params=params)
self.visit_url(f"/repositories_by_category/{category.id}")
self.check_for_strings(strings_displayed, strings_not_displayed)
def browse_repository(self, repository: Repository, strings_displayed=None, strings_not_displayed=None):
@@ -882,10 +867,7 @@ class ShedTwillTestCase(ShedApiTestCase):
self.check_for_strings(strings_displayed, strings_not_displayed)
def browse_tool_shed(self, url, strings_displayed=None, strings_not_displayed=None):
if self.is_v2:
url = "/repositories_by_category"
else:
url = "/repository/browse_valid_categories"
url = "/repositories_by_category"
self.visit_url(url)
self.check_for_strings(strings_displayed, strings_not_displayed)
@@ -929,14 +911,8 @@ class ShedTwillTestCase(ShedApiTestCase):
depends_on_changeset_revision=None,
changeset_revision=None,
):
if not self.is_v2:
# v2 doesn't display repository repository dependencies, they are deprecated
strings_displayed = [depends_on_repository.name, depends_on_repository.owner]
if depends_on_changeset_revision:
strings_displayed.append(depends_on_changeset_revision)
self.display_manage_repository_page(
repository, changeset_revision=changeset_revision, strings_displayed=strings_displayed
)
# v2 doesn't display repository repository dependencies, they are deprecated
pass
def check_repository_metadata(self, repository: Repository, tip_only=True):
if tip_only:
@@ -1228,9 +1204,7 @@ class ShedTwillTestCase(ShedApiTestCase):
params = {"id": repository.id}
if changeset_revision:
params["changeset_revision"] = changeset_revision
url = "/repository/manage_repository"
if self.is_v2:
url = f"/repositories/{repository.id}"
url = f"/repositories/{repository.id}"
self.visit_url(url, params=params)
self.check_for_strings(strings_displayed, strings_not_displayed)
@@ -1648,25 +1622,11 @@ class ShedTwillTestCase(ShedApiTestCase):
# Changeset revision should never be provided unless repository name also is.
assert repository_name is not None, "Changeset revision is present, but repository name is not - aborting."
url += f"/{changeset_revision}"
if self.is_v2:
# I think pagination broke this legacy test - so I added this
url += "?rows_per_page=250"
# I think pagination broke this legacy test - so I added this
url += "?rows_per_page=250"
self.visit_url(url)
self.check_for_strings(strings_displayed, strings_not_displayed)
if self.is_v2:
self.check_for_strings(strings_displayed_in_iframe, strings_not_displayed_in_iframe)
else:
# Now load the page that should be displayed inside the iframe and check for strings.
if encoded_repository_id:
params = {"id": encoded_repository_id, "operation": "view_or_manage_repository"}
if changeset_revision:
params["changeset_revision"] = changeset_revision
self.visit_url("/repository/view_repository", params=params)
self.check_for_strings(strings_displayed_in_iframe, strings_not_displayed_in_iframe)
elif encoded_user_id:
params = {"user_id": encoded_user_id, "operation": "repositories_by_user"}
self.visit_url("/repository/browse_repositories", params=params)
self.check_for_strings(strings_displayed_in_iframe, strings_not_displayed_in_iframe)
self.check_for_strings(strings_displayed_in_iframe, strings_not_displayed_in_iframe)
def load_changeset_in_tool_shed(
self, repository_id, changeset_revision, strings_displayed=None, strings_not_displayed=None
@@ -1758,13 +1718,8 @@ class ShedTwillTestCase(ShedApiTestCase):
return tip_ctx.rev() < 0
def reset_metadata_on_selected_repositories(self, repository_ids):
if self.is_v2:
for repository_id in repository_ids:
self.populator.reset_metadata(repository_id)
else:
self.visit_url("/admin/reset_metadata_on_selected_repositories_in_tool_shed")
kwd = dict(repository_ids=repository_ids)
self.submit_form(button="reset_metadata_on_selected_repositories_button", **kwd)
for repository_id in repository_ids:
self.populator.reset_metadata(repository_id)
def reset_metadata_on_installed_repositories(self, repositories):
assert self._installation_client
+2 -19
View File
@@ -1,21 +1,10 @@
import os
from collections.abc import (
Callable,
Generator,
)
from collections.abc import Generator
import pytest
from playwright.sync_api import Browser
from ..base.browser import ShedBrowser
from ..base.playwrightbrowser import PlaywrightShedBrowser
from ..base.twillbrowser import TwillShedBrowser
DEFAULT_BROWSER = "playwright"
def twill_browser() -> Generator[ShedBrowser, None, None]:
yield TwillShedBrowser()
def playwright_browser(browser: Browser) -> Generator[ShedBrowser, None, None]:
@@ -23,10 +12,4 @@ def playwright_browser(browser: Browser) -> Generator[ShedBrowser, None, None]:
yield PlaywrightShedBrowser(page)
test_browser = os.environ.get("TOOL_SHED_TEST_BROWSER", DEFAULT_BROWSER)
if test_browser == "twill":
shed_browser: Callable[..., Generator[ShedBrowser, None, None]] = pytest.fixture(scope="class")(twill_browser)
elif test_browser == "playwright":
shed_browser = pytest.fixture(scope="class")(playwright_browser)
else:
raise ValueError(f"Unrecognized value for TOOL_SHED_TEST_BROWSER: {test_browser}")
shed_browser = pytest.fixture(scope="class")(playwright_browser)
@@ -3,7 +3,6 @@ import logging
import pytest
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
repository_name = "filtering_0000"
@@ -22,15 +21,6 @@ class TestBasicRepositoryFeatures(ShedTwillTestCase):
self.login(email=common.test_user_2_email, username=common.test_user_2_name)
self.login(email=common.admin_email, username=common.admin_username)
@skip_if_api_v2
# no replicating the functionality in tool shed 2.0, use Planemo
# to create repositories.
def test_0005_create_repository_without_categories(self):
"""Verify that a repository cannot be created unless at least one category has been defined."""
strings_displayed = ["No categories have been configured in this instance of the Galaxy Tool Shed"]
self.visit_url("/repository/create_repository")
self.check_for_strings(strings_displayed=strings_displayed, strings_not_displayed=[])
def test_0010_create_categories(self):
"""Create categories for this test suite"""
self.create_category(
@@ -73,13 +63,6 @@ class TestBasicRepositoryFeatures(ShedTwillTestCase):
categories_to_remove=["Test 0000 Basic Repository Features 1"],
)
@skip_if_api_v2
def test_0030_grant_write_access(self):
"""Grant write access to another user"""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
self.grant_write_access(repository, usernames=[common.test_user_2_name])
self.revoke_write_access(repository, common.test_user_2_name)
def test_0035_upload_filtering_1_1_0(self):
"""Upload filtering_1.1.0.tar to the repository"""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
@@ -125,49 +108,6 @@ class TestBasicRepositoryFeatures(ShedTwillTestCase):
strings_displayed=strings,
)
@skip_if_api_v2
def test_0045_alter_repository_states(self):
"""Test toggling the malicious and deprecated repository flags."""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
self.login(email=common.admin_email, username=common.admin_username)
self.set_repository_malicious(
repository, set_malicious=True, strings_displayed=["The repository tip has been defined as malicious."]
)
self.set_repository_malicious(
repository,
set_malicious=False,
strings_displayed=["The repository tip has been defined as <b>not</b> malicious."],
)
self.login(email=common.test_user_1_email, username=common.test_user_1_name)
self.set_repository_deprecated(repository, strings_displayed=["has been marked as deprecated"])
strings_displayed = ["This repository has been marked as deprecated", "Mark repository as not deprecated"]
self.display_manage_repository_page(
repository,
strings_displayed=strings_displayed,
strings_not_displayed=["Reset all repository metadata"],
)
self.browse_repository(repository)
self.set_repository_deprecated(
repository, strings_displayed=["has been marked as not deprecated"], set_deprecated=False
)
strings_displayed = ["Mark repository as deprecated", "Reset all repository metadata"]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
@skip_if_api_v2
# probably not porting this functionality - just test
# with Twill for older UI and drop when that is all dropped
def test_0050_display_repository_tip_file(self):
"""Display the contents of filtering.xml in the repository tip revision"""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
if self._browser.is_twill:
self.display_repository_file_contents(
repository=repository,
filename="filtering.xml",
filepath=None,
strings_displayed=["1.1.0"],
strings_not_displayed=[],
)
def test_0055_upload_filtering_txt_file(self):
"""Upload filtering.txt file associated with tool version 1.1.0."""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
@@ -201,10 +141,7 @@ class TestBasicRepositoryFeatures(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
tip = self.get_repository_tip(repository)
self.check_for_valid_tools(repository)
if self.is_v2:
strings_displayed = []
else:
strings_displayed = ["Select a revision"]
strings_displayed: list[str] = []
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.check_count_of_metadata_revisions_associated_with_repository(repository, metadata_count=2)
tool_guid = f"{self.url.replace('http://', '').rstrip('/')}/repos/user1/filtering_0000/Filter1/2.2.0"
@@ -248,16 +185,6 @@ class TestBasicRepositoryFeatures(ShedTwillTestCase):
readme_content = self._escape_page_content_if_needed("Readme file for filtering 1.1.0")
self.display_manage_repository_page(repository, strings_displayed=[readme_content])
@skip_if_api_v2 # not re-implemented in the UI, there are API tests though
def test_0085_search_for_valid_filter_tool(self):
"""Search for the filtering tool by tool ID, name, and version."""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
tip_changeset = self.get_repository_tip(repository)
search_fields = dict(tool_id="Filter1", tool_name="filter", tool_version="2.2.0")
self.search_for_valid_tools(
search_fields=search_fields, strings_displayed=[tip_changeset], strings_not_displayed=[]
)
def test_0090_verify_repository_metadata(self):
"""Verify that resetting the metadata does not change it."""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
@@ -284,58 +211,12 @@ class TestBasicRepositoryFeatures(ShedTwillTestCase):
self.login(email="baduser@bx.psu.edu", username="repos")
test_user_1 = self.test_db_util.get_user("baduser@bx.psu.edu")
assert test_user_1 is None, 'Creating user with public name "repos" succeeded.'
if not self.is_v2:
# no longer use this terminology but the above test case ensures
# the important thing and caught a bug in v2
error_message = (
"The term 'repos' is a reserved word in the Tool Shed, so it cannot be used as a public user name."
)
self.check_for_strings(strings_displayed=[error_message])
def test_0105_contact_repository_owner(self):
""""""
# We no longer implement this.
pass
@skip_if_api_v2 # v2 doesn't implement repository deleting repositories
def test_0110_delete_filtering_repository(self):
"""Delete the filtering_0000 repository and verify that it no longer has any downloadable revisions."""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
self.login(email=common.admin_email, username=common.admin_username)
self.delete_repository(repository)
metadata = self.populator.get_metadata(repository, downloadable_only=False)
for _, value in metadata.root.items():
assert not value.downloadable
# Explicitly reload all metadata revisions from the database, to ensure that we have the current status of the downloadable flag.
# for metadata_revision in repository.metadata_revisions:
# self.test_db_util.refresh(metadata_revision)
# Marking a repository as deleted should result in no metadata revisions being downloadable.
# assert True not in [metadata.downloadable for metadata in self._db_repository(repository).metadata_revisions]
@skip_if_api_v2 # v2 doesn't implement repository deleting repositories
def test_0115_undelete_filtering_repository(self):
"""Undelete the filtering_0000 repository and verify that it now has two downloadable revisions."""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
self.login(email=common.admin_email, username=common.admin_username)
self.undelete_repository(repository)
# Explicitly reload all metadata revisions from the database, to ensure that we have the current status of the downloadable flag.
# for metadata_revision in repository.metadata_revisions:
# self.test_db_util.refresh(metadata_revision)
# Marking a repository as undeleted should result in all previously downloadable metadata revisions being downloadable again.
# In this case, there should be two downloadable revisions, one for filtering 1.1.0 and one for filtering 2.2.0.
assert True in [metadata.downloadable for metadata in self._db_repository(repository).metadata_revisions]
assert len(self._db_repository(repository).downloadable_revisions) == 2
@skip_if_api_v2 # not re-implementing in tool shed 2.0
def test_0120_enable_email_notifications(self):
"""Enable email notifications for test user 2 on filtering_0000."""
# Log in as test_user_2
self.login(email=common.test_user_2_email, username=common.test_user_2_name)
# Get the repository, so we can pass the encoded repository id and browse_repositories method to the set_email_alerts method.
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
strings_displayed = ["Total alerts added: 1, total alerts removed: 0"]
self.enable_email_alerts(repository, strings_displayed=strings_displayed)
def test_0125_upload_new_readme_file(self):
"""Upload a new readme file to the filtering_0000 repository and verify that there is no error."""
self.login(email=common.test_user_1_email, username=common.test_user_1_name)
@@ -1,7 +1,6 @@
import os
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
repository_name = "freebayes_0010"
@@ -53,9 +52,8 @@ class TestFreebayesRepository(ShedTwillTestCase):
)
strings_displayed = ["Metadata may have been defined", "This file requires an entry", "tool_data_table_conf"]
self.add_file_to_repository(repository, "freebayes/freebayes.xml", strings_displayed=strings_displayed)
if self.is_v2:
# opps... not good right?
self.populator.reset_metadata(repository)
# opps... not good right?
self.populator.reset_metadata(repository)
self.display_manage_repository_page(
repository, strings_displayed=[self.invalid_tools_labels], strings_not_displayed=["Valid tools"]
)
@@ -124,17 +122,3 @@ class TestFreebayesRepository(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
target = os.path.join("freebayes", "tool_dependencies.xml")
self.add_file_to_repository(repository, target)
@skip_if_api_v2
def test_0040_verify_tool_dependencies(self):
"""Verify that the uploaded tool_dependencies.xml specifies the correct package versions.
We are at step 7 - Check for the appropriate strings on the manage repository page.
Verify that the manage repository page now displays the valid tool dependencies, and that there are no invalid tools shown on the manage page.
"""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
strings_displayed = ["freebayes", "0.9.4_9696d0ce8a9", "samtools", "0.1.18", "Valid tools", "package"]
strings_not_displayed = [self.invalid_tools_labels]
self.display_manage_repository_page(
repository, strings_displayed=strings_displayed, strings_not_displayed=strings_not_displayed
)
@@ -1,5 +1,4 @@
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
column_maker_repository_name = "column_maker_0020"
@@ -69,25 +68,6 @@ class TestBasicRepositoryDependencies(ShedTwillTestCase):
repository=repository, repository_tuples=[repository_tuple], filepath=repository_dependencies_path
)
@skip_if_api_v2
def test_0030_verify_emboss_5_dependencies(self):
"""Verify that the emboss_5 repository now depends on the emboss_datatypes repository with correct name, owner, and changeset revision."""
repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
column_maker_repository = self._get_repository_by_name_and_owner(
column_maker_repository_name, common.test_user_1_name
)
changeset_revision = self.get_repository_tip(column_maker_repository)
strings_displayed = [
"Tool dependencies",
"emboss",
"5.0.0",
"package",
"user1",
changeset_revision,
"Repository dependencies",
]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
def test_0040_verify_repository_metadata(self):
"""Verify that resetting the metadata does not change it."""
emboss_repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
@@ -1,5 +1,4 @@
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
column_maker_repository_name = "column_maker_0030"
@@ -160,27 +159,6 @@ class TestRepositoryDependencyRevisions(ShedTwillTestCase):
repository=emboss_repository, repository_tuples=[emboss_tuple], filepath=repository_dependencies_path
)
@skip_if_api_v2
def test_0050_verify_repository_dependency_revisions(self):
"""Verify that different metadata revisions of the emboss repository have different repository dependencies."""
repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
repository_metadata = [
(metadata.metadata, metadata.changeset_revision) for metadata in self.get_repository_metadata(repository)
]
column_maker_repository = self._get_repository_by_name_and_owner(
column_maker_repository_name, common.test_user_1_name
)
column_maker_tip = self.get_repository_tip(column_maker_repository)
strings_displayed = []
# Iterate through all metadata revisions and check for repository dependencies.
for _metadata, changeset_revision in repository_metadata:
# Add the dependency description and bismark repository details to the strings to check.
strings_displayed = ["column_maker_0030", "user1", column_maker_tip]
strings_displayed.extend(["Tool dependencies", "emboss", "5.0.0", "package"])
self.display_manage_repository_page(
repository, changeset_revision=changeset_revision, strings_displayed=strings_displayed
)
def test_0055_verify_repository_metadata(self):
"""Verify that resetting the metadata does not change it."""
emboss_repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
@@ -1,5 +1,4 @@
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
freebayes_repository_name = "freebayes_0040"
@@ -127,13 +126,3 @@ class TestRepositoryCircularDependencies(ShedTwillTestCase):
)
for repository in [freebayes_repository, filtering_repository]:
self.verify_unchanged_repository_metadata(repository)
@skip_if_api_v2
def test_0040_verify_tool_dependencies(self):
"""Verify that freebayes displays tool dependencies."""
repository = self._get_repository_by_name_and_owner(freebayes_repository_name, common.test_user_1_name)
self.display_manage_repository_page(
repository,
strings_displayed=["freebayes", "0.9.4_9696d0ce8a9", "samtools", "0.1.18", "Valid tools", "package"],
strings_not_displayed=["Invalid tools"],
)
@@ -1,5 +1,4 @@
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
emboss_repository_name = "emboss_0050"
@@ -242,24 +241,6 @@ class TestRepositoryCircularDependenciesToNLevels(ShedTwillTestCase):
self.check_repository_dependency(filtering_repository, emboss_repository)
for repository in [bismark_repository, emboss_repository, column_repository]:
self.check_repository_dependency(freebayes_repository, repository)
if not self.is_v2:
strings_displayed = ["freebayes_0050 depends on freebayes_0050, emboss_0050, column_maker_0050."]
self.display_manage_repository_page(freebayes_repository, strings_displayed=strings_displayed)
@skip_if_api_v2
def test_0050_verify_tool_dependencies(self):
"""Check that freebayes and emboss display tool dependencies."""
freebayes_repository = self._get_repository_by_name_and_owner(
freebayes_repository_name, common.test_user_1_name
)
emboss_repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
self.display_manage_repository_page(
freebayes_repository,
strings_displayed=["freebayes", "0.9.4_9696d0ce8a9", "samtools", "0.1.18", "Tool dependencies", "package"],
)
self.display_manage_repository_page(
emboss_repository, strings_displayed=["Tool dependencies", "emboss", "5.0.0", "package"]
)
def test_0055_verify_repository_metadata(self):
"""Verify that resetting the metadata does not change it."""
@@ -1,8 +1,6 @@
import logging
import os
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
log = logging.getLogger(__name__)
@@ -43,11 +41,6 @@ class TestComplexRepositoryDependencies(ShedTwillTestCase):
strings_displayed=[],
)
self.add_file_to_repository(repository, "bwa/complex/tool_dependencies.xml")
if not self.is_v2:
# Visit the manage repository page for package_bwa_0_5_9_0100.
self.display_manage_repository_page(
repository, strings_displayed=["Tool dependencies", "will not be", "to this repository"]
)
def test_0010_create_bwa_base_repository(self):
"""Create and populate bwa_base_0100."""
@@ -183,43 +176,3 @@ class TestComplexRepositoryDependencies(ShedTwillTestCase):
version="0.5.9",
)
self.check_repository_dependency(base_repository, depends_on_repository=tool_repository)
if not self.is_v2:
self.display_manage_repository_page(
base_repository, strings_displayed=["bwa", "0.5.9", "package", changeset_revision]
)
@skip_if_api_v2
def test_0040_generate_tool_dependency(self):
"""Generate and upload a new tool_dependencies.xml file that specifies an arbitrary file on the filesystem, and verify that bwa_base depends on the new changeset revision."""
# The base_repository named bwa_base_repository_0100 is the dependent repository.
base_repository = self._get_repository_by_name_and_owner(bwa_base_repository_name, common.test_user_1_name)
# The repository named package_bwa_0_5_9_0100 is the required repository.
tool_repository = self._get_repository_by_name_and_owner(bwa_package_repository_name, common.test_user_1_name)
previous_changeset = self.get_repository_tip(tool_repository)
old_tool_dependency = self.get_filename(os.path.join("bwa", "complex", "readme", "tool_dependencies.xml"))
new_tool_dependency_path = self.generate_temp_path("test_1100", additional_paths=["tool_dependency"])
xml_filename = os.path.abspath(os.path.join(new_tool_dependency_path, "tool_dependencies.xml"))
# Generate a tool_dependencies.xml file that points to an arbitrary file in the local filesystem.
open(xml_filename, "w").write(
open(old_tool_dependency).read().replace("__PATH__", self.get_filename("bwa/complex"))
)
self.add_file_to_repository(tool_repository, xml_filename, "tool_dependencies.xml")
# Verify that the dependency display has been updated as a result of the new tool_dependencies.xml file.
repository_tip = self.get_repository_tip(tool_repository)
strings_displayed = ["bwa", "0.5.9", "package"]
strings_displayed.append(repository_tip)
strings_not_displayed = [previous_changeset]
self.display_manage_repository_page(
tool_repository, strings_displayed=strings_displayed, strings_not_displayed=strings_not_displayed
)
# Visit the manage page of the package_bwa_0_5_9_0100 to confirm the valid tool dependency definition.
self.display_manage_repository_page(
tool_repository, strings_displayed=strings_displayed, strings_not_displayed=strings_not_displayed
)
# Visit the manage page of the bwa_base_repository_0100 to confirm the valid tool dependency definition
# and the updated changeset revision (updated tip) of the package_bwa_0_5_9_0100 repository is displayed
# as the required repository revision. The original revision defined in the previously uploaded
# tool_dependencies.xml file will be updated.
self.display_manage_repository_page(
base_repository, strings_displayed=strings_displayed, strings_not_displayed=strings_not_displayed
)
@@ -99,8 +99,6 @@ class TestRepositoryMultipleOwners(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner(tool_repository_name, common.test_user_1_name)
strings_displayed = ["blastxml_to_top_descr_0120", "BLAST top hit descriptions", "Make a table from BLAST XML"]
strings_displayed.append("0.0.1")
if not self.is_v2:
strings_displayed.append("Valid tools")
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
def test_0025_create_repository_dependency(self):
@@ -1,7 +1,6 @@
import logging
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
log = logging.getLogger(__name__)
@@ -54,27 +53,3 @@ class TestToolHelpImages(ShedTwillTestCase):
"htseq_count/htseq_count.tar",
commit_message="Uploaded htseq_count.tar.",
)
@skip_if_api_v2
def test_0010_load_tool_page(self):
"""Load the tool page and check for the image.
We are at step 2
Visit the manage_repository page and the tool page, and look for the image url
similar to the following string:
src="/repository/static/images/<id>/count_modes.png"
"""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
# Get the repository tip.
changeset_revision = self.get_repository_tip(repository)
self.display_manage_repository_page(repository)
# Generate the image path.
image_path = f'src="/repository/static/images/{repository.id}/count_modes.png"'
# The repository uploaded in this test should only have one metadata revision, with one tool defined, which
# should be the tool that contains a link to the image.
repository_metadata = self._db_repository(repository).metadata_revisions[0].metadata
tool_path = repository_metadata["tools"][0]["tool_config"]
self.load_display_tool_page(
repository, tool_path, changeset_revision, strings_displayed=[image_path], strings_not_displayed=[]
)
@@ -123,9 +123,4 @@ class TestComplexPriorInstallation(ShedTwillTestCase):
matplotlib_repository = self._get_repository_by_name_and_owner(
matplotlib_repository_name, common.test_user_1_name
)
changeset_revision = self.get_repository_tip(numpy_repository)
self.check_repository_dependency(matplotlib_repository, depends_on_repository=numpy_repository)
if not self.is_v2:
self.display_manage_repository_page(
matplotlib_repository, strings_displayed=["numpy", "1.7", "package", changeset_revision]
)
@@ -86,11 +86,7 @@ class TestRepositoryCitableURLs(ShedTwillTestCase):
# Since twill does not load the contents of an iframe, we need to check that the iframe has been generated correctly,
# then directly load the url that the iframe should be loading and check for the expected strings.
# The iframe should point to /repository/browse_repositories?user_id=<encoded user ID>&operation=repositories_by_user
if self.is_v2:
strings_displayed = []
else:
strings_displayed = ["/repository/browse_repositories", encoded_user_id, "operation=repositories_by_user"]
strings_displayed.append(encoded_user_id)
strings_displayed: list[str] = []
strings_displayed_in_iframe = ["user1", "filtering_0420", repository_description]
self.load_citable_url(
username="user1",
@@ -116,19 +112,13 @@ class TestRepositoryCitableURLs(ShedTwillTestCase):
# Since twill does not load the contents of an iframe, we need to check that the iframe has been generated correctly,
# then directly load the url that the iframe should be loading and check for the expected strings.
# The iframe should point to /repository/bview_repository?id=<encoded repository ID>
if self.is_v2:
strings_displayed = []
else:
strings_displayed = ["/repository", "view_repository", "id=", encoded_repository_id]
strings_displayed: list[str] = []
strings_displayed_in_iframe = [
"user1",
"filtering_0420",
self._escape_page_content_if_needed(repository_long_description),
]
strings_displayed_in_iframe.append(self.get_repository_tip(repository))
if not self.is_v2:
strings_displayed_in_iframe.append("Link to this repository:")
strings_displayed_in_iframe.append(f"{self.url}/view/user1/filtering_0420")
self.load_citable_url(
username="user1",
repository_name="filtering_0420",
@@ -153,19 +143,13 @@ class TestRepositoryCitableURLs(ShedTwillTestCase):
# Since twill does not load the contents of an iframe, we need to check that the iframe has been generated correctly,
# then directly load the url that the iframe should be loading and check for the expected strings.
# The iframe should point to /repository/view_repository?id=<encoded repository ID>
if self.is_v2:
strings_displayed = []
else:
strings_displayed = ["/repository", "view_repository", f"id={encoded_repository_id}"]
strings_displayed: list[str] = []
strings_displayed_in_iframe = [
"user1",
"filtering_0420",
self._escape_page_content_if_needed(repository_long_description),
first_changeset_hash,
]
if not self.is_v2:
strings_displayed_in_iframe.append("Link to this repository revision:")
strings_displayed_in_iframe.append(f"{self.url}/view/user1/filtering_0420/{first_changeset_hash}")
self.load_citable_url(
username="user1",
repository_name="filtering_0420",
@@ -183,16 +167,7 @@ class TestRepositoryCitableURLs(ShedTwillTestCase):
encoded_user_id = self.security.encode_id(test_user_1.id)
encoded_repository_id = repository.id
invalid_changeset_hash = "invalid"
if not self.is_v2:
# Since twill does not load the contents of an iframe, we need to check that the iframe has been generated correctly,
# then directly load the url that the iframe should be loading and check for the expected strings.
# The iframe should point to /repository/view_repository?id=<encoded repository ID>&status=error
strings_displayed = ["/repository", "view_repository", f"id={encoded_repository_id}"]
strings_displayed.extend(
["The+change+log", "does+not+include+revision", invalid_changeset_hash, "status=error"]
)
else:
strings_displayed = ["The change log does not include revision " + invalid_changeset_hash]
strings_displayed = ["The change log does not include revision " + invalid_changeset_hash]
self.load_citable_url(
username="user1",
repository_name="filtering_0420",
@@ -213,16 +188,8 @@ class TestRepositoryCitableURLs(ShedTwillTestCase):
# Since twill does not load the contents of an iframe, we need to check that the iframe has been generated correctly,
# then directly load the url that the iframe should be loading and check for the expected strings.
# The iframe should point to /repository/browse_repositories?user_id=<encoded user ID>&operation=repositories_by_user
if not self.is_v2:
strings_displayed = ["/repository", "browse_repositories", "user1"]
strings_displayed.extend(
["list+of+repositories+owned", "does+not+include+one+named", "%21%21invalid%21%21", "status=error"]
)
strings_displayed_in_iframe = ["user1", "filtering_0420"]
strings_displayed_in_iframe.append("Repositories Owned by user1")
else:
strings_displayed = ["Repository user1/!!invalid!! is not found"]
strings_displayed_in_iframe = []
strings_displayed = ["Repository user1/!!invalid!! is not found"]
strings_displayed_in_iframe: list[str] = []
self.load_citable_url(
username="user1",
repository_name="!!invalid!!",
@@ -239,10 +206,7 @@ class TestRepositoryCitableURLs(ShedTwillTestCase):
We are at step 8.
Visit the following url and check for appropriate strings: <tool shed base url>/view/!!invalid!!
"""
if not self.is_v2:
strings_displayed = ["The tool shed", self.url, "contains no repositories owned by", "!!invalid!!"]
else:
strings_displayed = ["No repositories found"]
strings_displayed = ["No repositories found"]
self.load_citable_url(
username="!!invalid!!",
repository_name=None,
@@ -1,7 +1,6 @@
import logging
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
log = logging.getLogger(__name__)
@@ -83,37 +82,3 @@ class TestToolShedBrowseUtilities(ShedTwillTestCase):
"freebayes/freebayes.tar",
commit_message="Uploaded freebayes.tar.",
)
@skip_if_api_v2
def test_0030_browse_tools(self):
"""Load the page to browse tools.
We are at step 3.
Verify the existence of emboss tools in the browse tools page.
"""
repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
changeset_revision = self.get_repository_tip(repository)
strings_displayed = ["EMBOSS", "antigenic1", "5.0.0", changeset_revision, "user1", "emboss_0430"]
self.browse_tools(strings_displayed=strings_displayed)
@skip_if_api_v2
def test_0040_browse_tool_dependencies(self):
"""Browse tool dependencies and look for the right versions of freebayes and samtools.
We are at step 4.
Verify that the browse tool dependencies page shows the correct dependencies defined for freebayes_0430.
"""
freebayes_repository = self._get_repository_by_name_and_owner(
freebayes_repository_name, common.test_user_1_name
)
freebayes_changeset_revision = self.get_repository_tip(freebayes_repository)
strings_displayed = [
freebayes_changeset_revision,
"freebayes_0430",
"user1",
"0.9.4_9696d0ce8a96",
"freebayes",
"samtools",
"0.1.18",
]
self.browse_tool_dependencies(strings_displayed=strings_displayed)
@@ -125,15 +125,7 @@ class TestAutomaticDependencyRevision(ShedTwillTestCase):
a complex repository dependency on package_bwa_0_5_9_0460 without a specified changeset revision or tool shed url.
"""
repository = self._get_repository_by_name_and_owner("complex_dependency_test_1_0460", common.test_user_1_name)
package_repository = self._get_repository_by_name_and_owner("package_bwa_0_5_9_0460", common.test_user_1_name)
self.add_file_to_repository(repository, "0460_files/tool_dependencies.xml")
if not self.is_v2:
changeset_revision = self.get_repository_tip(package_repository)
strings_displayed = ["package_bwa_0_5_9_0460", "bwa", "0.5.9", "package", changeset_revision]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.display_repository_file_contents(
repository, filename="tool_dependencies.xml", strings_displayed=[changeset_revision]
)
def test_0025_populate_complex_dependency_test_2_0460(self):
"""Populate complex_dependency_test_2_0460.
@@ -142,19 +134,11 @@ class TestAutomaticDependencyRevision(ShedTwillTestCase):
a complex repository dependency on package_bwa_0_5_9_0460 without a specified changeset revision or tool shed url.
"""
repository = self._get_repository_by_name_and_owner("complex_dependency_test_2_0460", common.test_user_1_name)
package_repository = self._get_repository_by_name_and_owner("package_bwa_0_5_9_0460", common.test_user_1_name)
self.commit_tar_to_repository(
repository,
"0460_files/tool_dependencies_in_root.tar",
commit_message="Uploaded complex repository dependency definition.",
)
if not self.is_v2:
changeset_revision = self.get_repository_tip(package_repository)
strings_displayed = ["package_bwa_0_5_9_0460", "bwa", "0.5.9", "package", changeset_revision]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.display_repository_file_contents(
repository, filename="tool_dependencies.xml", strings_displayed=[changeset_revision]
)
def test_0030_populate_complex_dependency_test_3_0460(self):
"""Populate complex_dependency_test_3_0460.
@@ -163,22 +147,11 @@ class TestAutomaticDependencyRevision(ShedTwillTestCase):
specifies a complex repository dependency on package_bwa_0_5_9_0460 without a specified changeset revision or tool shed url.
"""
repository = self._get_repository_by_name_and_owner("complex_dependency_test_3_0460", common.test_user_1_name)
package_repository = self._get_repository_by_name_and_owner("package_bwa_0_5_9_0460", common.test_user_1_name)
self.commit_tar_to_repository(
repository,
"0460_files/tool_dependencies_in_subfolder.tar",
commit_message="Uploaded complex repository dependency definition.",
)
changeset_revision = self.get_repository_tip(package_repository)
if not self.is_v2:
strings_displayed = ["package_bwa_0_5_9_0460", "bwa", "0.5.9", "package", changeset_revision]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.display_repository_file_contents(
repository,
filename="tool_dependencies.xml",
filepath="subfolder",
strings_displayed=[changeset_revision],
)
def test_0035_create_repositories_for_url_upload(self):
"""Create and populate hg_tool_dependency_0460 and hg_subfolder_tool_dependency_0460.
@@ -245,15 +218,7 @@ class TestAutomaticDependencyRevision(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner(
"repository_dependency_test_1_0460", common.test_user_1_name
)
package_repository = self._get_repository_by_name_and_owner(bwa_repository_name, common.test_user_1_name)
self.add_file_to_repository(repository, "0460_files/repository_dependencies.xml")
changeset_revision = self.get_repository_tip(package_repository)
if not self.is_v2:
strings_displayed = [bwa_repository_name, "user1", changeset_revision]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.display_repository_file_contents(
repository, filename="repository_dependencies.xml", strings_displayed=[changeset_revision]
)
def test_0060_populate_repository_dependency_test_2_0460(self):
"""Populate repository_dependency_test_2_0460.
@@ -263,19 +228,11 @@ class TestAutomaticDependencyRevision(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner(
"repository_dependency_test_2_0460", common.test_user_1_name
)
package_repository = self._get_repository_by_name_and_owner(bwa_repository_name, common.test_user_1_name)
self.commit_tar_to_repository(
repository,
"0460_files/in_root/repository_dependencies_in_root.tar",
commit_message="Uploaded complex repository dependency definition.",
)
changeset_revision = self.get_repository_tip(package_repository)
if not self.is_v2:
strings_displayed = [bwa_repository_name, "user1", changeset_revision]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.display_repository_file_contents(
repository, filename="repository_dependencies.xml", strings_displayed=[changeset_revision]
)
def test_0065_populate_repository_dependency_test_3_0460(self):
"""Populate repository_dependency_test_3_0460.
@@ -286,22 +243,11 @@ class TestAutomaticDependencyRevision(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner(
"repository_dependency_test_3_0460", common.test_user_1_name
)
package_repository = self._get_repository_by_name_and_owner(bwa_repository_name, common.test_user_1_name)
self.commit_tar_to_repository(
repository,
"0460_files/in_subfolder/repository_dependencies_in_subfolder.tar",
commit_message="Uploaded complex repository dependency definition.",
)
changeset_revision = self.get_repository_tip(package_repository)
if not self.is_v2:
strings_displayed = [bwa_repository_name, "user1", changeset_revision]
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
self.display_repository_file_contents(
repository,
filename="repository_dependencies.xml",
filepath="subfolder",
strings_displayed=[changeset_revision],
)
def test_0070_create_repositories_for_url_upload(self):
"""Create and populate hg_repository_dependency_0460 and hg_subfolder_repository_dependency_0460.
@@ -1,7 +1,6 @@
import logging
from ..base import common
from ..base.api import skip_if_api_v2
from ..base.twilltestcase import ShedTwillTestCase
log = logging.getLogger(__name__)
@@ -97,23 +96,6 @@ class TestRepositoryAdminRole(ShedTwillTestCase):
repository = self._get_repository_by_name_and_owner("renamed_filtering_0530", common.test_user_1_name)
assert repository.name == "renamed_filtering_0530", "Repository was not renamed to renamed_filtering_0530."
@skip_if_api_v2
def test_0030_verify_access_denied(self):
"""Make sure a non-admin user can't modify the repository.
This is step 6 - Log into the Tool Shed as a user that is not the repository owner (e.g., user2) and make sure the repository
name and description cannot be changed.
"""
self.login(email=common.test_user_2_email, username=common.test_user_2_name)
repository = self._get_repository_by_name_and_owner("renamed_filtering_0530", common.test_user_1_name)
strings_not_displayed = ["Manage repository"]
strings_displayed = ["View repository"]
self.display_manage_repository_page(repository, strings_not_displayed=strings_not_displayed)
self.submit_form(form_no=0, button="edit_repository_button", description="This description has been modified.")
strings_displayed = ["You are not the owner of this repository, so you cannot administer it."]
strings_not_displayed = ["The repository information has been updated."]
self.check_for_strings(strings_displayed=strings_displayed, strings_not_displayed=strings_not_displayed)
def test_0035_grant_admin_role(self):
"""Grant the repository admin role to user2.
@@ -68,11 +68,6 @@ class TestGetUpdatedMetadata(ShedTwillTestCase):
freebayes_repository,
"0550_files/package_freebayes_1_0550.tgz",
)
if not self.is_v2:
# Visit the manage repository page for package_freebayes_0_5_9_0100.
self.display_manage_repository_page(
freebayes_repository, strings_displayed=["Tool dependencies", "will not be", "to this repository"]
)
def test_0010_create_samtools_repository(self):
"""Create and populate the package_samtools_0550 repository."""
@@ -110,12 +105,8 @@ class TestGetUpdatedMetadata(ShedTwillTestCase):
def test_0020_check_repository_dependency(self):
"""Make filtering depend on samtools and freebayes."""
freebayes = self._get_repository_by_name_and_owner(repositories["freebayes"]["name"], common.test_user_1_name)
samtools = self._get_repository_by_name_and_owner(repositories["samtools"]["name"], common.test_user_1_name)
filtering = self._get_repository_by_name_and_owner(repositories["filtering"]["name"], common.test_user_1_name)
strings_displayed = [freebayes.id, samtools.id]
if not self.is_v2:
self.display_manage_repository_page(filtering, strings_displayed=strings_displayed)
# Note: test not yet implemented in v2
pass
def test_0025_update_dependent_repositories(self):
"""
@@ -41,11 +41,6 @@ class TestToolWithToolDependencies(ShedTwillTestCase):
self.browse_tool_shed(url=self.url, strings_displayed=[category_name])
category = self.populator.get_category_with_name(category_name)
self.browse_category(category, strings_displayed=[repository_name])
if not self.is_v2:
strings_displayed = [repository_name, "Valid tools", "Tool dependencies"]
self.preview_repository_in_tool_shed(
repository_name, common.test_user_1_name, strings_displayed=strings_displayed
)
def test_0015_install_freebayes_repository(self):
"""Install the freebayes repository without installing tool dependencies."""
@@ -71,12 +71,6 @@ class TestToolWithRepositoryDependencies(ShedTwillTestCase):
self.browse_tool_shed(url=self.url, strings_displayed=["Test 0020 Basic Repository Dependencies"])
category = self.populator.get_category_with_name("Test 0020 Basic Repository Dependencies")
self.browse_category(category, strings_displayed=[emboss_repository_name])
if not self.is_v2:
self.preview_repository_in_tool_shed(
emboss_repository_name,
common.test_user_1_name,
strings_displayed=[emboss_repository_name, "Valid tools"],
)
def test_0015_install_emboss_repository(self):
"""Install the emboss repository without installing tool dependencies."""
@@ -138,12 +138,6 @@ class TestRepositoryWithDependencyRevisions(ShedTwillTestCase):
self.browse_tool_shed(url=self.url, strings_displayed=["Test 0030 Repository Dependency Revisions"])
category = self.populator.get_category_with_name("Test 0030 Repository Dependency Revisions")
self.browse_category(category, strings_displayed=[emboss_repository_name])
if not self.is_v2:
self.preview_repository_in_tool_shed(
emboss_repository_name,
common.test_user_1_name,
strings_displayed=[emboss_repository_name, "Valid tools"],
)
def test_0015_install_emboss_repository(self):
"""Install the emboss repository without installing tool dependencies."""
@@ -258,31 +258,11 @@ class TestInstallRepositoryCircularDependencies(ShedTwillTestCase):
self.check_repository_dependency(filtering_repository, emboss_repository)
for repository in [bismark_repository, emboss_repository, column_repository]:
self.check_repository_dependency(freebayes_repository, repository)
freebayes_dependencies = [
freebayes_repository,
emboss_repository,
column_repository,
]
strings_displayed = [
f"{freebayes_repository.name} depends on {', '.join(repo.name for repo in freebayes_dependencies)}."
]
if not self.is_v2:
self.display_manage_repository_page(freebayes_repository, strings_displayed=strings_displayed)
def test_0050_verify_tool_dependencies(self):
"""Check that freebayes and emboss display tool dependencies."""
freebayes_repository = self._get_repository_by_name_and_owner(
freebayes_repository_name, common.test_user_1_name
)
emboss_repository = self._get_repository_by_name_and_owner(emboss_repository_name, common.test_user_1_name)
if not self.is_v2:
self.display_manage_repository_page(
freebayes_repository,
strings_displayed=["freebayes", "0.9.4_9696d0ce8a9", "samtools", "0.1.18", "Tool dependencies"],
)
self.display_manage_repository_page(
emboss_repository, strings_displayed=["Tool dependencies", "emboss", "5.0.0", "package"]
)
# Note: test not yet implemented in v2
pass
def test_0055_install_column_repository(self):
"""Install column_maker with repository dependencies."""
@@ -71,19 +71,6 @@ class TestInstallRepositoryMultipleOwners(ShedTwillTestCase):
Check for appropriate strings, most importantly BlastXml, BlastNucDb, and BlastProtDb,
the datatypes that are defined in datatypes_conf.xml.
"""
repository = self._get_repository_by_name_and_owner(datatypes_repository_name, common.test_user_2_name)
strings_displayed = [
"BlastXml",
"BlastNucDb",
"BlastProtDb",
"application/xml",
"text/html",
"blastxml",
"blastdbn",
"blastdbp",
]
if not self.is_v2:
self.display_manage_repository_page(repository, strings_displayed=strings_displayed)
def test_0015_create_tool_repository(self):
"""Create and populate the blastxml_to_top_descr_0120 repository
@@ -58,17 +58,6 @@ class TestToolHelpImages(ShedTwillTestCase):
This is a duplicate of test method _0010 in test_0140_tool_help_images.
"""
repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
# Get the repository tip.
changeset_revision = self.get_repository_tip(repository)
# Generate the image path.
image_path = f'src="/repository/static/images/{repository.id}/count_modes.png"'
# The repository uploaded in this test should only have one metadata revision, with one tool defined, which
# should be the tool that contains a link to the image.
repository_metadata = self._db_repository(repository).metadata_revisions[0].metadata
tool_path = repository_metadata["tools"][0]["tool_config"]
# V2 is not going to have this page right? So... do we need this test at all or that route? Likely not?
if self._browser.is_twill and not self.is_v2:
self.load_display_tool_page(
repository, tool_path, changeset_revision, strings_displayed=[image_path], strings_not_displayed=[]
)
# TODO: replace with API-based tool metadata check or Vue route.
# load_display_tool_page depends on deleted Mako route (/repository/display_tool).
pass
@@ -132,12 +132,7 @@ class TestComplexPriorInstallation(ShedTwillTestCase):
matplotlib_repository = self._get_repository_by_name_and_owner(
matplotlib_repository_name, common.test_user_1_name
)
changeset_revision = self.get_repository_tip(numpy_repository)
self.check_repository_dependency(matplotlib_repository, depends_on_repository=numpy_repository)
if not self.is_v2:
self.display_manage_repository_page(
matplotlib_repository, strings_displayed=["numpy", "1.7", "package", changeset_revision]
)
def test_0025_install_matplotlib_repository(self):
"""Install the package_matplotlib_1_2_0170 repository.
@@ -1,7 +1,7 @@
"""Playwright tests for component showcase screenshots.
Run with:
TOOL_SHED_TEST_SCREENSHOTS=/tmp/component_screenshots TOOL_SHED_API_VERSION=v2 uv run pytest \
TOOL_SHED_TEST_SCREENSHOTS=/tmp/component_screenshots uv run pytest \
tool_shed/test/functional/test_component_showcase.py -v
Screenshots are saved to TOOL_SHED_TEST_SCREENSHOTS directory if set.
@@ -13,7 +13,6 @@ from pathlib import Path
from playwright.sync_api import expect
from ..base.api import skip_if_api_v1
from ..base.playwrighttestcase import PlaywrightTestCase
@@ -47,43 +46,36 @@ class TestComponentShowcase(PlaywrightTestCase):
# === General Components ===
@skip_if_api_v1
def test_loading_div(self):
"""Screenshot LoadingDiv component."""
self._navigate_to_showcase()
self._screenshot_component("LoadingDiv")
@skip_if_api_v1
def test_error_banner(self):
"""Screenshot ErrorBanner component."""
self._navigate_to_showcase()
self._screenshot_component("ErrorBanner")
@skip_if_api_v1
def test_repository_link(self):
"""Screenshot RepositoryLink component."""
self._navigate_to_showcase()
self._screenshot_component("RepositoryLink")
@skip_if_api_v1
def test_repository_actions(self):
"""Screenshot RepositoryActions component."""
self._navigate_to_showcase()
self._screenshot_component("RepositoryActions")
@skip_if_api_v1
def test_recently_created_repositories(self):
"""Screenshot RecentlyCreatedRepositories component."""
self._navigate_to_showcase()
self._screenshot_component("RecentlyCreatedRepositories")
@skip_if_api_v1
def test_landing_search_box(self):
"""Screenshot LandingSearchBox component."""
self._navigate_to_showcase()
self._screenshot_component("LandingSearchBox")
@skip_if_api_v1
def test_landing_info_sections(self):
"""Screenshot LandingInfoSections component."""
self._navigate_to_showcase()
@@ -91,37 +83,31 @@ class TestComponentShowcase(PlaywrightTestCase):
# === MetadataInspector Components ===
@skip_if_api_v1
def test_changeset_summary_table(self):
"""Screenshot ChangesetSummaryTable component."""
self._navigate_to_showcase()
self._screenshot_component("ChangesetSummaryTable")
@skip_if_api_v1
def test_json_diff_viewer(self):
"""Screenshot JsonDiffViewer component."""
self._navigate_to_showcase()
self._screenshot_component("JsonDiffViewer")
@skip_if_api_v1
def test_metadata_json_viewer(self):
"""Screenshot MetadataJsonViewer component."""
self._navigate_to_showcase()
self._screenshot_component("MetadataJsonViewer")
@skip_if_api_v1
def test_revisions_tab(self):
"""Screenshot RevisionsTab component."""
self._navigate_to_showcase()
self._screenshot_component("RevisionsTab")
@skip_if_api_v1
def test_overview_tab(self):
"""Screenshot OverviewTab component."""
self._navigate_to_showcase()
self._screenshot_component("OverviewTab")
@skip_if_api_v1
def test_tool_history_tab(self):
"""Screenshot ToolHistoryTab component."""
self._navigate_to_showcase()
@@ -1,13 +1,12 @@
"""Playwright tests for admin-only functionality.
Run with:
TOOL_SHED_API_VERSION=v2 uv run pytest \
uv run pytest \
tool_shed/test/functional/test_frontend_admin.py -v
"""
from playwright.sync_api import expect
from ..base.api import skip_if_api_v1
from ..base.playwrighttestcase import PlaywrightTestCase
TEST_CATEGORY_PREFIX = "admintestcategory"
@@ -17,7 +16,6 @@ TEST_REPO_PREFIX = "admintestcolumnmaker"
class TestFrontendAdmin(PlaywrightTestCase):
"""Frontend tests for admin-only pages and functionality."""
@skip_if_api_v1
def test_admin_page_loads_when_logged_in(self):
"""Verify admin page loads for admin user."""
self.login()
@@ -28,7 +26,6 @@ class TestFrontendAdmin(PlaywrightTestCase):
# Admin page should show re-index button
expect(page.get_by_role("button", name="Re-index search")).to_be_visible()
@skip_if_api_v1
def test_reindex_search(self):
"""Verify Re-index search button works and returns results."""
self.login()
@@ -52,7 +49,6 @@ class TestFrontendAdmin(PlaywrightTestCase):
expect(page.locator("text=repositories_indexed")).to_be_visible(timeout=30000)
expect(page.locator("text=tools_indexed")).to_be_visible()
@skip_if_api_v1
def test_admin_page_screenshot(self):
"""Capture screenshot of admin page with re-index results."""
self.login()
@@ -1,7 +1,6 @@
from playwright.sync_api import expect
from galaxy_test.base.api_util import random_name
from ..base.api import skip_if_api_v1
from ..base.playwrightbrowser import Locators
from ..base.playwrighttestcase import PlaywrightTestCase
@@ -9,7 +8,6 @@ TEST_PASSWORD = "testpass"
class TestFrontendLogin(PlaywrightTestCase):
@skip_if_api_v1
def test_register(self):
self.visit_url("/")
page = self._page
@@ -26,7 +24,6 @@ class TestFrontendLogin(PlaywrightTestCase):
)
expect(page.locator(Locators.login_submit_button)).to_be_visible()
@skip_if_api_v1
def test_create(self):
user = random_name(prefix="shduser")
self.create(
@@ -35,14 +32,12 @@ class TestFrontendLogin(PlaywrightTestCase):
username=user,
)
@skip_if_api_v1
def test_logout(self):
self._create_and_login()
self._playwright_browser.expect_logged_in()
self._playwright_browser.logout_if_logged_in()
self._playwright_browser.expect_not_logged_in()
@skip_if_api_v1
def test_change_password(self):
self._create_and_login()
@@ -1,6 +1,5 @@
from playwright.sync_api import expect
from ..base.api import skip_if_api_v1
from ..base.playwrighttestcase import PlaywrightTestCase
TEST_CATEGORY_PREFIX = "guitestcategory"
@@ -31,7 +30,6 @@ class TestFrontendRepositories(PlaywrightTestCase):
self.visit_url(f"/repositories/{repository.id}/metadata-inspector")
return repository
@skip_if_api_v1
def test_metadata_inspector_loads(self):
"""Verify Metadata Inspector page loads with repository data."""
self._setup_repo_and_visit_inspector()
@@ -43,7 +41,6 @@ class TestFrontendRepositories(PlaywrightTestCase):
expect(page.locator("[role=tab]").filter(has_text="Tool History")).to_be_visible()
expect(page.locator("[role=tab]").filter(has_text="Raw JSON")).to_be_visible()
@skip_if_api_v1
def test_metadata_inspector_default_tab(self):
"""Verify Revisions is the default landing tab."""
repository = self._setup_repo_and_visit_inspector()
@@ -52,7 +49,6 @@ class TestFrontendRepositories(PlaywrightTestCase):
# Revisions tab active by default, repo name visible
expect(page.locator("body")).to_contain_text(repository.name)
@skip_if_api_v1
def test_metadata_inspector_revisions_tab(self):
"""Verify Revisions tab shows changeset data."""
self._setup_repo_and_visit_inspector()
@@ -62,7 +58,6 @@ class TestFrontendRepositories(PlaywrightTestCase):
# RevisionsTab uses q-list with expansion items showing changeset hashes
expect(page.locator(".q-list")).to_be_visible()
@skip_if_api_v1
def test_metadata_inspector_reset_tab(self):
"""Verify Reset Metadata tab with admin login."""
self.login()
@@ -77,7 +72,6 @@ class TestFrontendRepositories(PlaywrightTestCase):
# Preview button should be visible
expect(page.get_by_role("button", name="Preview")).to_be_visible()
@skip_if_api_v1
def test_metadata_inspector_screenshots(self):
"""Capture screenshots of metadata inspector tabs."""
self.login()
@@ -114,7 +108,6 @@ class TestFrontendRepositories(PlaywrightTestCase):
page.wait_for_selector("text=Preview Results") # Wait for results
self.screenshot("metadata_inspector_reset_preview")
@skip_if_api_v1
def test_metadata_inspector_reset_full(self):
"""Perform a full metadata reset and verify completion."""
self.login()
@@ -15,10 +15,7 @@ from sqlalchemy import select
from galaxy_test.base import api_asserts
from tool_shed.test.base import test_db_util
from tool_shed.webapp import model
from ..base.api import (
ShedApiTestCase,
skip_if_api_v1,
)
from ..base.api import ShedApiTestCase
CORRUPTED_PATH = "/old/wrong/path/column_maker.xml"
@@ -88,7 +85,6 @@ def metadata_has_valid_paths(metadata_dict: dict, invalid_prefix: str = "/old/wr
class TestRepositoriesIntegration(ShedApiTestCase):
@skip_if_api_v1
def test_reset_metadata_dry_run_shows_corrupted_path_fix(self):
"""Verify dry_run=True shows before/after diff for corrupted tool_config paths.
@@ -133,7 +129,6 @@ class TestRepositoriesIntegration(ShedApiTestCase):
metadata_revision.metadata["tools"][0]["tool_config"] == fixture.corrupted_path
), "dry_run should not have modified the database"
@skip_if_api_v1
def test_reset_metadata_fixes_corrupted_path_when_not_dry_run(self):
"""Verify non-dry-run reset actually fixes corrupted tool_config paths."""
repository = self.populator.setup_column_maker_repo(prefix="integfix")
@@ -18,11 +18,7 @@ from tool_shed_client.schema import (
RepositoryRevisionMetadata,
UpdateRepositoryRequest,
)
from ..base.api import (
ShedApiTestCase,
skip_if_api_v1,
skip_if_api_v2,
)
from ..base.api import ShedApiTestCase
COLUMN_MAKER_PATH = resource_path(__name__, "../test_data/column_maker/column_maker.tar")
@@ -130,7 +126,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
assert repository.owner == repo.owner
assert repository.name == repo.name
@skip_if_api_v1
def test_index_pagination(self):
populator = self.populator
category1 = populator.new_category(prefix="paginatecat1")
@@ -161,7 +156,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
response = populator.repository_index_paginated(request)
assert response.total_results == 1
@skip_if_api_v1
def test_index_sorting(self):
populator = self.populator
category1 = populator.new_category(prefix="paginatecat1")
@@ -184,7 +178,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
assert "_a" in order_of_these[0]
assert "_z" in order_of_these[1]
@skip_if_api_v1
def test_allow_push(self):
populator = self.populator
request = {
@@ -216,7 +209,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
assert "sharewith" not in populator.get_usernames_allowed_to_push(repo)
assert "alsosharewith" in populator.get_usernames_allowed_to_push(repo)
@skip_if_api_v1
def test_set_malicious(self):
populator = self.populator
repository = populator.setup_column_maker_repo(prefix="repoformalicious")
@@ -231,7 +223,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
populator.unset_malicious(repository, only_revision.changeset_revision)
assert not populator.tip_is_malicious(repository)
@skip_if_api_v1
def test_set_deprecated(self):
populator = self.populator
repository = populator.setup_column_maker_repo(prefix="repofordeprecated")
@@ -300,7 +291,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
else:
raise AssertionError("Wrong number of repo tars returned...")
@skip_if_api_v1
def test_readmes(self):
populator = self.populator
repository = populator.setup_test_data_repo("column_maker_with_readme")
@@ -334,21 +324,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
api_asserts.assert_status_code_is_ok(response)
populator.assert_has_n_installable_revisions(repository, 3)
@skip_if_api_v2
def test_reset_all_v1(self):
populator = self.populator
repository = populator.setup_test_data_repo("column_maker_with_download_gaps")
populator.assert_has_n_installable_revisions(repository, 3)
# resetting one at a time or resetting everything via the web controllers works...
# resetting all at once via the API does not work - it breaks the repository
response = self.api_interactor.post(
"repositories/reset_metadata_on_repositories",
data={"payload": "can not be empty because bug in controller"},
)
api_asserts.assert_status_code_is_ok(response)
populator.assert_has_n_installable_revisions(repository, 3)
@skip_if_api_v1
def test_reset_all_v2(self):
populator = self.populator
repository = populator.setup_test_data_repo("column_maker_with_download_gaps")
@@ -357,7 +332,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
api_asserts.assert_status_code_is_ok(response)
populator.assert_has_n_installable_revisions(repository, 3)
@skip_if_api_v1
def test_reset_metadata_dry_run(self):
"""Verify dry_run=True returns success but doesn't modify repository."""
populator = self.populator
@@ -379,7 +353,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
# Revisions should still be there (nothing changed)
populator.assert_has_n_installable_revisions(repository, 3)
@skip_if_api_v1
def test_reset_metadata_verbose(self):
"""Verify verbose=True returns per-changeset details."""
populator = self.populator
@@ -401,7 +374,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
assert "numeric_revision" in detail
assert "comparison_result" in detail or "error" in detail
@skip_if_api_v1
def test_reset_metadata_dry_run_and_verbose(self):
"""Verify dry_run + verbose returns details without persisting."""
populator = self.populator
@@ -422,7 +394,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
# Verify repo unchanged
populator.assert_has_n_installable_revisions(repository, 3)
@skip_if_api_v1
def test_reset_metadata_legacy_endpoint_with_dry_run(self):
"""Verify legacy endpoint supports dry_run in request body."""
populator = self.populator
@@ -438,7 +409,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
assert result["dry_run"] is True
assert result["changeset_details"] is not None
@skip_if_api_v1
def test_reset_metadata_verbose_includes_before_after(self):
"""Verify verbose=True returns repository_metadata_before and after snapshots."""
populator = self.populator
@@ -469,7 +439,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
for tool in rev_data["tools"]:
assert "tool_config" in tool
@skip_if_api_v1
def test_reset_metadata_non_verbose_omits_before_after(self):
"""Verify verbose=False (default) omits before/after metadata."""
populator = self.populator
@@ -496,7 +465,6 @@ class TestShedRepositoriesApi(ShedApiTestCase):
assert only_revision
return only_revision
@skip_if_api_v1
def test_generate_frontend_fixtures(self):
"""Generate JSON fixture files for frontend unit tests.
@@ -3,10 +3,7 @@ from tool_shed_client.schema.trs import (
ToolClass,
)
from tool_shed_client.trs_util import encode_identifier
from ..base.api import (
ShedApiTestCase,
skip_if_api_v1,
)
from ..base.api import ShedApiTestCase
class TestShedToolsApi(ShedApiTestCase):
@@ -41,12 +38,10 @@ class TestShedToolsApi(ShedApiTestCase):
tool_search_hit = response.find_search_hit(repository)
assert tool_search_hit
@skip_if_api_v1
def test_trs_service_info(self):
service_info = self.api_interactor.get("ga4gh/trs/v2/service-info")
service_info.raise_for_status()
@skip_if_api_v1
def test_trs_tool_classes(self):
classes_response = self.api_interactor.get("ga4gh/trs/v2/toolClasses")
classes_response.raise_for_status()
@@ -56,7 +51,6 @@ class TestShedToolsApi(ShedApiTestCase):
class0 = classes[0]
assert ToolClass(**class0)
@skip_if_api_v1
def test_trs_tool_list(self):
populator = self.populator
repository = populator.setup_column_maker_repo(prefix="toolstrsindex")
@@ -67,7 +61,6 @@ class TestShedToolsApi(ShedApiTestCase):
tool_response.raise_for_status()
assert Tool(**tool_response.json())
@skip_if_api_v1
def test_trs_tool_parameter_json_schema(self):
populator = self.populator
repository = populator.setup_column_maker_repo(prefix="toolsparameterschema")
@@ -77,7 +70,6 @@ class TestShedToolsApi(ShedApiTestCase):
tool_response = self.api_interactor.get(url)
tool_response.raise_for_status()
@skip_if_api_v1
def test_tool_source(self):
populator = self.populator
repository = populator.setup_column_maker_repo(prefix="toolsource")
@@ -7,10 +7,7 @@ from tool_shed_client.schema import (
CreateUserRequest,
User,
)
from ..base.api import (
ShedApiTestCase,
skip_if_api_v1,
)
from ..base.api import ShedApiTestCase
from ..base.api_util import (
email_to_username,
ensure_user_with_email,
@@ -80,7 +77,6 @@ class TestShedUsersApi(ShedApiTestCase):
assert show_response.json()["username"] == username
assert show_response.json()["id"] == user_id
@skip_if_api_v1
def test_api_key_endpoints(self):
email = "testindexapi@bx.psu.edu"
password = "mycoolpassword123"
-9
View File
@@ -1,9 +0,0 @@
from galaxy.webapps.base.controller import BaseAPIController
from tool_shed.structured_app import ToolShedApp
class BaseShedAPIController(BaseAPIController):
app: ToolShedApp
def __init__(self, app: ToolShedApp):
super().__init__(app)
-44
View File
@@ -1,44 +0,0 @@
"""
API key retrieval through BaseAuth
Sample usage:
.. code-block::
curl --user zipzap@foo.com:password http://localhost:9009/api/authenticate/baseauth
Returns
.. code-block:: json
{
"api_key": "<some api key>"
}
"""
import logging
from galaxy.web import expose_api_anonymous_and_sessionless
from galaxy.webapps.galaxy.api import depends
from galaxy.webapps.galaxy.services.authenticate import AuthenticationService
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class ToolShedAuthenticationController(BaseShedAPIController):
authentication_service = depends(AuthenticationService)
@expose_api_anonymous_and_sessionless
def get_tool_shed_api_key(self, trans, **kwd):
"""
GET /api/authenticate/baseauth
returns an API key for authenticated user based on BaseAuth headers
:returns: api_key in json format
:rtype: dict
:raises: ObjectNotFound, HTTPBadRequest
"""
return self.authentication_service.get_api_key(trans.environ, trans.request)
-112
View File
@@ -1,112 +0,0 @@
import logging
from typing import (
Any,
)
import tool_shed.util.shed_util_common as suc
import tool_shed_client.schema
from galaxy import (
util,
web,
)
from galaxy.web import (
expose_api,
expose_api_anonymous_and_sessionless,
require_admin,
)
from galaxy.webapps.galaxy.api import depends
from tool_shed.managers.categories import CategoryManager
from tool_shed.managers.repositories import repositories_by_category
from tool_shed.webapp.model import Category
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class CategoriesController(BaseShedAPIController):
"""RESTful controller for interactions with categories in the Tool Shed."""
category_manager: CategoryManager = depends(CategoryManager)
@expose_api
@require_admin
def create(self, trans, payload, **kwd):
"""
POST /api/categories
Return a dictionary of information about the created category.
The following parameters are included in the payload:
:param name (required): the name of the category
:param description (optional): the description of the category (if not provided, the name will be used)
Example: POST /api/categories/?key=XXXYYYXXXYYY
Content-Disposition: form-data; name="name" Category_Name
Content-Disposition: form-data; name="description" Category_Description
"""
category_dict = dict(message="", status="ok")
request = tool_shed_client.schema.CreateCategoryRequest(
name=payload.get("name"),
description=payload.get("description", ""),
)
category: Category = self.category_manager.create(trans, request)
category_dict = self.category_manager.to_dict(category)
category_dict["message"] = f"Category '{str(category.name)}' has been created"
return category_dict
@expose_api_anonymous_and_sessionless
def get_repositories(self, trans, category_id, **kwd):
"""
GET /api/categories/{encoded_category_id}/repositories
Return information about the provided category and the repositories in that category.
:param id: the encoded id of the Category object
:param sort_key: the field by which the repositories should be sorted
:param sort_order: ascending or descending sort
:param page: the page number to return
Example: GET localhost:9009/api/categories/f9cad7b01a472135/repositories
"""
installable = util.asbool(kwd.get("installable", "false"))
sort_key = kwd.get("sort_key", "name")
sort_order = kwd.get("sort_order", "asc")
page = kwd.get("page", None)
category_dict = repositories_by_category(
self.app,
category_id,
page=page,
sort_key=sort_key,
sort_order=sort_order,
installable=installable,
)
category_dict["url"] = web.url_for(controller="categories", action="show", id=category_dict["id"])
return category_dict
@expose_api_anonymous_and_sessionless
def index(self, trans, deleted=False, **kwd) -> list[dict[str, Any]]:
"""
GET /api/categories
Return a list of dictionaries that contain information about each Category.
:param deleted: flag used to include deleted categories
Example: GET localhost:9009/api/categories
"""
deleted = util.asbool(deleted)
return self.category_manager.index(trans, deleted)
@expose_api_anonymous_and_sessionless
def show(self, trans, id, **kwd):
"""
GET /api/categories/{encoded_category_id}
Return a dictionary of information about a category.
:param id: the encoded id of the Category object
Example: GET localhost:9009/api/categories/f9cad7b01a472135
"""
category = suc.get_category(self.app, id)
if category is None:
category_dict = dict(message=f"Unable to locate category record for id {str(id)}.", status="error")
return category_dict
category_dict = self.category_manager.to_dict(category)
return category_dict
-25
View File
@@ -1,25 +0,0 @@
"""
API operations allowing clients to determine Tool Shed instance's
capabilities and configuration settings.
"""
import logging
from galaxy.web import expose_api_anonymous_and_sessionless
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class ConfigurationController(BaseShedAPIController):
@expose_api_anonymous_and_sessionless
def version(self, trans, **kwds):
"""
GET /api/version
Return a description of the version_major and version of Galaxy Tool Shed
(e.g. 15.07 and 15.07.dev).
:rtype: dict
:returns: dictionary with versions keyed as 'version_major' and 'version'
"""
return {"version_major": self.app.config.version_major, "version": self.app.config.version}
-180
View File
@@ -1,180 +0,0 @@
import logging
from collections.abc import Callable
from sqlalchemy import select
from galaxy import (
util,
web,
)
from galaxy.exceptions import (
AdminRequiredException,
ObjectNotFound,
RequestParameterMissingException,
)
from galaxy.util import (
pretty_print_time_interval,
UNKNOWN,
)
from galaxy.web import (
expose_api,
expose_api_anonymous_and_sessionless,
require_admin,
)
from tool_shed.managers import groups
from tool_shed.structured_app import ToolShedApp
from tool_shed.webapp.model import (
Category,
Repository,
RepositoryCategoryAssociation,
RepositoryMetadata,
User,
)
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class GroupsController(BaseShedAPIController):
"""RESTful controller for interactions with groups in the Tool Shed."""
def __init__(self, app: ToolShedApp):
super().__init__(app)
self.group_manager = groups.GroupManager()
def __get_value_mapper(self, trans) -> dict[str, Callable]:
value_mapper = {"id": trans.security.encode_id}
return value_mapper
@expose_api_anonymous_and_sessionless
def index(self, trans, deleted=False, **kwd):
"""
GET /api/groups
Return a list of dictionaries that contain information about each Group.
:param deleted: flag used to include deleted groups
Example: GET localhost:9009/api/groups
"""
group_dicts = []
deleted = util.asbool(deleted)
if deleted and not trans.user_is_admin:
raise AdminRequiredException("Only administrators can query deleted groups.")
for group in self.group_manager.list(trans, deleted):
group_dicts.append(self._populate(trans, group))
return group_dicts
@expose_api
@require_admin
def create(self, trans, payload, **kwd):
"""
POST /api/groups
Return a dictionary of information about the created group.
The following parameters are included in the payload:
:param name (required): the name of the group
:param description (optional): the description of the group
Example: POST /api/groups/?key=XXXYYYXXXYYY
Content-Disposition: form-data; name="name" Group_Name
Content-Disposition: form-data; name="description" Group_Description
"""
group_dict = dict(message="", status="ok")
if name := payload.get("name", ""):
description = payload.get("description", "")
if not description:
description = ""
else:
# TODO add description field to the model
group_dict = self.group_manager.create(trans, name=name).to_dict(
view="element", value_mapper=self.__get_value_mapper(trans)
)
else:
raise RequestParameterMissingException('Missing required parameter "name".')
return group_dict
@expose_api_anonymous_and_sessionless
def show(self, trans, encoded_id, **kwd):
"""
GET /api/groups/{encoded_group_id}
Return a dictionary of information about a group.
:param id: the encoded id of the Group object
Example: GET localhost:9009/api/groups/f9cad7b01a472135
"""
decoded_id = trans.security.decode_id(encoded_id)
group = self.group_manager.get(trans, decoded_id)
if group is None:
raise ObjectNotFound("Unable to locate group record with the given id.")
return self._populate(trans, group)
def _populate(self, trans, group):
"""
Turn the given group information from DB into a dict
and add other characteristics like members and repositories.
"""
group_dict = group.to_dict(view="collection", value_mapper=self.__get_value_mapper(trans))
group_members = []
group_repos = []
total_downloads = 0
for uga in group.users:
user = trans.sa_session.get(User, uga.user_id)
user_repos_count = 0
for repo in get_user_repositories(trans.sa_session, uga.user_id):
categories = []
for rca in repo.categories:
cat_dict = dict(name=rca.category.name, id=trans.app.security.encode_id(rca.category.id))
categories.append(cat_dict)
time_repo_created_full = repo.create_time.strftime("%Y-%m-%d %I:%M %p")
time_repo_updated_full = repo.update_time.strftime("%Y-%m-%d %I:%M %p")
time_repo_created = pretty_print_time_interval(repo.create_time, True)
time_repo_updated = pretty_print_time_interval(repo.update_time, True)
# TODO add user ratings
total_downloads += repo.times_downloaded
group_repos.append(
{
"name": repo.name,
"times_downloaded": repo.times_downloaded,
"owner": repo.user.username,
"time_created_full": time_repo_created_full,
"time_created": time_repo_created,
"time_updated_full": time_repo_updated_full,
"time_updated": time_repo_updated,
"description": repo.description,
"categories": categories,
}
)
user_repos_count += 1
encoded_user_id = trans.app.security.encode_id(user.id)
user_repos_url = web.url_for(
controller="repository", action="browse_repositories_by_user", user_id=encoded_user_id
)
time_created = pretty_print_time_interval(user.create_time, True)
member_dict = {
"id": encoded_user_id,
"username": user.username,
"user_repos_url": user_repos_url,
"user_repos_count": user_repos_count,
"user_tools_count": UNKNOWN,
"time_created": time_created,
}
group_members.append(member_dict)
group_dict["members"] = group_members
group_dict["total_members"] = len(group_members)
group_dict["repositories"] = group_repos
group_dict["total_repos"] = len(group_repos)
group_dict["total_downloads"] = total_downloads
return group_dict
def get_user_repositories(session, user_id):
stmt = (
select(Repository)
.where(Repository.user_id == user_id)
.join(RepositoryMetadata)
.join(User)
.outerjoin(RepositoryCategoryAssociation)
.outerjoin(Category)
)
return session.scalars(stmt)
-734
View File
@@ -1,734 +0,0 @@
import json
import logging
import os
from collections.abc import Callable
from io import StringIO
from time import strftime
from webob.compat import cgi_FieldStorage
from galaxy import (
util,
web,
)
from galaxy.exceptions import (
ActionInputError,
InsufficientPermissionsException,
MessageException,
ObjectNotFound,
RequestParameterInvalidException,
RequestParameterMissingException,
)
from galaxy.web import (
expose_api,
expose_api_anonymous_and_sessionless,
expose_api_raw_anonymous_and_sessionless,
)
from galaxy.webapps.base.controller import HTTPBadRequest
from tool_shed.managers.repositories import (
can_update_repo,
check_updates,
create_repository,
get_install_info,
get_ordered_installable_revisions,
get_repository_metadata_dict,
get_value_mapper,
index_repositories,
index_tool_ids,
IndexRequest,
reset_metadata_on_repository,
search,
to_element_dict,
UpdatesRequest,
upload_tar_and_set_metadata,
)
from tool_shed.metadata import repository_metadata_manager
from tool_shed.repository_types import util as rt_util
from tool_shed.util import (
metadata_util,
repository_util,
tool_util,
)
from tool_shed.webapp import model
from tool_shed.webapp.model.db import get_repository_by_name_and_owner
from tool_shed_client.schema import (
CreateRepositoryRequest,
LegacyInstallInfoTuple,
)
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class RepositoriesController(BaseShedAPIController):
"""RESTful controller for interactions with repositories in the Tool Shed."""
@web.legacy_expose_api
def add_repository_registry_entry(self, trans, payload, **kwd):
"""
POST /api/repositories/add_repository_registry_entry
Adds appropriate entries to the repository registry for the repository defined by the received name and owner.
:param key: the user's API key
The following parameters are included in the payload.
:param tool_shed_url (required): the base URL of the Tool Shed containing the Repository
:param name (required): the name of the Repository
:param owner (required): the owner of the Repository
"""
response_dict = {}
if not trans.user_is_admin:
response_dict["status"] = "error"
response_dict["message"] = "You are not authorized to add entries to this Tool Shed's repository registry."
return response_dict
tool_shed_url = payload.get("tool_shed_url", "")
if not tool_shed_url:
raise HTTPBadRequest(detail="Missing required parameter 'tool_shed_url'.")
tool_shed_url = tool_shed_url.rstrip("/")
name = payload.get("name", "")
if not name:
raise HTTPBadRequest(detail="Missing required parameter 'name'.")
owner = payload.get("owner", "")
if not owner:
raise HTTPBadRequest(detail="Missing required parameter 'owner'.")
repository = get_repository_by_name_and_owner(self.app.model.context, name, owner)
if repository is None:
error_message = f"Cannot locate repository with name {name} and owner {owner},"
log.debug(error_message)
response_dict["status"] = "error"
response_dict["message"] = error_message
return response_dict
# Update the repository registry.
self.app.repository_registry.add_entry(repository)
response_dict["status"] = "ok"
response_dict["message"] = (
f"Entries for repository {name} owned by {owner} have been added to the Tool Shed repository registry."
)
return response_dict
@web.legacy_expose_api_anonymous
def get_ordered_installable_revisions(self, trans, name=None, owner=None, **kwd):
"""
GET /api/repositories/get_ordered_installable_revisions
:param name: the name of the Repository
:param owner: the owner of the Repository
Returns the ordered list of changeset revision hash strings that are associated with installable revisions.
As in the changelog, the list is ordered oldest to newest.
"""
# Example URL: http://localhost:9009/api/repositories/get_ordered_installable_revisions?name=add_column&owner=test
if name is None:
name = kwd.get("name", None)
if owner is None:
owner = kwd.get("owner", None)
tsr_id = kwd.get("tsr_id", None)
return get_ordered_installable_revisions(self.app, name, owner, tsr_id)
@web.legacy_expose_api_anonymous
def get_repository_revision_install_info(
self, trans, name, owner, changeset_revision, **kwd
) -> LegacyInstallInfoTuple:
"""
GET /api/repositories/get_repository_revision_install_info
:param name: the name of the Repository
:param owner: the owner of the Repository
:param changeset_revision: the changeset_revision of the RepositoryMetadata object associated with the Repository
Returns a list of the following dictionaries
- a dictionary defining the Repository. For example::
{
"deleted": false,
"deprecated": false,
"description": "add_column hello",
"id": "f9cad7b01a472135",
"long_description": "add_column hello",
"name": "add_column",
"owner": "test",
"private": false,
"times_downloaded": 6,
"url": "/api/repositories/f9cad7b01a472135",
"user_id": "f9cad7b01a472135"
}
- a dictionary defining the Repository revision (RepositoryMetadata). For example::
{
"changeset_revision": "3a08cc21466f",
"downloadable": true,
"has_repository_dependencies": false,
"has_repository_dependencies_only_if_compiling_contained_td": false,
"id": "f9cad7b01a472135",
"includes_datatypes": false,
"includes_tool_dependencies": false,
"includes_tools": true,
"includes_tools_for_display_in_tool_panel": true,
"includes_workflows": false,
"malicious": false,
"repository_id": "f9cad7b01a472135",
"url": "/api/repository_revisions/f9cad7b01a472135",
"valid_tools": [{u'add_to_tool_panel': True,
u'description': u'data on any column using simple expressions',
u'guid': u'localhost:9009/repos/enis/sample_repo_1/Filter1/2.2.0',
u'id': u'Filter1',
u'name': u'Filter',
u'requirements': [],
u'tests': [{u'inputs': [[u'input', u'1.bed'], [u'cond', u"c1=='chr22'"]],
u'name': u'Test-1',
u'outputs': [[u'out_file1', u'filter1_test1.bed']],
u'required_files': [u'1.bed', u'filter1_test1.bed']}],
u'tool_config': u'database/community_files/000/repo_1/filtering.xml',
u'tool_type': u'default',
u'version': u'2.2.0',
u'version_string_cmd': None}]
}
- a dictionary including the additional information required to install the repository. For example::
{
"add_column": [
"add_column hello",
"http://test@localhost:9009/repos/test/add_column",
"3a08cc21466f",
"1",
"test",
{},
{}
]
}
"""
return get_install_info(trans, name, owner, changeset_revision)
@web.legacy_expose_api_anonymous
def get_installable_revisions(self, trans, **kwd):
"""
GET /api/repositories/get_installable_revisions
:param tsr_id: the encoded toolshed ID of the repository
Returns a list of lists of changesets, in the format [ [ 0, fbb391dc803c ], [ 1, 9d9ec4d9c03e ], [ 2, 9b5b20673b89 ], [ 3, e8c99ce51292 ] ].
"""
# Example URL: http://localhost:9009/api/repositories/get_installable_revisions?tsr_id=9d37e53072ff9fa4
if (tsr_id := kwd.get("tsr_id", None)) is not None:
repository = repository_util.get_repository_in_tool_shed(
self.app, tsr_id, eagerload_columns=[model.Repository.downloadable_revisions]
)
else:
error_message = "Error in the Tool Shed repositories API in get_ordered_installable_revisions: "
error_message += "missing or invalid parameter received."
log.debug(error_message)
return []
return repository.installable_revisions(self.app)
def __get_value_mapper(self, trans) -> dict[str, Callable]:
return get_value_mapper(self.app)
@expose_api_raw_anonymous_and_sessionless
def index(self, trans, deleted=False, owner=None, name=None, **kwd):
"""
GET /api/repositories
Displays a collection of repositories with optional criteria.
:param q: (optional)if present search on the given query will be performed
:type q: str
:param page: (optional)requested page of the search
:type page: int
:param page_size: (optional)requested page_size of the search
:type page_size: int
:param jsonp: (optional)flag whether to use jsonp format response, defaults to False
:type jsonp: bool
:param callback: (optional)name of the function to wrap callback in
used only when jsonp is true, defaults to 'callback'
:type callback: str
:param deleted: (optional)displays repositories that are or are not set to deleted.
:type deleted: bool
:param owner: (optional)the owner's public username.
:type owner: str
:param name: (optional)the repository name.
:type name: str
:param tool_ids: (optional) a tool GUID to find the repository for
:param tool_ids: str
:returns dict: object containing list of results
Examples:
GET http://localhost:9009/api/repositories
GET http://localhost:9009/api/repositories?q=fastq
"""
repository_dicts = []
deleted = util.asbool(deleted)
if q := kwd.get("q", ""):
page = kwd.get("page", 1)
page_size = kwd.get("page_size", 10)
try:
page = int(page)
page_size = int(page_size)
except ValueError:
raise RequestParameterInvalidException('The "page" and "page_size" parameters have to be integers.')
return_jsonp = util.asbool(kwd.get("jsonp", False))
callback = kwd.get("callback", "callback")
search_results = search(trans, q, page, page_size)
if return_jsonp:
response = str(f"{callback}({json.dumps(search_results)});")
else:
response = json.dumps(search_results)
return response
if (tool_ids := kwd.get("tool_ids", None)) is not None:
tool_ids = util.listify(tool_ids)
response = index_tool_ids(self.app, tool_ids)
return json.dumps(response)
else:
index_request = IndexRequest(owner=owner, name=name, deleted=deleted)
repositories = index_repositories(self.app, index_request)
repository_dicts = []
for repository in repositories:
repository_dict = repository.to_dict(view="collection", value_mapper=self.__get_value_mapper(trans))
repository_dict["category_ids"] = [
trans.security.encode_id(x.category.id) for x in repository.categories
]
repository_dicts.append(repository_dict)
return json.dumps(repository_dicts)
@web.legacy_expose_api
def remove_repository_registry_entry(self, trans, payload, **kwd):
"""
POST /api/repositories/remove_repository_registry_entry
Removes appropriate entries from the repository registry for the repository defined by the received name and owner.
:param key: the user's API key
The following parameters are included in the payload.
:param tool_shed_url (required): the base URL of the Tool Shed containing the Repository
:param name (required): the name of the Repository
:param owner (required): the owner of the Repository
"""
response_dict = {}
if not trans.user_is_admin:
response_dict["status"] = "error"
response_dict["message"] = (
"You are not authorized to remove entries from this Tool Shed's repository registry."
)
return response_dict
tool_shed_url = payload.get("tool_shed_url", "")
if not tool_shed_url:
raise HTTPBadRequest(detail="Missing required parameter 'tool_shed_url'.")
tool_shed_url = tool_shed_url.rstrip("/")
name = payload.get("name", "")
if not name:
raise HTTPBadRequest(detail="Missing required parameter 'name'.")
owner = payload.get("owner", "")
if not owner:
raise HTTPBadRequest(detail="Missing required parameter 'owner'.")
repository = get_repository_by_name_and_owner(self.app.model.context, name, owner)
if repository is None:
error_message = f"Cannot locate repository with name {name} and owner {owner},"
log.debug(error_message)
response_dict["status"] = "error"
response_dict["message"] = error_message
return response_dict
# Update the repository registry.
self.app.repository_registry.remove_entry(repository)
response_dict["status"] = "ok"
response_dict["message"] = (
f"Entries for repository {name} owned by {owner} have been removed from the Tool Shed repository registry."
)
return response_dict
@web.legacy_expose_api
def reset_metadata_on_repositories(self, trans, payload, **kwd):
"""
POST /api/repositories/reset_metadata_on_repositories
Resets all metadata on all repositories in the Tool Shed in an "orderly fashion". Since there are currently only two
repository types (tool_dependecy_definition and unrestricted), the order in which metadata is reset is repositories of
type tool_dependecy_definition first followed by repositories of type unrestricted, and only one pass is necessary. If
a new repository type is introduced, the process will undoubtedly need to be revisited. To facilitate this order, an
in-memory list of repository ids that have been processed is maintained.
:param key: the API key of the Tool Shed user.
:param my_writable (optional):
if the API key is associated with an admin user in the Tool Shed, setting this param value
to True will restrict resetting metadata to only repositories that are writable by the user
in addition to those repositories of type tool_dependency_definition. This param is ignored
if the current user is not an admin user, in which case this same restriction is automatic.
:param encoded_ids_to_skip (optional): a list of encoded repository ids for repositories that should not be processed.
:param skip_file (optional):
A local file name that contains the encoded repository ids associated with repositories to skip.
This param can be used as an alternative to the above encoded_ids_to_skip.
"""
def handle_repository(trans, repository, results):
log.debug(f"Resetting metadata on repository {repository.name}")
try:
rmm = repository_metadata_manager.RepositoryMetadataManager(
trans,
resetting_all_metadata_on_repository=True,
updating_installed_repository=False,
repository=repository,
persist=False,
)
rmm.reset_all_metadata_on_repository_in_tool_shed()
rmm_invalid_file_tups = rmm.get_invalid_file_tups()
if rmm_invalid_file_tups:
message = tool_util.generate_message_for_invalid_tools(
self.app, rmm_invalid_file_tups, repository, None, as_html=False
)
results["unsuccessful_count"] += 1
else:
message = f"Successfully reset metadata on repository {repository.name} owned by {repository.user.username}"
results["successful_count"] += 1
except Exception as e:
message = (
f"Error resetting metadata on repository {repository.name} owned by {repository.user.username}: {e}"
)
results["unsuccessful_count"] += 1
status = f"{repository.name} : {message}"
results["repository_status"].append(status)
return results
start_time = strftime("%Y-%m-%d %H:%M:%S")
results = dict(start_time=start_time, repository_status=[], successful_count=0, unsuccessful_count=0)
handled_repository_ids = []
encoded_ids_to_skip = payload.get("encoded_ids_to_skip", [])
skip_file = payload.get("skip_file", None)
if skip_file and os.path.exists(skip_file) and not encoded_ids_to_skip:
# Load the list of encoded_ids_to_skip from the skip_file.
# Contents of file must be 1 encoded repository id per line.
lines = open(skip_file, "rb").readlines()
for line in lines:
if line.startswith("#"):
# Skip comments.
continue
encoded_ids_to_skip.append(line.rstrip("\n"))
if trans.user_is_admin:
my_writable = util.asbool(payload.get("my_writable", False))
else:
my_writable = True
rmm = repository_metadata_manager.RepositoryMetadataManager(
trans,
resetting_all_metadata_on_repository=True,
updating_installed_repository=False,
persist=False,
)
# First reset metadata on all repositories of type repository_dependency_definition.
for repository in rmm.get_repositories_for_setting_metadata(my_writable=my_writable, order=False):
encoded_id = trans.security.encode_id(repository.id)
if encoded_id in encoded_ids_to_skip:
log.debug(
"Skipping repository with id %s because it is in encoded_ids_to_skip %s",
repository.id,
encoded_ids_to_skip,
)
elif repository.type == rt_util.TOOL_DEPENDENCY_DEFINITION and repository.id not in handled_repository_ids:
results = handle_repository(trans, repository, results)
# Now reset metadata on all remaining repositories.
for repository in rmm.get_repositories_for_setting_metadata(my_writable=my_writable, order=False):
encoded_id = trans.security.encode_id(repository.id)
if encoded_id in encoded_ids_to_skip:
log.debug(
"Skipping repository with id %s because it is in encoded_ids_to_skip %s",
repository.id,
encoded_ids_to_skip,
)
elif repository.type != rt_util.TOOL_DEPENDENCY_DEFINITION and repository.id not in handled_repository_ids:
results = handle_repository(trans, repository, results)
stop_time = strftime("%Y-%m-%d %H:%M:%S")
results["stop_time"] = stop_time
return json.dumps(results, sort_keys=True, indent=4)
@web.legacy_expose_api
def reset_metadata_on_repository(self, trans, payload, **kwd):
"""
POST /api/repositories/reset_metadata_on_repository
Resets all metadata on a specified repository in the Tool Shed.
:param key: the API key of the Tool Shed user.
The following parameters must be included in the payload.
:param repository_id: the encoded id of the repository on which metadata is to be reset.
"""
repository_id = payload.get("repository_id", None)
return reset_metadata_on_repository(trans, repository_id).model_dump()
@expose_api_anonymous_and_sessionless
def show(self, trans, id, **kwd):
"""
GET /api/repositories/{encoded_repository_id}
Returns information about a repository in the Tool Shed.
Example URL: http://localhost:9009/api/repositories/f9cad7b01a472135
:param id: the encoded id of the Repository object
:type id: encoded str
:returns: detailed repository information
:rtype: dict
:raises: ObjectNotFound, MalformedId
"""
repository = repository_util.get_repository_in_tool_shed(self.app, id)
if repository is None:
raise ObjectNotFound("Unable to locate repository for the given id.")
repository_dict = repository.to_dict(view="element", value_mapper=self.__get_value_mapper(trans))
# TODO the following property would be better suited in the to_dict method
repository_dict["category_ids"] = [trans.security.encode_id(x.category.id) for x in repository.categories]
return repository_dict
@expose_api_raw_anonymous_and_sessionless
def updates(self, trans, **kwd):
"""
GET /api/repositories/updates
Return a dictionary with boolean values for whether there are updates available
for the repository revision, newer installable revisions available,
the revision is the latest installable revision, and if the repository is deprecated.
:param owner: owner of the repository
:type owner: str
:param name: name of the repository
:type name: str
:param changeset_revision: changeset of the repository
:type changeset_revision: str
:param hexlify: flag whether to hexlify the response (for backward compatibility)
:type changeset: boolean
:returns: information about repository deprecations, updates, and upgrades
:rtype: dict
"""
name = kwd.get("name", None)
owner = kwd.get("owner", None)
changeset_revision = kwd.get("changeset_revision", None)
hexlify_this = util.asbool(kwd.get("hexlify", True))
request = UpdatesRequest(
name=name,
owner=owner,
changeset_revision=changeset_revision,
hexlify=hexlify_this,
)
return check_updates(trans.app, request)
@expose_api_anonymous_and_sessionless
def show_tools(self, trans, id, changeset, **kwd):
repository_metadata = metadata_util.get_repository_metadata_by_changeset_revision(self.app, id, changeset)
if repository_metadata is not None:
encoded_repository_metadata_id = trans.security.encode_id(repository_metadata.id)
repository_metadata_dict = repository_metadata.to_dict(
view="collection", value_mapper=self.__get_value_mapper(trans)
)
repository_metadata_dict["url"] = web.url_for(
controller="repository_revisions", action="show", id=encoded_repository_metadata_id
)
if "tools" in repository_metadata.metadata:
repository_metadata_dict["valid_tools"] = repository_metadata.metadata["tools"]
return repository_metadata_dict
else:
log.debug(
"Unable to locate repository_metadata record for repository id %s and changeset_revision %s",
id,
changeset,
)
return {}
@expose_api_anonymous_and_sessionless
def metadata(self, trans, id, **kwd):
"""
GET /api/repositories/{encoded_repository_id}/metadata
Returns information about a repository in the Tool Shed.
Example URL: http://localhost:9009/api/repositories/f9cad7b01a472135/metadata
:param id: the encoded id of the Repository object
:param downloadable_only: Return only downloadable revisions (defaults to True).
Added for test cases - shouldn't be considered part of the stable API.
:returns: A dictionary containing the specified repository's metadata, by changeset,
recursively including dependencies and their metadata.
:not found: Empty dictionary.
"""
recursive = util.asbool(kwd.get("recursive", "True"))
downloadable_only = util.asbool(kwd.get("downloadable_only", "True"))
return get_repository_metadata_dict(self.app, id, recursive, downloadable_only)
@expose_api
def update(self, trans, id, **kwd):
"""
PATCH /api/repositories/{encoded_repository_id}
Updates information about a repository in the Tool Shed.
:param id: the encoded id of the Repository object
:param payload: dictionary structure containing
'name': repo's name (optional)
'synopsis': repo's synopsis (optional)
'description': repo's description (optional)
'remote_repository_url': repo's remote repo (optional)
'homepage_url': repo's homepage url (optional)
'category_ids': list of existing encoded TS category ids the updated repo should be associated with (optional)
:type payload: dict
:returns: detailed repository information
:rtype: dict
:raises: RequestParameterInvalidException, InsufficientPermissionsException
"""
payload = kwd.get("payload", None)
if not payload:
raise RequestParameterMissingException("You did not specify any payload.")
name = payload.get("name", None)
synopsis = payload.get("synopsis", None)
description = payload.get("description", None)
remote_repository_url = payload.get("remote_repository_url", None)
homepage_url = payload.get("homepage_url", None)
category_ids = payload.get("category_ids", None)
if category_ids is not None:
# We need to know if it was actually passed, and listify turns None into []
category_ids = util.listify(category_ids)
update_kwds = dict(
name=name,
description=synopsis,
long_description=description,
remote_repository_url=remote_repository_url,
homepage_url=homepage_url,
category_ids=category_ids,
)
repo, message = repository_util.update_repository(trans, id, **update_kwds)
if repo is None:
if "You are not the owner" in message:
raise InsufficientPermissionsException(message)
else:
raise ActionInputError(message)
repository_dict = repo.to_dict(view="element", value_mapper=self.__get_value_mapper(trans))
repository_dict["category_ids"] = [trans.security.encode_id(x.category.id) for x in repo.categories]
return repository_dict
@expose_api
def create(self, trans, **kwd):
"""
POST /api/repositories:
Creates a new repository.
Only ``name`` and ``synopsis`` parameters are required.
:param payload: dictionary structure containing
'name': new repo's name (required)
'synopsis': new repo's synopsis (required)
'description': new repo's description (optional)
'remote_repository_url': new repo's remote repo (optional)
'homepage_url': new repo's homepage url (optional)
'category_ids[]': list of existing encoded TS category ids the new repo should be associated with (optional)
'type': new repo's type, defaults to ``unrestricted`` (optional)
:type payload: dict
:returns: detailed repository information
:rtype: dict
:raises: RequestParameterMissingException, RequestParameterInvalidException
"""
payload = kwd.get("payload", None)
if not payload:
raise RequestParameterMissingException("You did not specify any payload.")
name = payload.get("name", None)
if not name:
raise RequestParameterMissingException("Missing required parameter 'name'.")
synopsis = payload.get("synopsis", None)
if not synopsis:
raise RequestParameterMissingException("Missing required parameter 'synopsis'.")
description = payload.get("description", "")
remote_repository_url = payload.get("remote_repository_url", "")
homepage_url = payload.get("homepage_url", "")
repo_type = payload.get("type", rt_util.UNRESTRICTED)
if repo_type not in rt_util.types:
raise RequestParameterInvalidException("This repository type is not valid")
request = CreateRepositoryRequest(
name=name,
synopsis=synopsis,
description=description,
remote_repository_url=remote_repository_url,
homepage_url=homepage_url,
category_ids=payload.get("category_ids[]", ""),
type_=repo_type,
)
repo = create_repository(trans, request)
return to_element_dict(self.app, repo, include_categories=True)
@web.legacy_expose_api
def create_changeset_revision(self, trans, id, payload, **kwd):
"""
POST /api/repositories/{encoded_repository_id}/changeset_revision
Create a new tool shed repository commit - leaving PUT on parent
resource open for updating meta-attributes of the repository (and
Galaxy doesn't allow PUT multipart data anyway
https://trello.com/c/CQwmCeG6).
:param id: the encoded id of the Repository object
The following parameters may be included in the payload.
:param commit_message: hg commit message for update.
"""
# Example URL: http://localhost:9009/api/repositories/f9cad7b01a472135
repository = repository_util.get_repository_in_tool_shed(self.app, id)
if not can_update_repo(trans, repository):
trans.response.status = 400
return {
"err_msg": "You do not have permission to update this repository.",
}
file_data = payload.get("file")
# Code stolen from gx's upload_common.py
if isinstance(file_data, cgi_FieldStorage):
assert not isinstance(file_data.file, StringIO)
assert file_data.file.name != "<fdopen>"
local_filename = util.mkstemp_ln(file_data.file.name, "upload_file_data_")
file_data.file.close()
file_data = dict(filename=file_data.filename, local_filename=local_filename)
elif isinstance(file_data, dict) and "local_filename" not in file_data:
raise Exception("Uploaded file was encoded in a way not understood.")
commit_message = kwd.get("commit_message", "Uploaded")
uploaded_file_name = file_data["local_filename"]
try:
message = upload_tar_and_set_metadata(
trans,
trans.request.host,
repository,
uploaded_file_name,
commit_message,
)
rval = {"message": message}
except MessageException as e:
trans.response.status = e.status_code
rval = {"err_msg": str(e)}
if os.path.exists(uploaded_file_name):
os.remove(uploaded_file_name)
return rval
@@ -1,214 +0,0 @@
import logging
from collections.abc import Callable
from sqlalchemy import select
from galaxy import (
util,
web,
)
from galaxy.webapps.base.controller import HTTPBadRequest
from tool_shed.util import metadata_util
from tool_shed.webapp.model import RepositoryMetadata
from tool_shed.webapp.model.db import get_repository_by_name_and_owner
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class RepositoryRevisionsController(BaseShedAPIController):
"""RESTful controller for interactions with tool shed repository revisions."""
def __get_value_mapper(self, trans) -> dict[str, Callable]:
value_mapper = {
"id": trans.security.encode_id,
"repository_id": trans.security.encode_id,
"user_id": trans.security.encode_id,
}
return value_mapper
@web.legacy_expose_api_anonymous
def index(self, trans, **kwd):
"""
GET /api/repository_revisions
Displays a collection (list) of repository revisions.
"""
# Example URL: http://localhost:9009/api/repository_revisions
downloadable = kwd.get("downloadable", None)
malicious = kwd.get("malicious", None)
missing_test_components = kwd.get("missing_test_components", None)
includes_tools = kwd.get("includes_tools", None)
repository_metadata_dicts = []
all_repository_metadata = get_repository_metadata(
trans.sa_session, downloadable, malicious, missing_test_components, includes_tools
)
for repository_metadata in all_repository_metadata:
repository_metadata_dict = repository_metadata.to_dict(
view="collection", value_mapper=self.__get_value_mapper(trans)
)
repository_metadata_dict["url"] = web.url_for(
controller="repository_revisions", action="show", id=trans.security.encode_id(repository_metadata.id)
)
repository_metadata_dicts.append(repository_metadata_dict)
return repository_metadata_dicts
@web.legacy_expose_api_anonymous
def repository_dependencies(self, trans, id, **kwd):
"""
GET /api/repository_revisions/{encoded repository_metadata id}/repository_dependencies
Returns a list of dictionaries that each define a specific downloadable revision of a
repository in the Tool Shed. This method returns dictionaries with more information in
them than other methods in this controller. The information about repository_metdata is
enhanced to include information about the repository (e.g., name, owner, etc) associated
with the repository_metadata record.
:param id: the encoded id of the `RepositoryMetadata` object
"""
# Example URL: http://localhost:9009/api/repository_revisions/repository_dependencies/bb125606ff9ea620
repository_dependencies_dicts = []
repository_metadata = metadata_util.get_repository_metadata_by_id(trans.app, id)
if repository_metadata is None:
log.debug(f"Invalid repository_metadata id received: {id}")
return repository_dependencies_dicts
metadata = repository_metadata.metadata
if metadata is None:
log.debug(f"The repository_metadata record with id {id} has no metadata.")
return repository_dependencies_dicts
if "repository_dependencies" in metadata:
rd_tups = metadata["repository_dependencies"]["repository_dependencies"]
for rd_tup in rd_tups:
tool_shed, name, owner, changeset_revision = rd_tup[0:4]
repository_dependency = get_repository_by_name_and_owner(trans.sa_session, name, owner)
if repository_dependency is None:
log.debug(f"Cannot locate repository dependency {name} owned by {owner}.")
continue
repository_dependency_id = trans.security.encode_id(repository_dependency.id)
repository_dependency_repository_metadata = metadata_util.get_repository_metadata_by_changeset_revision(
trans.app, repository_dependency_id, changeset_revision
)
if repository_dependency_repository_metadata is None:
# The changeset_revision column in the repository_metadata table has been updated with a new
# value value, so find the changeset_revision to which we need to update.
new_changeset_revision = metadata_util.get_next_downloadable_changeset_revision(
trans.app, repository_dependency, changeset_revision
)
if new_changeset_revision != changeset_revision:
repository_dependency_repository_metadata = (
metadata_util.get_repository_metadata_by_changeset_revision(
trans.app, repository_dependency_id, new_changeset_revision
)
)
changeset_revision = new_changeset_revision
else:
decoded_repository_dependency_id = trans.security.decode_id(repository_dependency_id)
debug_msg = (
f"Cannot locate repository_metadata with id {decoded_repository_dependency_id} for repository dependency {name} owned by {owner} "
f"using either of these changeset_revisions: {changeset_revision}, {new_changeset_revision}."
)
log.debug(debug_msg)
continue
repository_dependency_metadata_dict = repository_dependency_repository_metadata.to_dict(
view="element", value_mapper=self.__get_value_mapper(trans)
)
repository_dependency_dict = repository_dependency.to_dict(
view="element", value_mapper=self.__get_value_mapper(trans)
)
# We need to be careful with the entries in our repository_dependency_dict here since this Tool Shed API
# controller is working with repository_metadata records. The above to_dict() method returns a dictionary
# with an id entry for the repository record. However, all of the other methods in this controller have
# the id entry associated with a repository_metadata record id. To avoid confusion, we'll update the
# repository_dependency_metadata_dict with entries from the repository_dependency_dict without using the
# Python dictionary update() method because we do not want to overwrite existing entries.
for k, v in repository_dependency_dict.items():
if k not in repository_dependency_metadata_dict:
repository_dependency_metadata_dict[k] = v
repository_dependency_metadata_dict["url"] = web.url_for(
controller="repositories", action="show", id=repository_dependency_id
)
repository_dependencies_dicts.append(repository_dependency_metadata_dict)
return repository_dependencies_dicts
@web.legacy_expose_api_anonymous
def show(self, trans, id, **kwd):
"""
GET /api/repository_revisions/{encoded_repository_metadata_id}
Displays information about a repository_metadata record in the Tool Shed.
:param id: the encoded id of the `RepositoryMetadata` object
"""
# Example URL: http://localhost:9009/api/repository_revisions/bb125606ff9ea620
repository_metadata = metadata_util.get_repository_metadata_by_id(trans.app, id)
if repository_metadata is None:
log.debug(f"Cannot locate repository_metadata with id {id}")
return {}
encoded_repository_id = trans.security.encode_id(repository_metadata.repository_id)
repository_metadata_dict = repository_metadata.to_dict(
view="element", value_mapper=self.__get_value_mapper(trans)
)
repository_metadata_dict["url"] = web.url_for(
controller="repositories", action="show", id=encoded_repository_id
)
return repository_metadata_dict
@web.legacy_expose_api
def update(self, trans, payload, **kwd):
"""
PUT /api/repository_revisions/{encoded_repository_metadata_id}/{payload}
Updates the value of specified columns of the repository_metadata table based on the key / value pairs in payload.
:param id: the encoded id of the `RepositoryMetadata` object
"""
repository_metadata_id = kwd.get("id", None)
if repository_metadata_id is None:
raise HTTPBadRequest(detail="Missing required parameter 'id'.")
repository_metadata = metadata_util.get_repository_metadata_by_id(trans.app, repository_metadata_id)
if repository_metadata is None:
decoded_repository_metadata_id = trans.security.decode_id(repository_metadata_id)
log.debug(f"Cannot locate repository_metadata with id {decoded_repository_metadata_id}")
return {}
else:
decoded_repository_metadata_id = repository_metadata.id
flush_needed = False
for key, new_value in payload.items():
if hasattr(repository_metadata, key):
# log information when setting attributes associated with the Tool Shed's install and test framework.
if key in ["includes_tools", "missing_test_components"]:
log.debug(
"Setting repository_metadata column %s to value %s for changeset_revision %s via the Tool Shed API.",
key,
new_value,
repository_metadata.changeset_revision,
)
setattr(repository_metadata, key, new_value)
flush_needed = True
if flush_needed:
log.debug(
"Updating repository_metadata record with id %s and changeset_revision %s.",
decoded_repository_metadata_id,
repository_metadata.changeset_revision,
)
trans.sa_session.add(repository_metadata)
trans.sa_session.commit()
trans.sa_session.refresh(repository_metadata)
repository_metadata_dict = repository_metadata.to_dict(
view="element", value_mapper=self.__get_value_mapper(trans)
)
repository_metadata_dict["url"] = web.url_for(
controller="repository_revisions", action="show", id=repository_metadata_id
)
return repository_metadata_dict
def get_repository_metadata(session, downloadable, malicious, missing_test_components, includes_tools):
stmt = select(RepositoryMetadata)
if downloadable is not None:
stmt = stmt.where(RepositoryMetadata.downloadable == util.asbool(downloadable))
if malicious is not None:
stmt = stmt.where(RepositoryMetadata.malicious == util.asbool(malicious))
if missing_test_components is not None:
stmt = stmt.where(RepositoryMetadata.missing_test_components == util.asbool(missing_test_components))
if includes_tools is not None:
stmt = stmt.where(RepositoryMetadata.includes_tools == util.asbool(includes_tools))
stmt = stmt.order_by(RepositoryMetadata.repository_id.desc())
return session.scalars(stmt)
-92
View File
@@ -1,92 +0,0 @@
import json
import logging
from galaxy import (
exceptions,
util,
)
from galaxy.web import (
expose_api,
expose_api_raw_anonymous_and_sessionless,
require_admin,
)
from tool_shed.managers.tools import search
from tool_shed.util.shed_index import build_index
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class ToolsController(BaseShedAPIController):
"""RESTful controller for interactions with tools in the Tool Shed."""
@expose_api
@require_admin
def build_search_index(self, trans, **kwd):
"""
PUT /api/tools/build_search_index
Not part of the stable API, just something to simplify bootstrapping tool sheds,
scripting, testing, etc...
"""
repos_indexed, tools_indexed = build_index(
trans.app.config.whoosh_index_dir,
trans.app.config.file_path,
trans.app.config.hgweb_config_dir,
trans.app.config.hgweb_repo_prefix,
trans.app.config.database_connection,
)
return {
"repositories_indexed": repos_indexed,
"tools_indexed": tools_indexed,
}
@expose_api_raw_anonymous_and_sessionless
def index(self, trans, **kwd):
"""
GET /api/tools
Displays a collection of tools with optional criteria.
:param q: (optional)if present search on the given query will be performed
:type q: str
:param page: (optional)requested page of the search
:type page: int
:param page_size: (optional)requested page_size of the search
:type page_size: int
:param jsonp: (optional)flag whether to use jsonp format response, defaults to False
:type jsonp: bool
:param callback: (optional)name of the function to wrap callback in
used only when jsonp is true, defaults to 'callback'
:type callback: str
:returns dict: object containing list of results and metadata
Examples:
GET http://localhost:9009/api/tools
GET http://localhost:9009/api/tools?q=fastq
"""
q = kwd.get("q", "")
if not q:
raise exceptions.NotImplemented(
'Listing of all the tools is not implemented. Provide parameter "q" to search instead.'
)
else:
page = kwd.get("page", 1)
page_size = kwd.get("page_size", 10)
try:
page = int(page)
page_size = int(page_size)
except ValueError:
raise exceptions.RequestParameterInvalidException('The "page" and "page_size" have to be integers.')
return_jsonp = util.asbool(kwd.get("jsonp", False))
callback = kwd.get("callback", "callback")
search_results = search(trans, q, page, page_size)
if return_jsonp:
response = str(f"{callback}({json.dumps(search_results)});")
else:
response = json.dumps(search_results)
return response
-87
View File
@@ -1,87 +0,0 @@
import logging
import tool_shed.util.shed_util_common as suc
from galaxy import (
util,
web,
)
from tool_shed.managers.users import (
api_create_user,
index,
)
from tool_shed_client.schema import CreateUserRequest
from . import BaseShedAPIController
log = logging.getLogger(__name__)
class UsersController(BaseShedAPIController):
"""RESTful controller for interactions with users in the Tool Shed."""
@web.expose_api
@web.require_admin
def create(self, trans, payload, **kwd):
"""
POST /api/users
Returns a dictionary of information about the created user.
: param key: the current Galaxy admin user's API key
The following parameters are included in the payload.
:param email (required): the email address of the user
:param password (required): the password of the user
:param username (required): the public username of the user
"""
# Get the information about the user to be created from the payload.
email = payload.get("email", "")
password = payload.get("password", "")
username = payload.get("username", "")
# Create the user.
request = CreateUserRequest(
email=email,
username=username,
password=password,
)
user = api_create_user(trans, request)
user_dict = user.dict()
user_dict["message"] = f"User '{str(user.username)}' has been created."
user_dict["url"] = web.url_for(controller="users", action="show", id=trans.security.encode_id(user.id))
return user_dict
def __get_value_mapper(self, trans):
value_mapper = {"id": trans.security.encode_id}
return value_mapper
@web.expose_api_anonymous_and_sessionless
def index(self, trans, deleted=False, **kwd):
"""
GET /api/users
Returns a list of dictionaries that contain information about each user.
"""
# Example URL: http://localhost:9009/api/users
user_dicts = []
deleted = util.asbool(deleted)
for user in index(trans.app, deleted):
user_dict = user.dict()
user_dict["url"] = web.url_for(controller="users", action="show", id=trans.security.encode_id(user.id))
user_dicts.append(user_dict)
return user_dicts
@web.expose_api_anonymous_and_sessionless
def show(self, trans, id, **kwd):
"""
GET /api/users/{encoded_user_id}
GET /api/users/current
Returns a dictionary of information about a user.
:param id: the encoded id of the User object.
"""
user = None
# user is requesting data about themselves
user = trans.user if id == "current" else suc.get_user(trans.app, id)
if user is None:
user_dict = dict(message=f"Unable to locate user record for id {str(id)}.", status="error")
return user_dict
user_dict = user.to_dict(view="element", value_mapper=self.__get_value_mapper(trans))
user_dict["url"] = web.url_for(controller="users", action="show", id=trans.security.encode_id(user.id))
return user_dict
+1 -4
View File
@@ -105,10 +105,7 @@ class UniverseApplication(ToolShedApp, SentryClientMixin, HaltableContainer):
self.hgweb_config_manager.hgweb_config_dir = self.config.hgweb_config_dir
self.hgweb_config_manager.hgweb_repo_prefix = self.config.hgweb_repo_prefix
# Initialize the repository registry.
if config.SHED_API_VERSION != "v2":
self.repository_registry = tool_shed.repository_registry.Registry(self)
else:
self.repository_registry = tool_shed.repository_registry.NullRepositoryRegistry(self)
self.repository_registry = tool_shed.repository_registry.NullRepositoryRegistry(self)
# Configure Sentry client if configured
self.configure_sentry_client()
# used for cachebusting -- refactor this into a *SINGLE* UniverseApplication base.
-125
View File
@@ -26,7 +26,6 @@ from galaxy.webapps.base.webapp import (
GalaxyWebTransaction,
)
from galaxy.webapps.util import wrap_if_allowed
from .config import SHED_API_VERSION
log = logging.getLogger(__name__)
@@ -125,130 +124,6 @@ def app_pair(global_conf, load_app_kwds=None, **kwargs):
webapp.add_route("/{action}", controller="repository", action="index")
# Enable 'hg clone' functionality on repos by letting hgwebapp handle the request
webapp.add_route("/repos/*path_info", controller="hg", action="handle_request", path_info="/")
# Add the web API. # A good resource for RESTful services - https://routes.readthedocs.io/en/latest/restful.html
if SHED_API_VERSION == "v1":
webapp.add_api_controllers("tool_shed.webapp.api", app)
webapp.mapper.connect(
"api_key_retrieval",
"/api/authenticate/baseauth/",
controller="authenticate",
action="get_tool_shed_api_key",
conditions=dict(method=["GET"]),
)
webapp.mapper.connect(
"group", "/api/groups/", controller="groups", action="index", conditions=dict(method=["GET"])
)
webapp.mapper.connect(
"group", "/api/groups/", controller="groups", action="create", conditions=dict(method=["POST"])
)
webapp.mapper.connect(
"group", "/api/groups/{encoded_id}", controller="groups", action="show", conditions=dict(method=["GET"])
)
webapp.mapper.resource(
"category",
"categories",
controller="categories",
name_prefix="category_",
path_prefix="/api",
parent_resources=dict(member_name="category", collection_name="categories"),
)
webapp.mapper.connect(
"repositories_in_category",
"/api/categories/{category_id}/repositories",
controller="categories",
action="get_repositories",
conditions=dict(method=["GET"]),
)
webapp.mapper.connect(
"show_updates_for_repository",
"/api/repositories/updates",
controller="repositories",
action="updates",
conditions=dict(method=["GET"]),
)
webapp.mapper.resource(
"repository",
"repositories",
controller="repositories",
collection={
"add_repository_registry_entry": "POST",
"get_repository_revision_install_info": "GET",
"get_ordered_installable_revisions": "GET",
"get_installable_revisions": "GET",
"remove_repository_registry_entry": "POST",
"reset_metadata_on_repositories": "POST",
"reset_metadata_on_repository": "POST",
},
name_prefix="repository_",
path_prefix="/api",
parent_resources=dict(member_name="repository", collection_name="repositories"),
)
webapp.mapper.resource(
"repository_revision",
"repository_revisions",
member={"repository_dependencies": "GET", "export": "POST"},
controller="repository_revisions",
name_prefix="repository_revision_",
path_prefix="/api",
parent_resources=dict(member_name="repository_revision", collection_name="repository_revisions"),
)
webapp.mapper.resource(
"user",
"users",
controller="users",
name_prefix="user_",
path_prefix="/api",
parent_resources=dict(member_name="user", collection_name="users"),
)
webapp.mapper.connect(
"update_repository",
"/api/repositories/{id}",
controller="repositories",
action="update",
conditions=dict(method=["PATCH", "PUT"]),
)
webapp.mapper.connect(
"repository_create_changeset_revision",
"/api/repositories/{id}/changeset_revision",
controller="repositories",
action="create_changeset_revision",
conditions=dict(method=["POST"]),
)
webapp.mapper.connect(
"repository_get_metadata",
"/api/repositories/{id}/metadata",
controller="repositories",
action="metadata",
conditions=dict(method=["GET"]),
)
webapp.mapper.connect(
"repository_show_tools",
"/api/repositories/{id}/{changeset}/show_tools",
controller="repositories",
action="show_tools",
conditions=dict(method=["GET"]),
)
webapp.mapper.connect(
"create_repository",
"/api/repositories",
controller="repositories",
action="create",
conditions=dict(method=["POST"]),
)
webapp.mapper.connect(
"tools",
"/api/tools/build_search_index",
controller="tools",
action="build_search_index",
conditions=dict(method=["PUT"]),
)
webapp.mapper.connect(
"tools", "/api/tools", controller="tools", action="index", conditions=dict(method=["GET"])
)
webapp.mapper.connect(
"version", "/api/version", controller="configuration", action="version", conditions=dict(method=["GET"])
)
webapp.finalize_config()
# Wrap the webapp in some useful middleware
if kwargs.get("middleware", True):
-1
View File
@@ -27,7 +27,6 @@ from galaxy.version import (
log = logging.getLogger(__name__)
TOOLSHED_APP_NAME = "tool_shed"
SHED_API_VERSION = os.environ.get("TOOL_SHED_API_VERSION", "v1")
class ToolShedAppConfiguration(BaseAppConfiguration, CommonConfigurationMixin):
+11 -31
View File
@@ -12,10 +12,7 @@ from fastapi import (
Depends,
FastAPI,
)
from fastapi.responses import (
HTMLResponse,
RedirectResponse,
)
from fastapi.responses import HTMLResponse
from fastapi.staticfiles import StaticFiles
from slowapi import (
_rate_limit_exceeded_handler,
@@ -61,7 +58,7 @@ api_tags_metadata = [
# Run vite with:
# pnpm dev
# Start tool shed with:
# TOOL_SHED_VITE_PORT=4040 TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
# TOOL_SHED_VITE_PORT=4040 ./run_tool_shed.sh
TOOL_SHED_VITE_PORT: Optional[str] = os.environ.get("TOOL_SHED_VITE_PORT", None)
TOOL_SHED_FRONTEND_TARGET: str = os.environ.get("TOOL_SHED_FRONTEND_TARGET") or "auto" # auto, src, or node
TOOL_SHED_USE_HMR: bool = TOOL_SHED_VITE_PORT is not None
@@ -111,12 +108,6 @@ def frontend_controller(app):
return app, index
def redirect_route(app, from_url: str, to_url: str):
@app.get(from_url)
def redirect():
return RedirectResponse(to_url)
def frontend_route(controller, path):
app, index = controller
app.get(path, response_class=HTMLResponse)(index)
@@ -148,11 +139,6 @@ FRONT_END_ROUTES = [
"/view/{username}/{repository_name}",
"/view/{username}/{repository_name}/{changeset_revision}",
]
LEGACY_ROUTES = {
"/user/create": "/register", # for twilltestcase
"/user/login": "/login", # for twilltestcase
}
limiter = Limiter(key_func=get_remote_address)
@@ -163,29 +149,23 @@ def initialize_fast_app(gx_webapp, tool_shed_app):
add_request_id_middleware(app)
app.state.limiter = limiter
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler) # type: ignore[arg-type]
from .config import SHED_API_VERSION
def mount_static(directory: Path):
name = directory.name
if directory.exists():
app.mount(f"/{name}", StaticFiles(directory=directory), name=name)
if SHED_API_VERSION == "v2":
controller = frontend_controller(app)
for route in FRONT_END_ROUTES:
frontend_route(controller, route)
controller = frontend_controller(app)
for route in FRONT_END_ROUTES:
frontend_route(controller, route)
for from_route, to_route in LEGACY_ROUTES.items():
redirect_route(app, from_route, to_route)
mount_static(FRONTEND / "static")
if TOOL_SHED_USE_HMR:
mount_static(FRONTEND / "node_modules")
else:
mount_static(find_frontend_target() / "assets")
mount_static(FRONTEND / "static")
if TOOL_SHED_USE_HMR:
mount_static(FRONTEND / "node_modules")
else:
mount_static(find_frontend_target() / "assets")
routes_package = "tool_shed.webapp.api" if SHED_API_VERSION == "v1" else "tool_shed.webapp.api2"
include_all_package_routers(app, routes_package)
include_all_package_routers(app, "tool_shed.webapp.api2")
wsgi_handler = WSGIMiddleware(gx_webapp)
tool_shed_app.haltables.append(("WSGI Middleware threadpool", wsgi_handler.executor.shutdown))
# https://github.com/abersheeran/a2wsgi/issues/44
+158 -115
View File
@@ -3,6 +3,7 @@
Vue 3 + TypeScript frontend for the Galaxy Tool Shed.
## Stack
- **Framework**: Vue 3 (Composition API + Options API mix)
- **Build**: Vite 4
- **UI**: Quasar 2
@@ -11,6 +12,7 @@ Vue 3 + TypeScript frontend for the Galaxy Tool Shed.
- **Router**: vue-router 4
## Structure
```
src/
├── api/ # API wrapper functions
@@ -43,29 +45,33 @@ pnpm lint
pnpm format
```
Backend must be running with `TOOL_SHED_API_VERSION=v2`:
Backend must be running:
```shell
# From galaxy root
TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
./run_tool_shed.sh
```
For rapid local dev with bootstrapped data:
```shell
TOOL_SHED_CONFIG_OVERRIDE_BOOTSTRAP_ADMIN_API_KEY=tsadminkey \
TOOL_SHED_CONFIG_CONFIG_HG_FOR_DEV=1 \
TOOL_SHED_VITE_PORT=4040 \
TOOL_SHED_API_VERSION=v2 \
./run_tool_shed.sh
```
## API Pattern
API calls use openapi-fetch typed client via `ToolShedApi()` in `src/schema/client.ts`:
```typescript
import { ToolShedApi } from "@/schema"
const { data } = await ToolShedApi().GET("/api/repositories", { params: { query: params } })
```
## Key Components
- `ShedToolbar.vue` - Main navigation toolbar
- `RepositoryPage.vue` - Single repository view
- `LandingPage.vue` - Homepage
@@ -73,7 +79,9 @@ const { data } = await ToolShedApi().GET("/api/repositories", { params: { query:
- `ComponentsShowcase.vue` - Developer page for widget demos
## Component Showcase
When creating reusable UI components, add examples to `src/components/pages/ComponentsShowcase.vue`. This page (accessible via `/showcase`) helps developers see components in isolation. Pattern:
```vue
<component-showcase title="MyComponent">
<component-showcase-example title="default">
@@ -86,6 +94,7 @@ When creating reusable UI components, add examples to `src/components/pages/Comp
```
When writing unit tests that cover special cases (edge cases, error states, long content, special characters, etc.), consider adding those same cases to the Component Showcase. This helps developers:
- Visually verify the component handles edge cases correctly
- See how the component looks in various states during development
- Document expected behavior for different scenarios
@@ -93,11 +102,13 @@ When writing unit tests that cover special cases (edge cases, error states, long
For example, if you test a component with long text or special characters, add showcase examples demonstrating those cases.
## Path Alias
`@/` maps to `src/` directory.
## Accessibility (WCAG 2.1 AA)
### Key Patterns
- **Skip link**: `App.vue` - hidden until focused, targets `#main-content`
- **Landmarks**: `role="banner"` on header, `role="main"` on page container
- **Live regions**: `ErrorBanner.vue` uses `role="alert"`, `LoadingDiv.vue` uses `role="status"`
@@ -105,22 +116,25 @@ For example, if you test a component with long text or special characters, add s
- **Focus indicators**: Global `:focus-visible` styles in `App.vue`
### Components with ARIA
| Component | ARIA Attrs |
|-----------|------------|
| `App.vue` | Skip link, landmarks, focus CSS |
| `ShedToolbar.vue` | `aria-label` on icon buttons, `aria-haspopup` on dropdowns |
| `ErrorBanner.vue` | `role="alert"`, `aria-live="assertive"` |
| `LoadingDiv.vue` | `role="status"`, `aria-live="polite"` |
| `RepositoryExplore.vue` | `aria-label` on FAB and icon buttons |
| `PaginatedRepositoriesGrid.vue` | `aria-label` on table |
| Component | ARIA Attrs |
| ------------------------------- | ---------------------------------------------------------- |
| `App.vue` | Skip link, landmarks, focus CSS |
| `ShedToolbar.vue` | `aria-label` on icon buttons, `aria-haspopup` on dropdowns |
| `ErrorBanner.vue` | `role="alert"`, `aria-live="assertive"` |
| `LoadingDiv.vue` | `role="status"`, `aria-live="polite"` |
| `RepositoryExplore.vue` | `aria-label` on FAB and icon buttons |
| `PaginatedRepositoriesGrid.vue` | `aria-label` on table |
### Quasar Notes
- `q-btn-dropdown` auto-manages `aria-expanded`
- `q-select` has built-in label association
- Use `aria-label` on icon-only `q-btn` components
- FABs (`q-fab`) need explicit `aria-label` on trigger
### Notification System
- `util.ts` `notify()` - uses Quasar Notify (toast messages)
- `ErrorBanner.vue` - inline persistent errors with dismiss
- `LoadingDiv.vue` - spinner with status message
@@ -128,30 +142,35 @@ For example, if you test a component with long text or special characters, add s
## Testing (Vitest + Vue Test Utils)
### Setup
Tests use Vitest with `@vue/test-utils` for component testing. Test files should be co-located with components (e.g., `MyComponent.vue``MyComponent.test.ts`) or in a `__tests__` directory.
### Best Practices for AI-Developed Tests
#### 1. Test Behavior, Not Implementation
Focus on what the component does from a user's perspective, not internal implementation details:
```typescript
// ✅ Good: Tests user-visible behavior
test('displays error message when API fails', async () => {
vi.mocked(ToolShedApi).mockReturnValue({ error: { status: 500 } })
const wrapper = mount(MyComponent)
await flushPromises()
expect(wrapper.text()).toContain('Error loading data')
test("displays error message when API fails", async () => {
vi.mocked(ToolShedApi).mockReturnValue({ error: { status: 500 } })
const wrapper = mount(MyComponent)
await flushPromises()
expect(wrapper.text()).toContain("Error loading data")
})
// ❌ Bad: Tests implementation details
test('calls fetchData method', () => {
const fetchDataSpy = vi.spyOn(wrapper.vm, 'fetchData')
// ...
test("calls fetchData method", () => {
const fetchDataSpy = vi.spyOn(wrapper.vm, "fetchData")
// ...
})
```
#### 2. Use Real Queries Over Test IDs
Prefer semantic queries (text, labels, roles) over test IDs:
```typescript
// ✅ Good: Uses accessible queries
const button = wrapper.find('button[aria-label="Submit"]')
@@ -162,204 +181,228 @@ const button = wrapper.find('[data-testid="submit-btn"]')
```
#### 3. Mock External Dependencies
Mock API calls, router, and Pinia stores at the module level:
```typescript
import { vi } from 'vitest'
import { ToolShedApi } from '@/schema'
vi.mock('@/schema', () => ({
ToolShedApi: vi.fn(),
Mock API calls, router, and Pinia stores at the module level:
```typescript
import { vi } from "vitest"
import { ToolShedApi } from "@/schema"
vi.mock("@/schema", () => ({
ToolShedApi: vi.fn(),
}))
test('loads repository data', async () => {
vi.mocked(ToolShedApi).mockReturnValue({
GET: vi.fn().mockResolvedValue({ data: { name: 'test-repo' } }),
})
// ...
test("loads repository data", async () => {
vi.mocked(ToolShedApi).mockReturnValue({
GET: vi.fn().mockResolvedValue({ data: { name: "test-repo" } }),
})
// ...
})
```
#### 4. Test Composition API Components Properly
For Composition API components, use `mount()` and interact with the component as a user would:
```typescript
import { mount } from '@vue/test-utils'
import { describe, it, expect, vi, beforeEach } from 'vitest'
import MyComponent from '@/components/MyComponent.vue'
describe('MyComponent', () => {
it('updates count when button clicked', async () => {
const wrapper = mount(MyComponent)
const button = wrapper.find('button')
await button.trigger('click')
expect(wrapper.text()).toContain('Count: 1')
})
For Composition API components, use `mount()` and interact with the component as a user would:
```typescript
import { mount } from "@vue/test-utils"
import { describe, it, expect, vi, beforeEach } from "vitest"
import MyComponent from "@/components/MyComponent.vue"
describe("MyComponent", () => {
it("updates count when button clicked", async () => {
const wrapper = mount(MyComponent)
const button = wrapper.find("button")
await button.trigger("click")
expect(wrapper.text()).toContain("Count: 1")
})
})
```
#### 5. Test Options API Components
For Options API components, avoid accessing `wrapper.vm` directly. Test through the template:
```typescript
// ✅ Good: Tests through template
test('shows message prop', () => {
const wrapper = mount(MyComponent, {
props: { message: 'Hello' },
})
expect(wrapper.text()).toContain('Hello')
test("shows message prop", () => {
const wrapper = mount(MyComponent, {
props: { message: "Hello" },
})
expect(wrapper.text()).toContain("Hello")
})
// ❌ Avoid: Direct vm access
expect(wrapper.vm.message).toBe('Hello')
expect(wrapper.vm.message).toBe("Hello")
```
#### 6. Mock Quasar Components When Needed
Quasar components can be mocked if they're not the focus of the test:
```typescript
import { mount, config } from '@vue/test-utils'
import { mount, config } from "@vue/test-utils"
config.global.stubs = {
'q-btn': { template: '<button><slot /></button>' },
'q-input': { template: '<input />' },
"q-btn": { template: "<button><slot /></button>" },
"q-input": { template: "<input />" },
}
```
#### 7. Test Pinia Stores in Isolation
Test stores separately from components:
```typescript
import { setActivePinia, createPinia } from 'pinia'
import { useAuthStore } from '@/stores/auth.store'
import { setActivePinia, createPinia } from "pinia"
import { useAuthStore } from "@/stores/auth.store"
describe('auth store', () => {
beforeEach(() => {
setActivePinia(createPinia())
})
describe("auth store", () => {
beforeEach(() => {
setActivePinia(createPinia())
})
it('sets user on login', () => {
const store = useAuthStore()
store.setUser({ id: 1, username: 'test' })
expect(store.user?.username).toBe('test')
})
it("sets user on login", () => {
const store = useAuthStore()
store.setUser({ id: 1, username: "test" })
expect(store.user?.username).toBe("test")
})
})
```
#### 8. Use `flushPromises()` for Async Operations
Wait for async operations to complete:
```typescript
import { flushPromises } from '@vue/test-utils'
test('loads data asynchronously', async () => {
const wrapper = mount(MyComponent)
await flushPromises()
expect(wrapper.text()).toContain('Loaded')
Wait for async operations to complete:
```typescript
import { flushPromises } from "@vue/test-utils"
test("loads data asynchronously", async () => {
const wrapper = mount(MyComponent)
await flushPromises()
expect(wrapper.text()).toContain("Loaded")
})
```
#### 9. Test Router Navigation
Mock `vue-router` and verify navigation calls:
```typescript
import { vi } from 'vitest'
import { vi } from "vitest"
const mockPush = vi.fn()
vi.mock('vue-router', () => ({
useRouter: () => ({ push: mockPush }),
vi.mock("vue-router", () => ({
useRouter: () => ({ push: mockPush }),
}))
test('navigates on click', async () => {
const wrapper = mount(MyComponent)
await wrapper.find('a').trigger('click')
expect(mockPush).toHaveBeenCalledWith('/expected-route')
test("navigates on click", async () => {
const wrapper = mount(MyComponent)
await wrapper.find("a").trigger("click")
expect(mockPush).toHaveBeenCalledWith("/expected-route")
})
```
#### 10. Keep Tests Focused and Independent
Each test should verify one behavior and be independent:
```typescript
// ✅ Good: Focused test
test('displays loading state', () => {
const wrapper = mount(MyComponent, {
props: { loading: true },
})
expect(wrapper.find('[role="status"]').exists()).toBe(true)
test("displays loading state", () => {
const wrapper = mount(MyComponent, {
props: { loading: true },
})
expect(wrapper.find('[role="status"]').exists()).toBe(true)
})
// ❌ Bad: Multiple concerns
test('component does everything', () => {
// Tests loading, error, success, navigation...
test("component does everything", () => {
// Tests loading, error, success, navigation...
})
```
#### 11. Use Descriptive Test Names
Test names should clearly describe what is being tested:
```typescript
// ✅ Good: Clear and descriptive
test('displays error banner when API returns 500', async () => {})
test('hides submit button when form is invalid', () => {})
test("displays error banner when API returns 500", async () => {})
test("hides submit button when form is invalid", () => {})
// ❌ Bad: Vague
test('works correctly', () => {})
test('component test', () => {})
test("works correctly", () => {})
test("component test", () => {})
```
#### 12. Clean Up After Tests
Reset mocks and clear state between tests:
```typescript
import { beforeEach, afterEach, vi } from 'vitest'
import { beforeEach, afterEach, vi } from "vitest"
beforeEach(() => {
vi.clearAllMocks()
vi.clearAllMocks()
})
afterEach(() => {
vi.restoreAllMocks()
vi.restoreAllMocks()
})
```
### Common Patterns
#### Testing Components with Props
```typescript
test('renders with required props', () => {
const wrapper = mount(MyComponent, {
props: {
title: 'Test Title',
count: 5,
},
})
expect(wrapper.text()).toContain('Test Title')
test("renders with required props", () => {
const wrapper = mount(MyComponent, {
props: {
title: "Test Title",
count: 5,
},
})
expect(wrapper.text()).toContain("Test Title")
})
```
#### Testing User Interactions
```typescript
test('calls handler on button click', async () => {
const handleClick = vi.fn()
const wrapper = mount(MyComponent, {
props: { onClick: handleClick },
})
await wrapper.find('button').trigger('click')
expect(handleClick).toHaveBeenCalledTimes(1)
test("calls handler on button click", async () => {
const handleClick = vi.fn()
const wrapper = mount(MyComponent, {
props: { onClick: handleClick },
})
await wrapper.find("button").trigger("click")
expect(handleClick).toHaveBeenCalledTimes(1)
})
```
#### Testing Conditional Rendering
```typescript
test('shows content when condition is true', () => {
const wrapper = mount(MyComponent, {
props: { show: true },
})
expect(wrapper.find('.content').exists()).toBe(true)
test("shows content when condition is true", () => {
const wrapper = mount(MyComponent, {
props: { show: true },
})
expect(wrapper.find(".content").exists()).toBe(true)
})
test('hides content when condition is false', () => {
const wrapper = mount(MyComponent, {
props: { show: false },
})
expect(wrapper.find('.content').exists()).toBe(false)
test("hides content when condition is false", () => {
const wrapper = mount(MyComponent, {
props: { show: false },
})
expect(wrapper.find(".content").exists()).toBe(false)
})
```
### AI-Specific Guidelines
When generating tests with AI:
1. **Avoid over-testing**: Don't test every method or computed property. Focus on user-facing behavior.
2. **Don't test framework code**: Vue, Quasar, and Pinia are already tested. Test your application logic.
3. **Test edge cases**: Include tests for error states, empty data, and boundary conditions.
+1 -1
View File
@@ -16,7 +16,7 @@ lint:
# how to get a test server running and populated with some initial data
# for the new tool shed frontend.
run_test_backend:
cd $(GALAXY_ROOT); TOOL_SHED_CONFIG_OVERRIDE_BOOTSTRAP_ADMIN_API_KEY=tsadminkey TOOL_SHED_VITE_PORT=4040 TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
cd $(GALAXY_ROOT); TOOL_SHED_CONFIG_OVERRIDE_BOOTSTRAP_ADMIN_API_KEY=tsadminkey TOOL_SHED_VITE_PORT=4040 ./run_tool_shed.sh
bootstrap_test_backend:
cd $(GALAXY_ROOT); . .venv/bin/activate; python scripts/bootstrap_test_shed.py
+3 -3
View File
@@ -4,7 +4,7 @@ You will need to start the Tool Shed backend from the galaxy root directory.
This is required if you want to develop against a local tool shed.
```shell
TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
./run_tool_shed.sh
```
Start the HMR dev server.
@@ -27,5 +27,5 @@ To run a local toolshed patched for rapid bootstrapping and in local Vite dev se
the following command should work.
```shell
TOOL_SHED_CONFIG_OVERRIDE_BOOTSTRAP_ADMIN_API_KEY=tsadminkey TOOL_SHED_CONFIG_CONFIG_HG_FOR_DEV=1 TOOL_SHED_VITE_PORT=4040 TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
```
TOOL_SHED_CONFIG_OVERRIDE_BOOTSTRAP_ADMIN_API_KEY=tsadminkey TOOL_SHED_CONFIG_CONFIG_HG_FOR_DEV=1 TOOL_SHED_VITE_PORT=4040 ./run_tool_shed.sh
```
@@ -4,7 +4,6 @@
* To regenerate these fixtures, run from packages/tool_shed:
*
* TOOL_SHED_FIXTURE_OUTPUT_DIR=lib/tool_shed/webapp/frontend/src/components/MetadataInspector/__fixtures__ \
* TOOL_SHED_API_VERSION=v2 \
* uv run pytest tool_shed/test/functional/test_shed_repositories.py::TestShedRepositoriesApi::test_generate_frontend_fixtures -v
*/
@@ -120,7 +119,7 @@ export function getFirstRevision(metadata: RepositoryMetadata): RepositoryRevisi
/** Get all tools from all revisions in metadata */
export function getAllTools(
metadata: RepositoryMetadata
metadata: RepositoryMetadata,
): Array<{ tool: components["schemas"]["RepositoryTool"]; revisionKey: string }> {
const tools: Array<{ tool: components["schemas"]["RepositoryTool"]; revisionKey: string }> = []
for (const [key, revision] of Object.entries(metadata)) {
@@ -3,15 +3,13 @@ Regenerate API test fixtures for MetadataInspector frontend components.
# Command
Run from `packages/tool_shed`:
```bash
TOOL_SHED_FIXTURE_OUTPUT_DIR=tool_shed/webapp/frontend/src/components/MetadataInspector/__fixtures__ \
TOOL_SHED_API_VERSION=v2 \
uv run pytest tool_shed/test/functional/test_shed_repositories.py::TestShedRepositoriesApi::test_generate_frontend_fixtures -v &&
cd tool_shed/webapp/frontend && npm run format
```
# Generated files
- `repository_metadata_column_maker.json` - Multi-revision repo with tools (RepositoryMetadata)
+13 -11
View File
@@ -3,6 +3,7 @@
Backend services for the Galaxy Tool Shed.
## Structure
```
tool_shed/
├── managers/ # Business logic (repositories, users, categories)
@@ -25,6 +26,7 @@ subproject (the directory where this file is located).
Always use `uv run python` and `uv run pytest` to run commands in this directory to ensure the correct environment is used.
Example:
```bash
uv run python -c "from selenium.webdriver.common.by import By; print(By.CLASS_NAME)"
uv run pytest tests/seleniumtests/test_has_driver.py -v
@@ -41,21 +43,21 @@ Run from this directory (`packages/tool_shed`):
uv run pytest tests/tool_shed/
# Functional tests (requires running shed)
TOOL_SHED_API_VERSION=v2 uv run pytest tool_shed/test/functional/test_shed_repositories.py -v
uv run pytest tool_shed/test/functional/test_shed_repositories.py -v
```
### Test Categories
| Type | Location | Server Required | Description |
|------|----------|-----------------|-------------|
| Unit | `tests/tool_shed/` | No | In-memory SQLite, mock app |
| Functional | `tool_shed/test/functional/` | Yes | API + Playwright browser tests |
| Type | Location | Server Required | Description |
| ---------- | ---------------------------- | --------------- | ------------------------------ |
| Unit | `tests/tool_shed/` | No | In-memory SQLite, mock app |
| Functional | `tool_shed/test/functional/` | Yes | API + Playwright browser tests |
### Quick Start: Functional Tests
```shell
# Terminal 1: Start shed with v2 API
TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
# Terminal 1: Start shed
./run_tool_shed.sh
# Terminal 2: Run tests
uv run pytest tool_shed/test/functional/test_frontend_login.py -v
@@ -71,13 +73,13 @@ uv run pytest tool_shed/test/functional/test_frontend_login.py -v
### Component Showcase & Fixtures
The component showcase (`/_component_showcase`) displays UI components with real API data from generated fixtures. These fixtures are shared between:
- **ComponentsShowcase.vue** - Live demos at `/_component_showcase`
- **Vitest unit tests** - Frontend component tests
```shell
# Regenerate fixtures from real API responses
TOOL_SHED_FIXTURE_OUTPUT_DIR=tool_shed/webapp/frontend/src/components/MetadataInspector/__fixtures__ \
TOOL_SHED_API_VERSION=v2 \
uv run pytest tool_shed/test/functional/test_shed_repositories.py::TestShedRepositoriesApi::test_generate_frontend_fixtures -v
# Format generated files
@@ -85,7 +87,6 @@ cd tool_shed/webapp/frontend && npm run format
# Capture component screenshots
TOOL_SHED_TEST_SCREENSHOTS=/tmp/screenshots \
TOOL_SHED_API_VERSION=v2 \
uv run pytest tool_shed/test/functional/test_component_showcase.py -v
```
@@ -95,19 +96,19 @@ Regenerate fixtures after: schema changes, API response changes, or metadata bug
```shell
# From galaxy root
TOOL_SHED_API_VERSION=v2 ./run_tool_shed.sh
./run_tool_shed.sh
# With dev conveniences (admin key, hg config, frontend dev server)
TOOL_SHED_CONFIG_OVERRIDE_BOOTSTRAP_ADMIN_API_KEY=tsadminkey \
TOOL_SHED_CONFIG_CONFIG_HG_FOR_DEV=1 \
TOOL_SHED_VITE_PORT=4040 \
TOOL_SHED_API_VERSION=v2 \
./run_tool_shed.sh
```
## API Schema
OpenAPI schema generated from FastAPI endpoints. To regenerate TypeScript types:
```shell
# From galaxy root
make update-client-api-schema
@@ -125,6 +126,7 @@ Frontend types live in `tool_shed/webapp/frontend/src/schema/schema.ts`.
## Frontend
See [tool_shed/webapp/frontend/CLAUDE.md](tool_shed/webapp/frontend/CLAUDE.md) for:
- Vue 3 / Quasar architecture
- Component patterns
- API client usage