diff --git a/doc/source/dev/debugging_tests.md b/doc/source/dev/debugging_tests.md index d78f1e0de7f..d464672b9b6 100644 --- a/doc/source/dev/debugging_tests.md +++ b/doc/source/dev/debugging_tests.md @@ -1,15 +1,16 @@ # Debugging Galaxy Tests -## Debugging Galaxy unit & integration tests in VS code +## Debugging Galaxy unit & integration tests in VS code The following instructions assume that you have cloned your Galaxy fork into ~/galaxy directory. 1. Start VS Code 2. Install the Python extension for VS Code: select Extensions on Activity Bar, on the far left-hand side -2. Create a VS Code workspace by selecting 'File -> Add Folder to Workspace...' from menu and adding the `~/galaxy` directory -3. Optionally, save the workspace by selecting 'File -> Save Workspace As...' and save as 'galaxy.code-workspace' in the `~/galaxy` directory -2. Add the following snippet to `~/galaxy/.vscode/settings.json` (create the file if it does not already exist) - ``` +3. Create a VS Code workspace by selecting 'File -> Add Folder to Workspace...' from menu and adding the `~/galaxy` directory +4. Optionally, save the workspace by selecting 'File -> Save Workspace As...' and save as 'galaxy.code-workspace' in the `~/galaxy` directory +5. Add the following snippet to `~/galaxy/.vscode/settings.json` (create the file if it does not already exist) + + ```json { "python.testing.unittestEnabled": false, "python.testing.nosetestsEnabled": false, @@ -25,10 +26,50 @@ The following instructions assume that you have cloned your Galaxy fork into ~/g "pythonTestExplorer.testFramework": "pytest", } ``` -3. Re-start VS Code -3. Choose the .venv Python as your Python Interpreter. First `Ctrl+Shift+P` then `>Python: Select Interpreter`. + +6. Re-start VS Code +7. Choose the .venv Python as your Python Interpreter. First `Ctrl+Shift+P` then `>Python: Select Interpreter`. +![VS Code Python Interpreter](select_python_interpreter.png) -4. Select Test on Activity Bar, on the far left hand side. You should see unit and integration tests under 'Python' (as shown in the image below). It may take a few seconds for the tests to load up. If they do not, click the 'Discover Tests' icon and wait for the tests to load. -![VS Code Tests](tests.png) -5. Expand integration/unit tests, select a test to display its source code in editor, add a breakpoint, and start the debugger by clicking on the debug icon (next to test name) -6. Enjoy your debugging session :-) +8. Select Test on Activity Bar, on the far left hand side. You should see unit and integration tests under 'Python' (as shown in the image below). It may take a few seconds for the tests to load up. If they do not, click the 'Discover Tests' icon and wait for the tests to load. +![VS Code Tests](tests.png) +9. Expand integration/unit tests, select a test to display its source code in editor, add a breakpoint, and start the debugger by clicking on the debug icon (next to test name) + +## Debugging tests within github actions + +Sometimes it is necessary to debug tests that work locally. There are a few different strategies one could employ. +The first step is often to exclude the workflow runs that are passing, so that you don't waste time waiting for your test. +To do this you can delete all the workflow files in .github/workflows/*.yaml except for the one that sets up the failing test. + +If the yaml file has a strategy section that sets up multiple tests keep just the one that contains the failing test. +If the test never finishes through in a `-s` in the run_tests.sh arguments, so you see live output. +This diff here adds `-s` and disables strategies that are already passing: + +```diff +diff --git a/.github/workflows/integration.yaml b/.github/workflows/integration.yaml +index 6647588dfb..c8d82957a1 100644 +--- a/.github/workflows/integration.yaml ++++ b/.github/workflows/integration.yaml +@@ -14,7 +14,7 @@ jobs: + fail-fast: false + matrix: + python-version: ['3.7'] +- subset: ['upload_datatype', 'extended_metadata', 'kubernetes', 'not (upload_datatype or extended_metadata or kubernetes)'] ++ subset: ['not (upload_datatype or extended_metadata or kubernetes)'] + services: + postgres: + image: postgres:13 +@@ -73,7 +73,7 @@ jobs: + if: matrix.subset == 'upload_datatype' + - name: Run tests + if: matrix.subset != 'kubernetes' +- run: './run_tests.sh -integration test/integration -- -k "${{ matrix.subset }}"' ++ run: './run_tests.sh -integration test/integration -- -s -k "${{ matrix.subset }}"' + working-directory: 'galaxy root' + - name: Run tests + if: matrix.subset == 'kubernetes' +``` + +If that's not enough and you need to snoop around the process, halting it and inspecting what is going on it's helpful to log into the github runner +and run the test manually. You can do this by injecting a step that inserts an ssh session via [tmate](https://github.com/mxschmitt/action-tmate#use-registered-public-ssh-keys), then use your favorite tools like py-spy or similar to look at the test while it's failing or getting stuck. + +Note that you probably don't want to do this on a PR branch, where all your tests are still going to run. Instead run this only against your fork.