DeepAudit/backend/docs/testing.md

Testing Guide

This document describes the testing strategy and how to run tests for XCodeReviewer.

Test Structure

tests/
├── test_api_endpoints.py       # API endpoint tests
├── test_e2e_workflows.py        # End-to-end workflow tests
├── test_llm_service.py          # LLM service tests
├── test_repository_scanner.py   # Repository scanning tests
├── test_report_generation.py    # Report generation tests
└── conftest.py                  # Shared fixtures

Test Categories

Unit Tests

Test individual components in isolation.

pytest tests/ -m unit

Integration Tests

Test interactions between components.

pytest tests/ -m integration

End-to-End Tests

Test complete user workflows.

pytest tests/ -m e2e

Running Tests

Run All Tests

# Using pytest directly
pytest

# Using test script
./scripts/run_tests.sh

Run Specific Test File

pytest tests/test_api_endpoints.py

Run Specific Test Class

pytest tests/test_api_endpoints.py::TestAuthEndpoints

Run Specific Test

pytest tests/test_api_endpoints.py::TestAuthEndpoints::test_register_user

Run with Coverage

pytest --cov=. --cov-report=html

Run in Parallel

pytest -n auto

Coverage Reports

Generate Coverage Report

pytest --cov=. --cov-report=html --cov-report=term

View HTML Report

open htmlcov/index.html  # macOS
xdg-open htmlcov/index.html  # Linux
start htmlcov/index.html  # Windows

Coverage Threshold

The project requires minimum 80% code coverage. Tests will fail if coverage is below this threshold.

Test Fixtures

Common fixtures are defined in conftest.py:

• db_session: Database session for tests
• test_client: HTTP client for API tests
• authenticated_user: Pre-authenticated test user
• auth_token: Authentication token

Writing Tests

Unit Test Example

def test_password_validation():
    """Test password validation logic"""
    from core.security import validate_password
    
    # Valid password
    assert validate_password("SecurePass123!")
    
    # Invalid password
    assert not validate_password("weak")

Integration Test Example

@pytest.mark.asyncio
async def test_create_project(test_client, auth_token):
    """Test project creation via API"""
    response = await test_client.post(
        "/api/v1/projects",
        headers={"Authorization": f"Bearer {auth_token}"},
        json={
            "name": "Test Project",
            "source_type": "github"
        }
    )
    assert response.status_code == 201

E2E Test Example

@pytest.mark.e2e
@pytest.mark.asyncio
async def test_complete_workflow(test_client, auth_token):
    """Test complete analysis workflow"""
    # Create project
    project = await create_project(test_client, auth_token)
    
    # Create task
    task = await create_task(test_client, auth_token, project["id"])
    
    # Generate report
    report = await generate_report(test_client, auth_token, task["id"])
    
    assert report["status"] == "completed"

Mocking

Mock External Services

from unittest.mock import patch, MagicMock

@patch('services.llm.openai_adapter.OpenAI')
def test_llm_call(mock_openai):
    """Test LLM service with mocked API"""
    mock_openai.return_value.chat.completions.create.return_value = MagicMock(
        choices=[MagicMock(message=MagicMock(content="Test response"))]
    )
    
    # Test code here

Mock Database

@pytest.fixture
async def mock_db():
    """Mock database session"""
    from unittest.mock import AsyncMock
    
    session = AsyncMock()
    session.commit = AsyncMock()
    session.rollback = AsyncMock()
    
    return session

Continuous Integration

Tests run automatically on:

• Pull requests
• Commits to main branch
• Scheduled daily runs

GitHub Actions Workflow

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.11'
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run tests
        run: ./scripts/run_tests.sh
      - name: Upload coverage
        uses: codecov/codecov-action@v2

Performance Testing

Load Testing with Locust

from locust import HttpUser, task, between

class APIUser(HttpUser):
    wait_time = between(1, 3)
    
    @task
    def list_projects(self):
        self.client.get("/api/v1/projects")

Run load test:

locust -f tests/load_test.py --host=http://localhost:8000

Security Testing

SQL Injection Tests

def test_sql_injection_protection(test_client, auth_token):
    """Test SQL injection protection"""
    response = test_client.get(
        "/api/v1/projects",
        params={"search": "'; DROP TABLE projects; --"}
    )
    assert response.status_code != 500

Authentication Tests

def test_unauthorized_access(test_client):
    """Test unauthorized access is blocked"""
    response = test_client.get("/api/v1/projects")
    assert response.status_code == 401

Debugging Tests

Run with Debug Output

pytest -vv --tb=long

Run with PDB

pytest --pdb

Run Specific Failed Tests

pytest --lf  # Last failed
pytest --ff  # Failed first

Best Practices

1. Test Naming: Use descriptive names that explain what is being tested
2. Arrange-Act-Assert: Structure tests clearly
3. One Assertion: Focus each test on one behavior
4. Independent Tests: Tests should not depend on each other
5. Fast Tests: Keep unit tests fast (< 1 second)
6. Mock External Services: Don't make real API calls in tests
7. Clean Up: Use fixtures to clean up test data
8. Coverage: Aim for 80%+ coverage
9. Documentation: Add docstrings to test functions
10. Markers: Use pytest markers to categorize tests

Troubleshooting

Tests Hanging

# Add timeout
pytest --timeout=30

Database Conflicts

# Use separate test database
export TEST_DATABASE_URL=sqlite+aiosqlite:///./test.db

Import Errors

# Add project root to PYTHONPATH
export PYTHONPATH="${PYTHONPATH}:$(pwd)"

Async Tests Failing

# Ensure pytest-asyncio is installed
pip install pytest-asyncio

Resources

• Pytest Documentation
• Coverage.py Documentation
• Testing Best Practices