From aa6d1f7b2d7488108b46db0188bbc4a2d0203a6f Mon Sep 17 00:00:00 2001 From: John Chilton Date: Sun, 22 Feb 2026 00:07:16 -0500 Subject: [PATCH] Add admin docs for AI agent configuration Co-Authored-By: Claude Opus 4.6 --- doc/source/admin/ai_agents.md | 319 ++++++++++++++++++++++++++++++++++ doc/source/admin/index.rst | 1 + 2 files changed, 320 insertions(+) create mode 100644 doc/source/admin/ai_agents.md diff --git a/doc/source/admin/ai_agents.md b/doc/source/admin/ai_agents.md new file mode 100644 index 00000000000..83fb8aa42de --- /dev/null +++ b/doc/source/admin/ai_agents.md @@ -0,0 +1,319 @@ +# AI Agent Configuration + +Galaxy includes a multi-agent AI system built on [pydantic-ai](https://github.com/pydantic/pydantic-ai). The agents provide specialized assistants for answering platform questions, diagnosing job errors, creating custom tools, recommending tools, and more. The entire system is gated behind AI API key configuration -- if no AI credentials are provided, the agent features are completely invisible to users. + +## Overview + +When AI is configured, Galaxy exposes two main user-facing features: + +- **ChatGXY**: A sidebar chat interface (visible in the Activity Bar) that routes user questions to specialized agents. +- **GalaxyWizard**: An error-analysis widget that appears on failed job pages to help users understand what went wrong. + +All AI configuration lives in `galaxy.yml` under the `galaxy:` section. There is no admin UI for toggling agents -- everything is controlled through configuration files. + +## Minimum Required Configuration + +The single most important setting is `ai_api_key`. Setting this value (or `inference_services` or `ai_api_base_url`) is what activates the entire agent system. Without at least one of these, no agent code loads, the ChatGXY sidebar entry is hidden, and the GalaxyWizard error-analysis widget does not appear. + +```yaml +galaxy: + # Required: API key for an AI provider (OpenAI by default) + ai_api_key: "sk-..." +``` + +That is all you need to get started with the default configuration (OpenAI, model `gpt-4o-mini`). + +## Configuration Settings + +All AI-related settings go under the `galaxy:` section in `galaxy.yml`: + +| Setting | Default | Description | +| -------------------- | -------- | -------------------------------------------------------------------------------------------------- | +| `ai_api_key` | (none) | API key for an AI provider. Required unless using `inference_services` or `ai_api_base_url`. | +| `ai_api_base_url` | (none) | Override the default OpenAI base URL for OpenAI-compatible backends (vLLM, Ollama, LiteLLM, etc.). | +| `ai_model` | `gpt-4o-mini` | Global model fallback for all agents. | +| `inference_services` | (none) | Per-agent configuration with fine-grained control over model, temperature, tokens, and API keys. | + +```{note} +The legacy config keys `openai_api_key` and `openai_model` still work as deprecated aliases for `ai_api_key` and `ai_model` respectively. They will be removed in a future release. Prefer the `ai_*` keys in new deployments. +``` + +## Supported AI Backends + +Galaxy supports multiple LLM providers through pydantic-ai's provider system. The model name prefix determines which provider is used: + +### OpenAI (default) + +Use bare model names like `gpt-4o` or prefixed as `openai:gpt-4o`. This is the default provider and requires only an API key. + +```yaml +galaxy: + ai_api_key: "sk-..." + ai_model: "gpt-4o" +``` + +### Anthropic / Claude + +Use the `anthropic:` prefix, e.g. `anthropic:claude-sonnet-4-5`. + +```yaml +galaxy: + inference_services: + default: + model: "anthropic:claude-sonnet-4-5" + api_key: "sk-ant-..." +``` + +```{warning} +Anthropic support requires the optional `pydantic-ai[anthropic]` Python package to be installed in Galaxy's virtual environment. If it is not installed, agents configured with an `anthropic:` model prefix will fail at runtime. +``` + +### Google / Gemini + +Use the `google:` prefix, e.g. `google:gemini-2.5-pro`. + +```yaml +galaxy: + inference_services: + default: + model: "google:gemini-2.5-pro" + api_key: "AIza..." +``` + +```{warning} +Google support requires the optional `pydantic-ai[google]` Python package to be installed in Galaxy's virtual environment. +``` + +### OpenAI-Compatible (vLLM, Ollama, LiteLLM, TACC) + +Use any model name combined with `api_base_url` to point at a self-hosted or institutional inference endpoint. The request is routed through the OpenAI-compatible API path. + +```yaml +galaxy: + ai_api_key: "not-needed-but-required-by-some-clients" + ai_api_base_url: "http://localhost:11434/v1/" + ai_model: "llama3.1" +``` + +```{note} +Not all models support structured output (JSON schema mode). The `custom_tool` agent requires structured output and will return a graceful error if the configured model lacks that capability. +``` + +## Per-Agent Configuration via `inference_services` + +The `inference_services` dictionary allows fine-grained control over individual agents. Each key is either `default` (applied to all agents as a fallback) or a specific agent type name. + +Supported keys within each agent block: + +| Key | Description | +| -------------- | --------------------------------------------------------------------------------------- | +| `model` | Model name with optional provider prefix (e.g. `gpt-4o`, `anthropic:claude-sonnet-4-5`) | +| `api_key` | API key override for this agent or default | +| `api_base_url` | Base URL override for this agent or default | +| `temperature` | Sampling temperature (0.0 - 1.0) | +| `max_tokens` | Maximum tokens in the response | + +### Example: Per-Agent Overrides + +Use a cheap model globally but a more capable model for agents that need it: + +```yaml +galaxy: + ai_api_key: "sk-..." + inference_services: + default: + model: "gpt-4o-mini" + temperature: 0.7 + custom_tool: + model: "openai:gpt-4o" + temperature: 0.4 + max_tokens: 2000 + error_analysis: + model: "openai:gpt-4o" + temperature: 0.2 + max_tokens: 2000 +``` + +### Example: Mixed Providers + +Use different providers for different agents: + +```yaml +galaxy: + inference_services: + default: + model: "anthropic:claude-sonnet-4-5" + api_key: "sk-ant-..." + temperature: 0.3 + custom_tool: + model: "openai:gpt-4o" + api_key: "sk-..." + temperature: 0.4 +``` + +### Example: Self-Hosted with Ollama + +```yaml +galaxy: + ai_api_key: "ollama" + ai_api_base_url: "http://localhost:11434/v1/" + ai_model: "llama3.1" + inference_services: + default: + model: "llama3.1" + api_base_url: "http://localhost:11434/v1/" + temperature: 0.7 +``` + +### Example: Institutional Endpoint (TACC, LiteLLM proxy) + +```yaml +galaxy: + inference_services: + default: + model: "llama-4-scout" + api_base_url: "http://litellm-proxy.internal:4000/v1/" + api_key: "internal-key" + temperature: 0.7 +``` + +## Configuration Cascade + +At runtime, each agent resolves its configuration through a four-level cascade. The precedence order is: + +1. **Agent-specific config** -- `inference_services..` (e.g. `inference_services.custom_tool.model`) +2. **Default inference config** -- `inference_services.default.` +3. **Global config** -- `ai_model`, `ai_api_key`, `ai_api_base_url` +4. **Hardcoded defaults** -- model `gpt-4o-mini`, no base URL override + +This means you can set a cheap model as the global default and override only the agents that need a more capable (and more expensive) model. + +## Enabling and Disabling Agents + +Galaxy registers the following agent types: + +| Agent Type | Default State | Purpose | +| --------------------- | ------------- | ---------------------------------------------------------------- | +| `router` | Enabled | Routes user queries to the appropriate specialized agent | +| `error_analysis` | Enabled | Diagnoses failed jobs and suggests fixes | +| `custom_tool` | Enabled | Generates custom Galaxy tools from natural language descriptions | +| `orchestrator` | Enabled | Coordinates multi-step workflow tasks | +| `tool_recommendation` | Enabled | Recommends tools from the toolbox for a given task | +| `dataset_analyzer` | Disabled | Analyzes dataset contents (beta) | + +## Prerequisites and Dependencies + +The core dependency is `pydantic-ai`, declared in Galaxy's `pyproject.toml`. It is installed automatically with Galaxy. For non-OpenAI providers, install the corresponding extras: + +```bash +# For Anthropic/Claude support +pip install 'pydantic-ai[anthropic]' + +# For Google/Gemini support +pip install 'pydantic-ai[google]' +``` + +The database migration for chat storage (`chat_exchange` and `chat_exchange_message` tables) runs as part of normal Galaxy schema migrations. No separate migration step is needed. + +## Verifying the Configuration + +### Check the Agent API Endpoint + +After configuring AI and restarting Galaxy, query the agents endpoint to verify that agents are available: + +```bash +curl -s -H "x-api-key: YOUR_GALAXY_API_KEY" \ + http://localhost:8080/api/ai/agents | python -m json.tool +``` + +You should see a list of enabled agents with their types. If AI is not configured, this endpoint returns an error indicating that the agent system is not available. + +### Check if ChatGXY Appears in the Sidebar + +Log in to the Galaxy web interface. If AI is properly configured, a **ChatGXY** entry should appear in the Activity Bar on the left side of the screen. If it does not appear: + +1. Verify that at least one of `ai_api_key`, `ai_api_base_url`, or `inference_services` is set in `galaxy.yml`. +2. Check that Galaxy was restarted after the configuration change. +3. Check the Galaxy server log for import errors related to `pydantic-ai`. + +### Check the Configuration API + +The frontend determines whether to show AI features by checking the `llm_api_configured` flag from the configuration API: + +```bash +curl -s http://localhost:8080/api/configuration | python -m json.tool | grep llm_api_configured +``` + +This should return `"llm_api_configured": true` when AI is active. + +## Troubleshooting + +### ChatGXY does not appear in the sidebar + +- Confirm that `ai_api_key`, `ai_api_base_url`, or `inference_services` is set in `galaxy.yml` under the `galaxy:` section. +- Restart Galaxy after any configuration change. +- Check that `pydantic-ai` is installed: `pip show pydantic-ai`. +- Check Galaxy's log for `Agent system is not available` errors, which indicate a missing or broken `pydantic-ai` installation. + +### "Agent system is not available" error from the API + +This means the `pydantic-ai` library failed to import. Verify it is installed in Galaxy's Python environment and that the version meets the minimum requirement. + +### Anthropic or Google models fail with ImportError + +Install the required provider extras: + +```bash +pip install 'pydantic-ai[anthropic]' # for anthropic: prefixed models +pip install 'pydantic-ai[google]' # for google: prefixed models +``` + +### Custom tool agent fails with "structured output not supported" + +The `custom_tool` agent requires a model that supports structured JSON output (JSON schema mode). Some models (e.g. certain DeepSeek variants) do not support this. Switch the `custom_tool` agent to a model that does, such as `gpt-4o` or `anthropic:claude-sonnet-4-5`. + +### Requests succeed but responses are empty or low quality + +- Check the `temperature` setting. Very low values (< 0.1) can produce repetitive output; very high values (> 0.9) can produce incoherent output. +- Check the `max_tokens` setting. If it is too low, responses may be truncated. +- Verify the model name is valid for your provider. An incorrect model name may silently fall back or return errors. + +### Self-hosted endpoint returns connection errors + +- Verify the `ai_api_base_url` or `inference_services.default.api_base_url` is reachable from the Galaxy server. +- The URL should include the path prefix expected by the API (typically `/v1/`). +- Check firewall rules if the inference service is on a different host. + +## Complete Configuration Example + +A production deployment using a LiteLLM proxy with per-agent model overrides: + +```yaml +galaxy: + # Global fallback + ai_api_key: "proxy-key-..." + ai_api_base_url: "http://litellm.internal:4000/v1/" + ai_model: "llama-4-scout" + + # Per-agent overrides + inference_services: + default: + model: "llama-4-scout" + api_base_url: "http://litellm.internal:4000/v1/" + temperature: 0.5 + custom_tool: + model: "openai:gpt-4o" + api_key: "sk-..." + temperature: 0.4 + max_tokens: 2000 + error_analysis: + model: "anthropic:claude-sonnet-4-5" + api_key: "sk-ant-..." + temperature: 0.2 + max_tokens: 2000 + + # Optionally disable beta agents + agents: + dataset_analyzer: + enabled: false +``` diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index a9dac17b0c2..30dd639d800 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -18,6 +18,7 @@ Galaxy Deployment & Administration jobs job_metrics authentication + ai_agents enable_headers_in_fetch_requests tool_panel data_tables