diff --git a/packages/kilo-docs/lib/nav/code-with-ai.ts b/packages/kilo-docs/lib/nav/code-with-ai.ts index e5d621b250f..d63d04942f4 100644 --- a/packages/kilo-docs/lib/nav/code-with-ai.ts +++ b/packages/kilo-docs/lib/nav/code-with-ai.ts @@ -36,6 +36,11 @@ export const CodeWithAiNav: NavSection[] = [ href: "/code-with-ai/agents/auto-model", children: "Auto Model", }, + { + href: "/code-with-ai/agents/custom-models", + children: "Custom Models", + platform: "new", + }, { href: "/code-with-ai/agents/free-and-budget-models", children: "Free & Budget Models", diff --git a/packages/kilo-docs/pages/ai-providers/lmstudio.md b/packages/kilo-docs/pages/ai-providers/lmstudio.md index e22058308b2..3ca78d716f4 100644 --- a/packages/kilo-docs/pages/ai-providers/lmstudio.md +++ b/packages/kilo-docs/pages/ai-providers/lmstudio.md @@ -71,6 +71,44 @@ Then set your default model: {% /tab %} {% /tabs %} +## Using Custom or Unlisted Models + +If the model you loaded in LM Studio doesn't appear in the Kilo model picker, you can register it as a custom model in your config file: + +```jsonc +{ + "model": "lmstudio/my-custom-model", + "provider": { + "lmstudio": { + "models": { + "my-custom-model": { + "name": "My Custom Model", + }, + }, + }, + }, +} +``` + +The model key (`my-custom-model`) must match the model identifier that LM Studio serves. If the display name you want differs from the API identifier, use the `id` field to set the API-facing name separately: + +```jsonc +{ + "provider": { + "lmstudio": { + "models": { + "my-llama": { + "id": "meta-llama-3.1-8b-instruct", + "name": "Llama 3.1 8B (Local)", + }, + }, + }, + }, +} +``` + +See [Custom Models](/docs/code-with-ai/agents/custom-models) for the full list of configuration fields and more examples. + ## Tips and Notes - **Resource Requirements:** Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose. diff --git a/packages/kilo-docs/pages/ai-providers/ollama.md b/packages/kilo-docs/pages/ai-providers/ollama.md index 33f8199e24e..3f2ba1887c2 100644 --- a/packages/kilo-docs/pages/ai-providers/ollama.md +++ b/packages/kilo-docs/pages/ai-providers/ollama.md @@ -116,6 +116,32 @@ Then set your default model: {% /tab %} {% /tabs %} +## Using Custom or Unlisted Models + +If your Ollama model doesn't appear in the Kilo model picker, register it as a custom model in your config file: + +```jsonc +{ + "model": "ollama/my-finetune:latest", + "provider": { + "ollama": { + "models": { + "my-finetune:latest": { + "name": "My Fine-tuned Model", + "tool_call": true, + "limit": { + "context": 32768, + "output": 8192, + }, + }, + }, + }, + }, +} +``` + +See [Custom Models](/docs/code-with-ai/agents/custom-models) for the full list of configuration fields and more examples. + ## Further Reading Refer to the [Ollama documentation](https://ollama.com/docs) for more information on installing, configuring and using Ollama. diff --git a/packages/kilo-docs/pages/ai-providers/openai-compatible.md b/packages/kilo-docs/pages/ai-providers/openai-compatible.md index 5e373573752..2e335fa9365 100644 --- a/packages/kilo-docs/pages/ai-providers/openai-compatible.md +++ b/packages/kilo-docs/pages/ai-providers/openai-compatible.md @@ -40,9 +40,25 @@ You'll find these settings in the Kilo Code settings panel (click the {% codicon {% /tab %} {% tab label="VSCode" %} -Open **Settings** (gear icon) and go to the **Providers** tab to add an OpenAI Compatible provider. Enter your API key and the provider's base URL. +1. Open **Settings** (gear icon) and go to the **Providers** tab. +2. Scroll to the bottom and click **Custom provider**. -The extension stores this in your `kilo.json` config file. You can also edit the config file directly — see the **CLI** tab for the file format. +![Custom provider button](/docs/img/custom-models/custom-provider-button.png) + +3. Fill in the custom provider dialog: + +![Custom provider configuration dialog](/docs/img/custom-models/custom-provider-details.png) + +- **Provider ID** — A unique identifier (e.g., `my-provider`). +- **Display name** — A human-readable name shown in the UI. +- **Base URL** — The provider's OpenAI-compatible API endpoint (e.g., `https://api.your-provider.com/v1`). Kilo auto-fetches available models when a valid URL is entered. +- **API key** — Your API key. Optional — leave empty if authentication is handled via headers. +- **Models** — Add models manually or select from the auto-fetched list. +- **Headers** (optional) — Custom HTTP headers as key-value pairs. + +4. Click **Submit** to save. The provider's models appear in the model picker. + +For additional model configuration (token limits, tool calling, variants), edit the `kilo.jsonc` config file directly — see the **CLI** tab or the [Custom Models](/docs/code-with-ai/agents/custom-models) guide. {% /tab %} {% tab label="CLI" %} diff --git a/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md b/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md new file mode 100644 index 00000000000..52d0473d577 --- /dev/null +++ b/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md @@ -0,0 +1,317 @@ +--- +title: "Custom Models" +description: "How to configure custom or unlisted models for any provider" +platform: new +--- + +# Custom Models + +Kilo Code ships with a curated list of models for each provider, but you can use **any model** your provider supports — including models that aren't in the built-in list. This is useful for: + +- Using a newly released model before it's added to the built-in catalog +- Running a custom or fine-tuned model via LM Studio, Ollama, or another local provider +- Connecting to a self-hosted model behind an OpenAI-compatible API +- Configuring model-specific options like token limits, pricing, or reasoning settings + +## Defining a Custom Model + +Add custom models under the `provider..models` key in your config file. The model key becomes the model ID you reference elsewhere. + +{% tabs %} +{% tab label="CLI" %} + +**Config file** (`~/.config/kilo/kilo.jsonc` or `./kilo.jsonc`): + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "model": "lmstudio/my-custom-model", + "provider": { + "lmstudio": { + "models": { + "my-custom-model": { + "name": "My Custom Model", + }, + }, + }, + }, +} +``` + +{% /tab %} +{% tab label="VSCode" %} + +1. Open **Settings** (gear icon) and go to the **Providers** tab. + +2. Scroll to the bottom of the provider list and click **Custom provider**. + +![Custom provider button in the Providers tab](/docs/img/custom-models/custom-provider-button.png) + +3. Fill in the custom provider dialog: + +![Custom provider configuration dialog](/docs/img/custom-models/custom-provider-details.png) + +- **Provider ID** — A unique identifier using lowercase letters, numbers, hyphens, or underscores (e.g., `myprovider`). This becomes the `provider_id` in the `provider_id/model_id` format. +- **Display name** — A human-readable name shown in the UI (e.g., `My AI Provider`). +- **Base URL** — The OpenAI-compatible API endpoint (e.g., `https://api.myprovider.com/v1`). When a valid URL is entered, Kilo automatically fetches available models from the endpoint. +- **API key** — Your provider's API key. Optional — leave empty if you manage authentication via headers. +- **Models** — Add models manually by ID and display name, or select from the auto-fetched list that appears after entering a valid base URL. +- **Headers** (optional) — Add custom HTTP headers as key-value pairs if your provider requires them. + +4. Click **Submit** to save. Your custom provider appears in the provider list and its models become available in the model picker. + +To edit an existing custom provider, click the **Edit provider** button next to it in the connected providers section. + +For additional model configuration (token limits, tool calling, reasoning, variants), edit the `kilo.jsonc` config file directly — see the **CLI** tab for the format. + +{% /tab %} +{% /tabs %} + +The `model` key uses the format `provider_id/model_id`, where: + +- **`provider_id`** is the key under `provider` (e.g., `lmstudio`, `ollama`, `openai`, `anthropic`, `openai-compatible`) +- **`model_id`** is the key under `provider..models` (e.g., `my-custom-model`) + +## Model Configuration Fields + +All fields are optional. When a model ID matches one already in the built-in catalog, your values are merged on top of the defaults — you only need to specify what you want to override. + +| Field | Type | Description | +| ------------- | --------- | ----------------------------------------------------------------------------- | +| `name` | `string` | Display name shown in the model picker | +| `id` | `string` | API-facing model ID sent to the provider. Defaults to the config key | +| `tool_call` | `boolean` | Whether the model supports tool/function calling | +| `reasoning` | `boolean` | Whether the model supports extended thinking | +| `temperature` | `boolean` | Whether the model supports the temperature parameter | +| `attachment` | `boolean` | Whether the model supports file attachments | +| `limit` | `object` | Token limits: `{ context, output, input? }` | +| `cost` | `object` | Pricing per million tokens: `{ input, output, cache_read?, cache_write? }` | +| `options` | `object` | Arbitrary provider-specific model options | +| `headers` | `object` | Custom HTTP headers to include in requests | +| `provider` | `object` | Override `{ npm?, api? }` — the AI SDK package or base API URL for this model | +| `variants` | `object` | Named variant configurations (e.g., different reasoning efforts) | + +## Examples + +### Local model with LM Studio + +Register a model that LM Studio serves under a custom name: + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "model": "lmstudio/deepseek-r1-0528", + "provider": { + "lmstudio": { + "models": { + "deepseek-r1-0528": { + "name": "DeepSeek R1 0528", + }, + }, + }, + }, +} +``` + +### Local model with Ollama + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "model": "ollama/my-finetune:latest", + "provider": { + "ollama": { + "models": { + "my-finetune:latest": { + "name": "My Fine-tuned Model", + "tool_call": true, + "limit": { + "context": 32768, + "output": 8192, + }, + }, + }, + }, + }, +} +``` + +### New or unlisted model from a cloud provider + +Use a model that's not yet in the built-in catalog: + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "model": "openai/gpt-6-preview", + "provider": { + "openai": { + "models": { + "gpt-6-preview": { + "name": "GPT-6 Preview", + "tool_call": true, + "reasoning": true, + "limit": { + "context": 200000, + "output": 32768, + }, + }, + }, + }, + }, +} +``` + +### OpenAI-compatible provider with a custom endpoint + +Connect to any provider that exposes an OpenAI-compatible API: + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "model": "openai-compatible/my-model", + "provider": { + "openai-compatible": { + "options": { + "apiKey": "{env:MY_PROVIDER_API_KEY}", + "baseURL": "https://api.my-provider.com/v1", + }, + "models": { + "my-model": { + "name": "My Custom Model", + "tool_call": true, + "limit": { + "context": 128000, + "output": 16384, + }, + }, + }, + }, + }, +} +``` + +### Configuring model options and variants + +Override options or define reasoning variants for a built-in model: + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "provider": { + "anthropic": { + "models": { + "claude-sonnet-4-20250514": { + "options": { + "thinking": { + "type": "enabled", + "budgetTokens": 16000, + }, + }, + "variants": { + "thinking-high": { + "thinking": { + "type": "enabled", + "budgetTokens": 32000, + }, + }, + "fast": { + "disabled": true, + }, + }, + }, + }, + }, + }, +} +``` + +### Using the `id` field to map model names + +If the model key in your config differs from what the provider expects, use the `id` field: + +```jsonc +{ + "$schema": "https://app.kilo.ai/config.json", + "model": "lmstudio/my-local-llama", + "provider": { + "lmstudio": { + "models": { + "my-local-llama": { + "id": "meta-llama-3.1-8b-instruct", + "name": "Llama 3.1 8B (Local)", + }, + }, + }, + }, +} +``` + +Here `my-local-llama` is the key you use in your config and model picker, while `meta-llama-3.1-8b-instruct` is the actual model identifier sent to the LM Studio API. + +## Model Loading Priority + +When Kilo starts, it resolves the active model in this order: + +1. The `--model` (or `-m`) command-line flag +2. The `model` key in your config file +3. The last used model from your previous session +4. The first available model using an internal priority + +The format for all of these is `provider_id/model_id`. + +## Provider-Level Options + +You can also set options that apply to all models from a provider: + +```jsonc +{ + "provider": { + "openai": { + "options": { + "apiKey": "{env:OPENAI_API_KEY}", + "baseURL": "https://my-proxy.example.com/v1", + "timeout": 120000, + }, + }, + }, +} +``` + +| Option | Type | Description | +| --------- | ----------------- | ------------------------------------------------------ | +| `apiKey` | `string` | API key (supports `{env:VAR}` syntax) | +| `baseURL` | `string` | Override the provider's base API URL | +| `timeout` | `number \| false` | Request timeout in milliseconds, or `false` to disable | + +## Filtering Available Models + +Control which models appear in the model picker for a provider using allowlists and blocklists: + +```jsonc +{ + "provider": { + "openai": { + "whitelist": ["gpt-5", "gpt-5-mini"], + "blacklist": ["gpt-4-turbo"], + }, + }, +} +``` + +- **`whitelist`** — only these model IDs are available from this provider +- **`blacklist`** — these model IDs are hidden from this provider + +## Troubleshooting + +**Model doesn't appear in the model picker:** + +- Verify the provider has valid credentials configured (API key, or local server running) +- Check that the model key matches what you set in `"model": "provider/model-key"` +- Run `kilo models` to list all available models and confirm your provider is active + +**Model errors or unexpected behavior:** + +- Set `tool_call: true` if you need the model to use tools (file editing, terminal, etc.) +- Set `limit.context` and `limit.output` to match the model's actual capabilities +- For local models, ensure your inference server is running and accessible at the configured URL diff --git a/packages/kilo-docs/pages/code-with-ai/agents/model-selection.md b/packages/kilo-docs/pages/code-with-ai/agents/model-selection.md index 43d9dbec509..932babfabbb 100644 --- a/packages/kilo-docs/pages/code-with-ai/agents/model-selection.md +++ b/packages/kilo-docs/pages/code-with-ai/agents/model-selection.md @@ -54,6 +54,8 @@ While the specifics change constantly, some principles stay consistent: **For local/private work**: Ollama and LM Studio let you run models locally. The tradeoff is usually speed and capability for privacy and zero API costs. +**Using an unlisted model?** You can register any model — including fine-tunes, newly released models, or custom local models — by adding it to your config file. See [Custom Models](/docs/code-with-ai/agents/custom-models) for details. + ## Context Windows Matter One thing that doesn't change: context window size matters for your workflow. diff --git a/packages/kilo-docs/pages/code-with-ai/platforms/cli.md b/packages/kilo-docs/pages/code-with-ai/platforms/cli.md index 91b40871124..fec996bc220 100644 --- a/packages/kilo-docs/pages/code-with-ai/platforms/cli.md +++ b/packages/kilo-docs/pages/code-with-ai/platforms/cli.md @@ -320,14 +320,18 @@ Project-level configuration takes precedence over global settings. Common configuration options include: -- **`model`** - Default model to use -- **`provider`** - Provider-specific settings (API keys, base URLs, custom models) +- **`model`** - Default model in `provider_id/model_id` format (e.g., `"anthropic/claude-sonnet-4-20250514"`) +- **`provider`** - Provider-specific settings (API keys, base URLs, [custom models](/docs/code-with-ai/agents/custom-models)) - **`mcp`** - MCP server configuration - **`permission`** - Tool permission settings (`allow` or `ask`) - **`instructions`** - Paths to instruction files (e.g., `["CONTRIBUTING.md", ".cursor/rules/*.md"]`) - **`formatter`** - Code formatter configuration - **`disabled_providers`** / **`enabled_providers`** - Control which providers are available +{% callout type="tip" %} +**Using a model that's not in the built-in list?** You can register any model by adding it under `provider..models` in your config file. See [Custom Models](/docs/code-with-ai/agents/custom-models) for full details and examples. +{% /callout %} + ### Environment Variables Use `{env:VARIABLE_NAME}` syntax in config files to reference environment variables: diff --git a/packages/kilo-docs/public/img/custom-models/custom-provider-button.png b/packages/kilo-docs/public/img/custom-models/custom-provider-button.png new file mode 100644 index 00000000000..4fabb5481ae Binary files /dev/null and b/packages/kilo-docs/public/img/custom-models/custom-provider-button.png differ diff --git a/packages/kilo-docs/public/img/custom-models/custom-provider-details.png b/packages/kilo-docs/public/img/custom-models/custom-provider-details.png new file mode 100644 index 00000000000..2a22661b9da Binary files /dev/null and b/packages/kilo-docs/public/img/custom-models/custom-provider-details.png differ