mirror of
https://github.com/Kilo-Org/kilocode.git
synced 2026-09-24 16:02:55 +08:00
docs(kilo-docs): add custom model configuration documentation (#8176)
Add a dedicated Custom Models page documenting how to register custom, unlisted, or fine-tuned models for any provider via the config file. Includes field reference, examples for LM Studio, Ollama, OpenAI, OpenAI-compatible endpoints, model options/variants, and the id field for name mapping. Covers the VS Code custom provider dialog with screenshots. Updates CLI, LM Studio, Ollama, OpenAI-compatible, and model selection pages with cross-references. Closes #8173
This commit is contained in:
@@ -36,6 +36,11 @@ export const CodeWithAiNav: NavSection[] = [
|
||||
href: "/code-with-ai/agents/auto-model",
|
||||
children: "Auto Model",
|
||||
},
|
||||
{
|
||||
href: "/code-with-ai/agents/custom-models",
|
||||
children: "Custom Models",
|
||||
platform: "new",
|
||||
},
|
||||
{
|
||||
href: "/code-with-ai/agents/free-and-budget-models",
|
||||
children: "Free & Budget Models",
|
||||
|
||||
@@ -71,6 +71,44 @@ Then set your default model:
|
||||
{% /tab %}
|
||||
{% /tabs %}
|
||||
|
||||
## Using Custom or Unlisted Models
|
||||
|
||||
If the model you loaded in LM Studio doesn't appear in the Kilo model picker, you can register it as a custom model in your config file:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"model": "lmstudio/my-custom-model",
|
||||
"provider": {
|
||||
"lmstudio": {
|
||||
"models": {
|
||||
"my-custom-model": {
|
||||
"name": "My Custom Model",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
The model key (`my-custom-model`) must match the model identifier that LM Studio serves. If the display name you want differs from the API identifier, use the `id` field to set the API-facing name separately:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"provider": {
|
||||
"lmstudio": {
|
||||
"models": {
|
||||
"my-llama": {
|
||||
"id": "meta-llama-3.1-8b-instruct",
|
||||
"name": "Llama 3.1 8B (Local)",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
See [Custom Models](/docs/code-with-ai/agents/custom-models) for the full list of configuration fields and more examples.
|
||||
|
||||
## Tips and Notes
|
||||
|
||||
- **Resource Requirements:** Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.
|
||||
|
||||
@@ -116,6 +116,32 @@ Then set your default model:
|
||||
{% /tab %}
|
||||
{% /tabs %}
|
||||
|
||||
## Using Custom or Unlisted Models
|
||||
|
||||
If your Ollama model doesn't appear in the Kilo model picker, register it as a custom model in your config file:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"model": "ollama/my-finetune:latest",
|
||||
"provider": {
|
||||
"ollama": {
|
||||
"models": {
|
||||
"my-finetune:latest": {
|
||||
"name": "My Fine-tuned Model",
|
||||
"tool_call": true,
|
||||
"limit": {
|
||||
"context": 32768,
|
||||
"output": 8192,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
See [Custom Models](/docs/code-with-ai/agents/custom-models) for the full list of configuration fields and more examples.
|
||||
|
||||
## Further Reading
|
||||
|
||||
Refer to the [Ollama documentation](https://ollama.com/docs) for more information on installing, configuring and using Ollama.
|
||||
|
||||
@@ -40,9 +40,25 @@ You'll find these settings in the Kilo Code settings panel (click the {% codicon
|
||||
{% /tab %}
|
||||
{% tab label="VSCode" %}
|
||||
|
||||
Open **Settings** (gear icon) and go to the **Providers** tab to add an OpenAI Compatible provider. Enter your API key and the provider's base URL.
|
||||
1. Open **Settings** (gear icon) and go to the **Providers** tab.
|
||||
2. Scroll to the bottom and click **Custom provider**.
|
||||
|
||||
The extension stores this in your `kilo.json` config file. You can also edit the config file directly — see the **CLI** tab for the file format.
|
||||

|
||||
|
||||
3. Fill in the custom provider dialog:
|
||||
|
||||

|
||||
|
||||
- **Provider ID** — A unique identifier (e.g., `my-provider`).
|
||||
- **Display name** — A human-readable name shown in the UI.
|
||||
- **Base URL** — The provider's OpenAI-compatible API endpoint (e.g., `https://api.your-provider.com/v1`). Kilo auto-fetches available models when a valid URL is entered.
|
||||
- **API key** — Your API key. Optional — leave empty if authentication is handled via headers.
|
||||
- **Models** — Add models manually or select from the auto-fetched list.
|
||||
- **Headers** (optional) — Custom HTTP headers as key-value pairs.
|
||||
|
||||
4. Click **Submit** to save. The provider's models appear in the model picker.
|
||||
|
||||
For additional model configuration (token limits, tool calling, variants), edit the `kilo.jsonc` config file directly — see the **CLI** tab or the [Custom Models](/docs/code-with-ai/agents/custom-models) guide.
|
||||
|
||||
{% /tab %}
|
||||
{% tab label="CLI" %}
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
---
|
||||
title: "Custom Models"
|
||||
description: "How to configure custom or unlisted models for any provider"
|
||||
platform: new
|
||||
---
|
||||
|
||||
# Custom Models
|
||||
|
||||
Kilo Code ships with a curated list of models for each provider, but you can use **any model** your provider supports — including models that aren't in the built-in list. This is useful for:
|
||||
|
||||
- Using a newly released model before it's added to the built-in catalog
|
||||
- Running a custom or fine-tuned model via LM Studio, Ollama, or another local provider
|
||||
- Connecting to a self-hosted model behind an OpenAI-compatible API
|
||||
- Configuring model-specific options like token limits, pricing, or reasoning settings
|
||||
|
||||
## Defining a Custom Model
|
||||
|
||||
Add custom models under the `provider.<provider_id>.models` key in your config file. The model key becomes the model ID you reference elsewhere.
|
||||
|
||||
{% tabs %}
|
||||
{% tab label="CLI" %}
|
||||
|
||||
**Config file** (`~/.config/kilo/kilo.jsonc` or `./kilo.jsonc`):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"model": "lmstudio/my-custom-model",
|
||||
"provider": {
|
||||
"lmstudio": {
|
||||
"models": {
|
||||
"my-custom-model": {
|
||||
"name": "My Custom Model",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
{% /tab %}
|
||||
{% tab label="VSCode" %}
|
||||
|
||||
1. Open **Settings** (gear icon) and go to the **Providers** tab.
|
||||
|
||||
2. Scroll to the bottom of the provider list and click **Custom provider**.
|
||||
|
||||

|
||||
|
||||
3. Fill in the custom provider dialog:
|
||||
|
||||

|
||||
|
||||
- **Provider ID** — A unique identifier using lowercase letters, numbers, hyphens, or underscores (e.g., `myprovider`). This becomes the `provider_id` in the `provider_id/model_id` format.
|
||||
- **Display name** — A human-readable name shown in the UI (e.g., `My AI Provider`).
|
||||
- **Base URL** — The OpenAI-compatible API endpoint (e.g., `https://api.myprovider.com/v1`). When a valid URL is entered, Kilo automatically fetches available models from the endpoint.
|
||||
- **API key** — Your provider's API key. Optional — leave empty if you manage authentication via headers.
|
||||
- **Models** — Add models manually by ID and display name, or select from the auto-fetched list that appears after entering a valid base URL.
|
||||
- **Headers** (optional) — Add custom HTTP headers as key-value pairs if your provider requires them.
|
||||
|
||||
4. Click **Submit** to save. Your custom provider appears in the provider list and its models become available in the model picker.
|
||||
|
||||
To edit an existing custom provider, click the **Edit provider** button next to it in the connected providers section.
|
||||
|
||||
For additional model configuration (token limits, tool calling, reasoning, variants), edit the `kilo.jsonc` config file directly — see the **CLI** tab for the format.
|
||||
|
||||
{% /tab %}
|
||||
{% /tabs %}
|
||||
|
||||
The `model` key uses the format `provider_id/model_id`, where:
|
||||
|
||||
- **`provider_id`** is the key under `provider` (e.g., `lmstudio`, `ollama`, `openai`, `anthropic`, `openai-compatible`)
|
||||
- **`model_id`** is the key under `provider.<provider_id>.models` (e.g., `my-custom-model`)
|
||||
|
||||
## Model Configuration Fields
|
||||
|
||||
All fields are optional. When a model ID matches one already in the built-in catalog, your values are merged on top of the defaults — you only need to specify what you want to override.
|
||||
|
||||
| Field | Type | Description |
|
||||
| ------------- | --------- | ----------------------------------------------------------------------------- |
|
||||
| `name` | `string` | Display name shown in the model picker |
|
||||
| `id` | `string` | API-facing model ID sent to the provider. Defaults to the config key |
|
||||
| `tool_call` | `boolean` | Whether the model supports tool/function calling |
|
||||
| `reasoning` | `boolean` | Whether the model supports extended thinking |
|
||||
| `temperature` | `boolean` | Whether the model supports the temperature parameter |
|
||||
| `attachment` | `boolean` | Whether the model supports file attachments |
|
||||
| `limit` | `object` | Token limits: `{ context, output, input? }` |
|
||||
| `cost` | `object` | Pricing per million tokens: `{ input, output, cache_read?, cache_write? }` |
|
||||
| `options` | `object` | Arbitrary provider-specific model options |
|
||||
| `headers` | `object` | Custom HTTP headers to include in requests |
|
||||
| `provider` | `object` | Override `{ npm?, api? }` — the AI SDK package or base API URL for this model |
|
||||
| `variants` | `object` | Named variant configurations (e.g., different reasoning efforts) |
|
||||
|
||||
## Examples
|
||||
|
||||
### Local model with LM Studio
|
||||
|
||||
Register a model that LM Studio serves under a custom name:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"model": "lmstudio/deepseek-r1-0528",
|
||||
"provider": {
|
||||
"lmstudio": {
|
||||
"models": {
|
||||
"deepseek-r1-0528": {
|
||||
"name": "DeepSeek R1 0528",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### Local model with Ollama
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"model": "ollama/my-finetune:latest",
|
||||
"provider": {
|
||||
"ollama": {
|
||||
"models": {
|
||||
"my-finetune:latest": {
|
||||
"name": "My Fine-tuned Model",
|
||||
"tool_call": true,
|
||||
"limit": {
|
||||
"context": 32768,
|
||||
"output": 8192,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### New or unlisted model from a cloud provider
|
||||
|
||||
Use a model that's not yet in the built-in catalog:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"model": "openai/gpt-6-preview",
|
||||
"provider": {
|
||||
"openai": {
|
||||
"models": {
|
||||
"gpt-6-preview": {
|
||||
"name": "GPT-6 Preview",
|
||||
"tool_call": true,
|
||||
"reasoning": true,
|
||||
"limit": {
|
||||
"context": 200000,
|
||||
"output": 32768,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### OpenAI-compatible provider with a custom endpoint
|
||||
|
||||
Connect to any provider that exposes an OpenAI-compatible API:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"model": "openai-compatible/my-model",
|
||||
"provider": {
|
||||
"openai-compatible": {
|
||||
"options": {
|
||||
"apiKey": "{env:MY_PROVIDER_API_KEY}",
|
||||
"baseURL": "https://api.my-provider.com/v1",
|
||||
},
|
||||
"models": {
|
||||
"my-model": {
|
||||
"name": "My Custom Model",
|
||||
"tool_call": true,
|
||||
"limit": {
|
||||
"context": 128000,
|
||||
"output": 16384,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### Configuring model options and variants
|
||||
|
||||
Override options or define reasoning variants for a built-in model:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"provider": {
|
||||
"anthropic": {
|
||||
"models": {
|
||||
"claude-sonnet-4-20250514": {
|
||||
"options": {
|
||||
"thinking": {
|
||||
"type": "enabled",
|
||||
"budgetTokens": 16000,
|
||||
},
|
||||
},
|
||||
"variants": {
|
||||
"thinking-high": {
|
||||
"thinking": {
|
||||
"type": "enabled",
|
||||
"budgetTokens": 32000,
|
||||
},
|
||||
},
|
||||
"fast": {
|
||||
"disabled": true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### Using the `id` field to map model names
|
||||
|
||||
If the model key in your config differs from what the provider expects, use the `id` field:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://app.kilo.ai/config.json",
|
||||
"model": "lmstudio/my-local-llama",
|
||||
"provider": {
|
||||
"lmstudio": {
|
||||
"models": {
|
||||
"my-local-llama": {
|
||||
"id": "meta-llama-3.1-8b-instruct",
|
||||
"name": "Llama 3.1 8B (Local)",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Here `my-local-llama` is the key you use in your config and model picker, while `meta-llama-3.1-8b-instruct` is the actual model identifier sent to the LM Studio API.
|
||||
|
||||
## Model Loading Priority
|
||||
|
||||
When Kilo starts, it resolves the active model in this order:
|
||||
|
||||
1. The `--model` (or `-m`) command-line flag
|
||||
2. The `model` key in your config file
|
||||
3. The last used model from your previous session
|
||||
4. The first available model using an internal priority
|
||||
|
||||
The format for all of these is `provider_id/model_id`.
|
||||
|
||||
## Provider-Level Options
|
||||
|
||||
You can also set options that apply to all models from a provider:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"provider": {
|
||||
"openai": {
|
||||
"options": {
|
||||
"apiKey": "{env:OPENAI_API_KEY}",
|
||||
"baseURL": "https://my-proxy.example.com/v1",
|
||||
"timeout": 120000,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
| Option | Type | Description |
|
||||
| --------- | ----------------- | ------------------------------------------------------ |
|
||||
| `apiKey` | `string` | API key (supports `{env:VAR}` syntax) |
|
||||
| `baseURL` | `string` | Override the provider's base API URL |
|
||||
| `timeout` | `number \| false` | Request timeout in milliseconds, or `false` to disable |
|
||||
|
||||
## Filtering Available Models
|
||||
|
||||
Control which models appear in the model picker for a provider using allowlists and blocklists:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"provider": {
|
||||
"openai": {
|
||||
"whitelist": ["gpt-5", "gpt-5-mini"],
|
||||
"blacklist": ["gpt-4-turbo"],
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
- **`whitelist`** — only these model IDs are available from this provider
|
||||
- **`blacklist`** — these model IDs are hidden from this provider
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Model doesn't appear in the model picker:**
|
||||
|
||||
- Verify the provider has valid credentials configured (API key, or local server running)
|
||||
- Check that the model key matches what you set in `"model": "provider/model-key"`
|
||||
- Run `kilo models` to list all available models and confirm your provider is active
|
||||
|
||||
**Model errors or unexpected behavior:**
|
||||
|
||||
- Set `tool_call: true` if you need the model to use tools (file editing, terminal, etc.)
|
||||
- Set `limit.context` and `limit.output` to match the model's actual capabilities
|
||||
- For local models, ensure your inference server is running and accessible at the configured URL
|
||||
@@ -54,6 +54,8 @@ While the specifics change constantly, some principles stay consistent:
|
||||
|
||||
**For local/private work**: Ollama and LM Studio let you run models locally. The tradeoff is usually speed and capability for privacy and zero API costs.
|
||||
|
||||
**Using an unlisted model?** You can register any model — including fine-tunes, newly released models, or custom local models — by adding it to your config file. See [Custom Models](/docs/code-with-ai/agents/custom-models) for details.
|
||||
|
||||
## Context Windows Matter
|
||||
|
||||
One thing that doesn't change: context window size matters for your workflow.
|
||||
|
||||
@@ -320,14 +320,18 @@ Project-level configuration takes precedence over global settings.
|
||||
|
||||
Common configuration options include:
|
||||
|
||||
- **`model`** - Default model to use
|
||||
- **`provider`** - Provider-specific settings (API keys, base URLs, custom models)
|
||||
- **`model`** - Default model in `provider_id/model_id` format (e.g., `"anthropic/claude-sonnet-4-20250514"`)
|
||||
- **`provider`** - Provider-specific settings (API keys, base URLs, [custom models](/docs/code-with-ai/agents/custom-models))
|
||||
- **`mcp`** - MCP server configuration
|
||||
- **`permission`** - Tool permission settings (`allow` or `ask`)
|
||||
- **`instructions`** - Paths to instruction files (e.g., `["CONTRIBUTING.md", ".cursor/rules/*.md"]`)
|
||||
- **`formatter`** - Code formatter configuration
|
||||
- **`disabled_providers`** / **`enabled_providers`** - Control which providers are available
|
||||
|
||||
{% callout type="tip" %}
|
||||
**Using a model that's not in the built-in list?** You can register any model by adding it under `provider.<provider_id>.models` in your config file. See [Custom Models](/docs/code-with-ai/agents/custom-models) for full details and examples.
|
||||
{% /callout %}
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Use `{env:VARIABLE_NAME}` syntax in config files to reference environment variables:
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 100 KiB |
Reference in New Issue
Block a user