docs(kilo-docs): add custom model configuration documentation (#8176)

Add a dedicated Custom Models page documenting how to register custom,
unlisted, or fine-tuned models for any provider via the config file.
Includes field reference, examples for LM Studio, Ollama, OpenAI,
OpenAI-compatible endpoints, model options/variants, and the id field
for name mapping. Covers the VS Code custom provider dialog with
screenshots. Updates CLI, LM Studio, Ollama, OpenAI-compatible, and
model selection pages with cross-references.

Closes #8173
This commit is contained in:
Marius
2026-04-02 13:34:23 +02:00
committed by GitHub
parent a1a29f2a5e
commit 754bf787a9
9 changed files with 412 additions and 4 deletions
@@ -36,6 +36,11 @@ export const CodeWithAiNav: NavSection[] = [
href: "/code-with-ai/agents/auto-model",
children: "Auto Model",
},
{
href: "/code-with-ai/agents/custom-models",
children: "Custom Models",
platform: "new",
},
{
href: "/code-with-ai/agents/free-and-budget-models",
children: "Free & Budget Models",
@@ -71,6 +71,44 @@ Then set your default model:
{% /tab %}
{% /tabs %}
## Using Custom or Unlisted Models
If the model you loaded in LM Studio doesn't appear in the Kilo model picker, you can register it as a custom model in your config file:
```jsonc
{
"model": "lmstudio/my-custom-model",
"provider": {
"lmstudio": {
"models": {
"my-custom-model": {
"name": "My Custom Model",
},
},
},
},
}
```
The model key (`my-custom-model`) must match the model identifier that LM Studio serves. If the display name you want differs from the API identifier, use the `id` field to set the API-facing name separately:
```jsonc
{
"provider": {
"lmstudio": {
"models": {
"my-llama": {
"id": "meta-llama-3.1-8b-instruct",
"name": "Llama 3.1 8B (Local)",
},
},
},
},
}
```
See [Custom Models](/docs/code-with-ai/agents/custom-models) for the full list of configuration fields and more examples.
## Tips and Notes
- **Resource Requirements:** Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.
@@ -116,6 +116,32 @@ Then set your default model:
{% /tab %}
{% /tabs %}
## Using Custom or Unlisted Models
If your Ollama model doesn't appear in the Kilo model picker, register it as a custom model in your config file:
```jsonc
{
"model": "ollama/my-finetune:latest",
"provider": {
"ollama": {
"models": {
"my-finetune:latest": {
"name": "My Fine-tuned Model",
"tool_call": true,
"limit": {
"context": 32768,
"output": 8192,
},
},
},
},
},
}
```
See [Custom Models](/docs/code-with-ai/agents/custom-models) for the full list of configuration fields and more examples.
## Further Reading
Refer to the [Ollama documentation](https://ollama.com/docs) for more information on installing, configuring and using Ollama.
@@ -40,9 +40,25 @@ You'll find these settings in the Kilo Code settings panel (click the {% codicon
{% /tab %}
{% tab label="VSCode" %}
Open **Settings** (gear icon) and go to the **Providers** tab to add an OpenAI Compatible provider. Enter your API key and the provider's base URL.
1. Open **Settings** (gear icon) and go to the **Providers** tab.
2. Scroll to the bottom and click **Custom provider**.
The extension stores this in your `kilo.json` config file. You can also edit the config file directly — see the **CLI** tab for the file format.
![Custom provider button](/docs/img/custom-models/custom-provider-button.png)
3. Fill in the custom provider dialog:
![Custom provider configuration dialog](/docs/img/custom-models/custom-provider-details.png)
- **Provider ID** — A unique identifier (e.g., `my-provider`).
- **Display name** — A human-readable name shown in the UI.
- **Base URL** — The provider's OpenAI-compatible API endpoint (e.g., `https://api.your-provider.com/v1`). Kilo auto-fetches available models when a valid URL is entered.
- **API key** — Your API key. Optional — leave empty if authentication is handled via headers.
- **Models** — Add models manually or select from the auto-fetched list.
- **Headers** (optional) — Custom HTTP headers as key-value pairs.
4. Click **Submit** to save. The provider's models appear in the model picker.
For additional model configuration (token limits, tool calling, variants), edit the `kilo.jsonc` config file directly — see the **CLI** tab or the [Custom Models](/docs/code-with-ai/agents/custom-models) guide.
{% /tab %}
{% tab label="CLI" %}
@@ -0,0 +1,317 @@
---
title: "Custom Models"
description: "How to configure custom or unlisted models for any provider"
platform: new
---
# Custom Models
Kilo Code ships with a curated list of models for each provider, but you can use **any model** your provider supports — including models that aren't in the built-in list. This is useful for:
- Using a newly released model before it's added to the built-in catalog
- Running a custom or fine-tuned model via LM Studio, Ollama, or another local provider
- Connecting to a self-hosted model behind an OpenAI-compatible API
- Configuring model-specific options like token limits, pricing, or reasoning settings
## Defining a Custom Model
Add custom models under the `provider.<provider_id>.models` key in your config file. The model key becomes the model ID you reference elsewhere.
{% tabs %}
{% tab label="CLI" %}
**Config file** (`~/.config/kilo/kilo.jsonc` or `./kilo.jsonc`):
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"model": "lmstudio/my-custom-model",
"provider": {
"lmstudio": {
"models": {
"my-custom-model": {
"name": "My Custom Model",
},
},
},
},
}
```
{% /tab %}
{% tab label="VSCode" %}
1. Open **Settings** (gear icon) and go to the **Providers** tab.
2. Scroll to the bottom of the provider list and click **Custom provider**.
![Custom provider button in the Providers tab](/docs/img/custom-models/custom-provider-button.png)
3. Fill in the custom provider dialog:
![Custom provider configuration dialog](/docs/img/custom-models/custom-provider-details.png)
- **Provider ID** — A unique identifier using lowercase letters, numbers, hyphens, or underscores (e.g., `myprovider`). This becomes the `provider_id` in the `provider_id/model_id` format.
- **Display name** — A human-readable name shown in the UI (e.g., `My AI Provider`).
- **Base URL** — The OpenAI-compatible API endpoint (e.g., `https://api.myprovider.com/v1`). When a valid URL is entered, Kilo automatically fetches available models from the endpoint.
- **API key** — Your provider's API key. Optional — leave empty if you manage authentication via headers.
- **Models** — Add models manually by ID and display name, or select from the auto-fetched list that appears after entering a valid base URL.
- **Headers** (optional) — Add custom HTTP headers as key-value pairs if your provider requires them.
4. Click **Submit** to save. Your custom provider appears in the provider list and its models become available in the model picker.
To edit an existing custom provider, click the **Edit provider** button next to it in the connected providers section.
For additional model configuration (token limits, tool calling, reasoning, variants), edit the `kilo.jsonc` config file directly — see the **CLI** tab for the format.
{% /tab %}
{% /tabs %}
The `model` key uses the format `provider_id/model_id`, where:
- **`provider_id`** is the key under `provider` (e.g., `lmstudio`, `ollama`, `openai`, `anthropic`, `openai-compatible`)
- **`model_id`** is the key under `provider.<provider_id>.models` (e.g., `my-custom-model`)
## Model Configuration Fields
All fields are optional. When a model ID matches one already in the built-in catalog, your values are merged on top of the defaults — you only need to specify what you want to override.
| Field | Type | Description |
| ------------- | --------- | ----------------------------------------------------------------------------- |
| `name` | `string` | Display name shown in the model picker |
| `id` | `string` | API-facing model ID sent to the provider. Defaults to the config key |
| `tool_call` | `boolean` | Whether the model supports tool/function calling |
| `reasoning` | `boolean` | Whether the model supports extended thinking |
| `temperature` | `boolean` | Whether the model supports the temperature parameter |
| `attachment` | `boolean` | Whether the model supports file attachments |
| `limit` | `object` | Token limits: `{ context, output, input? }` |
| `cost` | `object` | Pricing per million tokens: `{ input, output, cache_read?, cache_write? }` |
| `options` | `object` | Arbitrary provider-specific model options |
| `headers` | `object` | Custom HTTP headers to include in requests |
| `provider` | `object` | Override `{ npm?, api? }` — the AI SDK package or base API URL for this model |
| `variants` | `object` | Named variant configurations (e.g., different reasoning efforts) |
## Examples
### Local model with LM Studio
Register a model that LM Studio serves under a custom name:
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"model": "lmstudio/deepseek-r1-0528",
"provider": {
"lmstudio": {
"models": {
"deepseek-r1-0528": {
"name": "DeepSeek R1 0528",
},
},
},
},
}
```
### Local model with Ollama
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"model": "ollama/my-finetune:latest",
"provider": {
"ollama": {
"models": {
"my-finetune:latest": {
"name": "My Fine-tuned Model",
"tool_call": true,
"limit": {
"context": 32768,
"output": 8192,
},
},
},
},
},
}
```
### New or unlisted model from a cloud provider
Use a model that's not yet in the built-in catalog:
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"model": "openai/gpt-6-preview",
"provider": {
"openai": {
"models": {
"gpt-6-preview": {
"name": "GPT-6 Preview",
"tool_call": true,
"reasoning": true,
"limit": {
"context": 200000,
"output": 32768,
},
},
},
},
},
}
```
### OpenAI-compatible provider with a custom endpoint
Connect to any provider that exposes an OpenAI-compatible API:
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"model": "openai-compatible/my-model",
"provider": {
"openai-compatible": {
"options": {
"apiKey": "{env:MY_PROVIDER_API_KEY}",
"baseURL": "https://api.my-provider.com/v1",
},
"models": {
"my-model": {
"name": "My Custom Model",
"tool_call": true,
"limit": {
"context": 128000,
"output": 16384,
},
},
},
},
},
}
```
### Configuring model options and variants
Override options or define reasoning variants for a built-in model:
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"provider": {
"anthropic": {
"models": {
"claude-sonnet-4-20250514": {
"options": {
"thinking": {
"type": "enabled",
"budgetTokens": 16000,
},
},
"variants": {
"thinking-high": {
"thinking": {
"type": "enabled",
"budgetTokens": 32000,
},
},
"fast": {
"disabled": true,
},
},
},
},
},
},
}
```
### Using the `id` field to map model names
If the model key in your config differs from what the provider expects, use the `id` field:
```jsonc
{
"$schema": "https://app.kilo.ai/config.json",
"model": "lmstudio/my-local-llama",
"provider": {
"lmstudio": {
"models": {
"my-local-llama": {
"id": "meta-llama-3.1-8b-instruct",
"name": "Llama 3.1 8B (Local)",
},
},
},
},
}
```
Here `my-local-llama` is the key you use in your config and model picker, while `meta-llama-3.1-8b-instruct` is the actual model identifier sent to the LM Studio API.
## Model Loading Priority
When Kilo starts, it resolves the active model in this order:
1. The `--model` (or `-m`) command-line flag
2. The `model` key in your config file
3. The last used model from your previous session
4. The first available model using an internal priority
The format for all of these is `provider_id/model_id`.
## Provider-Level Options
You can also set options that apply to all models from a provider:
```jsonc
{
"provider": {
"openai": {
"options": {
"apiKey": "{env:OPENAI_API_KEY}",
"baseURL": "https://my-proxy.example.com/v1",
"timeout": 120000,
},
},
},
}
```
| Option | Type | Description |
| --------- | ----------------- | ------------------------------------------------------ |
| `apiKey` | `string` | API key (supports `{env:VAR}` syntax) |
| `baseURL` | `string` | Override the provider's base API URL |
| `timeout` | `number \| false` | Request timeout in milliseconds, or `false` to disable |
## Filtering Available Models
Control which models appear in the model picker for a provider using allowlists and blocklists:
```jsonc
{
"provider": {
"openai": {
"whitelist": ["gpt-5", "gpt-5-mini"],
"blacklist": ["gpt-4-turbo"],
},
},
}
```
- **`whitelist`** — only these model IDs are available from this provider
- **`blacklist`** — these model IDs are hidden from this provider
## Troubleshooting
**Model doesn't appear in the model picker:**
- Verify the provider has valid credentials configured (API key, or local server running)
- Check that the model key matches what you set in `"model": "provider/model-key"`
- Run `kilo models` to list all available models and confirm your provider is active
**Model errors or unexpected behavior:**
- Set `tool_call: true` if you need the model to use tools (file editing, terminal, etc.)
- Set `limit.context` and `limit.output` to match the model's actual capabilities
- For local models, ensure your inference server is running and accessible at the configured URL
@@ -54,6 +54,8 @@ While the specifics change constantly, some principles stay consistent:
**For local/private work**: Ollama and LM Studio let you run models locally. The tradeoff is usually speed and capability for privacy and zero API costs.
**Using an unlisted model?** You can register any model — including fine-tunes, newly released models, or custom local models — by adding it to your config file. See [Custom Models](/docs/code-with-ai/agents/custom-models) for details.
## Context Windows Matter
One thing that doesn't change: context window size matters for your workflow.
@@ -320,14 +320,18 @@ Project-level configuration takes precedence over global settings.
Common configuration options include:
- **`model`** - Default model to use
- **`provider`** - Provider-specific settings (API keys, base URLs, custom models)
- **`model`** - Default model in `provider_id/model_id` format (e.g., `"anthropic/claude-sonnet-4-20250514"`)
- **`provider`** - Provider-specific settings (API keys, base URLs, [custom models](/docs/code-with-ai/agents/custom-models))
- **`mcp`** - MCP server configuration
- **`permission`** - Tool permission settings (`allow` or `ask`)
- **`instructions`** - Paths to instruction files (e.g., `["CONTRIBUTING.md", ".cursor/rules/*.md"]`)
- **`formatter`** - Code formatter configuration
- **`disabled_providers`** / **`enabled_providers`** - Control which providers are available
{% callout type="tip" %}
**Using a model that's not in the built-in list?** You can register any model by adding it under `provider.<provider_id>.models` in your config file. See [Custom Models](/docs/code-with-ai/agents/custom-models) for full details and examples.
{% /callout %}
### Environment Variables
Use `{env:VARIABLE_NAME}` syntax in config files to reference environment variables:
Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 100 KiB