diff --git a/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md b/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md index db8dc74c64..834ec18ce1 100644 --- a/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md +++ b/packages/kilo-docs/pages/code-with-ai/agents/custom-models.md @@ -84,6 +84,7 @@ All fields are optional. When a model ID matches one already in the built-in cat | `reasoning` | `boolean` | Whether the model supports extended thinking | | `temperature` | `boolean` | Whether the model supports the temperature parameter | | `attachment` | `boolean` | Whether the model supports file attachments | +| `modalities` | `object` | Optional. Supported input and output types: `{ input, output }` | | `limit` | `object` | Token limits: `{ context, output, input? }` | | `cost` | `object` | Pricing per million tokens: `{ input, output, cache_read?, cache_write? }` | | `options` | `object` | Arbitrary provider-specific model options | @@ -91,6 +92,26 @@ All fields are optional. When a model ID matches one already in the built-in cat | `provider` | `object` | Override `{ npm?, api? }` — the AI SDK package or base API URL for this model | | `variants` | `object` | Named variant configurations (e.g., different reasoning efforts) | +### Modalities (modalities) + +The `modalities` object declares which content types the model can receive and produce. It is optional — omit it to use defaults from the catalog or fallback to text-only. When `modalities` is provided, both `input` and `output` arrays are required. Each array can include `text`, `image`, `audio`, `video`, or `pdf`. + +| Sub-field | Type | Required | Description | +| --------- | ------- | ---------------- | ------------------------------------------------ | +| `input` | `array` | Yes (if present) | Content types the model accepts from the user | +| `output` | `array` | Yes (if present) | Content types the model can generate in response | + +For a standard text model that can also inspect images, use: + +```jsonc +"modalities": { + "input": ["text", "image"], + "output": ["text"] +} +``` + +If `modalities` is omitted and the model ID matches a models.dev catalog entry for that provider, Kilo uses the catalog's modalities. For completely custom models with no catalog match, Kilo defaults to text input and text output only. Set `attachment: true` alongside image, audio, video, or PDF input modalities when the provider supports sending those files as attachments. + ### Token Limits (limit) The `limit` object controls how Kilo manages the model's context window and output length. These values are specified in **tokens**.