feat: add per-model OpenAI Responses API toggle (#27683)

chatd hardcoded `WithUseResponsesAPI()`, so the provider SDK's static
known-model list decided whether an OpenAI model spoke the Responses API
or Chat Completions. A model absent from that list silently fell back to
Chat Completions until the fantasy fork was patched.

This exposes the SDK's `WithResponsesAPIFunc` hook as a per-model
setting, `openai_config.use_responses_api`, stored in the existing
`chat_model_configs.options` JSONB. Unset keeps the known-model list,
`true` forces Responses, `false` forces Chat Completions. There is no
migration.

It sits in a new construction-time `openai_config` section rather than
in `provider_options.openai` because it selects the API when the client
is built, while `provider_options` holds per-request parameters. That
placement is also load-bearing: a config setting only this field would
otherwise materialize an OpenAI request-options struct and turn on
provider-side response storage, since `Store` defaults to true there.

Three places independently decided the transport and would silently
disagree with the client actually built:

| Site | Effect when it disagrees |
| --- | --- |
| `ModelFromConfig` | the transport being overridden |
| `AcceptsFilePartMediaType` | text attachments dropped, since Responses
natively accepts only images and PDFs |
| `UsesResponsesOptions` | the SDK type-asserts the concrete options
struct, so every OpenAI option is discarded |

They share one predicate here, `chatopenai.UsesResponsesAPI`, with the
override threaded to each. The rest of the stack removes that threading
by resolving the transport once and carrying it. Compaction overrides
and the quickgen debug model built clients without `ConfigOptions`, so
they now pass it and pick up both this setting and the existing
Anthropic beta headers.

The toggle also makes transport-conditional option handling
admin-switchable, so two hardening changes ride along.
`ServiceTierFromChat` now maps every tier the codersdk enum advertises
(`auto`, `default`, `flex`, `scale`, `priority`); it previously returned
nil for `default` and `scale`, so flipping a model to Responses silently
dropped a configured `service_tier` that the API accepts (fantasy
forwards the value unchanged). And a new
`TestProviderOptionsTransportParity` pins, per `provider_options.openai`
field, which transport honors it, against a table in ARCHITECTURE.md, so
a field honored on one transport and silently ignored on the other fails
the test unless recorded as intentional.

Review rounds also caught two lifecycle gaps around the new field.
`isZeroChatModelCallConfig` now inspects `OpenAIConfig`, so a stored
options blob whose only setting is this toggle survives into GET/list
responses instead of reading as `model_config: null`;
`TestIsZeroChatModelCallConfigCoversEveryField` sets each config field
in isolation and fails if any field is invisible to the zero check. And
the model editor's update path sends an explicit empty `model_config`
when an edit clears the last field, since an omitted property preserves
the stored options server-side; covered by the
`EditClearingLastOptionSendsEmptyConfig` story.

Azure keeps following the known-model list, because the Azure provider
exposes no equivalent hook. The model editor renders Azure with the
OpenAI option schema, so instead of shipping a visible but inert
control, the option schema generator gains a `providers` struct tag that
it emits as `visible_for_providers`. Gating uses the raw provider type
rather than the alias table, so the control appears only for
openai-typed providers. No hand-written frontend field: the editor
renders it from the generated schema.

Closes
https://linear.app/codercom/issue/CODAGT-874/add-completionsresponses-api-toggle-in-model-editor

> Mux prepared this PR on Mike's behalf.
This commit is contained in:
Michael Suchacz
2026-08-04 09:30:27 +02:00
committed by GitHub
parent 84cdc17602
commit c6cee10e8b
39 changed files with 1155 additions and 212 deletions
+7
View File
@@ -1511,9 +1511,16 @@ type ChatModelCallConfig struct {
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty" description:"Penalty for tokens based on their frequency in the output"`
Cost *ModelCostConfig `json:"cost,omitempty" description:"Optional pricing metadata for this model"`
ReasoningEffort *ChatModelReasoningEffortConfig `json:"reasoning_effort,omitempty" description:"Default and max reasoning effort for the model"`
OpenAIConfig *ChatModelOpenAIConfig `json:"openai_config,omitempty" description:"OpenAI client construction settings" providers:"openai"`
ProviderOptions *ChatModelProviderOptions `json:"provider_options,omitempty" description:"Provider-specific option overrides"`
}
// ChatModelOpenAIConfig holds settings applied once when the OpenAI client
// is built, not per request.
type ChatModelOpenAIConfig struct {
UseResponsesAPI *bool `json:"use_responses_api,omitempty" label:"Use Responses API" description:"Override which OpenAI API this model uses. Leave unset to decide from the provider SDK's known-model list, true to force the Responses API, false to force Chat Completions. Azure OpenAI providers ignore this and always follow the known-model list."`
}
// UnmarshalJSON accepts both the current nested cost object and the previous
// top-level pricing keys so legacy stored model_config JSON continues to load.
func (c *ChatModelCallConfig) UnmarshalJSON(data []byte) error {
+22
View File
@@ -584,6 +584,28 @@ func TestChatModelCallConfig_UnmarshalStrict(t *testing.T) {
require.NoError(t, json.Unmarshal([]byte(`{"bogus_setting": true}`), &decoded))
}
func TestChatModelCallConfig_UseResponsesAPIRoundTrip(t *testing.T) {
t.Parallel()
var decoded codersdk.ChatModelCallConfig
err := decoded.UnmarshalStrict([]byte(`{"openai_config": {"use_responses_api": true}}`))
require.NoError(t, err)
require.NotNil(t, decoded.OpenAIConfig)
require.NotNil(t, decoded.OpenAIConfig.UseResponsesAPI)
require.True(t, *decoded.OpenAIConfig.UseResponsesAPI)
raw, err := json.Marshal(decoded)
require.NoError(t, err)
require.Contains(t, string(raw), `"use_responses_api":true`)
var unset codersdk.ChatModelCallConfig
require.NoError(t, unset.UnmarshalStrict([]byte(`{"openai_config": {}}`)))
require.Nil(t, unset.OpenAIConfig.UseResponsesAPI)
raw, err = json.Marshal(unset)
require.NoError(t, err)
require.NotContains(t, string(raw), "use_responses_api")
}
func TestChatCostSummary_JSONRoundTrip(t *testing.T) {
t.Parallel()