mirror of
https://github.com/coder/coder.git
synced 2026-10-16 14:48:56 +08:00
fix: replay stateless OpenAI reasoning across chat turns (#30298)
OpenAI Responses chats default to `store=false`, and chatd only replayed reasoning as an `item_reference`, which needs provider-side storage. Reasoning was therefore dropped between tool steps and turns, so reasoning models lost their chain of thought mid-task. This pins coder/fantasy `coder_2_33` at 6ba8681, which includes fantasy #63 (and #64, #65, and #66). #63 captures each completed reasoning item (ID, `encrypted_content`, summary) and replays it in full regardless of `store`. In chatd: - Every Responses call requests `reasoning.encrypted_content`. The `store` default and configured includes are unchanged, and Chat Completions requests are untouched. - Prompt rebuilds keep reasoning parts without visible text when they carry finalized encrypted state, so reasoning survives durable history and coderd restarts. - The provider-switch sanitizer also strips OpenAI reasoning from history produced by a different or unknown provider instance, since reasoning IDs and encrypted content only resolve where they were issued. - A compaction model override on a different provider instance, or with a different model, gets a compaction prompt without the chat's OpenAI reasoning. - Each saved reasoning part records the provider instance, model, and credential (the deployment's or the user's personal API key) that produced it, and the sanitizer keeps it only when all three match the current turn, because another model, or the organization behind another credential, can reject the encrypted content. Switching models, adding or removing a personal key, or editing a config's model or provider drops the earlier reasoning; switching back restores it. The record is never returned by the API. Reasoning saved before this change has no record and follows its model config. - If OpenAI rejects replayed reasoning it cannot decrypt (`invalid_encrypted_content`), for example after the deployment's key is replaced with one from another organization, the request is retried once without reasoning, and the reasoning that step produces is saved with the provider's next rejection epoch. Epochs are kept separately for each provider, model, and credential. Later steps and turns leave out reasoning from earlier epochs, on any replica and after restarts, so only the first request after the change is rejected and newer reasoning keeps replaying. A retried step that returns no reasoning saves its epoch on its first part. The advisor's prompt never includes OpenAI reasoning, as on `main`. ## Testing Rebased onto main at98332c739bas 5735acb36ed (same tree as merging the pre-rebase head a378071862a with main; older SHAs below refer to the pre-rebase history). 3c17d807a33 then adds the model to the reasoning record, 21156b2b32e pins Fantasy d55a867, which treats a response that echoes `store: false` as unstored, bf795daa923 adds the retry and the advisor change, 7eb8b1d029a moves the pin to 88cffb5 (`coder_2_33` merged into #63; its OpenAI provider code is identical to d55a867), and f4f40734a41 pins `coder_2_33` at 6ba8681 once #63 merged. That adds fantasy #66, which replays the Responses message phase on assistant text. abccede81f7 makes the retry remember rejected reasoning in the process. The branch was then rebased onto main atabc2faed60, so every SHA above refers to the pre-rebase history.5fb3ce1689replaces that memory with the credential and a rejection epoch saved on each reasoning part, and62b5cb532akeeps epochs per provider, model, and credential and saves the epoch when the retried step returns no reasoning.1057beb327hides the four internal fields from the Swagger spec and API reference, and29a705e099merges main atd68b2c8b25. Its only conflicts were the fantasy pin (kept at 6ba8681, whose `coder_2_33` history includes main's 3510552) and test imports. - Tests in `chatprompt`, `chatprovider`, `chatsanitize`, the provider-switch sanitizer, and `integration_responses_test.go` cover tool-step and durable replay, legacy and incomplete metadata, provider switches, and a web search chat switching from `store=false` to `store=true`. Integration tests also cover the retry, a later tool step after it, the next turn on a fresh server against the same database (with and without reasoning in the retried step), a personal key added mid-chat, and the advisor prompt. Red-green verified, including a config whose model is edited in place. The compaction override's model check has unit tests and UAT coverage but no integration test. `go vet`, targeted race tests, and the pre-commit hook (gen, fmt, lint, build) pass. Local pre-push hooks were skipped; CI covers them. - Remote UAT on dogfood with real gpt-5-mini and gpt-5.4 behind a logging proxy that checks every request body. Rounds 1 and 2 failed on dangling item references (both fixed). Rounds 3 to 5 passed (ee3c0bf3d60, 0580870b5ff, a378071862a): tool-step and durable replay, restarts, interrupts, provider switches and config repoints, web search history, and compaction overrides. Both test providers shared one upstream account, so cross-provider checks cover request shape, not a real rejection. Round 6 at 21156b2b32e (model record) with gpt-5-mini and gpt-5-nano: PASS. 151 Responses requests, all 200; no request carried reasoning from a different model except an older, unrecorded part that follows its config by design. Same-model config switches, switching back, edits to a config's model, a mid-turn switch, restarts, and a same-provider compaction override with a different model all behaved as described. Round 7 at bf795daa923 (retry and advisor): PASS. Through AI Gateway, a corrupted saved blob is rejected with HTTP 400 `invalid_encrypted_content`. All 14 rejected requests were retried once without reasoning and returned 200, a different 400 was not retried, and advisor requests on the chat model, a same-provider override, and another provider carried no reasoning. - f4f40734a41 changes only the pin. Its new OpenAI code is fantasy #66, which UAT rounds 1 to 7 did not run. A live check of fantasy 6ba8681 through AI Gateway, two turns with tool calls on gpt-5.5 and gpt-5-mini with `store` false and true: all 18 requests returned 200, and the gpt-5.5 requests replayed encrypted reasoning followed by assistant messages with the `commentary` and `final_answer` phases. `go test ./coderd/x/chatd/... ./codersdk/` passes (21 packages) on the same fantasy tree. - UAT round 8 at abccede81f7 (remembered rejection, and fantasy #66 through chatd) with gpt-5.5, gpt-5-mini, and gpt-5-nano: PASS. 110 requests; the 6 rejections were 5 corrupted saved blobs, each retried once without reasoning and answered 200, and one unrelated `temperature` error that was not retried. After the first rejection, later steps and turns in that chat were never rejected again and replayed newer reasoning. A coderd restart cost one more rejection, which also dropped reasoning saved after the first one, and a second chat was unaffected. gpt-5.5 requests replayed assistant messages with the phases the model returned, with `store` false and true. - UAT round 9 at62b5cb532a(saved rejection epoch, personal key switch) with gpt-5.5, gpt-5-mini, and gpt-5-nano: PASS. 158 Responses and 7 Chat Completions requests; the 6 rejections were corrupted saved blobs, each retried once without reasoning and answered 200, and an unrelated `temperature` error was not retried. After a coderd restart, and on a second coderd against the same database, the chat was not rejected again and kept replaying the reasoning saved after the rejection. Adding a personal key left out the deployment key's reasoning before sending, and removing it brought that reasoning back. A retried step without reasoning, two chats rejected at once, compaction, a model switch after a rejection, and a switch to Chat Completions and back all behaved as described. ## Review record - Independent full-branch test audit atbbe1a1b27a: PASS. - Codex through a378071862: compaction overrides kept foreign reasoning and configs could be repointed to another provider (both fixed); a request for documentation TODOs and two behaviors that already exist on `main` (declined). - Codex on 5735acb36e: two findings. Reasoning replayed after a same-provider model switch: fixed in 3c17d807a33. A provider's base URL edited in place keeps its reasoning: covered by the retry in bf795daa923. - Codex on 21156b2b32, bf795daa92, and 7eb8b1d029: no major issues. Codex on f4f40734a4: rejected reasoning stayed in the chat's history, so each later request in that chat was rejected and retried again without any reasoning. Fixed in abccede81f7. Codex on abccede81f: that memory was lost on a restart or another replica, where the next rejection also dropped newer reasoning, and it grew without bound. Fixed in5fb3ce1689. Codex on5fb3ce1689: a rejection advanced the epoch for every model and credential on the provider, and a retried step without reasoning lost the new epoch. Both fixed in62b5cb532a. Codex on62b5cb532a: keep reasoning out of a config switched from Responses to Chat Completions (declined: the Chat Completions converter already skips reasoning parts, confirmed with a probe test). Codex on29a705e099: no major issues. - ethanndickson: retry without reasoning on `invalid_encrypted_content`, and keep OpenAI reasoning out of the advisor prompt. Both fixed in bf795daa923. Then: in-process memory is not enough because dogfood restarts on every merge and runs several replicas. Fixed in5fb3ce1689with his proposed rejection epoch in chat history, plus the credential record, and tested with the next turn on a fresh server. Then: hide the internal fields from Swagger, fixed in1057beb327. Approved on62b5cb532a. - coder/fantasy#63 review: honoring an echoed `store: false` is fixed in d55a867; replaying web searches that had no preceding reasoning is a pre-existing gap, tracked in #30528. - CI on 5735acb36ed: green after rerunning two failures outside this change (a `cli` `TestSSH` data race, and a `test-go-pg` job timeout that also fails on `main`). CI on 21156b2b32e: all checks pass. CI on bf795daa923: all checks pass. CI on 7eb8b1d029a: all checks pass. CI on f4f40734a41: all checks pass. CI on abccede81f7 did not run, because the branch conflicted with `main` (#30492 changed the provider-switch sanitizer to return database errors). CI on62b5cb532a: all checks pass. CI on29a705e099: all checks pass. > Xum, an AI coding agent, implemented, tested, and opened this PR on behalf of @ibetitsmike.
This commit is contained in:
@@ -1495,7 +1495,9 @@ func partsToMessageParts(
|
||||
})
|
||||
case codersdk.ChatMessagePartTypeReasoning:
|
||||
opts := providerMetadataToOptions(logger, part.ProviderMetadata)
|
||||
if strings.TrimSpace(part.Text) == "" && !chatsanitize.HasAnthropicSignedReasoningOptions(opts) {
|
||||
if strings.TrimSpace(part.Text) == "" &&
|
||||
!chatsanitize.HasAnthropicSignedReasoningOptions(opts) &&
|
||||
!chatsanitize.HasOpenAIReasoningOptions(opts) {
|
||||
continue
|
||||
}
|
||||
result = append(result, fantasy.ReasoningPart{
|
||||
@@ -1913,6 +1915,9 @@ var partNulFields = []partNulField{
|
||||
{name: "ToolCallID", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.ToolCallID }},
|
||||
{name: "ToolName", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.ToolName }},
|
||||
{name: "ParsedCommands", policy: nulReject, grid: func(p *codersdk.ChatMessagePart) [][]string { return p.ParsedCommands }},
|
||||
{name: "ProviderIdentity", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.ProviderIdentity }},
|
||||
{name: "ProviderModel", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.ProviderModel }},
|
||||
{name: "ProviderCredential", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.ProviderCredential }},
|
||||
{name: "SourceID", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.SourceID }},
|
||||
{name: "URL", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.URL }},
|
||||
{name: "MediaType", policy: nulReject, str: func(p *codersdk.ChatMessagePart) *string { return &p.MediaType }},
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"charm.land/fantasy"
|
||||
fantasyanthropic "charm.land/fantasy/providers/anthropic"
|
||||
fantasygoogle "charm.land/fantasy/providers/google"
|
||||
fantasyopenai "charm.land/fantasy/providers/openai"
|
||||
"github.com/google/uuid"
|
||||
"github.com/sqlc-dev/pqtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
@@ -39,42 +40,108 @@ func testMsg(role codersdk.ChatMessageRole, raw pqtype.NullRawMessage) database.
|
||||
}
|
||||
}
|
||||
|
||||
func TestConvertMessagesWithFilesPreservesEmptyRedactedReasoning(t *testing.T) {
|
||||
func TestConvertMessagesWithFilesPreservesEmptyReplayableReasoning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
metadata, err := json.Marshal(fantasy.ProviderMetadata{
|
||||
fantasyanthropic.Name: &fantasyanthropic.ReasoningOptionMetadata{
|
||||
RedactedData: "redacted-payload",
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
content, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
|
||||
encrypted := "encrypted-blob"
|
||||
tests := []struct {
|
||||
name string
|
||||
metadata fantasy.ProviderMetadata
|
||||
kept bool
|
||||
}{
|
||||
{
|
||||
Type: codersdk.ChatMessagePartTypeReasoning,
|
||||
ProviderMetadata: metadata,
|
||||
name: "AnthropicRedacted",
|
||||
metadata: fantasy.ProviderMetadata{
|
||||
fantasyanthropic.Name: &fantasyanthropic.ReasoningOptionMetadata{
|
||||
RedactedData: "redacted-payload",
|
||||
},
|
||||
},
|
||||
kept: true,
|
||||
},
|
||||
codersdk.ChatMessageText("done"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
prompt, err := chatprompt.ConvertMessagesWithFiles(context.Background(), []database.ChatMessage{
|
||||
{
|
||||
Role: database.ChatMessageRoleAssistant,
|
||||
Visibility: database.ChatMessageVisibilityBoth,
|
||||
Content: content,
|
||||
ContentVersion: chatprompt.CurrentContentVersion,
|
||||
name: "OpenAIFinalized",
|
||||
metadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{
|
||||
ItemID: "rs_1",
|
||||
EncryptedContent: &encrypted,
|
||||
Summary: []string{},
|
||||
Finalized: true,
|
||||
},
|
||||
},
|
||||
kept: true,
|
||||
},
|
||||
}, nil, slogtest.Make(t, nil), nil, uuid.NullUUID{})
|
||||
require.NoError(t, err)
|
||||
require.Len(t, prompt, 1)
|
||||
require.Len(t, prompt[0].Content, 2)
|
||||
{
|
||||
name: "OpenAIUnfinalized",
|
||||
metadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{
|
||||
ItemID: "rs_1",
|
||||
EncryptedContent: &encrypted,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "OpenAIFinalizedWithoutEncryptedContent",
|
||||
metadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{
|
||||
ItemID: "rs_1",
|
||||
Finalized: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "OpenAIFinalizedWithEmptyEncryptedContent",
|
||||
metadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{
|
||||
ItemID: "rs_1",
|
||||
EncryptedContent: new(""),
|
||||
Finalized: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "OpenAIWithoutItemID",
|
||||
metadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{
|
||||
EncryptedContent: &encrypted,
|
||||
Finalized: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
reasoning, ok := fantasy.AsMessagePart[fantasy.ReasoningPart](prompt[0].Content[0])
|
||||
require.True(t, ok)
|
||||
require.Empty(t, reasoning.Text)
|
||||
reasoningMetadata := fantasyanthropic.GetReasoningMetadata(reasoning.ProviderOptions)
|
||||
require.NotNil(t, reasoningMetadata)
|
||||
require.Equal(t, "redacted-payload", reasoningMetadata.RedactedData)
|
||||
content, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
|
||||
chatprompt.PartFromContent(fantasy.ReasoningContent{ProviderMetadata: tt.metadata}),
|
||||
codersdk.ChatMessageText("done"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
prompt, err := chatprompt.ConvertMessagesWithFiles(context.Background(), []database.ChatMessage{
|
||||
{
|
||||
Role: database.ChatMessageRoleAssistant,
|
||||
Visibility: database.ChatMessageVisibilityBoth,
|
||||
Content: content,
|
||||
ContentVersion: chatprompt.CurrentContentVersion,
|
||||
},
|
||||
}, nil, slogtest.Make(t, nil), nil, uuid.NullUUID{})
|
||||
require.NoError(t, err)
|
||||
require.Len(t, prompt, 1)
|
||||
if !tt.kept {
|
||||
require.Len(t, prompt[0].Content, 1)
|
||||
_, ok := fantasy.AsMessagePart[fantasy.TextPart](prompt[0].Content[0])
|
||||
require.True(t, ok)
|
||||
return
|
||||
}
|
||||
require.Len(t, prompt[0].Content, 2)
|
||||
|
||||
reasoning, ok := fantasy.AsMessagePart[fantasy.ReasoningPart](prompt[0].Content[0])
|
||||
require.True(t, ok)
|
||||
require.Empty(t, reasoning.Text)
|
||||
require.Equal(t, fantasy.ProviderOptions(tt.metadata), reasoning.ProviderOptions)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestConvertMessagesWithFilesRoundTripsAnthropicInterleavedWebSearch(t *testing.T) {
|
||||
|
||||
@@ -960,7 +960,27 @@ func ProviderOptionsForCall(
|
||||
) fantasy.ProviderOptions {
|
||||
options := providerOptionsFromChatModelConfig(model, config.ProviderOptions)
|
||||
effort := ResolveReasoningEffort(requestedEffort, config.ReasoningEffort)
|
||||
return applyReasoningEffort(model, options, effort)
|
||||
options = applyReasoningEffort(model, options, effort)
|
||||
return ensureOpenAIResponsesIncludes(model, options)
|
||||
}
|
||||
|
||||
// ensureOpenAIResponsesIncludes requests encrypted reasoning on every
|
||||
// Responses call, so reasoning can be replayed without provider-side storage.
|
||||
// An options block created here leaves Store unset, keeping the provider's
|
||||
// default.
|
||||
func ensureOpenAIResponsesIncludes(
|
||||
model Model,
|
||||
options fantasy.ProviderOptions,
|
||||
) fantasy.ProviderOptions {
|
||||
if !model.Valid() || !model.transport.UsesResponses() {
|
||||
return options
|
||||
}
|
||||
if options == nil {
|
||||
options = fantasy.ProviderOptions{}
|
||||
}
|
||||
responsesOptions := ensureProviderOptions[fantasyopenai.ResponsesProviderOptions](options, fantasyopenai.Name)
|
||||
responsesOptions.Include = chatopenai.EnsureResponseIncludes(responsesOptions.Include)
|
||||
return options
|
||||
}
|
||||
|
||||
func providerOptionsFromChatModelConfig(
|
||||
|
||||
@@ -312,3 +312,93 @@ func TestModelFromConfig_OpenAIReasoningModelOverride(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Encrypted reasoning is the only replayable reasoning state when responses
|
||||
// are not stored, so every Responses call must request it without changing
|
||||
// the configured or default store setting.
|
||||
func TestProviderOptionsForCall_OpenAIResponsesEncryptedReasoning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const encryptedReasoning = string(fantasyopenai.IncludeReasoningEncryptedContent)
|
||||
const fileSearchResults = string(fantasyopenai.IncludeFileSearchCallResults)
|
||||
effort := &codersdk.ChatModelReasoningEffortConfig{
|
||||
Default: new(codersdk.ChatModelReasoningEffortHigh),
|
||||
Max: new(codersdk.ChatModelReasoningEffortHigh),
|
||||
}
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
useResponses bool
|
||||
config codersdk.ChatModelCallConfig
|
||||
wantInclude []any
|
||||
wantStore any
|
||||
}{
|
||||
{
|
||||
name: "Default",
|
||||
useResponses: true,
|
||||
wantInclude: []any{encryptedReasoning},
|
||||
wantStore: false,
|
||||
},
|
||||
{
|
||||
name: "EffortOnly",
|
||||
useResponses: true,
|
||||
config: codersdk.ChatModelCallConfig{ReasoningEffort: effort},
|
||||
wantInclude: []any{encryptedReasoning},
|
||||
wantStore: false,
|
||||
},
|
||||
{
|
||||
name: "ConfiguredStoreAndInclude",
|
||||
useResponses: true,
|
||||
config: codersdk.ChatModelCallConfig{
|
||||
ReasoningEffort: effort,
|
||||
ProviderOptions: &codersdk.ChatModelProviderOptions{
|
||||
OpenAI: &codersdk.ChatModelOpenAIProviderOptions{
|
||||
Store: new(true),
|
||||
Include: []string{fileSearchResults},
|
||||
},
|
||||
},
|
||||
},
|
||||
wantInclude: []any{fileSearchResults, encryptedReasoning},
|
||||
wantStore: true,
|
||||
},
|
||||
{
|
||||
name: "ChatCompletions",
|
||||
config: codersdk.ChatModelCallConfig{ReasoningEffort: effort},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
bodies := make(chan []byte, 1)
|
||||
serverURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
bodies <- req.RawBody
|
||||
return chattest.OpenAINonStreamingResponse("ok")
|
||||
})
|
||||
model, err := chatprovider.ModelFromConfig(
|
||||
fantasyopenai.Name, "gpt-5",
|
||||
chatprovider.ProviderAPIKeys{
|
||||
ByProvider: map[string]string{fantasyopenai.Name: "test-key"},
|
||||
BaseURLByProvider: map[string]string{fantasyopenai.Name: serverURL},
|
||||
},
|
||||
chatprovider.UserAgent(), nil, nil,
|
||||
&codersdk.ChatModelCallConfig{OpenAIConfig: &codersdk.ChatModelOpenAIConfig{UseResponsesAPI: &tc.useResponses}},
|
||||
)
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = model.LanguageModel().Generate(t.Context(), fantasy.Call{
|
||||
Prompt: []fantasy.Message{{Role: fantasy.MessageRoleUser, Content: []fantasy.MessagePart{fantasy.TextPart{Text: "hello"}}}},
|
||||
ProviderOptions: chatprovider.ProviderOptionsForCall(model, tc.config, nil),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
var body map[string]any
|
||||
require.NoError(t, json.Unmarshal(<-bodies, &body))
|
||||
if !tc.useResponses {
|
||||
require.NotContains(t, body, "include")
|
||||
return
|
||||
}
|
||||
require.Equal(t, tc.wantInclude, body["include"])
|
||||
require.Equal(t, tc.wantStore, body["store"])
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
package chatsanitize
|
||||
|
||||
import (
|
||||
"charm.land/fantasy"
|
||||
fantasyopenai "charm.land/fantasy/providers/openai"
|
||||
)
|
||||
|
||||
// HasOpenAIReasoningOptions reports whether options carry finalized encrypted
|
||||
// reasoning that can be replayed without server-side storage.
|
||||
func HasOpenAIReasoningOptions(options fantasy.ProviderOptions) bool {
|
||||
metadata := fantasyopenai.GetReasoningMetadata(options)
|
||||
return metadata != nil && metadata.ItemID != "" && metadata.Finalized &&
|
||||
metadata.EncryptedContent != nil && *metadata.EncryptedContent != ""
|
||||
}
|
||||
|
||||
// HasOpenAIReasoningState reports whether options carry an OpenAI reasoning
|
||||
// item. Its item ID and encrypted content only resolve on the provider
|
||||
// instance that issued them.
|
||||
func HasOpenAIReasoningState(options fantasy.ProviderOptions) bool {
|
||||
metadata := fantasyopenai.GetReasoningMetadata(options)
|
||||
return metadata != nil && metadata.ItemID != ""
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
type ErrorResponse struct {
|
||||
StatusCode int
|
||||
Type string
|
||||
Code string
|
||||
Message string
|
||||
}
|
||||
|
||||
@@ -19,12 +20,14 @@ type ErrorResponse struct {
|
||||
func writeErrorResponse(t testing.TB, w http.ResponseWriter, errResp *ErrorResponse) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(errResp.StatusCode)
|
||||
body := map[string]interface{}{
|
||||
"error": map[string]interface{}{
|
||||
"type": errResp.Type,
|
||||
"message": errResp.Message,
|
||||
},
|
||||
errorBody := map[string]interface{}{
|
||||
"type": errResp.Type,
|
||||
"message": errResp.Message,
|
||||
}
|
||||
if errResp.Code != "" {
|
||||
errorBody["code"] = errResp.Code
|
||||
}
|
||||
body := map[string]interface{}{"error": errorBody}
|
||||
if err := json.NewEncoder(w).Encode(body); err != nil {
|
||||
t.Errorf("writeErrorResponse: failed to encode error response: %v", err)
|
||||
}
|
||||
|
||||
@@ -21,7 +21,8 @@ func sameCompactionProviderIdentity(chatConfig, overrideConfig database.ChatMode
|
||||
}
|
||||
|
||||
// sanitizeCompactionPrompt adapts a prompt built for the chat model to a
|
||||
// differing compaction model. The input messages are never mutated; the
|
||||
// differing compaction model. reasoningModel is the model whose OpenAI
|
||||
// reasoning the prompt may carry. The input messages are never mutated; the
|
||||
// assistant generation keeps using the original prompt.
|
||||
func sanitizeCompactionPrompt(
|
||||
ctx context.Context,
|
||||
@@ -31,11 +32,16 @@ func sanitizeCompactionPrompt(
|
||||
configuredProvider string,
|
||||
chatConfig database.ChatModelConfig,
|
||||
overrideConfig database.ChatModelConfig,
|
||||
reasoningModel string,
|
||||
) []fantasy.Message {
|
||||
messages := prompt
|
||||
if !sameCompactionProviderIdentity(chatConfig, overrideConfig) {
|
||||
sameProvider := sameCompactionProviderIdentity(chatConfig, overrideConfig)
|
||||
if !sameProvider {
|
||||
messages = flattenProviderExecutedToolParts(ctx, logger, messages)
|
||||
}
|
||||
if !sameProvider || compactionModel.ModelID() != reasoningModel {
|
||||
messages = dropOpenAIReasoningParts(ctx, logger, messages)
|
||||
}
|
||||
messages = replaceUnsupportedFileParts(ctx, logger, messages, compactionModel.AcceptsFilePartMediaType)
|
||||
messages = replaceUnsupportedToolMedia(ctx, logger, messages, compactionModel, configuredProvider)
|
||||
sanitized, stats := chatsanitize.SanitizeAnthropicProviderToolHistory(
|
||||
@@ -121,6 +127,40 @@ func flattenProviderExecutedToolParts(
|
||||
return out
|
||||
}
|
||||
|
||||
// dropOpenAIReasoningParts removes OpenAI reasoning parts from a copy of
|
||||
// messages, since another provider cannot resolve their item IDs and another
|
||||
// model or organization can reject their encrypted content. Messages emptied
|
||||
// by the drop are removed.
|
||||
func dropOpenAIReasoningParts(
|
||||
ctx context.Context,
|
||||
logger slog.Logger,
|
||||
messages []fantasy.Message,
|
||||
) []fantasy.Message {
|
||||
dropped := 0
|
||||
out := make([]fantasy.Message, 0, len(messages))
|
||||
for _, msg := range messages {
|
||||
parts := make([]fantasy.MessagePart, 0, len(msg.Content))
|
||||
for _, part := range msg.Content {
|
||||
if reasoning, ok := part.(fantasy.ReasoningPart); ok && chatsanitize.HasOpenAIReasoningState(reasoning.ProviderOptions) {
|
||||
dropped++
|
||||
continue
|
||||
}
|
||||
parts = append(parts, part)
|
||||
}
|
||||
if len(parts) == 0 && len(msg.Content) > 0 {
|
||||
continue
|
||||
}
|
||||
msg.Content = parts
|
||||
out = append(out, msg)
|
||||
}
|
||||
if dropped > 0 {
|
||||
logger.Debug(ctx, "dropped OpenAI reasoning from prompt",
|
||||
slog.F("dropped_parts", dropped),
|
||||
)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// stringifyToolResultOutput renders a tool result as prompt text. Media
|
||||
// payloads are summarized so base64 data does not enter the prompt.
|
||||
func stringifyToolResultOutput(output fantasy.ToolResultOutputContent) string {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"testing"
|
||||
|
||||
"charm.land/fantasy"
|
||||
fantasyopenai "charm.land/fantasy/providers/openai"
|
||||
"github.com/google/uuid"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
@@ -72,7 +73,7 @@ func TestSanitizeCompactionPrompt_FlattensForeignProviderExecutedToolParts(t *te
|
||||
}
|
||||
|
||||
compactionModel := chatprovider.NewModel(&chattest.FakeModel{ProviderName: "openai", ModelName: "gpt-4.1-mini"}, nil)
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(uuid.New()), configWithProvider(uuid.New()))
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(uuid.New()), configWithProvider(uuid.New()), compactionModel.ModelID())
|
||||
|
||||
require.Len(t, sanitized, 3)
|
||||
// Provider-executed parts are flattened to text so the summary keeps
|
||||
@@ -94,6 +95,61 @@ func TestSanitizeCompactionPrompt_FlattensForeignProviderExecutedToolParts(t *te
|
||||
}, prompt[1].Content[1])
|
||||
}
|
||||
|
||||
func TestSanitizeCompactionPrompt_OpenAIReasoning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
encrypted := "encrypted-blob"
|
||||
finalized := fantasy.ReasoningPart{ProviderOptions: fantasy.ProviderOptions{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{ItemID: "rs_final", EncryptedContent: &encrypted, Finalized: true},
|
||||
}}
|
||||
legacy := fantasy.ReasoningPart{Text: "summary", ProviderOptions: fantasy.ProviderOptions{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{ItemID: "rs_legacy", Summary: []string{"summary"}},
|
||||
}}
|
||||
prompt := []fantasy.Message{
|
||||
{Role: fantasy.MessageRoleUser, Content: []fantasy.MessagePart{fantasy.TextPart{Text: "hi"}}},
|
||||
{Role: fantasy.MessageRoleAssistant, Content: []fantasy.MessagePart{finalized, fantasy.TextPart{Text: "answer"}}},
|
||||
{Role: fantasy.MessageRoleAssistant, Content: []fantasy.MessagePart{legacy}},
|
||||
}
|
||||
compactionModel := chatprovider.NewModel(&chattest.FakeModel{ProviderName: "openai", ModelName: "gpt-5-mini"}, nil)
|
||||
|
||||
t.Run("ForeignProviderDropsReasoning", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx := testutil.Context(t, testutil.WaitShort)
|
||||
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true})
|
||||
|
||||
// Reasoning IDs and encrypted content only resolve on the provider
|
||||
// instance that issued them.
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(uuid.New()), configWithProvider(uuid.New()), compactionModel.ModelID())
|
||||
|
||||
require.Len(t, sanitized, 2)
|
||||
require.Equal(t, []fantasy.MessagePart{fantasy.TextPart{Text: "answer"}}, sanitized[1].Content)
|
||||
require.Equal(t, finalized, prompt[1].Content[0])
|
||||
})
|
||||
|
||||
t.Run("SameModelKeepsReasoning", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx := testutil.Context(t, testutil.WaitShort)
|
||||
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true})
|
||||
sharedProviderID := uuid.New()
|
||||
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(sharedProviderID), configWithProvider(sharedProviderID), compactionModel.ModelID())
|
||||
|
||||
require.Equal(t, prompt, sanitized)
|
||||
})
|
||||
|
||||
t.Run("SameProviderDifferentModelDropsReasoning", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
ctx := testutil.Context(t, testutil.WaitShort)
|
||||
logger := slogtest.Make(t, &slogtest.Options{IgnoreErrors: true})
|
||||
sharedProviderID := uuid.New()
|
||||
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(sharedProviderID), configWithProvider(sharedProviderID), "gpt-5.4")
|
||||
|
||||
require.Len(t, sanitized, 2)
|
||||
require.Equal(t, []fantasy.MessagePart{fantasy.TextPart{Text: "answer"}}, sanitized[1].Content)
|
||||
})
|
||||
}
|
||||
|
||||
func TestSanitizeCompactionPrompt_DropsNonAssistantProviderExecutedParts(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -121,7 +177,7 @@ func TestSanitizeCompactionPrompt_DropsNonAssistantProviderExecutedParts(t *test
|
||||
}
|
||||
|
||||
compactionModel := chatprovider.NewModel(&chattest.FakeModel{ProviderName: "openai", ModelName: "gpt-4.1-mini"}, nil)
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(uuid.New()), configWithProvider(uuid.New()))
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(uuid.New()), configWithProvider(uuid.New()), compactionModel.ModelID())
|
||||
|
||||
require.Len(t, sanitized, 1)
|
||||
require.Equal(t, fantasy.MessageRoleUser, sanitized[0].Role)
|
||||
@@ -151,7 +207,7 @@ func TestSanitizeCompactionPrompt_ReplacesUnsupportedFileParts(t *testing.T) {
|
||||
// placeholder while the prompt stays otherwise intact.
|
||||
compactionModel := chatprovider.NewModel(&chattest.FakeModel{ProviderName: "mistral", ModelName: "mistral-large"}, nil)
|
||||
sharedProviderID := uuid.New()
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(sharedProviderID), configWithProvider(sharedProviderID))
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(sharedProviderID), configWithProvider(sharedProviderID), compactionModel.ModelID())
|
||||
|
||||
require.Len(t, sanitized, 1)
|
||||
require.Len(t, sanitized[0].Content, 2)
|
||||
@@ -182,7 +238,7 @@ func TestSanitizeCompactionPrompt_ReplacesUnsupportedToolMedia(t *testing.T) {
|
||||
// not, and the filter must run even when both models share a provider.
|
||||
compactionModel := chatprovider.NewModel(&chattest.FakeModel{ProviderName: "openai-compat", ModelName: "model"}, nil)
|
||||
sharedProviderID := uuid.New()
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, "google", configWithProvider(sharedProviderID), configWithProvider(sharedProviderID))
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, "google", configWithProvider(sharedProviderID), configWithProvider(sharedProviderID), compactionModel.ModelID())
|
||||
|
||||
require.Len(t, sanitized, 1)
|
||||
result, ok := sanitized[0].Content[0].(fantasy.ToolResultPart)
|
||||
@@ -218,7 +274,7 @@ func TestSanitizeCompactionPrompt_SameProviderKeepsProviderExecutedParts(t *test
|
||||
|
||||
compactionModel := chatprovider.NewModel(&chattest.FakeModel{ProviderName: "openai", ModelName: "gpt-4.1-mini"}, nil)
|
||||
sharedProviderID := uuid.New()
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(sharedProviderID), configWithProvider(sharedProviderID))
|
||||
sanitized := sanitizeCompactionPrompt(ctx, logger, prompt, compactionModel, compactionModel.Provider(), configWithProvider(sharedProviderID), configWithProvider(sharedProviderID), compactionModel.ModelID())
|
||||
|
||||
require.Len(t, sanitized, 1)
|
||||
require.Len(t, sanitized[0].Content, 2)
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
package chatd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync/atomic"
|
||||
|
||||
"charm.land/fantasy"
|
||||
"github.com/google/uuid"
|
||||
"github.com/openai/openai-go/v3"
|
||||
|
||||
"cdr.dev/slog/v3"
|
||||
"github.com/coder/coder/v2/coderd/x/chatd/chatprovider"
|
||||
)
|
||||
|
||||
func isInvalidEncryptedContentError(err error) bool {
|
||||
var apiErr *openai.Error
|
||||
return errors.As(err, &apiErr) && apiErr.Code == "invalid_encrypted_content"
|
||||
}
|
||||
|
||||
// withEncryptedReasoningFallback wraps OpenAI Responses models in an
|
||||
// encryptedReasoningFallbackModel that sets rejected when it retries.
|
||||
func (p *Server) withEncryptedReasoningFallback(model chatprovider.Model, chatID uuid.UUID, rejected *atomic.Bool) chatprovider.Model {
|
||||
if !model.Transport().UsesResponses() {
|
||||
return model
|
||||
}
|
||||
return model.WithLanguageModel(&encryptedReasoningFallbackModel{
|
||||
LanguageModel: model.LanguageModel(),
|
||||
logger: p.logger.With(slog.F("chat_id", chatID)),
|
||||
rejected: rejected,
|
||||
})
|
||||
}
|
||||
|
||||
// encryptedReasoningFallbackModel retries a request without OpenAI reasoning
|
||||
// when the provider rejects its encrypted content. Encrypted reasoning only
|
||||
// decrypts for the OpenAI organization that produced it, which can change
|
||||
// without a provider, model, or credential change, for example when an admin
|
||||
// replaces the deployment key with one from another organization. The
|
||||
// rejected reasoning stays in the chat history, so the step stamps its new
|
||||
// reasoning with the next epoch, and later steps leave out reasoning from
|
||||
// earlier epochs.
|
||||
type encryptedReasoningFallbackModel struct {
|
||||
fantasy.LanguageModel
|
||||
logger slog.Logger
|
||||
rejected *atomic.Bool
|
||||
}
|
||||
|
||||
// Stream retries only when the rejection arrives before any output.
|
||||
func (m *encryptedReasoningFallbackModel) Stream(ctx context.Context, call fantasy.Call) (fantasy.StreamResponse, error) {
|
||||
stream, err := m.LanguageModel.Stream(ctx, call)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return func(yield func(fantasy.StreamPart) bool) {
|
||||
var rejection error
|
||||
started := false
|
||||
for part := range stream {
|
||||
if !started && part.Type == fantasy.StreamPartTypeError && isInvalidEncryptedContentError(part.Error) {
|
||||
rejection = part.Error
|
||||
break
|
||||
}
|
||||
started = started || part.Type != fantasy.StreamPartTypeWarnings
|
||||
if !yield(part) {
|
||||
return
|
||||
}
|
||||
}
|
||||
if rejection == nil {
|
||||
return
|
||||
}
|
||||
m.rejected.Store(true)
|
||||
m.logger.Warn(ctx, "retrying model call without rejected encrypted reasoning", slogError(rejection))
|
||||
call.Prompt = dropOpenAIReasoningParts(ctx, m.logger, call.Prompt)
|
||||
retry, err := m.LanguageModel.Stream(ctx, call)
|
||||
if err != nil {
|
||||
yield(fantasy.StreamPart{Type: fantasy.StreamPartTypeError, Error: err})
|
||||
return
|
||||
}
|
||||
for part := range retry {
|
||||
if !yield(part) {
|
||||
return
|
||||
}
|
||||
}
|
||||
}, nil
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"charm.land/fantasy"
|
||||
@@ -71,7 +72,11 @@ type generationPrepared struct {
|
||||
|
||||
// ResolvedProvider is the configured provider identity used to label
|
||||
// user-facing errors. See chatloop.GenerateAssistantOptions.ErrorProvider.
|
||||
ResolvedProvider string
|
||||
ResolvedProvider string
|
||||
ReasoningProvenance reasoningProvenance
|
||||
// ReasoningRejected reports whether the provider rejected the
|
||||
// encrypted reasoning replayed into this step's model call.
|
||||
ReasoningRejected *atomic.Bool
|
||||
|
||||
StageModel chatloop.StageModel
|
||||
|
||||
@@ -93,6 +98,17 @@ type generationPrepared struct {
|
||||
Debug *generationDebug
|
||||
}
|
||||
|
||||
// stepReasoningProvenance returns the provenance to stamp on the step's
|
||||
// reasoning. A rejection starts the provider's next epoch, so later steps
|
||||
// leave out the reasoning the rejected call replayed.
|
||||
func (p generationPrepared) stepReasoningProvenance() reasoningProvenance {
|
||||
provenance := p.ReasoningProvenance
|
||||
if p.ReasoningRejected != nil && p.ReasoningRejected.Load() {
|
||||
provenance.Epoch++
|
||||
}
|
||||
return provenance
|
||||
}
|
||||
|
||||
// generationCompaction contains compaction inputs prepared for generation.
|
||||
type generationCompaction struct {
|
||||
// Override, when non-nil, is the compaction model override resolved at
|
||||
@@ -966,6 +982,7 @@ func (s *taskStarter) generateAssistant(
|
||||
logger: s.opts.Logger,
|
||||
contentVersion: chatprompt.CurrentContentVersion,
|
||||
hookRewrittenToolCalls: preflight.Overrides,
|
||||
reasoningProvenance: prepared.stepReasoningProvenance(),
|
||||
})
|
||||
if err != nil {
|
||||
return s.finishGenerationError(ctx, machine, input, err, requireGenerationAttempt(attempt.number))
|
||||
@@ -1195,11 +1212,12 @@ func (s *taskStarter) executeLocalTools(
|
||||
chathooks.RestoreToolCallOrder(outcome.Content, decision.localToolCalls)
|
||||
step := stepDataFromPersisted(outcome)
|
||||
messages, err := buildCommitStepMessages(buildCommitStepMessagesInput{
|
||||
modelConfigID: prepared.ModelConfigID,
|
||||
step: step,
|
||||
toolNameToConfigID: prepared.ToolNameToConfigID,
|
||||
logger: s.opts.Logger,
|
||||
contentVersion: chatprompt.CurrentContentVersion,
|
||||
modelConfigID: prepared.ModelConfigID,
|
||||
step: step,
|
||||
toolNameToConfigID: prepared.ToolNameToConfigID,
|
||||
logger: s.opts.Logger,
|
||||
contentVersion: chatprompt.CurrentContentVersion,
|
||||
reasoningProvenance: prepared.stepReasoningProvenance(),
|
||||
})
|
||||
if err != nil {
|
||||
return s.finishGenerationError(ctx, machine, input, err, requireGenerationAttempt(attempt.number))
|
||||
@@ -1297,6 +1315,7 @@ func (s *taskStarter) generateCompaction(
|
||||
overrideModel.resolvedProvider,
|
||||
prepared.Compaction.ChatModelConfig,
|
||||
overrideModel.dbConfig,
|
||||
prepared.ReasoningProvenance.Model,
|
||||
)
|
||||
}
|
||||
preResult, err := s.server.hooks.Trigger(ctx, chathooks.ChatFor(prepared.Chat, input.hookTurnID()), chathooks.Message{}, agenthooks.EventPreCompact, dispatch.CapacityClassGeneration)
|
||||
|
||||
@@ -138,6 +138,7 @@ func (server *Server) prepareGeneration(
|
||||
// The chat config keeps driving compaction, sanitization, and debug
|
||||
// attribution even when computer use swaps the resolved call below.
|
||||
modelConfig := resolved.dbConfig
|
||||
providerIdentity := modelConfigProviderIdentity(modelConfig, chatprovider.NormalizeProvider(string(resolved.route.Provider.Type)))
|
||||
|
||||
// Computer-use turns swap in a specialized model, so the substitution
|
||||
// must happen before anything model-sensitive runs: file-part
|
||||
@@ -174,6 +175,11 @@ func (server *Server) prepareGeneration(
|
||||
}
|
||||
model := resolved.model
|
||||
callConfig := resolved.callConfig
|
||||
reasoningSource := reasoningProvenance{
|
||||
ProviderIdentity: providerIdentity,
|
||||
Model: model.ModelID(),
|
||||
Credential: reasoningCredential(resolved.route.ProviderAuth),
|
||||
}
|
||||
modelRoute := resolved.route
|
||||
|
||||
currentPlanMode := chat.PlanMode
|
||||
@@ -232,7 +238,10 @@ func (server *Server) prepareGeneration(
|
||||
if advisorRuntime == nil {
|
||||
return
|
||||
}
|
||||
advisorPromptSnapshot = slices.Clone(msgs)
|
||||
// The advisor reads the transcript as text. Replayed OpenAI
|
||||
// reasoning can be foreign to the advisor model, and textualized
|
||||
// tool calls would leave it without its required following item.
|
||||
advisorPromptSnapshot = dropOpenAIReasoningParts(ctx, logger, msgs)
|
||||
}
|
||||
|
||||
currentChat := chat
|
||||
@@ -324,11 +333,13 @@ func (server *Server) prepareGeneration(
|
||||
planPathBlock string
|
||||
)
|
||||
|
||||
// Drop provider-executed tool history produced by a different provider
|
||||
// before building the prompt. A provider that shares another's wire format
|
||||
// (e.g. Bedrock and Anthropic) can still reject the other's
|
||||
// provider-executed blocks, so a mid-chat provider switch must not replay
|
||||
// them.
|
||||
// Drop provider-executed tool history produced by a different provider,
|
||||
// and OpenAI reasoning state produced by a different provider, model,
|
||||
// or credential, or before the latest rejection of that source's
|
||||
// encrypted reasoning, before building the prompt. A provider
|
||||
// that shares another's wire format (e.g. Bedrock and Anthropic) can
|
||||
// still reject the other's provider-specific blocks, so a mid-chat
|
||||
// provider switch must not replay them.
|
||||
//
|
||||
// The pending-user segment (trailing user rows the assistant has not
|
||||
// answered yet) is handled separately from here through prompt
|
||||
@@ -341,7 +352,8 @@ func (server *Server) prepareGeneration(
|
||||
// sanitizing everything, since the tail is user-only and the
|
||||
// sanitizer only rewrites assistant rows.
|
||||
pendingRowsStart := pendingUserSegmentStart(promptRows)
|
||||
sanitizedHead, err := server.sanitizeForeignProviderExecutedToolRows(ctx, logger, promptRows[:pendingRowsStart], chat.OwnerID, modelConfig.ID)
|
||||
reasoningSource.Epoch = currentReasoningEpoch(promptRows[:pendingRowsStart], reasoningSource)
|
||||
sanitizedHead, err := server.sanitizeForeignProviderStateRows(ctx, logger, promptRows[:pendingRowsStart], chat.OwnerID, reasoningSource)
|
||||
if err != nil {
|
||||
return generationPrepared{}, err
|
||||
}
|
||||
@@ -906,6 +918,8 @@ func (server *Server) prepareGeneration(
|
||||
ProviderTools: providerTools,
|
||||
ModelBuildOptions: modelOpts,
|
||||
ResolvedProvider: resolved.resolvedProvider,
|
||||
ReasoningProvenance: reasoningSource,
|
||||
ReasoningRejected: resolved.reasoningRejected,
|
||||
StageModel: resolved.stageModel(),
|
||||
ModelConfigID: modelConfig.ID,
|
||||
CallTemplate: resolved.newCall(),
|
||||
|
||||
@@ -3,7 +3,10 @@ package chatd_test
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
@@ -14,6 +17,7 @@ import (
|
||||
|
||||
"github.com/coder/coder/v2/coderd/database"
|
||||
"github.com/coder/coder/v2/coderd/database/db2sdk"
|
||||
"github.com/coder/coder/v2/coderd/database/dbgen"
|
||||
"github.com/coder/coder/v2/coderd/database/dbtestutil"
|
||||
dbpubsub "github.com/coder/coder/v2/coderd/database/pubsub"
|
||||
"github.com/coder/coder/v2/coderd/x/chatd"
|
||||
@@ -25,47 +29,144 @@ import (
|
||||
func TestOpenAIResponsesNoStaleWebSearchReplay(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
destinationStore bool
|
||||
summary string
|
||||
}{
|
||||
{name: "stateless", summary: "checked provider-side search state"},
|
||||
{name: "switch to stored", destinationStore: true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db, ps := dbtestutil.NewDB(t)
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
|
||||
const (
|
||||
reasoningID = "rs_no_stale_reasoning"
|
||||
webSearchID = "ws_no_stale_search"
|
||||
)
|
||||
var recorder responsesRequestRecorder
|
||||
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
if !req.Stream {
|
||||
return chattest.OpenAINonStreamingResponse("title")
|
||||
}
|
||||
|
||||
requestNumber := recorder.record(req)
|
||||
switch requestNumber {
|
||||
case 1:
|
||||
resp := chattest.OpenAIStreamingResponse(
|
||||
chattest.OpenAITextChunks("search result summary")...,
|
||||
)
|
||||
resp.ResponseID = "resp_no_stale_first"
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{
|
||||
ID: reasoningID,
|
||||
Summary: tc.summary,
|
||||
EncryptedContent: "encrypted-no-stale",
|
||||
}
|
||||
resp.WebSearch = &chattest.OpenAIWebSearchCall{
|
||||
ID: webSearchID,
|
||||
Query: "coder changelog",
|
||||
}
|
||||
return resp
|
||||
default:
|
||||
resp := chattest.OpenAIStreamingResponse(
|
||||
chattest.OpenAITextChunks("follow-up answer")...,
|
||||
)
|
||||
resp.ResponseID = "resp_no_stale_second"
|
||||
return resp
|
||||
}
|
||||
})
|
||||
|
||||
user, org, _ := seedChatDependenciesWithProvider(t, db, "openai", openAIURL)
|
||||
model := insertOpenAIResponsesModelConfig(t, db, user.ID, false, true)
|
||||
followupModel := model
|
||||
if tc.destinationStore {
|
||||
followupModel = insertOpenAIResponsesModelConfig(t, db, user.ID, true, false)
|
||||
}
|
||||
factory := chattest.NewMockAIBridgeTransport(t, openAIURL)
|
||||
server := newOpenAIResponsesTestServer(t, db, ps, func(cfg *chatd.Config) {
|
||||
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(factory)
|
||||
})
|
||||
|
||||
chat, err := server.CreateChat(ctx, chatd.CreateOptions{
|
||||
OrganizationID: org.ID,
|
||||
OwnerID: user.ID,
|
||||
Title: uniqueResponsesTitle(t, "no-stale"),
|
||||
ModelConfigID: model.ID,
|
||||
InitialUserContent: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("search for the latest Coder docs"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
require.Len(t, recorder.all(), 1)
|
||||
|
||||
_, err = server.SendMessage(ctx, chatd.SendMessageOptions{
|
||||
ChatID: chat.ID,
|
||||
CreatedBy: user.ID,
|
||||
ModelConfigID: followupModel.ID,
|
||||
Content: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("summarize the result without searching again"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
|
||||
requests := recorder.all()
|
||||
require.Len(t, requests, 2)
|
||||
followup := requests[1]
|
||||
require.NotNil(t, followup.Store)
|
||||
require.Equal(t, tc.destinationStore, *followup.Store)
|
||||
require.NotEmpty(t, followup.Prompt)
|
||||
requireNoResponsesProviderItemReplay(t, followup.Prompt, webSearchID)
|
||||
require.NotContains(t, promptItemTypes(followup.Prompt), "web_search_call")
|
||||
requireInlineReasoningItem(t, followup.Prompt, reasoningID, "encrypted-no-stale",
|
||||
tc.summary)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAIResponsesStatelessReasoningReplay(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db, ps := dbtestutil.NewDB(t)
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
|
||||
const (
|
||||
reasoningID = "rs_no_stale_reasoning"
|
||||
webSearchID = "ws_no_stale_search"
|
||||
reasoningID = "rs_stateless_reasoning"
|
||||
blob = "encrypted-stateless"
|
||||
)
|
||||
var recorder responsesRequestRecorder
|
||||
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
if !req.Stream {
|
||||
return chattest.OpenAINonStreamingResponse("title")
|
||||
}
|
||||
|
||||
requestNumber := recorder.record(req)
|
||||
switch requestNumber {
|
||||
switch recorder.record(req) {
|
||||
case 1:
|
||||
resp := chattest.OpenAIStreamingResponse(
|
||||
chattest.OpenAITextChunks("search result summary")...,
|
||||
chattest.OpenAIToolCallChunk("list_templates", `{}`),
|
||||
)
|
||||
resp.ResponseID = "resp_no_stale_first"
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{
|
||||
ID: reasoningID,
|
||||
Summary: "checked provider-side search state",
|
||||
EncryptedContent: "encrypted-no-stale",
|
||||
}
|
||||
resp.WebSearch = &chattest.OpenAIWebSearchCall{
|
||||
ID: webSearchID,
|
||||
Query: "coder changelog",
|
||||
EncryptedContent: blob,
|
||||
}
|
||||
return resp
|
||||
default:
|
||||
resp := chattest.OpenAIStreamingResponse(
|
||||
chattest.OpenAITextChunks("follow-up answer")...,
|
||||
return chattest.OpenAIStreamingResponse(
|
||||
chattest.OpenAITextChunks("done")...,
|
||||
)
|
||||
resp.ResponseID = "resp_no_stale_second"
|
||||
return resp
|
||||
}
|
||||
})
|
||||
|
||||
user, org, _ := seedChatDependenciesWithProvider(t, db, "openai", openAIURL)
|
||||
model := insertOpenAIResponsesModelConfig(t, db, user.ID, false, true)
|
||||
// An empty call config is the default deployment shape: no stored
|
||||
// responses and no requested reasoning summary.
|
||||
model := insertChatModelConfigWithCallConfig(t, db, user.ID, "openai", "gpt-4o",
|
||||
codersdk.ChatModelCallConfig{})
|
||||
factory := chattest.NewMockAIBridgeTransport(t, openAIURL)
|
||||
server := newOpenAIResponsesTestServer(t, db, ps, func(cfg *chatd.Config) {
|
||||
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(factory)
|
||||
@@ -74,23 +175,23 @@ func TestOpenAIResponsesNoStaleWebSearchReplay(t *testing.T) {
|
||||
chat, err := server.CreateChat(ctx, chatd.CreateOptions{
|
||||
OrganizationID: org.ID,
|
||||
OwnerID: user.ID,
|
||||
Title: uniqueResponsesTitle(t, "no-stale"),
|
||||
Title: uniqueResponsesTitle(t, "stateless-reasoning"),
|
||||
ModelConfigID: model.ID,
|
||||
InitialUserContent: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("search for the latest Coder docs"),
|
||||
codersdk.ChatMessageText("list the templates"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
require.Len(t, recorder.all(), 1)
|
||||
require.Len(t, recorder.all(), 2)
|
||||
|
||||
_, err = server.SendMessage(ctx, chatd.SendMessageOptions{
|
||||
ChatID: chat.ID,
|
||||
CreatedBy: user.ID,
|
||||
ModelConfigID: model.ID,
|
||||
Content: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("summarize the result without searching again"),
|
||||
codersdk.ChatMessageText("thanks"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
@@ -98,13 +199,432 @@ func TestOpenAIResponsesNoStaleWebSearchReplay(t *testing.T) {
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
|
||||
requests := recorder.all()
|
||||
require.Len(t, requests, 2)
|
||||
followup := requests[1]
|
||||
require.NotNil(t, followup.Store)
|
||||
require.False(t, *followup.Store)
|
||||
require.NotEmpty(t, followup.Prompt)
|
||||
requireNoResponsesProviderItemReplay(t, followup.Prompt, reasoningID, webSearchID)
|
||||
require.NotContains(t, promptItemTypes(followup.Prompt), "web_search_call")
|
||||
require.Len(t, requests, 3)
|
||||
for _, request := range requests {
|
||||
require.NotNil(t, request.Store)
|
||||
require.False(t, *request.Store)
|
||||
require.Contains(t, request.Include, "reasoning.encrypted_content")
|
||||
}
|
||||
// requests[1] continues the tool step from in-memory history;
|
||||
// requests[2] rebuilds the prompt from persisted messages.
|
||||
for _, request := range requests[1:] {
|
||||
reasoningIndex := requireInlineReasoningItem(t, request.Prompt, reasoningID, blob, "")
|
||||
require.NotContains(t, promptItemTypes(request.Prompt), "item_reference")
|
||||
functionCallIndex := slices.IndexFunc(request.Prompt, func(item interface{}) bool {
|
||||
itemMap, ok := item.(map[string]interface{})
|
||||
return ok && chattest.StringResponseField(itemMap, "type") == "function_call"
|
||||
})
|
||||
require.NotEqual(t, -1, functionCallIndex, "missing function_call item")
|
||||
require.Less(t, reasoningIndex, functionCallIndex, "types=%v", promptItemTypes(request.Prompt))
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAIResponsesReasoningNotReplayedAfterModelConfigChange(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
change func(cfg *database.UpdateChatModelConfigParams, otherProvider uuid.UUID)
|
||||
}{
|
||||
{
|
||||
name: "ProviderMoved",
|
||||
change: func(cfg *database.UpdateChatModelConfigParams, otherProvider uuid.UUID) {
|
||||
cfg.AIProviderID = uuid.NullUUID{UUID: otherProvider, Valid: true}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ModelChanged",
|
||||
change: func(cfg *database.UpdateChatModelConfigParams, _ uuid.UUID) {
|
||||
cfg.Model = "gpt-4.1"
|
||||
},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db, ps := dbtestutil.NewDB(t)
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
|
||||
const (
|
||||
reasoningID = "rs_changed_config"
|
||||
blob = "encrypted-changed-config"
|
||||
)
|
||||
var recorder responsesRequestRecorder
|
||||
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
if !req.Stream {
|
||||
return chattest.OpenAINonStreamingResponse("title")
|
||||
}
|
||||
resp := chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("answer")...)
|
||||
if recorder.record(req) == 1 {
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{
|
||||
ID: reasoningID,
|
||||
EncryptedContent: blob,
|
||||
}
|
||||
}
|
||||
return resp
|
||||
})
|
||||
|
||||
user, org, _ := seedChatDependenciesWithProvider(t, db, "openai", openAIURL)
|
||||
model := insertChatModelConfigWithCallConfig(t, db, user.ID, "openai", "gpt-4o",
|
||||
codersdk.ChatModelCallConfig{})
|
||||
otherProvider := dbgen.ChatProvider(t, db, database.ChatProvider{
|
||||
Provider: "openai",
|
||||
BaseUrl: openAIURL,
|
||||
})
|
||||
factory := chattest.NewMockAIBridgeTransport(t, openAIURL)
|
||||
server := newOpenAIResponsesTestServer(t, db, ps, func(cfg *chatd.Config) {
|
||||
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(factory)
|
||||
})
|
||||
|
||||
chat, err := server.CreateChat(ctx, chatd.CreateOptions{
|
||||
OrganizationID: org.ID,
|
||||
OwnerID: user.ID,
|
||||
Title: uniqueResponsesTitle(t, "changed-config"),
|
||||
ModelConfigID: model.ID,
|
||||
InitialUserContent: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("hello"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
require.Len(t, recorder.all(), 1)
|
||||
|
||||
update := database.UpdateChatModelConfigParams{
|
||||
ID: model.ID,
|
||||
Model: model.Model,
|
||||
DisplayName: model.DisplayName,
|
||||
Enabled: model.Enabled,
|
||||
IsDefault: model.IsDefault,
|
||||
ContextLimit: model.ContextLimit,
|
||||
CompressionThreshold: model.CompressionThreshold,
|
||||
Options: model.Options,
|
||||
AIProviderID: model.AIProviderID,
|
||||
}
|
||||
tc.change(&update, otherProvider.ID)
|
||||
_, err = db.UpdateChatModelConfig(ctx, update)
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = server.SendMessage(ctx, chatd.SendMessageOptions{
|
||||
ChatID: chat.ID,
|
||||
CreatedBy: user.ID,
|
||||
ModelConfigID: model.ID,
|
||||
Content: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("thanks"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
|
||||
requests := recorder.all()
|
||||
require.Len(t, requests, 2)
|
||||
followup := requests[1]
|
||||
require.NotEmpty(t, followup.Prompt)
|
||||
require.NotContains(t, promptItemTypes(followup.Prompt), "reasoning")
|
||||
requireNoResponsesProviderItemReplay(t, followup.Prompt, reasoningID)
|
||||
encoded, err := json.Marshal(followup.Prompt)
|
||||
require.NoError(t, err)
|
||||
require.NotContains(t, string(encoded), blob)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAIResponsesRetriesWithoutRejectedEncryptedReasoning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
// freshReasoningAt is the request whose response carries the
|
||||
// first reasoning produced after the rejection.
|
||||
freshReasoningAt int
|
||||
}{
|
||||
{name: "RetryReturnsReasoning", freshReasoningAt: 3},
|
||||
{name: "RetryReturnsNoReasoning", freshReasoningAt: 4},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db, ps := dbtestutil.NewDB(t)
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
|
||||
const (
|
||||
staleID = "rs_rejected_reasoning"
|
||||
staleBlob = "encrypted-by-another-organization"
|
||||
freshID = "rs_fresh_reasoning"
|
||||
freshBlob = "encrypted-by-current-organization"
|
||||
)
|
||||
var recorder responsesRequestRecorder
|
||||
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
if !req.Stream {
|
||||
return chattest.OpenAINonStreamingResponse("title")
|
||||
}
|
||||
n := recorder.record(req)
|
||||
// Later turns reach an organization that cannot decrypt the
|
||||
// reasoning from the first turn.
|
||||
if n > 1 && slices.ContainsFunc(req.Prompt, func(item interface{}) bool {
|
||||
itemMap, ok := item.(map[string]interface{})
|
||||
return ok && chattest.StringResponseField(itemMap, "id") == staleID
|
||||
}) {
|
||||
resp := chattest.OpenAIErrorResponse(http.StatusBadRequest, "invalid_request_error",
|
||||
"The encrypted content could not be verified.")
|
||||
resp.Error.Code = "invalid_encrypted_content"
|
||||
return resp
|
||||
}
|
||||
resp := chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("done")...)
|
||||
if n == 3 {
|
||||
resp = chattest.OpenAIStreamingResponse(chattest.OpenAIToolCallChunk("list_templates", `{}`))
|
||||
}
|
||||
switch n {
|
||||
case 1:
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{ID: staleID, EncryptedContent: staleBlob}
|
||||
case tc.freshReasoningAt:
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{ID: freshID, EncryptedContent: freshBlob}
|
||||
}
|
||||
return resp
|
||||
})
|
||||
|
||||
user, org, _ := seedChatDependenciesWithProvider(t, db, "openai", openAIURL)
|
||||
model := insertChatModelConfigWithCallConfig(t, db, user.ID, "openai", "gpt-4o",
|
||||
codersdk.ChatModelCallConfig{})
|
||||
factory := chattest.NewMockAIBridgeTransport(t, openAIURL)
|
||||
newServer := func() *chatd.Server {
|
||||
return newOpenAIResponsesTestServer(t, db, ps, func(cfg *chatd.Config) {
|
||||
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(factory)
|
||||
})
|
||||
}
|
||||
server := newServer()
|
||||
sendMessage := func(server *chatd.Server, chatID uuid.UUID, text string) {
|
||||
t.Helper()
|
||||
_, err := server.SendMessage(ctx, chatd.SendMessageOptions{
|
||||
ChatID: chatID,
|
||||
CreatedBy: user.ID,
|
||||
ModelConfigID: model.ID,
|
||||
Content: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText(text),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chatID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chatID)
|
||||
}
|
||||
|
||||
chat, err := server.CreateChat(ctx, chatd.CreateOptions{
|
||||
OrganizationID: org.ID,
|
||||
OwnerID: user.ID,
|
||||
Title: uniqueResponsesTitle(t, "rejected-reasoning"),
|
||||
ModelConfigID: model.ID,
|
||||
InitialUserContent: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("hello"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
sendMessage(server, chat.ID, "list the templates")
|
||||
|
||||
// The next turn runs on a server that never saw the rejection,
|
||||
// like a restarted or different replica.
|
||||
require.NoError(t, server.Close())
|
||||
sendMessage(newServer(), chat.ID, "thanks")
|
||||
|
||||
// Only the first request after the change is rejected.
|
||||
requests := recorder.all()
|
||||
require.Len(t, requests, 5)
|
||||
requireInlineReasoningItem(t, requests[1].Prompt, staleID, staleBlob, "")
|
||||
retry := requests[2]
|
||||
require.NotEmpty(t, retry.Prompt)
|
||||
require.NotContains(t, promptItemTypes(retry.Prompt), "reasoning")
|
||||
// requests[3] is the step after the tool call; requests[4] is
|
||||
// the next turn on the fresh server.
|
||||
for i, request := range requests[3:] {
|
||||
requireNoResponsesProviderItemReplay(t, request.Prompt, staleID)
|
||||
if i+3 < tc.freshReasoningAt {
|
||||
require.NotContains(t, promptItemTypes(request.Prompt), "reasoning")
|
||||
continue
|
||||
}
|
||||
requireInlineReasoningItem(t, request.Prompt, freshID, freshBlob, "")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAIResponsesReasoningNotReplayedAfterCredentialChange(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db, ps := dbtestutil.NewDB(t)
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
|
||||
const (
|
||||
userAPIKey = "sk-user-reasoning"
|
||||
deploymentID = "rs_deployment_key"
|
||||
deploymentBlob = "encrypted-by-deployment-organization"
|
||||
personalKeyID = "rs_personal_key"
|
||||
personalKeyBlob = "encrypted-by-personal-organization"
|
||||
)
|
||||
var (
|
||||
recorder responsesRequestRecorder
|
||||
authMu sync.Mutex
|
||||
authByCall = map[int]string{}
|
||||
)
|
||||
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
if !req.Stream {
|
||||
return chattest.OpenAINonStreamingResponse("title")
|
||||
}
|
||||
n := recorder.record(req)
|
||||
authMu.Lock()
|
||||
authByCall[n] = req.Header.Get("Authorization")
|
||||
authMu.Unlock()
|
||||
resp := chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("answer")...)
|
||||
switch n {
|
||||
case 1:
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{ID: deploymentID, EncryptedContent: deploymentBlob}
|
||||
case 2:
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{ID: personalKeyID, EncryptedContent: personalKeyBlob}
|
||||
}
|
||||
return resp
|
||||
})
|
||||
|
||||
user, org, _ := seedChatDependenciesWithProvider(t, db, "openai", openAIURL)
|
||||
model := insertChatModelConfigWithCallConfig(t, db, user.ID, "openai", "gpt-4o",
|
||||
codersdk.ChatModelCallConfig{})
|
||||
factory := chattest.NewMockAIBridgeTransport(t, openAIURL)
|
||||
server := newOpenAIResponsesTestServer(t, db, ps, func(cfg *chatd.Config) {
|
||||
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(factory)
|
||||
})
|
||||
|
||||
chat, err := server.CreateChat(ctx, chatd.CreateOptions{
|
||||
OrganizationID: org.ID,
|
||||
OwnerID: user.ID,
|
||||
Title: uniqueResponsesTitle(t, "credential-change"),
|
||||
ModelConfigID: model.ID,
|
||||
InitialUserContent: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("hello"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
|
||||
_, err = db.UpsertUserAIProviderKey(ctx, database.UpsertUserAIProviderKeyParams{
|
||||
ID: uuid.New(),
|
||||
UserID: user.ID,
|
||||
AIProviderID: model.AIProviderID.UUID,
|
||||
APIKey: userAPIKey,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
for _, text := range []string{"switched keys", "thanks"} {
|
||||
_, err = server.SendMessage(ctx, chatd.SendMessageOptions{
|
||||
ChatID: chat.ID,
|
||||
CreatedBy: user.ID,
|
||||
ModelConfigID: model.ID,
|
||||
Content: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText(text),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
}
|
||||
|
||||
requests := recorder.all()
|
||||
require.Len(t, requests, 3)
|
||||
authMu.Lock()
|
||||
require.NotEqual(t, "Bearer "+userAPIKey, authByCall[1])
|
||||
require.Equal(t, "Bearer "+userAPIKey, authByCall[2])
|
||||
authMu.Unlock()
|
||||
// Reasoning from the deployment's key is left out before the
|
||||
// personal key's organization can reject it.
|
||||
require.NotContains(t, promptItemTypes(requests[1].Prompt), "reasoning")
|
||||
requireNoResponsesProviderItemReplay(t, requests[1].Prompt, deploymentID)
|
||||
requireInlineReasoningItem(t, requests[2].Prompt, personalKeyID, personalKeyBlob, "")
|
||||
requireNoResponsesProviderItemReplay(t, requests[2].Prompt, deploymentID)
|
||||
}
|
||||
|
||||
func TestOpenAIResponsesAdvisorPromptOmitsReasoning(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
db, ps := dbtestutil.NewDB(t)
|
||||
ctx := testutil.Context(t, testutil.WaitLong)
|
||||
|
||||
const (
|
||||
reasoningID = "rs_parent_reasoning"
|
||||
blob = "encrypted-parent-reasoning"
|
||||
)
|
||||
var recorder responsesRequestRecorder
|
||||
openAIURL := chattest.NewOpenAI(t, func(req *chattest.OpenAIRequest) chattest.OpenAIResponse {
|
||||
if !req.Stream {
|
||||
return chattest.OpenAINonStreamingResponse("title")
|
||||
}
|
||||
switch recorder.record(req) {
|
||||
case 1:
|
||||
resp := chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("answer")...)
|
||||
resp.Reasoning = &chattest.OpenAIReasoningItem{
|
||||
ID: reasoningID,
|
||||
EncryptedContent: blob,
|
||||
}
|
||||
return resp
|
||||
case 2:
|
||||
return chattest.OpenAIStreamingResponse(
|
||||
chattest.OpenAIToolCallChunk("advisor", `{"question":"what next?"}`),
|
||||
)
|
||||
case 3:
|
||||
return chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("advice")...)
|
||||
default:
|
||||
return chattest.OpenAIStreamingResponse(chattest.OpenAITextChunks("done")...)
|
||||
}
|
||||
})
|
||||
|
||||
user, org, _ := seedChatDependenciesWithProvider(t, db, "openai", openAIURL)
|
||||
model := insertChatModelConfigWithCallConfig(t, db, user.ID, "openai", "gpt-4o",
|
||||
codersdk.ChatModelCallConfig{})
|
||||
seedAdvisorConfig(ctx, t, db, codersdk.AdvisorConfig{
|
||||
Enabled: true,
|
||||
MaxUsesPerRun: 1,
|
||||
MaxOutputTokens: 1024,
|
||||
})
|
||||
factory := chattest.NewMockAIBridgeTransport(t, openAIURL)
|
||||
server := newOpenAIResponsesTestServer(t, db, ps, func(cfg *chatd.Config) {
|
||||
cfg.AIBridgeTransportFactory = chatAIGatewayTransportFactoryPointer(factory)
|
||||
})
|
||||
|
||||
chat, err := server.CreateChat(ctx, chatd.CreateOptions{
|
||||
OrganizationID: org.ID,
|
||||
OwnerID: user.ID,
|
||||
Title: uniqueResponsesTitle(t, "advisor-reasoning"),
|
||||
ModelConfigID: model.ID,
|
||||
InitialUserContent: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("hello"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
|
||||
_, err = server.SendMessage(ctx, chatd.SendMessageOptions{
|
||||
ChatID: chat.ID,
|
||||
CreatedBy: user.ID,
|
||||
ModelConfigID: model.ID,
|
||||
Content: []codersdk.ChatMessagePart{
|
||||
codersdk.ChatMessageText("ask the advisor"),
|
||||
},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
waitForChatProcessed(ctx, t, db, chat.ID, server)
|
||||
requireResponsesChatWaiting(ctx, t, db, chat.ID)
|
||||
|
||||
requests := recorder.all()
|
||||
require.Len(t, requests, 4)
|
||||
requireInlineReasoningItem(t, requests[1].Prompt, reasoningID, blob, "")
|
||||
advisorRequest := requests[2]
|
||||
require.NotEmpty(t, advisorRequest.Prompt)
|
||||
require.NotContains(t, promptItemTypes(advisorRequest.Prompt), "reasoning")
|
||||
encoded, err := json.Marshal(advisorRequest.Prompt)
|
||||
require.NoError(t, err)
|
||||
require.Contains(t, string(encoded), "what next?")
|
||||
require.NotContains(t, string(encoded), blob)
|
||||
requireInlineReasoningItem(t, requests[3].Prompt, reasoningID, blob, "")
|
||||
}
|
||||
|
||||
func TestOpenAIResponsesPersistsProviderResponseID(t *testing.T) {
|
||||
@@ -320,8 +840,9 @@ func TestOpenAIResponsesNarrationKeepsPhase(t *testing.T) {
|
||||
}
|
||||
|
||||
type recordedResponsesRequest struct {
|
||||
Prompt []interface{}
|
||||
Store *bool
|
||||
Prompt []interface{}
|
||||
Store *bool
|
||||
Include []string
|
||||
}
|
||||
|
||||
type responsesRequestRecorder struct {
|
||||
@@ -338,9 +859,14 @@ func (r *responsesRequestRecorder) record(req *chattest.OpenAIRequest) int {
|
||||
value := *req.Store
|
||||
store = &value
|
||||
}
|
||||
var body struct {
|
||||
Include []string `json:"include"`
|
||||
}
|
||||
_ = json.Unmarshal(req.RawBody, &body)
|
||||
r.requests = append(r.requests, recordedResponsesRequest{
|
||||
Prompt: append([]interface{}(nil), req.Prompt...),
|
||||
Store: store,
|
||||
Prompt: append([]interface{}(nil), req.Prompt...),
|
||||
Store: store,
|
||||
Include: body.Include,
|
||||
})
|
||||
return len(r.requests)
|
||||
}
|
||||
@@ -456,12 +982,17 @@ func assertNoResponsesProviderItemReplay(
|
||||
if key == "type" && text == "web_search_call" {
|
||||
require.FailNow(t, "prompt replayed web_search_call provider item")
|
||||
}
|
||||
if key == "type" && text == "item_reference" {
|
||||
require.FailNow(t, "prompt replayed provider item reference")
|
||||
}
|
||||
if key == "id" || key == "call_id" || key == "item_id" {
|
||||
if _, isStale := staleIDs[text]; isStale {
|
||||
require.FailNowf(t, "prompt replayed stale provider item ID",
|
||||
"field %q contained stale provider ID %q", key, text)
|
||||
}
|
||||
if strings.HasPrefix(text, "ws_") || strings.HasPrefix(text, "rs_") {
|
||||
// Finalized reasoning is replayed inline, so only hosted
|
||||
// tool items are provider-managed state here.
|
||||
if strings.HasPrefix(text, "ws_") {
|
||||
require.FailNowf(t, "prompt replayed provider item ID",
|
||||
"field %q contained provider-managed ID %q", key, text)
|
||||
}
|
||||
@@ -476,6 +1007,39 @@ func assertNoResponsesProviderItemReplay(
|
||||
}
|
||||
}
|
||||
|
||||
// requireInlineReasoningItem asserts that the prompt replays the reasoning
|
||||
// item in full and returns its index. An empty summary must still be sent
|
||||
// as an empty array.
|
||||
func requireInlineReasoningItem(
|
||||
t *testing.T,
|
||||
prompt []interface{},
|
||||
id string,
|
||||
encryptedContent string,
|
||||
summary string,
|
||||
) int {
|
||||
t.Helper()
|
||||
for index, item := range prompt {
|
||||
itemMap, ok := item.(map[string]interface{})
|
||||
if !ok || chattest.StringResponseField(itemMap, "type") != "reasoning" ||
|
||||
chattest.StringResponseField(itemMap, "id") != id {
|
||||
continue
|
||||
}
|
||||
require.Equal(t, encryptedContent, chattest.StringResponseField(itemMap, "encrypted_content"))
|
||||
summaryItems, ok := itemMap["summary"].([]interface{})
|
||||
require.True(t, ok, "reasoning summary must be an array, got %T", itemMap["summary"])
|
||||
var texts []string
|
||||
for _, summaryItem := range summaryItems {
|
||||
summaryMap, ok := summaryItem.(map[string]interface{})
|
||||
require.True(t, ok)
|
||||
texts = append(texts, chattest.StringResponseField(summaryMap, "text"))
|
||||
}
|
||||
require.Equal(t, summary, strings.Join(texts, ""))
|
||||
return index
|
||||
}
|
||||
require.FailNowf(t, "missing inline reasoning item", "id=%q types=%v", id, promptItemTypes(prompt))
|
||||
return -1
|
||||
}
|
||||
|
||||
func requirePromptItemReferenceOrder(
|
||||
t *testing.T,
|
||||
prompt []interface{},
|
||||
|
||||
@@ -33,6 +33,14 @@ type buildCommitStepMessagesInput struct {
|
||||
logger slog.Logger
|
||||
contentVersion int16
|
||||
hookRewrittenToolCalls map[string]json.RawMessage
|
||||
reasoningProvenance reasoningProvenance
|
||||
}
|
||||
|
||||
func stampReasoningProvenance(part *codersdk.ChatMessagePart, provenance reasoningProvenance) {
|
||||
part.ProviderIdentity = provenance.ProviderIdentity
|
||||
part.ProviderModel = provenance.Model
|
||||
part.ProviderCredential = provenance.Credential
|
||||
part.ReasoningEpoch = provenance.Epoch
|
||||
}
|
||||
|
||||
type stepMessagesForCommit struct {
|
||||
@@ -50,6 +58,19 @@ func buildCommitStepMessages(input buildCommitStepMessagesInput) (stepMessagesFo
|
||||
|
||||
assistantBlocks, toolResults := splitStepContent(input.step.Content)
|
||||
assistantParts := buildAssistantParts(input.logger, assistantBlocks, toolResults, input.step, input.toolNameToConfigID, input.hookRewrittenToolCalls)
|
||||
stamped := false
|
||||
for i := range assistantParts {
|
||||
if assistantParts[i].Type == codersdk.ChatMessagePartTypeReasoning {
|
||||
stampReasoningProvenance(&assistantParts[i], input.reasoningProvenance)
|
||||
stamped = true
|
||||
}
|
||||
}
|
||||
// Without a stamp, a step that produced no reasoning would lose the
|
||||
// rejection epoch, and the next step would replay the rejected
|
||||
// reasoning again.
|
||||
if !stamped && input.reasoningProvenance.Epoch > 0 && len(assistantParts) > 0 {
|
||||
stampReasoningProvenance(&assistantParts[0], input.reasoningProvenance)
|
||||
}
|
||||
|
||||
messages := make([]chatstate.Message, 0, 1+len(toolResults))
|
||||
if len(assistantParts) > 0 {
|
||||
|
||||
@@ -33,6 +33,12 @@ func TestBuildCommitStepMessages_AssistantTextAndReasoning(t *testing.T) {
|
||||
modelConfigID: modelConfigID,
|
||||
contentVersion: chatprompt.CurrentContentVersion,
|
||||
logger: slog.Make(),
|
||||
reasoningProvenance: reasoningProvenance{
|
||||
ProviderIdentity: "provider-a",
|
||||
Model: "model-a",
|
||||
Credential: reasoningCredentialUser,
|
||||
Epoch: 2,
|
||||
},
|
||||
step: stepData{
|
||||
Content: []fantasy.Content{
|
||||
fantasy.ReasoningContent{Text: "thinking"},
|
||||
@@ -56,8 +62,16 @@ func TestBuildCommitStepMessages_AssistantTextAndReasoning(t *testing.T) {
|
||||
require.Equal(t, "thinking", parts[0].Text)
|
||||
require.Equal(t, startedAt, requireNotNilTime(t, parts[0].CreatedAt))
|
||||
require.Equal(t, completedAt, requireNotNilTime(t, parts[0].CompletedAt))
|
||||
require.Equal(t, "provider-a", parts[0].ProviderIdentity)
|
||||
require.Equal(t, "model-a", parts[0].ProviderModel)
|
||||
require.Equal(t, reasoningCredentialUser, parts[0].ProviderCredential)
|
||||
require.Equal(t, 2, parts[0].ReasoningEpoch)
|
||||
require.Equal(t, codersdk.ChatMessagePartTypeText, parts[1].Type)
|
||||
require.Equal(t, "hello", parts[1].Text)
|
||||
require.Empty(t, parts[1].ProviderIdentity)
|
||||
require.Empty(t, parts[1].ProviderModel)
|
||||
require.Empty(t, parts[1].ProviderCredential)
|
||||
require.Zero(t, parts[1].ReasoningEpoch)
|
||||
}
|
||||
|
||||
func TestBuildCommitStepMessages_LocalToolResultsBecomeToolMessages(t *testing.T) {
|
||||
|
||||
@@ -2,6 +2,7 @@ package chatd
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync/atomic"
|
||||
|
||||
"charm.land/fantasy"
|
||||
"golang.org/x/xerrors"
|
||||
@@ -72,6 +73,9 @@ type resolvedModelCall struct {
|
||||
resolvedEffort string
|
||||
route aiGatewayModelRoute
|
||||
debugEnabled bool
|
||||
// reasoningRejected reports whether the provider rejected encrypted
|
||||
// reasoning replayed into a call to model.
|
||||
reasoningRejected *atomic.Bool
|
||||
}
|
||||
|
||||
// stageModel uses the configured provider type rather than
|
||||
@@ -182,6 +186,8 @@ func (p *Server) resolveModelCall(ctx context.Context, spec modelCallSpec) (reso
|
||||
return resolvedModelCall{}, xerrors.Errorf("create model: %w", err)
|
||||
}
|
||||
model = p.withThinkingDropBlock(model, out.route.Provider.ID, clientCallConfig, spec.chat.ID)
|
||||
out.reasoningRejected = &atomic.Bool{}
|
||||
model = p.withEncryptedReasoningFallback(model, spec.chat.ID, out.reasoningRejected)
|
||||
|
||||
if out.debugEnabled {
|
||||
model = model.WithLanguageModel(chatdebug.WrapModel(model.LanguageModel(), debugSvc, chatdebug.RecorderOptions{
|
||||
|
||||
@@ -1,22 +1,27 @@
|
||||
package chatd
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"charm.land/fantasy"
|
||||
"github.com/google/uuid"
|
||||
"golang.org/x/xerrors"
|
||||
|
||||
"cdr.dev/slog/v3"
|
||||
"github.com/coder/coder/v2/coderd/database"
|
||||
"github.com/coder/coder/v2/coderd/x/chatd/chatprompt"
|
||||
"github.com/coder/coder/v2/coderd/x/chatd/chatsanitize"
|
||||
"github.com/coder/coder/v2/codersdk"
|
||||
)
|
||||
|
||||
// providerSwitchStripStats counts provider-executed tool history removed
|
||||
// during a provider switch.
|
||||
// providerSwitchStripStats counts provider-specific history removed during a
|
||||
// provider switch.
|
||||
type providerSwitchStripStats struct {
|
||||
RemovedToolCalls int
|
||||
RemovedToolResults int
|
||||
RemovedReasoning int
|
||||
DroppedMessages int
|
||||
}
|
||||
|
||||
@@ -33,20 +38,100 @@ func modelConfigProviderIdentity(modelConfig database.ChatModelConfig, normalize
|
||||
return normalizedProvider
|
||||
}
|
||||
|
||||
// stripForeignProviderExecutedToolRows drops provider-executed tool blocks
|
||||
// (calls and results) from assistant rows whose producing provider differs
|
||||
// from targetIdentity. Rows with an unknown origin are treated as foreign
|
||||
// (fail closed). Rows emptied by stripping are dropped; rows that fail to parse
|
||||
// or re-marshal are kept unchanged.
|
||||
const (
|
||||
reasoningCredentialDeployment = "deployment"
|
||||
reasoningCredentialUser = "user"
|
||||
)
|
||||
|
||||
// reasoningProvenance identifies where OpenAI reasoning state was produced.
|
||||
// Reasoning item IDs only resolve on the issuing provider instance, and
|
||||
// encrypted content can be rejected by a different model or by the
|
||||
// organization behind a different credential. Epoch counts rejections of
|
||||
// the chat's encrypted reasoning from the same provider, model, and
|
||||
// credential, so reasoning replayed into a rejected call is never replayed
|
||||
// again.
|
||||
type reasoningProvenance struct {
|
||||
ProviderIdentity string
|
||||
Model string
|
||||
Credential string
|
||||
Epoch int
|
||||
}
|
||||
|
||||
// sameSource reports whether p and o name the same provider, model, and
|
||||
// credential, regardless of epoch.
|
||||
func (p reasoningProvenance) sameSource(o reasoningProvenance) bool {
|
||||
return p.ProviderIdentity == o.ProviderIdentity && p.Model == o.Model && p.Credential == o.Credential
|
||||
}
|
||||
|
||||
// stampedReasoningProvenance returns the provenance recorded on part.
|
||||
func stampedReasoningProvenance(part codersdk.ChatMessagePart) reasoningProvenance {
|
||||
return reasoningProvenance{
|
||||
ProviderIdentity: part.ProviderIdentity,
|
||||
Model: part.ProviderModel,
|
||||
Credential: part.ProviderCredential,
|
||||
Epoch: part.ReasoningEpoch,
|
||||
}
|
||||
}
|
||||
|
||||
// reasoningCredential names the credential an AI Gateway route sends.
|
||||
func reasoningCredential(auth aiGatewayProviderAuth) string {
|
||||
if len(auth.Headers) > 0 {
|
||||
return reasoningCredentialUser
|
||||
}
|
||||
return reasoningCredentialDeployment
|
||||
}
|
||||
|
||||
// unstampedReasoningProvenance is the provenance assumed for reasoning saved
|
||||
// without a stamp: the row's model config, the target's credential, and the
|
||||
// first epoch.
|
||||
func unstampedReasoningProvenance(rowOrigin, target reasoningProvenance) reasoningProvenance {
|
||||
return reasoningProvenance{
|
||||
ProviderIdentity: rowOrigin.ProviderIdentity,
|
||||
Model: rowOrigin.Model,
|
||||
Credential: target.Credential,
|
||||
}
|
||||
}
|
||||
|
||||
// currentReasoningEpoch returns the highest epoch recorded in rows for the
|
||||
// provider, model, and credential of target. A rejected request only
|
||||
// replayed reasoning from that source, so the rejection says nothing about
|
||||
// reasoning from other models or credentials.
|
||||
func currentReasoningEpoch(rows []database.ChatMessage, target reasoningProvenance) int {
|
||||
epoch := 0
|
||||
for _, row := range rows {
|
||||
if row.Role != database.ChatMessageRoleAssistant || !bytes.Contains(row.Content.RawMessage, []byte(`"reasoning_epoch"`)) {
|
||||
continue
|
||||
}
|
||||
parts, err := chatprompt.ParseContent(row)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for _, part := range parts {
|
||||
if source := stampedReasoningProvenance(part); source.sameSource(target) {
|
||||
epoch = max(epoch, source.Epoch)
|
||||
}
|
||||
}
|
||||
}
|
||||
return epoch
|
||||
}
|
||||
|
||||
// stripForeignProviderStateRows drops provider-executed tool blocks (calls and
|
||||
// results) from assistant rows whose producing provider differs from the
|
||||
// target's, and OpenAI reasoning state whose provenance differs from target.
|
||||
// A reasoning part's recorded provenance takes precedence over its row's
|
||||
// origin, since the row's model config may have been changed. Rows with an
|
||||
// unknown origin are treated as foreign (fail closed). Rows emptied by
|
||||
// stripping are dropped; rows that fail to parse or re-marshal are kept
|
||||
// unchanged.
|
||||
//
|
||||
// See modelConfigProviderIdentity for how identity is derived.
|
||||
func stripForeignProviderExecutedToolRows(
|
||||
func stripForeignProviderStateRows(
|
||||
rows []database.ChatMessage,
|
||||
targetIdentity string,
|
||||
originProvider func(uuid.NullUUID) (string, bool),
|
||||
target reasoningProvenance,
|
||||
originOf func(uuid.NullUUID) reasoningProvenance,
|
||||
) ([]database.ChatMessage, providerSwitchStripStats) {
|
||||
var stats providerSwitchStripStats
|
||||
if targetIdentity == "" || len(rows) == 0 {
|
||||
if target.ProviderIdentity == "" || len(rows) == 0 {
|
||||
return rows, stats
|
||||
}
|
||||
|
||||
@@ -56,7 +141,10 @@ func stripForeignProviderExecutedToolRows(
|
||||
out = append(out, row)
|
||||
continue
|
||||
}
|
||||
if origin, ok := originProvider(row.ModelConfigID); ok && origin == targetIdentity {
|
||||
origin := originOf(row.ModelConfigID)
|
||||
nativeRow := origin.ProviderIdentity == target.ProviderIdentity
|
||||
// Only stamped reasoning parts can be foreign when the row origin matches.
|
||||
if unstampedReasoningProvenance(origin, target) == target && !bytes.Contains(row.Content.RawMessage, []byte(`"provider_identity"`)) {
|
||||
out = append(out, row)
|
||||
continue
|
||||
}
|
||||
@@ -68,23 +156,26 @@ func stripForeignProviderExecutedToolRows(
|
||||
}
|
||||
|
||||
kept := make([]codersdk.ChatMessagePart, 0, len(parts))
|
||||
var removedCalls, removedResults int
|
||||
var removedCalls, removedResults, removedReasoning int
|
||||
for _, part := range parts {
|
||||
switch {
|
||||
case part.Type == codersdk.ChatMessagePartTypeToolCall && part.ProviderExecuted:
|
||||
case !nativeRow && part.Type == codersdk.ChatMessagePartTypeToolCall && part.ProviderExecuted:
|
||||
removedCalls++
|
||||
case part.Type == codersdk.ChatMessagePartTypeToolResult && part.ProviderExecuted:
|
||||
case !nativeRow && part.Type == codersdk.ChatMessagePartTypeToolResult && part.ProviderExecuted:
|
||||
removedResults++
|
||||
case part.Type == codersdk.ChatMessagePartTypeReasoning && isForeignReasoning(part, origin, target):
|
||||
removedReasoning++
|
||||
default:
|
||||
kept = append(kept, part)
|
||||
}
|
||||
}
|
||||
if removedCalls == 0 && removedResults == 0 {
|
||||
if removedCalls == 0 && removedResults == 0 && removedReasoning == 0 {
|
||||
out = append(out, row)
|
||||
continue
|
||||
}
|
||||
stats.RemovedToolCalls += removedCalls
|
||||
stats.RemovedToolResults += removedResults
|
||||
stats.RemovedReasoning += removedReasoning
|
||||
if len(kept) == 0 {
|
||||
stats.DroppedMessages++
|
||||
continue
|
||||
@@ -102,28 +193,40 @@ func stripForeignProviderExecutedToolRows(
|
||||
return out, stats
|
||||
}
|
||||
|
||||
func (server *Server) sanitizeForeignProviderExecutedToolRows(
|
||||
func isForeignReasoning(part codersdk.ChatMessagePart, rowOrigin, target reasoningProvenance) bool {
|
||||
source := unstampedReasoningProvenance(rowOrigin, target)
|
||||
if part.ProviderIdentity != "" {
|
||||
source = stampedReasoningProvenance(part)
|
||||
}
|
||||
return source != target && hasOpenAIReasoningState(part)
|
||||
}
|
||||
|
||||
func hasOpenAIReasoningState(part codersdk.ChatMessagePart) bool {
|
||||
if len(part.ProviderMetadata) == 0 {
|
||||
return false
|
||||
}
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal(part.ProviderMetadata, &raw); err != nil {
|
||||
return false
|
||||
}
|
||||
options, err := fantasy.UnmarshalProviderOptions(raw)
|
||||
return err == nil && chatsanitize.HasOpenAIReasoningState(options)
|
||||
}
|
||||
|
||||
func (server *Server) sanitizeForeignProviderStateRows(
|
||||
ctx context.Context,
|
||||
logger slog.Logger,
|
||||
rows []database.ChatMessage,
|
||||
ownerID uuid.UUID,
|
||||
modelConfigID uuid.UUID,
|
||||
target reasoningProvenance,
|
||||
) ([]database.ChatMessage, error) {
|
||||
targetCfg, targetProvider, err := server.resolveModelConfigAndNormalizedProvider(ctx, ownerID, modelConfigID)
|
||||
if err != nil && !modelConfigUnavailable(err) {
|
||||
return nil, xerrors.Errorf("resolve target model config: %w", err)
|
||||
}
|
||||
if err != nil || targetProvider == "" {
|
||||
logger.Debug(ctx, "skipping provider-switch sanitization: target provider unresolved",
|
||||
slog.F("model_config_id", modelConfigID),
|
||||
slog.Error(err),
|
||||
)
|
||||
if target.ProviderIdentity == "" {
|
||||
return rows, nil
|
||||
}
|
||||
targetIdentity := modelConfigProviderIdentity(targetCfg, targetProvider)
|
||||
|
||||
// An unavailable origin config maps to "", so its rows are foreign.
|
||||
origins := make(map[uuid.UUID]string)
|
||||
// An unavailable origin config maps to the zero provenance, so its rows
|
||||
// are foreign.
|
||||
origins := make(map[uuid.UUID]reasoningProvenance)
|
||||
for _, row := range rows {
|
||||
id := row.ModelConfigID
|
||||
if row.Role != database.ChatMessageRoleAssistant || !id.Valid {
|
||||
@@ -135,25 +238,29 @@ func (server *Server) sanitizeForeignProviderExecutedToolRows(
|
||||
originCfg, provider, err := server.resolveModelConfigAndNormalizedProvider(ctx, ownerID, id.UUID)
|
||||
switch {
|
||||
case err == nil:
|
||||
origins[id.UUID] = modelConfigProviderIdentity(originCfg, provider)
|
||||
origins[id.UUID] = reasoningProvenance{
|
||||
ProviderIdentity: modelConfigProviderIdentity(originCfg, provider),
|
||||
Model: originCfg.Model,
|
||||
}
|
||||
case modelConfigUnavailable(err):
|
||||
origins[id.UUID] = ""
|
||||
origins[id.UUID] = reasoningProvenance{}
|
||||
default:
|
||||
return nil, xerrors.Errorf("resolve origin model config %s: %w", id.UUID, err)
|
||||
}
|
||||
}
|
||||
originProvider := func(id uuid.NullUUID) (string, bool) {
|
||||
identity := origins[id.UUID]
|
||||
return identity, identity != ""
|
||||
originOf := func(id uuid.NullUUID) reasoningProvenance {
|
||||
return origins[id.UUID]
|
||||
}
|
||||
|
||||
sanitized, stats := stripForeignProviderExecutedToolRows(rows, targetIdentity, originProvider)
|
||||
sanitized, stats := stripForeignProviderStateRows(rows, target, originOf)
|
||||
if stats != (providerSwitchStripStats{}) {
|
||||
logger.Debug(ctx, "stripped foreign provider-executed tool history",
|
||||
logger.Debug(ctx, "stripped foreign provider state from history",
|
||||
slog.F("phase", "provider_switch"),
|
||||
slog.F("target_provider_identity", targetIdentity),
|
||||
slog.F("target_provider_identity", target.ProviderIdentity),
|
||||
slog.F("target_model", target.Model),
|
||||
slog.F("removed_tool_calls", stats.RemovedToolCalls),
|
||||
slog.F("removed_tool_results", stats.RemovedToolResults),
|
||||
slog.F("removed_reasoning", stats.RemovedReasoning),
|
||||
slog.F("dropped_messages", stats.DroppedMessages),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -4,6 +4,8 @@ import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"charm.land/fantasy"
|
||||
fantasyopenai "charm.land/fantasy/providers/openai"
|
||||
"github.com/google/uuid"
|
||||
"github.com/sqlc-dev/pqtype"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -13,23 +15,32 @@ import (
|
||||
"github.com/coder/coder/v2/codersdk"
|
||||
)
|
||||
|
||||
func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
func TestStripForeignProviderStateRows(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
anthropic = "anthropic"
|
||||
bedrock = "bedrock"
|
||||
openai = "openai"
|
||||
)
|
||||
const model = "model-a"
|
||||
source := func(identity string) reasoningProvenance {
|
||||
return reasoningProvenance{ProviderIdentity: identity, Model: model}
|
||||
}
|
||||
anthropic := source("anthropic")
|
||||
bedrock := source("bedrock")
|
||||
openai := source("openai")
|
||||
|
||||
anthropicCfg := uuid.New()
|
||||
openAICfg := uuid.New()
|
||||
unknownCfg := uuid.New()
|
||||
|
||||
vllmProviderID := uuid.New()
|
||||
togetherProviderID := uuid.New()
|
||||
vllm := source(uuid.NewString())
|
||||
together := source(uuid.NewString())
|
||||
vllmCfg := uuid.New()
|
||||
|
||||
openAIInstance := source(uuid.NewString())
|
||||
otherOpenAIInstance := source(uuid.NewString())
|
||||
openAIInstanceOtherModel := reasoningProvenance{ProviderIdentity: openAIInstance.ProviderIdentity, Model: "model-b"}
|
||||
openAIInstanceCfg := uuid.New()
|
||||
openAIInstanceOtherCfg := uuid.New()
|
||||
openAIInstanceOtherModelCfg := uuid.New()
|
||||
|
||||
peCall := func(id string) codersdk.ChatMessagePart {
|
||||
p := codersdk.ChatMessageToolCall(id, "web_search", json.RawMessage(`{"query":"x"}`))
|
||||
p.ProviderExecuted = true
|
||||
@@ -44,6 +55,17 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
return codersdk.ChatMessageToolCall(id, "read_file", json.RawMessage(`{}`))
|
||||
}
|
||||
text := codersdk.ChatMessageText
|
||||
encryptedReasoning := "encrypted-blob"
|
||||
openAIReasoning := chatprompt.PartFromContent(fantasy.ReasoningContent{
|
||||
ProviderMetadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{
|
||||
ItemID: "rs_1",
|
||||
EncryptedContent: &encryptedReasoning,
|
||||
Summary: []string{},
|
||||
Finalized: true,
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
assistantRow := func(t *testing.T, cfg uuid.UUID, parts ...codersdk.ChatMessagePart) database.ChatMessage {
|
||||
t.Helper()
|
||||
@@ -67,20 +89,18 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
origin := func(providerByConfig map[uuid.UUID]string) func(uuid.NullUUID) (string, bool) {
|
||||
return func(id uuid.NullUUID) (string, bool) {
|
||||
if !id.Valid {
|
||||
return "", false
|
||||
}
|
||||
provider, ok := providerByConfig[id.UUID]
|
||||
return provider, ok
|
||||
}
|
||||
}
|
||||
resolver := origin(map[uuid.UUID]string{
|
||||
sourceByConfig := map[uuid.UUID]reasoningProvenance{
|
||||
anthropicCfg: anthropic,
|
||||
openAICfg: openai,
|
||||
vllmCfg: vllmProviderID.String(),
|
||||
})
|
||||
vllmCfg: vllm,
|
||||
|
||||
openAIInstanceCfg: openAIInstance,
|
||||
openAIInstanceOtherCfg: openAIInstance,
|
||||
openAIInstanceOtherModelCfg: openAIInstanceOtherModel,
|
||||
}
|
||||
resolver := func(id uuid.NullUUID) reasoningProvenance {
|
||||
return sourceByConfig[id.UUID]
|
||||
}
|
||||
|
||||
partsOf := func(t *testing.T, row database.ChatMessage) []codersdk.ChatMessagePart {
|
||||
t.Helper()
|
||||
@@ -95,7 +115,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, anthropicCfg, peCall("ws"), peResult("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, anthropic, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, anthropic, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
})
|
||||
@@ -106,7 +126,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, anthropicCfg, peCall("ws"), peResult("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, bedrock, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, bedrock, resolver)
|
||||
require.Len(t, got, 2)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("done")}, partsOf(t, got[1]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedToolCalls: 1, RemovedToolResults: 1}, stats)
|
||||
@@ -119,7 +139,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
assistantRow(t, anthropicCfg, peCall("ws")),
|
||||
userRow(t, "again"),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, bedrock, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, bedrock, resolver)
|
||||
require.Len(t, got, 2)
|
||||
require.Equal(t, database.ChatMessageRoleUser, got[0].Role)
|
||||
require.Equal(t, database.ChatMessageRoleUser, got[1].Role)
|
||||
@@ -132,7 +152,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
assistantRow(t, openAICfg, peCall("os"), peResult("os"), text("openai")),
|
||||
assistantRow(t, anthropicCfg, peCall("as"), peResult("as"), text("anthropic")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, anthropic, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, anthropic, resolver)
|
||||
require.Len(t, got, 2)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("openai")}, partsOf(t, got[0]))
|
||||
require.Equal(t, rows[1], got[1])
|
||||
@@ -144,7 +164,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, anthropicCfg, text("hello"), localCall("local")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, bedrock, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, bedrock, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
})
|
||||
@@ -154,7 +174,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, anthropicCfg, peCall("ws"), peResult("ws")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, "", resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, reasoningProvenance{}, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
})
|
||||
@@ -164,7 +184,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, unknownCfg, peResult("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, bedrock, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, bedrock, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedToolResults: 1}, stats)
|
||||
@@ -178,7 +198,7 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
Content: pqtype.NullRawMessage{RawMessage: []byte("{not json"), Valid: true},
|
||||
ContentVersion: chatprompt.ContentVersionV1,
|
||||
}}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, bedrock, resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, bedrock, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
})
|
||||
@@ -189,22 +209,196 @@ func TestStripForeignProviderExecutedToolRows(t *testing.T) {
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, vllmCfg, peCall("ws"), peResult("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, togetherProviderID.String(), resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, together, resolver)
|
||||
require.Len(t, got, 2)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("done")}, partsOf(t, got[1]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedToolCalls: 1, RemovedToolResults: 1}, stats)
|
||||
})
|
||||
|
||||
t.Run("same instance different config keeps OpenAI reasoning", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, openAIInstanceOtherCfg, openAIReasoning, localCall("local")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, openAIInstance, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
})
|
||||
|
||||
t.Run("foreign instance drops OpenAI reasoning", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
legacyReasoning := chatprompt.PartFromContent(fantasy.ReasoningContent{
|
||||
Text: "summary",
|
||||
ProviderMetadata: fantasy.ProviderMetadata{
|
||||
fantasyopenai.Name: &fantasyopenai.ResponsesReasoningMetadata{ItemID: "rs_legacy"},
|
||||
},
|
||||
})
|
||||
rows := []database.ChatMessage{
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, openAIInstanceCfg, openAIReasoning, legacyReasoning, codersdk.ChatMessageReasoning("plain"), text("visible"), localCall("local")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, otherOpenAIInstance, resolver)
|
||||
require.Len(t, got, 2)
|
||||
require.Equal(t, rows[0], got[0])
|
||||
require.Equal(t, []codersdk.ChatMessagePart{codersdk.ChatMessageReasoning("plain"), text("visible"), localCall("local")}, partsOf(t, got[1]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 2}, stats)
|
||||
})
|
||||
|
||||
t.Run("unknown origin drops OpenAI reasoning", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, unknownCfg, openAIReasoning, text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, openAIInstance, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 1}, stats)
|
||||
})
|
||||
|
||||
stampedReasoning := func(source reasoningProvenance) codersdk.ChatMessagePart {
|
||||
p := openAIReasoning
|
||||
p.ProviderIdentity = source.ProviderIdentity
|
||||
p.ProviderModel = source.Model
|
||||
return p
|
||||
}
|
||||
|
||||
t.Run("stamped foreign reasoning dropped from native row", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceCfg, stampedReasoning(otherOpenAIInstance), peCall("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, openAIInstance, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{peCall("ws"), text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 1}, stats)
|
||||
})
|
||||
|
||||
t.Run("stamped native reasoning kept after config moved", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceCfg, stampedReasoning(otherOpenAIInstance), peCall("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, otherOpenAIInstance, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{stampedReasoning(otherOpenAIInstance), text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedToolCalls: 1}, stats)
|
||||
})
|
||||
|
||||
t.Run("stamped reasoning from another model dropped", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceCfg, stampedReasoning(openAIInstanceOtherModel), peCall("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, openAIInstance, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{peCall("ws"), text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 1}, stats)
|
||||
})
|
||||
|
||||
t.Run("unstamped reasoning from another model on same instance dropped", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceOtherModelCfg, openAIReasoning, peCall("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, openAIInstance, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{peCall("ws"), text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 1}, stats)
|
||||
})
|
||||
|
||||
t.Run("unstamped reasoning follows row origin", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceCfg, stampedReasoning(openAIInstance), openAIReasoning, text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, openAIInstance, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
|
||||
got, stats = stripForeignProviderStateRows(rows, otherOpenAIInstance, resolver)
|
||||
require.Len(t, got, 1)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("done")}, partsOf(t, got[0]))
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 2}, stats)
|
||||
})
|
||||
|
||||
t.Run("same instance keeps provider blocks", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
rows := []database.ChatMessage{
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, vllmCfg, peCall("ws"), peResult("ws"), text("done")),
|
||||
}
|
||||
got, stats := stripForeignProviderExecutedToolRows(rows, vllmProviderID.String(), resolver)
|
||||
got, stats := stripForeignProviderStateRows(rows, vllm, resolver)
|
||||
require.Equal(t, rows, got)
|
||||
require.Zero(t, stats)
|
||||
})
|
||||
|
||||
stampedFull := func(source reasoningProvenance) codersdk.ChatMessagePart {
|
||||
p := stampedReasoning(source)
|
||||
p.ProviderCredential = source.Credential
|
||||
p.ReasoningEpoch = source.Epoch
|
||||
return p
|
||||
}
|
||||
|
||||
t.Run("stamped reasoning from another credential dropped", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
deployment := openAIInstance
|
||||
deployment.Credential = reasoningCredentialDeployment
|
||||
user := openAIInstance
|
||||
user.Credential = reasoningCredentialUser
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(deployment), text("deployment")),
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(user), text("user")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, user, resolver)
|
||||
require.Len(t, got, 2)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("deployment")}, partsOf(t, got[0]))
|
||||
require.Equal(t, rows[1], got[1])
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 1}, stats)
|
||||
})
|
||||
|
||||
t.Run("reasoning from an earlier epoch dropped", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
current := openAIInstance
|
||||
current.Epoch = 1
|
||||
rows := []database.ChatMessage{
|
||||
assistantRow(t, openAIInstanceCfg, openAIReasoning, text("unstamped")),
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(openAIInstance), text("rejected")),
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(current), text("current")),
|
||||
}
|
||||
got, stats := stripForeignProviderStateRows(rows, current, resolver)
|
||||
require.Len(t, got, 3)
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("unstamped")}, partsOf(t, got[0]))
|
||||
require.Equal(t, []codersdk.ChatMessagePart{text("rejected")}, partsOf(t, got[1]))
|
||||
require.Equal(t, rows[2], got[2])
|
||||
require.Equal(t, providerSwitchStripStats{RemovedReasoning: 2}, stats)
|
||||
})
|
||||
|
||||
t.Run("current epoch is per provider model and credential", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
at := func(source reasoningProvenance, credential string, epoch int) reasoningProvenance {
|
||||
source.Credential = credential
|
||||
source.Epoch = epoch
|
||||
return source
|
||||
}
|
||||
deployment := at(openAIInstance, reasoningCredentialDeployment, 0)
|
||||
// A step with no reasoning records its epoch on its first part.
|
||||
marker := text("no reasoning")
|
||||
stampReasoningProvenance(&marker, at(openAIInstance, reasoningCredentialDeployment, 3))
|
||||
rows := []database.ChatMessage{
|
||||
userRow(t, "hi"),
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(at(openAIInstance, reasoningCredentialDeployment, 2)), openAIReasoning),
|
||||
assistantRow(t, openAIInstanceCfg, marker),
|
||||
assistantRow(t, openAIInstanceOtherModelCfg, stampedFull(at(openAIInstanceOtherModel, reasoningCredentialDeployment, 4))),
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(at(openAIInstance, reasoningCredentialUser, 5))),
|
||||
assistantRow(t, openAIInstanceCfg, stampedFull(at(otherOpenAIInstance, reasoningCredentialDeployment, 6))),
|
||||
}
|
||||
require.Equal(t, 3, currentReasoningEpoch(rows, deployment))
|
||||
require.Equal(t, 4, currentReasoningEpoch(rows, at(openAIInstanceOtherModel, reasoningCredentialDeployment, 0)))
|
||||
require.Equal(t, 5, currentReasoningEpoch(rows, at(openAIInstance, reasoningCredentialUser, 0)))
|
||||
require.Equal(t, 6, currentReasoningEpoch(rows, at(otherOpenAIInstance, reasoningCredentialDeployment, 0)))
|
||||
require.Zero(t, currentReasoningEpoch(rows, at(vllm, reasoningCredentialDeployment, 0)))
|
||||
})
|
||||
}
|
||||
|
||||
func TestModelConfigProviderIdentity(t *testing.T) {
|
||||
@@ -265,8 +459,8 @@ func TestPendingUserSegmentDerivedBeforeProviderSwitchSanitization(t *testing.T)
|
||||
{Role: database.ChatMessageRoleUser, Content: userContent("pending"), ContentVersion: chatprompt.ContentVersionV1},
|
||||
}
|
||||
|
||||
origin := func(id uuid.NullUUID) (string, bool) {
|
||||
return "openai", id.Valid
|
||||
origin := func(uuid.NullUUID) reasoningProvenance {
|
||||
return reasoningProvenance{ProviderIdentity: "openai"}
|
||||
}
|
||||
|
||||
// Segment first: only the truly unanswered tail is pending, and the
|
||||
@@ -278,13 +472,13 @@ func TestPendingUserSegmentDerivedBeforeProviderSwitchSanitization(t *testing.T)
|
||||
// The sanitizer drops the emptied foreign assistant row from the
|
||||
// head; the answered user row stays in the head regardless, and the
|
||||
// segment derived above is unaffected.
|
||||
head, stats := stripForeignProviderExecutedToolRows(rows[:start], "anthropic", origin)
|
||||
head, stats := stripForeignProviderStateRows(rows[:start], reasoningProvenance{ProviderIdentity: "anthropic"}, origin)
|
||||
require.Equal(t, 1, stats.DroppedMessages)
|
||||
require.Len(t, head, 1)
|
||||
require.Equal(t, database.ChatMessageRoleUser, head[0].Role)
|
||||
|
||||
// Deriving the segment after sanitization would find no assistant
|
||||
// row at all and therefore no pending segment.
|
||||
sanitizedAll, _ := stripForeignProviderExecutedToolRows(rows, "anthropic", origin)
|
||||
sanitizedAll, _ := stripForeignProviderStateRows(rows, reasoningProvenance{ProviderIdentity: "anthropic"}, origin)
|
||||
require.Equal(t, len(sanitizedAll), pendingUserSegmentStart(sanitizedAll))
|
||||
}
|
||||
|
||||
@@ -506,6 +506,26 @@ type ChatMessagePart struct {
|
||||
// (e.g. Anthropic cache control hints) as raw JSON. Internal
|
||||
// only: stripped by db2sdk before API responses.
|
||||
ProviderMetadata json.RawMessage `json:"provider_metadata,omitempty" typescript:"-"`
|
||||
// ProviderIdentity records the provider that produced a reasoning
|
||||
// part, because a model config's provider can change after the
|
||||
// message was written. Internal only: stripped before API responses.
|
||||
ProviderIdentity string `json:"provider_identity,omitempty" typescript:"-" swaggerignore:"true"`
|
||||
// ProviderModel records the model that produced a reasoning part,
|
||||
// because a model config's model can change after the message was
|
||||
// written. Internal only: stripped before API responses.
|
||||
ProviderModel string `json:"provider_model,omitempty" typescript:"-" swaggerignore:"true"`
|
||||
// ProviderCredential records whether a reasoning part was produced
|
||||
// with the deployment's credential or the user's personal API key,
|
||||
// because encrypted reasoning only decrypts for the organization
|
||||
// behind that credential. Internal only: stripped before API responses.
|
||||
ProviderCredential string `json:"provider_credential,omitempty" typescript:"-" swaggerignore:"true"`
|
||||
// ReasoningEpoch records how many times the provider had rejected the
|
||||
// chat's encrypted reasoning from this provider, model, and credential
|
||||
// before this reasoning part was produced. A step that produced no
|
||||
// reasoning after a rejection records its epoch, provider, model, and
|
||||
// credential on its first part instead. Internal only: stripped before
|
||||
// API responses.
|
||||
ReasoningEpoch int `json:"reasoning_epoch,omitempty" typescript:"-" swaggerignore:"true"`
|
||||
// ProviderExecuted indicates the tool call was executed by
|
||||
// the provider (e.g. Anthropic computer use).
|
||||
ProviderExecuted bool `json:"provider_executed,omitempty" variants:"tool-call?,tool-result?"`
|
||||
@@ -596,6 +616,10 @@ type ChatMessagePart struct {
|
||||
// the advisor streaming callbacks.
|
||||
func (p *ChatMessagePart) StripInternal() {
|
||||
p.ProviderMetadata = nil
|
||||
p.ProviderIdentity = ""
|
||||
p.ProviderModel = ""
|
||||
p.ProviderCredential = ""
|
||||
p.ReasoningEpoch = 0
|
||||
if p.FileID.Valid {
|
||||
p.Data = nil
|
||||
}
|
||||
|
||||
+17
-5
@@ -199,14 +199,22 @@ func TestChatMessagePart_StripInternal(t *testing.T) {
|
||||
t.Run("StripsProviderMetadata", func(t *testing.T) {
|
||||
t.Parallel()
|
||||
part := codersdk.ChatMessagePart{
|
||||
Type: codersdk.ChatMessagePartTypeToolCall,
|
||||
ToolCallID: "call-1",
|
||||
ToolName: "some_tool",
|
||||
Args: json.RawMessage(`{"key":"value"}`),
|
||||
ProviderMetadata: json.RawMessage(`{"type":"ephemeral"}`),
|
||||
Type: codersdk.ChatMessagePartTypeToolCall,
|
||||
ToolCallID: "call-1",
|
||||
ToolName: "some_tool",
|
||||
Args: json.RawMessage(`{"key":"value"}`),
|
||||
ProviderMetadata: json.RawMessage(`{"type":"ephemeral"}`),
|
||||
ProviderIdentity: "provider-1",
|
||||
ProviderModel: "model-1",
|
||||
ProviderCredential: "user",
|
||||
ReasoningEpoch: 2,
|
||||
}
|
||||
part.StripInternal()
|
||||
assert.Nil(t, part.ProviderMetadata)
|
||||
assert.Empty(t, part.ProviderIdentity)
|
||||
assert.Empty(t, part.ProviderModel)
|
||||
assert.Empty(t, part.ProviderCredential)
|
||||
assert.Zero(t, part.ReasoningEpoch)
|
||||
// Public fields preserved.
|
||||
assert.Equal(t, codersdk.ChatMessagePartTypeToolCall, part.Type)
|
||||
assert.Equal(t, "call-1", part.ToolCallID)
|
||||
@@ -303,6 +311,10 @@ func TestChatMessagePartVariantTags(t *testing.T) {
|
||||
excludedFields := map[string]string{
|
||||
"type": "discriminant, added automatically by codegen",
|
||||
"provider_metadata": "internal only, stripped by db2sdk before API responses",
|
||||
"provider_identity": "internal only, stripped by db2sdk before API responses",
|
||||
"provider_model": "internal only, stripped by db2sdk before API responses",
|
||||
"provider_credential": "internal only, stripped by db2sdk before API responses",
|
||||
"reasoning_epoch": "internal only, stripped by db2sdk before API responses",
|
||||
"context_file_content": "internal only, stripped before API responses (typescript:\"-\")",
|
||||
"context_file_os": "internal only, used during prompt expansion (typescript:\"-\")",
|
||||
"context_file_directory": "internal only, used during prompt expansion (typescript:\"-\")",
|
||||
|
||||
@@ -111,8 +111,12 @@ replace github.com/spf13/afero => github.com/aslilac/afero v0.0.0-20250403163713
|
||||
// with no results so chatd keeps them in saved history.
|
||||
// 13) coder/fantasy#66, capture OpenAI Responses message phase on text
|
||||
// parts and replay it on assistant input messages in both store modes.
|
||||
// See: https://github.com/coder/fantasy/commits/351055280157
|
||||
replace charm.land/fantasy => github.com/coder/fantasy v0.0.0-20261009165329-351055280157
|
||||
// 14) coder/fantasy#63, capture finalized OpenAI Responses reasoning
|
||||
// (item ID, encrypted content, summary) and replay it in full
|
||||
// regardless of store. Skip hosted search references from unstored
|
||||
// source responses.
|
||||
// See: https://github.com/coder/fantasy/commits/6ba868142390
|
||||
replace charm.land/fantasy => github.com/coder/fantasy v0.0.0-20261011102151-6ba868142390
|
||||
|
||||
// coder/coder uses a fork of charmbracelet's fork of the Anthropic Go SDK
|
||||
// with performance improvements and Bedrock header cleanup.
|
||||
|
||||
@@ -333,8 +333,8 @@ github.com/coder/bubbletea v1.2.2-0.20241212190825-007a1cdb2c41 h1:SBN/DA63+ZHwu
|
||||
github.com/coder/bubbletea v1.2.2-0.20241212190825-007a1cdb2c41/go.mod h1:I9ULxr64UaOSUv7hcb3nX4kowodJCVS7vt7VVJk/kW4=
|
||||
github.com/coder/clistat v1.2.1 h1:P9/10njXMyj5cWzIU5wkRsSy5LVQH49+tcGMsAgWX0w=
|
||||
github.com/coder/clistat v1.2.1/go.mod h1:m7SC0uj88eEERgvF8Kn6+w6XF21BeSr+15f7GoLAw0A=
|
||||
github.com/coder/fantasy v0.0.0-20261009165329-351055280157 h1:t1DbZyWlLOPISsLzcFFnKfvSToWpXFIL1pkpFovlNh0=
|
||||
github.com/coder/fantasy v0.0.0-20261009165329-351055280157/go.mod h1:Hd5DBPecZ+CGc8O9bFAV03S7jbEmqd9xXVmjKCzpD7g=
|
||||
github.com/coder/fantasy v0.0.0-20261011102151-6ba868142390 h1:l0MyL3NG85LofRJj7pvXaPINmUqGI89dTQXxg2fJMNs=
|
||||
github.com/coder/fantasy v0.0.0-20261011102151-6ba868142390/go.mod h1:Hd5DBPecZ+CGc8O9bFAV03S7jbEmqd9xXVmjKCzpD7g=
|
||||
github.com/coder/flog v1.1.0 h1:kbAes1ai8fIS5OeV+QAnKBQE22ty1jRF/mcAwHpLBa4=
|
||||
github.com/coder/flog v1.1.0/go.mod h1:UQlQvrkJBvnRGo69Le8E24Tcl5SJleAAR7gYEHzAmdQ=
|
||||
github.com/coder/go-httpstat v0.0.0-20230801153223-321c88088322 h1:m0lPZjlQ7vdVpRBPKfYIFlmgevoTkBxB10wv6l2gOaU=
|
||||
|
||||
Reference in New Issue
Block a user