feat(litellm): add LiteLLM as AI gateway provider (#4739)

* feat: add LiteLLM as AI gateway provider

* fix: add litellm to attachments, provider store, utils, and block guards

* fix: add frontend model discovery pipeline for litellm provider

Add API route, contract, query hook case, and ProviderModelsLoader
entry so litellm models are fetched and synced to the store on
workspace load, matching the vllm/ollama/openrouter/fireworks pattern.

Also fixes defaultModel to empty string and adds litellm/ prefix
early-return in blocks/utils.ts (reviewer feedback).

* fix: remove azureEndpoint fallback from LiteLLM provider

Copy-paste artifact from vLLM provider. LiteLLM should only use
LITELLM_BASE_URL, not fall back to azureEndpoint which could cause
requests to be routed to the wrong server.

* fix(litellm): close audit gaps from PR #4644

- byok.ts: add litellm branch to getApiKeyWithBYOK so workflow
  block execution can resolve the proxy key instead of throwing
  "API key is required for litellm ..."
- check-api-validation-contracts.ts: bump route baseline 755 -> 756
  to account for the new /api/providers/litellm/models route
- .env.example: document LITELLM_BASE_URL / LITELLM_API_KEY
- copilot edit-workflow validation: include LiteLLM in the list of
  user-configured prefixed providers shown to the model
- providers/utils.ts: drop stray optional-chain on providers.litellm
  to match the vllm pattern
- lint: apply biome formatting fixes (multi-line if, SVG path,
  multi-line DYNAMIC_MODEL_PROVIDERS)

* fix(litellm): final parity gaps from second audit

- blocks/utils.ts getModelOptions(): include litellm models in the
  combined model dropdown — was previously dropping any
  proxy-discovered models from the agent block model picker.
- get-blocks-metadata-tool.ts mockProvidersState: add litellm bucket
  so the server-side copilot block-metadata fallback can render
  model options when the providers store is not initialized.
- blocks/utils.test.ts: add litellm to mock providers state (initial
  + beforeEach reset) and add a parallel store-bucket guard test
  mirroring the vLLM case.
- providers/utils.test.ts: add parallel getApiKey test for litellm.

* feat(litellm): use official LiteLLM brand icon and color

- icons.tsx: replace the placeholder letterform with the official
  LiteLLM brand mark embedded as a PNG data URI in an SVG image.
- models.ts: set color: #040229 on the litellm provider definition
  to match the brand background.

* chore(litellm): validate /v1/models response with shared schema in initialize()

Match the API route handler — both code paths now run the same
vllmUpstreamResponseSchema.parse() over the upstream /v1/models
JSON instead of a raw type-cast, so malformed upstream payloads
surface a descriptive ZodError instead of a downstream TypeError.

Addresses Greptile review feedback on PR #4739.

---------

Co-authored-by: RheagalFire <arishalam121@gmail.com>
This commit is contained in:
Waleed
2026-05-26 10:38:24 -07:00
committed by GitHub
co-authored by RheagalFire
parent d62f9cabde
commit 81bf93b184
23 changed files with 899 additions and 8 deletions
+2 -2
View File
@@ -9,8 +9,8 @@ const QUERY_HOOKS_DIR = path.join(ROOT, 'apps/sim/hooks/queries')
const SELECTOR_HOOKS_DIR = path.join(ROOT, 'apps/sim/hooks/selectors')
const BASELINE = {
totalRoutes: 755,
zodRoutes: 755,
totalRoutes: 756,
zodRoutes: 756,
nonZodRoutes: 0,
} as const