mirror of
https://github.com/cline/cline.git
synced 2026-09-21 05:10:09 +08:00
10f7b8ca9e
* migrate custom model config section * changeset --------- Co-authored-by: Elephant Lumps <celestial_vault@mac.mynetworksettings.com> Co-authored-by: Elephant Lumps <celestial_vault@Elephants-MacBook-Pro.local>
66 lines
2.1 KiB
Plaintext
66 lines
2.1 KiB
Plaintext
---
|
|
title: "LiteLLM & Cline (using Codestral)"
|
|
description: "Learn how to set up and run LiteLLM with Cline using the Codestral model. This guide covers Docker setup, configuration, and integration with Cline."
|
|
---
|
|
|
|
### Using LiteLLM with Cline
|
|
|
|
This guide demonstrates how to run a demo for LiteLLM starting with the Codestral model for use with Cline.
|
|
|
|
#### Prerequisites
|
|
|
|
- [Docker CLI or Docker Desktop](https://www.docker.com/get-started/) installed to run the LiteLLM image locally
|
|
- For this example config: A Codestral API Key (different from the Mistral API Keys)
|
|
|
|
#### Setup
|
|
|
|
1. **Create a `.env` file and fill in the appropriate field**
|
|
|
|
```bash
|
|
# Tip: Use the following command to generate a random alphanumeric key:
|
|
# openssl rand -base64 32 | tr -dc 'A-Za-z0-9' | head -c 32
|
|
LITELLM_MASTER_KEY=YOUR_LITELLM_MASTER_KEY
|
|
CODESTRAL_API_KEY=YOUR_CODESTRAL_API_KEY
|
|
```
|
|
|
|
_Note: Although this is limited to localhost, it's a good practice set LITELLM_MASTER_KEY to something secure_
|
|
|
|
2. **Configuration**
|
|
|
|
We'll need to create a `config.yaml` file to contain our LiteLLM configuration. In this case we'll just have one model, 'codestral-latest' and label it 'codestral'
|
|
|
|
```yaml
|
|
model_list:
|
|
- model_name: codestral
|
|
litellm_params:
|
|
model: codestral/codestral-latest
|
|
api_key: os.environ/CODESTRAL_API_KEY
|
|
```
|
|
|
|
#### Running the Demo
|
|
|
|
1. **Startup the LiteLLM docker container**
|
|
|
|
```bash
|
|
docker run \
|
|
--env-file .env \
|
|
-v $(pwd)/config.yaml:/app/config.yaml \
|
|
-p 127.0.0.1:4000:4000 \
|
|
ghcr.io/berriai/litellm:main-latest \
|
|
--config /app/config.yaml --detailed_debug
|
|
```
|
|
|
|
2. **Setup Cline**
|
|
|
|
Once the LiteLLM server is up and running you can set it up in Cline:
|
|
|
|
- Base URL should be `http://0.0.0.0:4000/v1`
|
|
- API Key should be the one you set in `.env` for LITELLM_MASTER_KEY
|
|
- Model ID is `codestral` or whatever you named it under `config.yaml`
|
|
|
|
#### Getting Help
|
|
|
|
- [LiteLLM Documentation](https://docs.litellm.ai/)
|
|
- [Mistral AI Console](https://console.mistral.ai/)
|
|
- [Cline Discord Community](https://discord.gg/cline)
|