diff --git a/.changeset/tiny-doors-cover.md b/.changeset/tiny-doors-cover.md new file mode 100644 index 0000000000..e5a8c9b9a5 --- /dev/null +++ b/.changeset/tiny-doors-cover.md @@ -0,0 +1,5 @@ +--- +"claude-dev": minor +--- + +Migrate custom model config section to new docs diff --git a/docs/custom-model-configs/aws-bedrock-with-credentials-authentication.mdx b/docs/custom-model-configs/aws-bedrock-with-credentials-authentication.mdx new file mode 100644 index 0000000000..b9fe7b69e6 --- /dev/null +++ b/docs/custom-model-configs/aws-bedrock-with-credentials-authentication.mdx @@ -0,0 +1,121 @@ +--- +title: "AWS Bedrock" +description: "Learn how to set up AWS Bedrock with Cline using credentials authentication. This guide covers AWS environment setup, regional access verification, and secure integration with the Cline VS Code extension." +--- + +### Overview + +- **AWS Bedrock:** A fully managed service that offers access to leading generative AI models (e.g., Anthropic Claude, Amazon Titan) through AWS.\ + [Learn more about AWS Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html). +- **Cline:** A VS Code extension that acts as a coding assistant by integrating with AI models—empowering developers to generate code, debug, and analyze data. +- **Enterprise Focus:** This guide is tailored for organizations with established AWS environments (using IAM roles, AWS SSO, AWS Organizations, etc.) to ensure secure and compliant usage. + +--- + +### Step 1: Prepare Your AWS Environment + +#### 1.1 Create or Use an IAM Role/User + +1. **Sign in to the AWS Management Console:**\ + [AWS Console](https://aws.amazon.com/console/) +2. **Access IAM:** + - Search for **IAM (Identity and Access Management)** in the AWS Console. + - Either create a new IAM user or use your enterprise's AWS SSO to assume a dedicated role for Bedrock access. + - [AWS IAM User Guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) + +#### 1.2 Attach the Required Policies + +1. **Attach the Managed Policy:** + - Attach the **`AmazonBedrockFullAccess`** managed policy to your user/role.\ + [View AmazonBedrockFullAccess Policy Details](https://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html) +2. **Confirm Additional Permissions:** + - Ensure your policy includes permissions for model invocation (e.g., `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream`), model listing, and AWS Marketplace actions (like `aws-marketplace:Subscribe`). + - _Enterprise Tip:_ Apply least-privilege practices by scoping resource ARNs and using [Service Control Policies (SCPs)](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html) to restrict access where necessary. + +--- + +### Step 2: Verify Regional and Model Access + +#### 2.1 Choose and Confirm a Region + +1. **Select a Region:**\ + AWS Bedrock is available in multiple regions (e.g., US East, Europe, Asia Pacific). Choose the region that meets your latency and compliance needs.\ + [AWS Global Infrastructure](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) +2. **Verify Model Access:** + - In the AWS Bedrock console, confirm that the models your team requires (e.g., Anthropic Claude, Amazon Titan) are marked as "Access granted." + - **Note:** Some advanced models might require an [Inference Profile](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html) if not available on-demand. + +#### 2.2 Set Up AWS Marketplace Subscriptions (if needed) + +1. **Subscribe to Third-Party Models:** + - Navigate to the AWS Bedrock console and locate the model subscription section. + - For models from third-party providers (e.g., Anthropic), accept the terms to subscribe. + - [AWS Marketplace](https://aws.amazon.com/marketplace/) +2. **Enterprise Tip:** + - Model subscriptions are often managed centrally. Confirm with your cloud team if a standard subscription process is in place. + +--- + +### Step 3: Configure the Cline VS Code Extension + +#### 3.1 Install and Open Cline + +1. **Install VS Code:**\ + Download from the [VS Code website](https://code.visualstudio.com/). +2. **Install the Cline Extension:** + - Open VS Code. + - Go to the Extensions Marketplace (`Ctrl+Shift+X` or `Cmd+Shift+X`). + - Search for **Cline** and install it. + +#### 3.2 Configure Cline Settings + +1. **Open Cline Settings:** + - Click on the settings ⚙️ to select your API Provider. +2. **Select AWS Bedrock as the API Provider:** + - From the API Provider dropdown, choose **AWS Bedrock**. +3. **Enter Your AWS Credentials:** + - Input your **Access Key** and **Secret Key** (or use temporary credentials if using AWS SSO). + - Specify the correct **AWS Region** (e.g., `us-east-1` or your enterprise-approved region). +4. **Select a Model:** + - Choose an on-demand model (e.g., **anthropic.claude-3-5-sonnet-20241022-v2:0**). +5. **Save and Test:** + - Click **Done/Save** to apply your settings. + - Test the integration by sending a simple prompt (e.g., "Generate a Python function to check if a number is prime."). + +--- + +### Step 4: Security, Monitoring, and Best Practices + +1. **Secure Access:** + - Prefer AWS SSO/federated roles over long-lived IAM credentials. + - [AWS IAM Best Practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) +2. **Enhance Network Security:** + - Consider setting up [AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/userguide/endpoint-services-overview.html) to securely connect to Bedrock. +3. **Monitor and Log Activity:** + - Enable AWS CloudTrail to log Bedrock API calls. + - Use CloudWatch to monitor metrics like invocation count, latency, and token usage. + - Set up alerts for abnormal activity. +4. **Handle Errors and Manage Costs:** + - Implement exponential backoff for throttling errors. + - Use AWS Cost Explorer and set billing alerts to track usage.\ + [AWS Cost Management](https://docs.aws.amazon.com/cost-management/latest/userguide/what-is-aws-cost-management.html) +5. **Regular Audits and Compliance:** + - Periodically review IAM roles and CloudTrail logs. + - Follow internal data privacy and governance policies. + +--- + +### Conclusion + +By following these steps, your enterprise team can securely integrate AWS Bedrock with the Cline VS Code extension to accelerate development: + +1. **Prepare Your AWS Environment:** Create or use a secure IAM role/user, attach the `AmazonBedrockFullAccess` policy, and ensure necessary permissions. +2. **Verify Region and Model Access:** Confirm that your selected region supports your required models and subscribe via AWS Marketplace if needed. +3. **Configure Cline in VS Code:** Install and set up Cline with your AWS credentials and choose an appropriate model. +4. **Implement Security and Monitoring:** Use best practices for IAM, network security, monitoring, and cost management. + +For further details, consult the [AWS Bedrock Documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) and coordinate with your internal cloud team. Happy coding! + +--- + +_This guide will be updated as AWS Bedrock and Cline evolve. Always refer to the latest documentation and internal policies for up-to-date practices._ diff --git a/docs/custom-model-configs/aws-bedrock-with-profile-authentication.mdx b/docs/custom-model-configs/aws-bedrock-with-profile-authentication.mdx new file mode 100644 index 0000000000..b46b4be00a --- /dev/null +++ b/docs/custom-model-configs/aws-bedrock-with-profile-authentication.mdx @@ -0,0 +1,42 @@ +--- +title: "AWS Bedrock w/ Profile Authentication" +description: "Learn how to configure AWS Bedrock to use AWS Profiles for authentication with Cline, focusing on SSO/Federated roles for secure access." +--- + +### Overview + +Cline offers the option of utilizing AWS credentials or AWS profiles to access AWS Bedrock services. SSO/Federated roles are suggested over Legacy IAM configuration; this guide describes how to configure your environment so that Cline uses SSO roles for authentication. + +--- + +### Configuration Steps + +1. Install the [latest version](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) of AWS CLI + + - Follow the AWS docs to install your OS-specific version of AWS CLI + +2. [Configure IAM authentication](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html) with the AWS CLI + + - If you do not already have AWS access through the IAM Identity Center, follow the [IAM User Guide](https://docs.aws.amazon.com/singlesignon/latest/userguide/getting-started.html) to set up IAM users and roles. Ensure you have a `PowerUserAccess` role. + - If you have access to AWS through your employer, open your AWS access portal and find the appropriate account. Ensure you have `PowerUserAccess` permissions. + - Open the `Access keys` link and note the `SSO start URL` and `SSO region`, which are needed in the next step + +3. Continue configuring your profile using [the `aws configure sso` CLI wizard](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html#cli-configure-sso-configure) + + - Once configured, use the following command to authenticate the AWS CLI: `aws sso login --profile ` + - Note which profile name you attach to your AWS account, this is needed to configure Cline in the following steps + +4. If you haven't already done so, install VSCode and the Cline extension. Consult the [Getting Started](/getting-started) page for guidance. + +5. Open the Cline extension, then click on the settings button ⚙️ to select your API Provider. + - From the API Provider dropdown, select AWS Bedrock + - Select the AWS Profile radio button, then enter the AWS Profile Name from step 3 + - Select your AWS Region from the dropdown menu + - Selecting the cross-region inference checkbox is required for some models + + + AWS Bedrock configuration in Cline settings showing profile authentication setup + diff --git a/docs/custom-model-configs/gcp-vertex-ai.mdx b/docs/custom-model-configs/gcp-vertex-ai.mdx new file mode 100644 index 0000000000..76531d95b8 --- /dev/null +++ b/docs/custom-model-configs/gcp-vertex-ai.mdx @@ -0,0 +1,206 @@ +--- +title: "GCP Vertex AI" +description: "Configure GCP Vertex AI with Cline to access leading generative AI models like Claude 3.5 Sonnet v2. This guide covers GCP environment setup, authentication, and secure integration for enterprise teams." +--- + +### Overview + +**GCP Vertex AI:**\ +A fully managed service that provides access to leading generative AI models—such as Anthropic's Claude 3.5 Sonnet v2—through Google Cloud.\ +[Learn more about GCP Vertex AI](https://cloud.google.com/vertex-ai). + +This guide is tailored for organizations with established GCP environments (leveraging IAM roles, service accounts, and best practices in resource management) to ensure secure and compliant usage. + +--- + +### Step 1: Prepare Your GCP Environment + +#### 1.1 Create or Use a GCP Project + +- **Sign in to the GCP Console:**\ + [Google Cloud Console](https://console.cloud.google.com/) +- **Select or Create a Project:**\ + Use an existing project or create a new one dedicated to Vertex AI. + +#### 1.2 Set Up IAM Permissions and Service Accounts + +- **Assign Required Roles:** + + - Grant your user (or service account) the **Vertex AI User** role (`roles/aiplatform.user`) + - For service accounts, also attach the **Vertex AI Service Agent** role (`roles/aiplatform.serviceAgent`) to enable certain operations + - Consider additional predefined roles as needed: + - Vertex AI Platform Express Admin + - Vertex AI Platform Express User + - Vertex AI Migration Service User + +- **Cross-Project Resource Access:** + - For BigQuery tables in different projects, assign the **BigQuery Data Viewer** role + - For Cloud Storage buckets in different projects, assign the **Storage Object Viewer** role + - For external data sources, refer to the [GCP Vertex AI Access Control documentation](https://cloud.google.com/vertex-ai/docs/general/access-control) + +--- + +### Step 2: Verify Regional and Model Access + +#### 2.1 Choose and Confirm a Region + +Vertex AI supports eight regions. Select a region that meets your latency, compliance, and capacity needs. Examples include: + +- **us-east5 (Columbus, Ohio)** +- **us-east1 (South Carolina)** +- **us-east4 (Northern Virginia)** +- **us-central1 (Iowa)** +- **us-west1 (The Dalles, Oregon)** +- **us-west4 (Las Vegas, Nevada)** +- **europe-west1 (Belgium)** +- **asia-southeast1 (Singapore)** + +#### 2.2 Enable the Claude 3.5 Sonnet v2 Model + +- **Open Vertex AI Model Garden:**\ + In the Cloud Console, navigate to **Vertex AI → Model Garden** +- **Enable Claude 3.5 Sonnet v2:**\ + Locate the model card for Claude 3.5 Sonnet v2 and click **Enable** + +--- + +### Step 3: Configure the Cline VS Code Extension + +#### 3.1 Install and Open Cline + +- **Download VS Code:**\ + [Download Visual Studio Code](https://code.visualstudio.com/) +- **Install the Cline Extension:** + - Open VS Code + - Navigate to the Extensions Marketplace (Ctrl+Shift+X or Cmd+Shift+X) + - Search for **Cline** and install the extension + + + Cline extension in VS Code + + +#### 3.2 Configure Cline Settings + +- **Open Cline Settings:**\ + Click the settings ⚙️ icon within the Cline extension +- **Set API Provider:**\ + Choose **GCP Vertex AI** from the API Provider dropdown +- **Enter Your Google Cloud Project ID:**\ + Provide the project ID you set up earlier +- **Select the Region:**\ + Choose one of the supported regions (e.g., `us-east5`) +- **Select the Model:**\ + From the available list, choose **Claude 3.5 Sonnet v2** +- **Save and Test:**\ + Save your settings and test by sending a simple prompt (e.g., "Generate a Python function to check if a number is prime.") + +--- + +### Step 4: Authentication and Credentials Setup + +#### Option A: Using Your Google Account (User Credentials) + +1. **Install the Google Cloud CLI:**\ + Follow the [installation guide](https://cloud.google.com/sdk/docs/install) +2. **Initialize and Authenticate:** + + ```bash + gcloud init + gcloud auth application-default login + ``` + + - This sets up Application Default Credentials (ADC) using your Google account + +3. **Restart VS Code:**\ + Ensure VS Code is restarted so that the Cline extension picks up the new credentials + +#### Option B: Using a Service Account (JSON Key) + +1. **Create a Service Account:** + + - In the GCP Console, navigate to **IAM & Admin > Service Accounts** + - Create a new service account (e.g., "vertex-ai-client") + +2. **Assign Roles:** + + - Attach **Vertex AI User** (`roles/aiplatform.user`) + - Attach **Vertex AI Service Agent** (`roles/aiplatform.serviceAgent`) + - Optionally, add other roles as required + +3. **Generate a JSON Key:** + + - In the Service Accounts section, manage keys for your service account and download the JSON key + +4. **Set the Environment Variable:** + + ```bash + export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/service-account-key.json" + ``` + + - This instructs Google Cloud client libraries (and Cline) to use this key + +5. **Restart VS Code:**\ + Launch VS Code from a terminal where the `GOOGLE_APPLICATION_CREDENTIALS` variable is set + +--- + +### Step 5: Security, Monitoring, and Best Practices + +#### 5.1 Enforce Least Privilege + +- **Principle of Least Privilege:**\ + Only grant the minimum necessary permissions. Custom roles can offer finer control compared to broad predefined roles +- **Best Practices:**\ + Refer to [GCP IAM Best Practices](https://cloud.google.com/iam/docs/best-practices) + +#### 5.2 Manage Resource Access + +- **Project vs. Resource-Level Access:**\ + Access can be managed at both levels. Note that resource-level permissions (e.g., for BigQuery or Cloud Storage) add to, but do not override, project-level policies + +#### 5.3 Monitor Usage and Quotas + +- **Model Observability Dashboard:** + + - In the Vertex AI Console, navigate to the **Model Observability** dashboard + - Monitor metrics such as request throughput, latency, and error rates (including 429 quota errors) + +- **Quota Management:** + - If you encounter 429 errors, check the **IAM & Admin > Quotas** page + - Request a quota increase if necessary\ + [Learn more about GCP Vertex AI Quotas](https://cloud.google.com/vertex-ai/docs/quotas) + +#### 5.4 Service Agents and Cross-Project Considerations + +- **Service Agents:**\ + Be aware of the different service agents: + + - Vertex AI Service Agent + - Vertex AI RAG Data Service Agent + - Vertex AI Custom Code Service Agent + - Vertex AI Extension Service Agent + +- **Cross-Project Access:**\ + For resources in other projects (e.g., BigQuery, Cloud Storage), ensure that the appropriate roles (BigQuery Data Viewer, Storage Object Viewer) are assigned + +--- + +### Conclusion + +By following these steps, your enterprise team can securely integrate GCP Vertex AI with the Cline VS Code extension to harness the power of **Claude 3.5 Sonnet v2**: + +- **Prepare Your GCP Environment:**\ + Create or use a project, configure IAM with least privilege, and ensure necessary roles (including the Vertex AI Service Agent role) are attached +- **Verify Regional and Model Access:**\ + Confirm that your chosen region supports Claude 3.5 Sonnet v2 and that the model is enabled +- **Configure Cline in VS Code:**\ + Install Cline, enter your project ID, select the appropriate region, and choose the model +- **Set Up Authentication:**\ + Use either user credentials (via `gcloud auth application-default login`) or a service account with a JSON key +- **Implement Security and Monitoring:**\ + Adhere to best practices for IAM, manage resource access carefully, and monitor usage with the Model Observability dashboard + +For further details, please consult the [GCP Vertex AI Documentation](https://cloud.google.com/vertex-ai/docs) and your internal security policies.\ +Happy coding! + +_This guide will be updated as GCP Vertex AI and Cline evolve. Always refer to the latest documentation for current practices._ diff --git a/docs/custom-model-configs/litellm-and-cline-using-codestral.mdx b/docs/custom-model-configs/litellm-and-cline-using-codestral.mdx new file mode 100644 index 0000000000..a1af3483a3 --- /dev/null +++ b/docs/custom-model-configs/litellm-and-cline-using-codestral.mdx @@ -0,0 +1,65 @@ +--- +title: "LiteLLM & Cline (using Codestral)" +description: "Learn how to set up and run LiteLLM with Cline using the Codestral model. This guide covers Docker setup, configuration, and integration with Cline." +--- + +### Using LiteLLM with Cline + +This guide demonstrates how to run a demo for LiteLLM starting with the Codestral model for use with Cline. + +#### Prerequisites + +- [Docker CLI or Docker Desktop](https://www.docker.com/get-started/) installed to run the LiteLLM image locally +- For this example config: A Codestral API Key (different from the Mistral API Keys) + +#### Setup + +1. **Create a `.env` file and fill in the appropriate field** + + ```bash + # Tip: Use the following command to generate a random alphanumeric key: + # openssl rand -base64 32 | tr -dc 'A-Za-z0-9' | head -c 32 + LITELLM_MASTER_KEY=YOUR_LITELLM_MASTER_KEY + CODESTRAL_API_KEY=YOUR_CODESTRAL_API_KEY + ``` + + _Note: Although this is limited to localhost, it's a good practice set LITELLM_MASTER_KEY to something secure_ + +2. **Configuration** + + We'll need to create a `config.yaml` file to contain our LiteLLM configuration. In this case we'll just have one model, 'codestral-latest' and label it 'codestral' + + ```yaml + model_list: + - model_name: codestral + litellm_params: + model: codestral/codestral-latest + api_key: os.environ/CODESTRAL_API_KEY + ``` + +#### Running the Demo + +1. **Startup the LiteLLM docker container** + + ```bash + docker run \ + --env-file .env \ + -v $(pwd)/config.yaml:/app/config.yaml \ + -p 127.0.0.1:4000:4000 \ + ghcr.io/berriai/litellm:main-latest \ + --config /app/config.yaml --detailed_debug + ``` + +2. **Setup Cline** + + Once the LiteLLM server is up and running you can set it up in Cline: + + - Base URL should be `http://0.0.0.0:4000/v1` + - API Key should be the one you set in `.env` for LITELLM_MASTER_KEY + - Model ID is `codestral` or whatever you named it under `config.yaml` + +#### Getting Help + +- [LiteLLM Documentation](https://docs.litellm.ai/) +- [Mistral AI Console](https://console.mistral.ai/) +- [Cline Discord Community](https://discord.gg/cline) diff --git a/docs/docs.json b/docs/docs.json index 02544422ed..dbaf33e3e7 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -88,6 +88,15 @@ { "group": "Improving Your Prompting Skills", "pages": ["prompting/prompt-engineering-guide", "prompting/cline-memory-bank"] + }, + { + "group": "Custom Model Configurations", + "pages": [ + "custom-model-configs/aws-bedrock-with-credentials-authentication", + "custom-model-configs/aws-bedrock-with-profile-authentication", + "custom-model-configs/gcp-vertex-ai", + "custom-model-configs/litellm-and-cline-using-codestral" + ] } ] },