mirror of
https://github.com/aiclientproxy/proxycast.git
synced 2026-09-24 23:10:56 +08:00
chore: bump version to 0.48.4
This commit is contained in:
@@ -0,0 +1,121 @@
|
||||
# ProxyCast 测试体系
|
||||
|
||||
> 基于 Anthropic AI Agent 评估指南与 Orchids Bridge 项目实践
|
||||
|
||||
## 概述
|
||||
|
||||
ProxyCast 作为 AI API 代理和 Agent 集成平台,需要一套完整的测试体系来确保:
|
||||
- API 代理的正确性和稳定性
|
||||
- 凭证池管理的可靠性
|
||||
- Aster Agent 集成的功能完整性
|
||||
- 协议转换的准确性
|
||||
|
||||
## 测试分层
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ ProxyCast 测试金字塔 │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────┐ │
|
||||
│ │ E2E │ 端到端测试 │
|
||||
│ │ 测试 │ (Tauri + 前端) │
|
||||
│ ─┴─────────┴─ │
|
||||
│ ┌─────────────┐ │
|
||||
│ │ 集成测试 │ API 服务器、凭证池 │
|
||||
│ ─┴─────────────┴─ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ 单元测试 │ 转换器、Provider、工具 │
|
||||
│ ─┴─────────────────┴─ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
docs/test/
|
||||
├── README.md # 本文件 - 测试体系概览
|
||||
├── unit-tests.md # 单元测试指南
|
||||
├── integration-tests.md # 集成测试指南
|
||||
├── e2e-tests.md # 端到端测试指南
|
||||
├── agent-evaluation.md # Agent 评估指南(核心文档)
|
||||
└── test-cases/ # 测试用例模板
|
||||
├── converter-tests.md # 协议转换器测试用例
|
||||
├── provider-tests.md # Provider 测试用例
|
||||
└── agent-tests.md # Agent 测试用例
|
||||
```
|
||||
|
||||
## 文档索引
|
||||
|
||||
| 文档 | 说明 | 适用场景 |
|
||||
|------|------|----------|
|
||||
| [unit-tests.md](unit-tests.md) | 单元测试指南 | 独立模块测试 |
|
||||
| [integration-tests.md](integration-tests.md) | 集成测试指南 | 模块间协作测试 |
|
||||
| [e2e-tests.md](e2e-tests.md) | E2E 测试指南 | 完整用户流程测试 |
|
||||
| [agent-evaluation.md](agent-evaluation.md) | Agent 评估指南 | AI Agent 行为评估 |
|
||||
| [test-cases/converter-tests.md](test-cases/converter-tests.md) | 转换器测试用例 | OpenAI ↔ Claude 转换 |
|
||||
| [test-cases/provider-tests.md](test-cases/provider-tests.md) | Provider 测试用例 | OAuth 和 API 调用 |
|
||||
| [test-cases/agent-tests.md](test-cases/agent-tests.md) | Agent 测试用例 | Aster Agent 集成 |
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 运行 Rust 测试
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo test
|
||||
```
|
||||
|
||||
### 运行前端测试
|
||||
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
|
||||
### 运行代码检查
|
||||
|
||||
```bash
|
||||
# Rust
|
||||
cd src-tauri && cargo clippy
|
||||
|
||||
# 前端
|
||||
npm run lint
|
||||
```
|
||||
|
||||
## 核心测试模块
|
||||
|
||||
| 模块 | 测试重点 | 文档 |
|
||||
|------|----------|------|
|
||||
| 协议转换 | OpenAI ↔ Claude 转换正确性 | [converter-tests.md](test-cases/converter-tests.md) |
|
||||
| Provider 系统 | OAuth 刷新、API 调用 | [provider-tests.md](test-cases/provider-tests.md) |
|
||||
| 凭证池 | 轮询、健康检查、负载均衡 | [integration-tests.md](integration-tests.md) |
|
||||
| Aster Agent | 流式响应、工具调用 | [agent-tests.md](test-cases/agent-tests.md) |
|
||||
|
||||
## 测试原则
|
||||
|
||||
基于 [Anthropic AI Agent 评估指南](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) 和 Orchids Bridge 项目实践:
|
||||
|
||||
1. **评估结果,而非路径** - Agent 可能找到更好的方法,不要过度约束执行路径
|
||||
2. **平衡问题集** - 测试"应该做"和"不应该做"两种情况
|
||||
3. **隔离测试环境** - 每个测试独立状态,避免测试间污染
|
||||
4. **从 Bug 到测试** - 每个修复的 Bug 都应该有对应测试用例
|
||||
5. **处理非确定性** - 使用 pass@k 和 pass^k 指标评估 Agent 行为
|
||||
6. **多层防护** - 结合自动评估、监控、人工审查
|
||||
|
||||
## 评分器类型
|
||||
|
||||
| 类型 | 适用场景 | 优点 | 缺点 |
|
||||
|------|----------|------|------|
|
||||
| **代码评分器** | 确定性验证 | 快速、可复现 | 对有效变体脆弱 |
|
||||
| **模型评分器** | 语义评估 | 灵活、可扩展 | 非确定性、需校准 |
|
||||
| **人工评分器** | 复杂判断 | 金标准质量 | 昂贵、慢 |
|
||||
|
||||
## 评估指标
|
||||
|
||||
```
|
||||
pass@k = P(至少 1 次成功 | k 次尝试) = 1 - (1 - p)^k
|
||||
pass^k = P(全部成功 | k 次尝试) = p^k
|
||||
```
|
||||
|
||||
- **pass@k**:适用于"找到一个解决方案就行"的场景
|
||||
- **pass^k**:适用于"每次都必须成功"的场景
|
||||
@@ -0,0 +1,272 @@
|
||||
# ProxyCast Agent 评估指南
|
||||
|
||||
> 基于 Anthropic AI Agent 评估指南的实践
|
||||
|
||||
## 概述
|
||||
|
||||
ProxyCast 集成了 Aster Agent,需要专门的评估体系来确保 Agent 行为的正确性和稳定性。本指南基于 Anthropic 官方评估指南和 Orchids Bridge 项目的实践经验。
|
||||
|
||||
## 核心概念
|
||||
|
||||
### 评估术语
|
||||
|
||||
| 术语 | 定义 | ProxyCast 示例 |
|
||||
|------|------|----------------|
|
||||
| **Task** | 单个测试任务 | "使用 Agent 读取文件并总结" |
|
||||
| **Trial** | 对任务的一次尝试 | 同一任务运行 5 次 |
|
||||
| **Grader** | 评分器 | 代码检查、LLM 判断 |
|
||||
| **Transcript** | 完整记录 | Agent 的所有消息和工具调用 |
|
||||
| **Outcome** | 最终结果 | 任务是否完成 |
|
||||
|
||||
### 评分器类型
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ 评分器类型 │
|
||||
├─────────────────┬─────────────────┬─────────────────────────────┤
|
||||
│ 代码评分器 │ 模型评分器 │ 人工评分器 │
|
||||
├─────────────────┼─────────────────┼─────────────────────────────┤
|
||||
│ • 工具调用验证 │ • 回答质量评估 │ • 复杂任务评审 │
|
||||
│ • 输出格式检查 │ • 语义相似度 │ • 边界情况判断 │
|
||||
│ • 状态断言 │ • 多轮对话评估 │ • 用户体验评估 │
|
||||
└─────────────────┴─────────────────┴─────────────────────────────┘
|
||||
```
|
||||
|
||||
## 评估场景
|
||||
|
||||
### 1. 工具调用评估
|
||||
|
||||
验证 Agent 正确调用工具:
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod agent_tool_tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_file_read_tool_call() {
|
||||
let agent = create_test_agent().await;
|
||||
|
||||
let response = agent.chat("请读取 /test/file.txt 的内容").await;
|
||||
|
||||
// 验证工具调用
|
||||
assert!(response.tool_calls.iter().any(|tc| {
|
||||
tc.name == "read_file" &&
|
||||
tc.args.get("path") == Some(&"/test/file.txt".into())
|
||||
}));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_no_unnecessary_tool_calls() {
|
||||
let agent = create_test_agent().await;
|
||||
|
||||
// 简单问题不应该调用工具
|
||||
let response = agent.chat("1 + 1 等于多少?").await;
|
||||
|
||||
assert!(response.tool_calls.is_empty());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 流式响应评估
|
||||
|
||||
验证流式输出的正确性:
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_streaming_response_format() {
|
||||
let agent = create_test_agent().await;
|
||||
let mut stream = agent.chat_stream("你好").await;
|
||||
|
||||
let mut events = Vec::new();
|
||||
while let Some(event) = stream.next().await {
|
||||
events.push(event);
|
||||
}
|
||||
|
||||
// 验证事件序列
|
||||
assert!(events.iter().any(|e| matches!(e, StreamEvent::Start)));
|
||||
assert!(events.iter().any(|e| matches!(e, StreamEvent::Delta(_))));
|
||||
assert!(events.iter().any(|e| matches!(e, StreamEvent::Stop)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_streaming_content_accumulation() {
|
||||
let agent = create_test_agent().await;
|
||||
let mut stream = agent.chat_stream("写一首短诗").await;
|
||||
|
||||
let mut content = String::new();
|
||||
while let Some(event) = stream.next().await {
|
||||
if let StreamEvent::Delta(delta) = event {
|
||||
content.push_str(&delta);
|
||||
}
|
||||
}
|
||||
|
||||
// 验证内容非空且有意义
|
||||
assert!(!content.is_empty());
|
||||
assert!(content.len() > 20);
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 错误处理评估
|
||||
|
||||
验证 Agent 正确处理错误:
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_invalid_tool_graceful_handling() {
|
||||
let agent = create_test_agent().await;
|
||||
|
||||
// 请求不存在的文件
|
||||
let response = agent.chat("读取 /nonexistent/file.txt").await;
|
||||
|
||||
// Agent 应该优雅处理错误
|
||||
assert!(response.content.contains("文件不存在") ||
|
||||
response.content.contains("无法找到"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_timeout_handling() {
|
||||
let agent = create_test_agent_with_timeout(Duration::from_secs(1)).await;
|
||||
|
||||
// 长时间任务应该超时
|
||||
let result = agent.chat("执行一个需要很长时间的任务").await;
|
||||
|
||||
assert!(result.is_err() || result.unwrap().content.contains("超时"));
|
||||
}
|
||||
```
|
||||
|
||||
## 评估指标
|
||||
|
||||
### pass@k 与 pass^k
|
||||
|
||||
```
|
||||
pass@k = P(至少 1 次成功 | k 次尝试)
|
||||
pass^k = P(全部成功 | k 次尝试)
|
||||
```
|
||||
|
||||
**应用场景**:
|
||||
- **pass@k**:代码生成、创意任务(找到一个解决方案即可)
|
||||
- **pass^k**:关键操作、用户交互(每次都必须成功)
|
||||
|
||||
### 评估脚本
|
||||
|
||||
```rust
|
||||
async fn evaluate_task(task: &Task, trials: usize) -> EvalResult {
|
||||
let mut successes = 0;
|
||||
let mut transcripts = Vec::new();
|
||||
|
||||
for _ in 0..trials {
|
||||
let agent = create_fresh_agent().await;
|
||||
let transcript = agent.run_task(task).await;
|
||||
|
||||
let passed = task.grader.evaluate(&transcript);
|
||||
if passed {
|
||||
successes += 1;
|
||||
}
|
||||
|
||||
transcripts.push(transcript);
|
||||
}
|
||||
|
||||
EvalResult {
|
||||
task_id: task.id.clone(),
|
||||
trials,
|
||||
successes,
|
||||
pass_at_k: 1.0 - (1.0 - successes as f64 / trials as f64).powi(trials as i32),
|
||||
pass_pow_k: (successes as f64 / trials as f64).powi(trials as i32),
|
||||
transcripts,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 测试套件组织
|
||||
|
||||
### 能力评估 vs 回归评估
|
||||
|
||||
| 类型 | 目标 | 初始通过率 | 用途 |
|
||||
|------|------|-----------|------|
|
||||
| **能力评估** | Agent 能做什么? | 低 | 推动改进 |
|
||||
| **回归评估** | Agent 还能做以前能做的吗? | ~100% | 防止退化 |
|
||||
|
||||
### 测试套件结构
|
||||
|
||||
```
|
||||
tests/agent/
|
||||
├── capability/ # 能力评估
|
||||
│ ├── file_operations.rs # 文件操作能力
|
||||
│ ├── code_generation.rs # 代码生成能力
|
||||
│ └── reasoning.rs # 推理能力
|
||||
├── regression/ # 回归评估
|
||||
│ ├── basic_chat.rs # 基础对话
|
||||
│ ├── tool_calls.rs # 工具调用
|
||||
│ └── streaming.rs # 流式响应
|
||||
└── edge_cases/ # 边界情况
|
||||
├── error_handling.rs
|
||||
└── timeout.rs
|
||||
```
|
||||
|
||||
## 评估原则
|
||||
|
||||
### 1. 评估结果,而非路径
|
||||
|
||||
```rust
|
||||
// ❌ 错误:检查具体的工具调用顺序
|
||||
fn test_bad() {
|
||||
assert_eq!(transcript[0].tool, "list_files");
|
||||
assert_eq!(transcript[1].tool, "read_file");
|
||||
}
|
||||
|
||||
// ✅ 正确:检查最终结果
|
||||
fn test_good() {
|
||||
assert!(outcome.file_content.contains("expected content"));
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 平衡问题集
|
||||
|
||||
```rust
|
||||
// 测试"应该做"
|
||||
#[test]
|
||||
fn test_should_read_file_when_asked() { ... }
|
||||
|
||||
// 测试"不应该做"
|
||||
#[test]
|
||||
fn test_should_not_read_file_without_permission() { ... }
|
||||
```
|
||||
|
||||
### 3. 从 Bug 到测试
|
||||
|
||||
每个修复的 Bug 都应该有对应的测试用例:
|
||||
|
||||
```rust
|
||||
// Bug: Agent 在文件不存在时无限重试
|
||||
// 修复后添加测试
|
||||
#[test]
|
||||
fn test_no_infinite_retry_on_missing_file() {
|
||||
let agent = create_test_agent();
|
||||
let response = agent.chat("读取 /nonexistent.txt").await;
|
||||
|
||||
// 验证重试次数有限
|
||||
assert!(response.tool_calls.len() <= 3);
|
||||
}
|
||||
```
|
||||
|
||||
## 运行评估
|
||||
|
||||
```bash
|
||||
# 运行所有 Agent 评估
|
||||
cd src-tauri && cargo test agent::
|
||||
|
||||
# 运行能力评估
|
||||
cargo test agent::capability::
|
||||
|
||||
# 运行回归评估
|
||||
cargo test agent::regression::
|
||||
|
||||
# 运行多次试验
|
||||
cargo test agent:: -- --test-threads=1 --nocapture
|
||||
```
|
||||
|
||||
## 下一步
|
||||
|
||||
- [测试用例:Agent](test-cases/agent-tests.md)
|
||||
- [单元测试指南](unit-tests.md)
|
||||
@@ -0,0 +1,264 @@
|
||||
# ProxyCast E2E 测试指南
|
||||
|
||||
> 端到端测试验证完整用户流程
|
||||
|
||||
## 概述
|
||||
|
||||
E2E 测试模拟真实用户操作,验证从前端到后端的完整流程。ProxyCast 使用 Tauri 框架,E2E 测试需要覆盖:
|
||||
- 桌面应用启动和初始化
|
||||
- 用户界面交互
|
||||
- API 代理完整流程
|
||||
- 凭证管理流程
|
||||
|
||||
## 测试框架
|
||||
|
||||
### Tauri E2E 测试
|
||||
|
||||
使用 `tauri-driver` 进行自动化测试:
|
||||
|
||||
```bash
|
||||
# 安装依赖
|
||||
cargo install tauri-driver
|
||||
|
||||
# 运行 E2E 测试
|
||||
npm run test:e2e
|
||||
```
|
||||
|
||||
### 测试配置
|
||||
|
||||
```javascript
|
||||
// playwright.config.ts
|
||||
import { defineConfig } from '@playwright/test';
|
||||
|
||||
export default defineConfig({
|
||||
testDir: './tests/e2e',
|
||||
timeout: 30000,
|
||||
use: {
|
||||
baseURL: 'tauri://localhost',
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## 测试场景
|
||||
|
||||
### 1. 应用启动流程
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('应用启动', () => {
|
||||
test('应用正常启动并显示主界面', async ({ page }) => {
|
||||
// 等待应用加载
|
||||
await page.waitForSelector('[data-testid="main-layout"]');
|
||||
|
||||
// 验证核心组件存在
|
||||
await expect(page.locator('[data-testid="sidebar"]')).toBeVisible();
|
||||
await expect(page.locator('[data-testid="content-area"]')).toBeVisible();
|
||||
});
|
||||
|
||||
test('首次启动显示欢迎引导', async ({ page }) => {
|
||||
// 清除本地存储模拟首次启动
|
||||
await page.evaluate(() => localStorage.clear());
|
||||
await page.reload();
|
||||
|
||||
await expect(page.locator('[data-testid="welcome-modal"]')).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 2. 凭证管理流程
|
||||
|
||||
```typescript
|
||||
test.describe('凭证管理', () => {
|
||||
test('添加 Kiro 凭证', async ({ page }) => {
|
||||
// 打开凭证管理
|
||||
await page.click('[data-testid="credentials-tab"]');
|
||||
await page.click('[data-testid="add-credential-btn"]');
|
||||
|
||||
// 选择 Provider
|
||||
await page.click('[data-testid="provider-kiro"]');
|
||||
|
||||
// 上传凭证文件
|
||||
const fileInput = page.locator('input[type="file"]');
|
||||
await fileInput.setInputFiles('./tests/fixtures/test-credential.json');
|
||||
|
||||
// 验证凭证添加成功
|
||||
await expect(page.locator('[data-testid="credential-item"]')).toBeVisible();
|
||||
await expect(page.locator('text=test@example.com')).toBeVisible();
|
||||
});
|
||||
|
||||
test('删除凭证', async ({ page }) => {
|
||||
// 假设已有凭证
|
||||
await page.click('[data-testid="credentials-tab"]');
|
||||
|
||||
// 删除凭证
|
||||
await page.click('[data-testid="credential-menu"]');
|
||||
await page.click('[data-testid="delete-credential"]');
|
||||
await page.click('[data-testid="confirm-delete"]');
|
||||
|
||||
// 验证凭证已删除
|
||||
await expect(page.locator('[data-testid="credential-item"]')).not.toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 3. API 代理流程
|
||||
|
||||
```typescript
|
||||
test.describe('API 代理', () => {
|
||||
test('启动代理服务器', async ({ page }) => {
|
||||
await page.click('[data-testid="server-tab"]');
|
||||
await page.click('[data-testid="start-server-btn"]');
|
||||
|
||||
// 等待服务器启动
|
||||
await expect(page.locator('text=服务器运行中')).toBeVisible();
|
||||
await expect(page.locator('[data-testid="server-port"]')).toContainText('8080');
|
||||
});
|
||||
|
||||
test('代理请求成功', async ({ page, request }) => {
|
||||
// 启动服务器
|
||||
await page.click('[data-testid="start-server-btn"]');
|
||||
await page.waitForSelector('text=服务器运行中');
|
||||
|
||||
// 发送测试请求
|
||||
const response = await request.post('http://localhost:8080/v1/chat/completions', {
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Authorization': 'Bearer test-key',
|
||||
},
|
||||
data: {
|
||||
model: 'gpt-4',
|
||||
messages: [{ role: 'user', content: 'Hello' }],
|
||||
},
|
||||
});
|
||||
|
||||
expect(response.ok()).toBeTruthy();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### 4. Agent 对话流程
|
||||
|
||||
```typescript
|
||||
test.describe('Agent 对话', () => {
|
||||
test('发送消息并接收响应', async ({ page }) => {
|
||||
await page.click('[data-testid="agent-tab"]');
|
||||
|
||||
// 输入消息
|
||||
await page.fill('[data-testid="message-input"]', '你好,请介绍一下自己');
|
||||
await page.click('[data-testid="send-btn"]');
|
||||
|
||||
// 等待响应
|
||||
await expect(page.locator('[data-testid="assistant-message"]')).toBeVisible({
|
||||
timeout: 30000,
|
||||
});
|
||||
});
|
||||
|
||||
test('流式响应正确显示', async ({ page }) => {
|
||||
await page.click('[data-testid="agent-tab"]');
|
||||
await page.fill('[data-testid="message-input"]', '写一首短诗');
|
||||
await page.click('[data-testid="send-btn"]');
|
||||
|
||||
// 验证流式显示(内容逐渐增加)
|
||||
const messageEl = page.locator('[data-testid="assistant-message"]');
|
||||
|
||||
let prevLength = 0;
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await page.waitForTimeout(500);
|
||||
const text = await messageEl.textContent();
|
||||
expect(text?.length).toBeGreaterThan(prevLength);
|
||||
prevLength = text?.length || 0;
|
||||
}
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## 测试数据管理
|
||||
|
||||
### Fixtures
|
||||
|
||||
```
|
||||
tests/
|
||||
├── fixtures/
|
||||
│ ├── test-credential.json # 测试凭证
|
||||
│ ├── mock-responses/ # Mock API 响应
|
||||
│ │ ├── chat-completion.json
|
||||
│ │ └── streaming-response.txt
|
||||
│ └── test-config.json # 测试配置
|
||||
└── e2e/
|
||||
└── *.spec.ts
|
||||
```
|
||||
|
||||
### Mock 服务
|
||||
|
||||
```typescript
|
||||
// tests/mocks/api-server.ts
|
||||
import { setupServer } from 'msw/node';
|
||||
import { rest } from 'msw';
|
||||
|
||||
export const mockServer = setupServer(
|
||||
rest.post('*/v1/chat/completions', (req, res, ctx) => {
|
||||
return res(
|
||||
ctx.json({
|
||||
id: 'test-id',
|
||||
choices: [{
|
||||
message: { role: 'assistant', content: 'Mock response' },
|
||||
}],
|
||||
})
|
||||
);
|
||||
})
|
||||
);
|
||||
```
|
||||
|
||||
## 运行 E2E 测试
|
||||
|
||||
```bash
|
||||
# 构建应用
|
||||
npm run build
|
||||
|
||||
# 运行 E2E 测试
|
||||
npm run test:e2e
|
||||
|
||||
# 运行特定测试
|
||||
npm run test:e2e -- --grep "凭证管理"
|
||||
|
||||
# 生成测试报告
|
||||
npm run test:e2e -- --reporter=html
|
||||
```
|
||||
|
||||
## CI/CD 集成
|
||||
|
||||
```yaml
|
||||
# .github/workflows/e2e.yml
|
||||
name: E2E Tests
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
e2e:
|
||||
runs-on: macos-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Setup Rust
|
||||
uses: dtolnay/rust-toolchain@stable
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Build app
|
||||
run: npm run build
|
||||
|
||||
- name: Run E2E tests
|
||||
run: npm run test:e2e
|
||||
```
|
||||
|
||||
## 下一步
|
||||
|
||||
- [Agent 评估指南](agent-evaluation.md)
|
||||
- [测试用例:Agent](test-cases/agent-tests.md)
|
||||
@@ -0,0 +1,229 @@
|
||||
# ProxyCast 集成测试指南
|
||||
|
||||
> 测试模块间的协作和数据流
|
||||
|
||||
## 概述
|
||||
|
||||
集成测试验证多个模块协同工作的正确性,主要覆盖:
|
||||
- API 服务器端点
|
||||
- 凭证池管理
|
||||
- Provider 与服务层交互
|
||||
- 数据库操作
|
||||
|
||||
## 测试场景
|
||||
|
||||
### 1. API 服务器集成
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod api_integration_tests {
|
||||
use super::*;
|
||||
use axum::http::StatusCode;
|
||||
use tower::ServiceExt;
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_chat_completion_endpoint() {
|
||||
let app = create_test_app().await;
|
||||
|
||||
let request = Request::builder()
|
||||
.method("POST")
|
||||
.uri("/v1/chat/completions")
|
||||
.header("Content-Type", "application/json")
|
||||
.header("Authorization", "Bearer test-key")
|
||||
.body(Body::from(r#"{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello"}]
|
||||
}"#))
|
||||
.unwrap();
|
||||
|
||||
let response = app.oneshot(request).await.unwrap();
|
||||
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_streaming_response() {
|
||||
let app = create_test_app().await;
|
||||
|
||||
let request = Request::builder()
|
||||
.method("POST")
|
||||
.uri("/v1/chat/completions")
|
||||
.header("Content-Type", "application/json")
|
||||
.body(Body::from(r#"{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"stream": true
|
||||
}"#))
|
||||
.unwrap();
|
||||
|
||||
let response = app.oneshot(request).await.unwrap();
|
||||
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(
|
||||
response.headers().get("content-type").unwrap(),
|
||||
"text/event-stream"
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 凭证池集成
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod credential_pool_tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_credential_rotation() {
|
||||
let pool = CredentialPool::new();
|
||||
|
||||
// 添加多个凭证
|
||||
pool.add_credential(create_test_credential("cred1")).await;
|
||||
pool.add_credential(create_test_credential("cred2")).await;
|
||||
pool.add_credential(create_test_credential("cred3")).await;
|
||||
|
||||
// 验证轮询
|
||||
let first = pool.get_next().await.unwrap();
|
||||
let second = pool.get_next().await.unwrap();
|
||||
let third = pool.get_next().await.unwrap();
|
||||
let fourth = pool.get_next().await.unwrap();
|
||||
|
||||
// 第四次应该回到第一个
|
||||
assert_eq!(first.id, fourth.id);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_unhealthy_credential_skipped() {
|
||||
let pool = CredentialPool::new();
|
||||
|
||||
let healthy = create_test_credential("healthy");
|
||||
let unhealthy = create_test_credential("unhealthy");
|
||||
|
||||
pool.add_credential(healthy.clone()).await;
|
||||
pool.add_credential(unhealthy.clone()).await;
|
||||
|
||||
// 标记为不健康
|
||||
pool.mark_unhealthy(&unhealthy.id).await;
|
||||
|
||||
// 应该只返回健康的凭证
|
||||
for _ in 0..10 {
|
||||
let cred = pool.get_next().await.unwrap();
|
||||
assert_eq!(cred.id, healthy.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Provider 与数据库集成
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod provider_db_tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_token_persistence() {
|
||||
let db = create_test_db().await;
|
||||
let provider = KiroProvider::new(db.clone());
|
||||
|
||||
// 刷新 Token
|
||||
let token = provider.refresh_token("test-refresh-token").await.unwrap();
|
||||
|
||||
// 验证 Token 被保存到数据库
|
||||
let saved = db.get_token("kiro", "test-id").await.unwrap();
|
||||
assert_eq!(saved.access_token, token.access_token);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_credential_state_sync() {
|
||||
let db = create_test_db().await;
|
||||
let service = ProviderPoolService::new(db.clone());
|
||||
|
||||
// 添加凭证
|
||||
service.add_credential(create_test_credential()).await.unwrap();
|
||||
|
||||
// 验证数据库状态
|
||||
let credentials = db.list_credentials("kiro").await.unwrap();
|
||||
assert_eq!(credentials.len(), 1);
|
||||
assert_eq!(credentials[0].status, "active");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 测试环境设置
|
||||
|
||||
### 测试数据库
|
||||
|
||||
```rust
|
||||
async fn create_test_db() -> Database {
|
||||
let db = Database::new(":memory:").await.unwrap();
|
||||
db.run_migrations().await.unwrap();
|
||||
db
|
||||
}
|
||||
```
|
||||
|
||||
### Mock HTTP 服务
|
||||
|
||||
```rust
|
||||
use wiremock::{MockServer, Mock, ResponseTemplate};
|
||||
use wiremock::matchers::{method, path};
|
||||
|
||||
async fn setup_mock_oauth_server() -> MockServer {
|
||||
let mock_server = MockServer::start().await;
|
||||
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/oauth/token"))
|
||||
.respond_with(ResponseTemplate::new(200)
|
||||
.set_body_json(json!({
|
||||
"access_token": "test-token",
|
||||
"expires_in": 3600
|
||||
})))
|
||||
.mount(&mock_server)
|
||||
.await;
|
||||
|
||||
mock_server
|
||||
}
|
||||
```
|
||||
|
||||
## 测试数据管理
|
||||
|
||||
### Fixtures
|
||||
|
||||
```rust
|
||||
fn create_test_credential(id: &str) -> Credential {
|
||||
Credential {
|
||||
id: id.to_string(),
|
||||
provider: "kiro".to_string(),
|
||||
email: "test@example.com".to_string(),
|
||||
access_token: Some("test-access-token".to_string()),
|
||||
refresh_token: Some("test-refresh-token".to_string()),
|
||||
expires_at: Some(Utc::now() + Duration::hours(1)),
|
||||
status: "active".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn create_expired_credential(id: &str) -> Credential {
|
||||
let mut cred = create_test_credential(id);
|
||||
cred.expires_at = Some(Utc::now() - Duration::hours(1));
|
||||
cred
|
||||
}
|
||||
```
|
||||
|
||||
## 运行集成测试
|
||||
|
||||
```bash
|
||||
# 运行所有集成测试
|
||||
cd src-tauri && cargo test --test integration
|
||||
|
||||
# 运行特定测试
|
||||
cargo test --test integration test_credential_rotation
|
||||
|
||||
# 并行运行(注意数据库隔离)
|
||||
cargo test --test integration -- --test-threads=1
|
||||
```
|
||||
|
||||
## 下一步
|
||||
|
||||
- [E2E 测试指南](e2e-tests.md)
|
||||
- [Agent 评估指南](agent-evaluation.md)
|
||||
@@ -0,0 +1,350 @@
|
||||
# Agent 测试用例
|
||||
|
||||
> Aster Agent 集成的测试用例
|
||||
|
||||
## 概述
|
||||
|
||||
Agent 测试验证 Aster Agent 在 ProxyCast 中的集成,包括:
|
||||
- 基础对话功能
|
||||
- 流式响应
|
||||
- 工具调用
|
||||
- 错误处理
|
||||
- 状态管理
|
||||
|
||||
## 测试用例
|
||||
|
||||
### 1. 基础对话
|
||||
|
||||
#### TC-AGENT-001: 简单对话
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_simple_chat() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
let response = state.chat("你好").await.unwrap();
|
||||
|
||||
assert!(!response.content.is_empty());
|
||||
assert_eq!(response.role, "assistant");
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-002: 多轮对话
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_multi_turn_chat() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
// 第一轮
|
||||
let r1 = state.chat("我叫小明").await.unwrap();
|
||||
assert!(!r1.content.is_empty());
|
||||
|
||||
// 第二轮 - 应该记住上下文
|
||||
let r2 = state.chat("我叫什么名字?").await.unwrap();
|
||||
assert!(r2.content.contains("小明"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-003: 系统提示词
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_system_prompt() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
state.set_system_prompt("你是一个诗人,只用诗歌回答问题").await;
|
||||
|
||||
let response = state.chat("今天天气怎么样?").await.unwrap();
|
||||
|
||||
// 响应应该有诗歌风格(包含换行或韵律)
|
||||
assert!(response.content.contains('\n') || response.content.len() > 50);
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 流式响应
|
||||
|
||||
#### TC-AGENT-010: 流式输出
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_streaming_output() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
let mut stream = state.chat_stream("写一首短诗").await.unwrap();
|
||||
|
||||
let mut chunks = Vec::new();
|
||||
while let Some(chunk) = stream.next().await {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
|
||||
// 应该有多个 chunk
|
||||
assert!(chunks.len() > 1);
|
||||
|
||||
// 合并后应该是完整内容
|
||||
let full_content: String = chunks.iter()
|
||||
.filter_map(|c| c.as_text())
|
||||
.collect();
|
||||
assert!(!full_content.is_empty());
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-011: 流式事件顺序
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_streaming_event_order() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
let mut stream = state.chat_stream("你好").await.unwrap();
|
||||
|
||||
let mut events = Vec::new();
|
||||
while let Some(event) = stream.next().await {
|
||||
events.push(event);
|
||||
}
|
||||
|
||||
// 验证事件顺序
|
||||
let has_start = events.iter().any(|e| matches!(e, StreamEvent::Start));
|
||||
let has_delta = events.iter().any(|e| matches!(e, StreamEvent::Delta(_)));
|
||||
let has_stop = events.iter().any(|e| matches!(e, StreamEvent::Stop));
|
||||
|
||||
assert!(has_start);
|
||||
assert!(has_delta);
|
||||
assert!(has_stop);
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-012: 流式取消
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_streaming_cancellation() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
let mut stream = state.chat_stream("写一篇长文章").await.unwrap();
|
||||
|
||||
// 只读取前几个 chunk
|
||||
let mut count = 0;
|
||||
while let Some(_) = stream.next().await {
|
||||
count += 1;
|
||||
if count >= 3 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// 取消流
|
||||
drop(stream);
|
||||
|
||||
// 状态应该正确清理
|
||||
assert!(state.is_idle().await);
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 工具调用
|
||||
|
||||
#### TC-AGENT-020: 文件读取工具
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_file_read_tool() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
// 创建测试文件
|
||||
let test_file = create_temp_file("test content").await;
|
||||
|
||||
let response = state.chat(&format!("读取文件 {}", test_file.path())).await.unwrap();
|
||||
|
||||
// 应该调用了读取工具并返回内容
|
||||
assert!(response.content.contains("test content") ||
|
||||
response.tool_calls.iter().any(|tc| tc.name == "read_file"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-021: 文件写入工具
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_file_write_tool() {
|
||||
let state = create_test_agent_state().await;
|
||||
let temp_dir = create_temp_dir().await;
|
||||
let file_path = temp_dir.join("output.txt");
|
||||
|
||||
let response = state.chat(&format!(
|
||||
"在 {} 创建一个文件,内容是 'Hello World'",
|
||||
file_path.display()
|
||||
)).await.unwrap();
|
||||
|
||||
// 验证文件被创建
|
||||
assert!(file_path.exists());
|
||||
let content = std::fs::read_to_string(&file_path).unwrap();
|
||||
assert!(content.contains("Hello World"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-022: 工具调用失败处理
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_tool_call_failure() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
// 请求读取不存在的文件
|
||||
let response = state.chat("读取 /nonexistent/file.txt").await.unwrap();
|
||||
|
||||
// Agent 应该优雅处理错误
|
||||
assert!(response.content.contains("不存在") ||
|
||||
response.content.contains("找不到") ||
|
||||
response.content.contains("无法"));
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 错误处理
|
||||
|
||||
#### TC-AGENT-030: 网络错误恢复
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_network_error_recovery() {
|
||||
let state = create_test_agent_state_with_flaky_network().await;
|
||||
|
||||
// 第一次可能失败
|
||||
let result1 = state.chat("你好").await;
|
||||
|
||||
// 重试应该成功
|
||||
let result2 = state.chat("你好").await;
|
||||
|
||||
assert!(result1.is_ok() || result2.is_ok());
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-031: 超时处理
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_timeout_handling() {
|
||||
let state = create_test_agent_state_with_timeout(Duration::from_secs(1)).await;
|
||||
|
||||
// 长任务应该超时
|
||||
let result = state.chat("执行一个需要很长时间的复杂任务").await;
|
||||
|
||||
assert!(result.is_err() ||
|
||||
result.unwrap().content.contains("超时"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-032: 无效输入处理
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_invalid_input() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
// 空消息
|
||||
let result = state.chat("").await;
|
||||
assert!(result.is_err() || !result.unwrap().content.is_empty());
|
||||
|
||||
// 超长消息
|
||||
let long_msg = "x".repeat(1_000_000);
|
||||
let result = state.chat(&long_msg).await;
|
||||
// 应该处理或拒绝,不应该崩溃
|
||||
assert!(result.is_ok() || result.is_err());
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 状态管理
|
||||
|
||||
#### TC-AGENT-040: 会话隔离
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_session_isolation() {
|
||||
let state1 = create_test_agent_state().await;
|
||||
let state2 = create_test_agent_state().await;
|
||||
|
||||
// 在 state1 中设置上下文
|
||||
state1.chat("我叫小明").await.unwrap();
|
||||
|
||||
// state2 不应该知道这个信息
|
||||
let response = state2.chat("我叫什么名字?").await.unwrap();
|
||||
assert!(!response.content.contains("小明"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-041: 会话清理
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_session_cleanup() {
|
||||
let state = create_test_agent_state().await;
|
||||
|
||||
// 建立上下文
|
||||
state.chat("我叫小明").await.unwrap();
|
||||
|
||||
// 清理会话
|
||||
state.clear_session().await;
|
||||
|
||||
// 上下文应该被清除
|
||||
let response = state.chat("我叫什么名字?").await.unwrap();
|
||||
assert!(!response.content.contains("小明"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-AGENT-042: 并发请求
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_concurrent_requests() {
|
||||
let state = Arc::new(create_test_agent_state().await);
|
||||
|
||||
let handles: Vec<_> = (0..5).map(|i| {
|
||||
let state = state.clone();
|
||||
tokio::spawn(async move {
|
||||
state.chat(&format!("问题 {}", i)).await
|
||||
})
|
||||
}).collect();
|
||||
|
||||
let results: Vec<_> = futures::future::join_all(handles).await;
|
||||
|
||||
// 所有请求应该成功或有序失败
|
||||
for result in results {
|
||||
assert!(result.is_ok());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 测试矩阵
|
||||
|
||||
| 测试 ID | 场景 | 类型 | 优先级 |
|
||||
|---------|------|------|--------|
|
||||
| TC-AGENT-001 | 简单对话 | 功能 | P0 |
|
||||
| TC-AGENT-002 | 多轮对话 | 功能 | P0 |
|
||||
| TC-AGENT-010 | 流式输出 | 功能 | P0 |
|
||||
| TC-AGENT-020 | 文件读取 | 工具 | P1 |
|
||||
| TC-AGENT-030 | 网络错误 | 错误处理 | P1 |
|
||||
| TC-AGENT-040 | 会话隔离 | 状态 | P1 |
|
||||
|
||||
## 测试辅助函数
|
||||
|
||||
```rust
|
||||
async fn create_test_agent_state() -> AsterAgentState {
|
||||
let config = AsterConfig {
|
||||
model: "test-model".into(),
|
||||
api_key: "test-key".into(),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
AsterAgentState::new(config).await.unwrap()
|
||||
}
|
||||
|
||||
async fn create_temp_file(content: &str) -> TempFile {
|
||||
let file = TempFile::new().await.unwrap();
|
||||
file.write_all(content.as_bytes()).await.unwrap();
|
||||
file
|
||||
}
|
||||
```
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo test agent::
|
||||
```
|
||||
@@ -0,0 +1,260 @@
|
||||
# 协议转换器测试用例
|
||||
|
||||
> OpenAI ↔ Claude 协议转换的测试用例
|
||||
|
||||
## 概述
|
||||
|
||||
协议转换器是 ProxyCast 的核心模块,负责在不同 API 格式之间转换。测试需要覆盖:
|
||||
- 消息格式转换
|
||||
- 流式响应转换
|
||||
- 工具调用转换
|
||||
- 边界情况处理
|
||||
|
||||
## 测试用例
|
||||
|
||||
### 1. 消息格式转换
|
||||
|
||||
#### TC-CONV-001: 基础消息转换
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_openai_to_claude_basic_message() {
|
||||
let openai_msg = OpenAIMessage {
|
||||
role: "user".to_string(),
|
||||
content: "Hello, world!".to_string(),
|
||||
};
|
||||
|
||||
let claude_msg = convert_to_claude(&openai_msg);
|
||||
|
||||
assert_eq!(claude_msg.role, "user");
|
||||
assert_eq!(claude_msg.content, "Hello, world!");
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-002: System 消息处理
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_system_message_extraction() {
|
||||
let messages = vec![
|
||||
OpenAIMessage { role: "system".into(), content: "You are helpful.".into() },
|
||||
OpenAIMessage { role: "user".into(), content: "Hi".into() },
|
||||
];
|
||||
|
||||
let (system, user_msgs) = extract_system_message(&messages);
|
||||
|
||||
assert_eq!(system, Some("You are helpful.".to_string()));
|
||||
assert_eq!(user_msgs.len(), 1);
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-003: 多轮对话转换
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_multi_turn_conversation() {
|
||||
let messages = vec![
|
||||
OpenAIMessage { role: "user".into(), content: "Hello".into() },
|
||||
OpenAIMessage { role: "assistant".into(), content: "Hi there!".into() },
|
||||
OpenAIMessage { role: "user".into(), content: "How are you?".into() },
|
||||
];
|
||||
|
||||
let claude_msgs = convert_messages(&messages);
|
||||
|
||||
assert_eq!(claude_msgs.len(), 3);
|
||||
assert_eq!(claude_msgs[0].role, "user");
|
||||
assert_eq!(claude_msgs[1].role, "assistant");
|
||||
assert_eq!(claude_msgs[2].role, "user");
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 流式响应转换
|
||||
|
||||
#### TC-CONV-010: SSE 事件格式
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_sse_event_format() {
|
||||
let delta = TextDelta { text: "Hello".to_string() };
|
||||
let sse = format_sse_event(&delta);
|
||||
|
||||
assert!(sse.starts_with("data: "));
|
||||
assert!(sse.ends_with("\n\n"));
|
||||
assert!(sse.contains("\"delta\""));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-011: 流式开始事件
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_stream_start_event() {
|
||||
let event = create_stream_start_event("msg-123");
|
||||
|
||||
assert_eq!(event.event_type, "message_start");
|
||||
assert!(event.data.contains("msg-123"));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-012: 流式结束事件
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_stream_stop_event() {
|
||||
let event = create_stream_stop_event("end_turn");
|
||||
|
||||
assert_eq!(event.event_type, "message_stop");
|
||||
assert!(event.data.contains("end_turn"));
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 工具调用转换
|
||||
|
||||
#### TC-CONV-020: 工具定义转换
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_tool_definition_conversion() {
|
||||
let openai_tool = OpenAITool {
|
||||
r#type: "function".into(),
|
||||
function: OpenAIFunction {
|
||||
name: "get_weather".into(),
|
||||
description: "Get weather info".into(),
|
||||
parameters: json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": { "type": "string" }
|
||||
}
|
||||
}),
|
||||
},
|
||||
};
|
||||
|
||||
let claude_tool = convert_tool(&openai_tool);
|
||||
|
||||
assert_eq!(claude_tool.name, "get_weather");
|
||||
assert_eq!(claude_tool.description, "Get weather info");
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-021: 工具调用响应转换
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_tool_call_response_conversion() {
|
||||
let claude_tool_use = ClaudeToolUse {
|
||||
id: "tool-123".into(),
|
||||
name: "get_weather".into(),
|
||||
input: json!({"location": "Beijing"}),
|
||||
};
|
||||
|
||||
let openai_tool_call = convert_tool_call(&claude_tool_use);
|
||||
|
||||
assert_eq!(openai_tool_call.id, "tool-123");
|
||||
assert_eq!(openai_tool_call.function.name, "get_weather");
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-022: 工具结果转换
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_tool_result_conversion() {
|
||||
let openai_result = OpenAIToolResult {
|
||||
tool_call_id: "tool-123".into(),
|
||||
content: "Sunny, 25°C".into(),
|
||||
};
|
||||
|
||||
let claude_result = convert_tool_result(&openai_result);
|
||||
|
||||
assert_eq!(claude_result.tool_use_id, "tool-123");
|
||||
assert_eq!(claude_result.content, "Sunny, 25°C");
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 边界情况
|
||||
|
||||
#### TC-CONV-030: 空消息处理
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_empty_message_content() {
|
||||
let msg = OpenAIMessage {
|
||||
role: "user".into(),
|
||||
content: "".into(),
|
||||
};
|
||||
|
||||
let result = convert_to_claude(&msg);
|
||||
|
||||
// 空内容应该被正确处理
|
||||
assert!(result.content.is_empty());
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-031: 特殊字符处理
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_special_characters() {
|
||||
let msg = OpenAIMessage {
|
||||
role: "user".into(),
|
||||
content: "Hello\n\t\"world\"\\test".into(),
|
||||
};
|
||||
|
||||
let result = convert_to_claude(&msg);
|
||||
|
||||
// 特殊字符应该被保留
|
||||
assert!(result.content.contains('\n'));
|
||||
assert!(result.content.contains('\t'));
|
||||
assert!(result.content.contains('"'));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-032: Unicode 处理
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_unicode_content() {
|
||||
let msg = OpenAIMessage {
|
||||
role: "user".into(),
|
||||
content: "你好世界 🌍 مرحبا".into(),
|
||||
};
|
||||
|
||||
let result = convert_to_claude(&msg);
|
||||
|
||||
assert_eq!(result.content, "你好世界 🌍 مرحبا");
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-CONV-033: 大消息处理
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_large_message() {
|
||||
let large_content = "x".repeat(100_000);
|
||||
let msg = OpenAIMessage {
|
||||
role: "user".into(),
|
||||
content: large_content.clone(),
|
||||
};
|
||||
|
||||
let result = convert_to_claude(&msg);
|
||||
|
||||
assert_eq!(result.content.len(), 100_000);
|
||||
}
|
||||
```
|
||||
|
||||
## 测试矩阵
|
||||
|
||||
| 测试 ID | 场景 | 输入 | 期望输出 | 优先级 |
|
||||
|---------|------|------|----------|--------|
|
||||
| TC-CONV-001 | 基础消息 | user 消息 | 正确转换 | P0 |
|
||||
| TC-CONV-002 | System 消息 | system + user | 正确提取 | P0 |
|
||||
| TC-CONV-010 | SSE 格式 | 文本增量 | 正确格式 | P0 |
|
||||
| TC-CONV-020 | 工具定义 | OpenAI 工具 | Claude 工具 | P1 |
|
||||
| TC-CONV-030 | 空消息 | 空内容 | 不崩溃 | P1 |
|
||||
| TC-CONV-032 | Unicode | 多语言 | 正确保留 | P1 |
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo test converter::
|
||||
```
|
||||
@@ -0,0 +1,283 @@
|
||||
# Provider 测试用例
|
||||
|
||||
> OAuth 认证和 API 调用的测试用例
|
||||
|
||||
## 概述
|
||||
|
||||
Provider 模块负责与各个 AI 服务提供商的交互,包括:
|
||||
- OAuth 认证流程
|
||||
- Token 刷新
|
||||
- API 调用
|
||||
- 错误处理
|
||||
|
||||
## 测试用例
|
||||
|
||||
### 1. Kiro Provider
|
||||
|
||||
#### TC-KIRO-001: 凭证加载
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_kiro_credential_loading() {
|
||||
let credential_json = r#"{
|
||||
"access_token": "test-access",
|
||||
"refresh_token": "test-refresh",
|
||||
"expires_at": "2026-01-30T12:00:00Z"
|
||||
}"#;
|
||||
|
||||
let cred = KiroCredential::from_json(credential_json).unwrap();
|
||||
|
||||
assert_eq!(cred.access_token, "test-access");
|
||||
assert_eq!(cred.refresh_token, "test-refresh");
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-KIRO-002: Token 刷新
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_kiro_token_refresh() {
|
||||
let mock_server = setup_mock_oauth_server().await;
|
||||
let provider = KiroProvider::new_with_endpoint(&mock_server.uri());
|
||||
|
||||
let new_token = provider.refresh_token("old-refresh-token").await.unwrap();
|
||||
|
||||
assert!(!new_token.access_token.is_empty());
|
||||
assert!(new_token.expires_at > Utc::now());
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-KIRO-003: 过期 Token 检测
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_kiro_token_expiry_check() {
|
||||
let expired_cred = KiroCredential {
|
||||
access_token: "test".into(),
|
||||
refresh_token: "test".into(),
|
||||
expires_at: Utc::now() - Duration::hours(1),
|
||||
};
|
||||
|
||||
assert!(expired_cred.is_expired());
|
||||
|
||||
let valid_cred = KiroCredential {
|
||||
access_token: "test".into(),
|
||||
refresh_token: "test".into(),
|
||||
expires_at: Utc::now() + Duration::hours(1),
|
||||
};
|
||||
|
||||
assert!(!valid_cred.is_expired());
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Gemini Provider
|
||||
|
||||
#### TC-GEMINI-001: OAuth 流程
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_gemini_oauth_flow() {
|
||||
let mock_server = setup_mock_google_oauth().await;
|
||||
let provider = GeminiProvider::new_with_endpoint(&mock_server.uri());
|
||||
|
||||
let auth_url = provider.get_auth_url();
|
||||
|
||||
assert!(auth_url.contains("accounts.google.com"));
|
||||
assert!(auth_url.contains("scope="));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-GEMINI-002: API 调用
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_gemini_api_call() {
|
||||
let mock_server = setup_mock_gemini_api().await;
|
||||
let provider = GeminiProvider::new_with_endpoint(&mock_server.uri());
|
||||
|
||||
let response = provider.chat(&[
|
||||
Message { role: "user".into(), content: "Hello".into() }
|
||||
]).await.unwrap();
|
||||
|
||||
assert!(!response.content.is_empty());
|
||||
}
|
||||
```
|
||||
|
||||
### 3. OpenAI Provider
|
||||
|
||||
#### TC-OPENAI-001: API Key 验证
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn test_openai_api_key_validation() {
|
||||
// 有效的 API Key
|
||||
assert!(OpenAIProvider::validate_api_key("sk-1234567890abcdef"));
|
||||
|
||||
// 无效的 API Key
|
||||
assert!(!OpenAIProvider::validate_api_key("invalid"));
|
||||
assert!(!OpenAIProvider::validate_api_key(""));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-OPENAI-002: 流式响应处理
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_openai_streaming() {
|
||||
let mock_server = setup_mock_openai_streaming().await;
|
||||
let provider = OpenAIProvider::new_with_endpoint(&mock_server.uri());
|
||||
|
||||
let mut stream = provider.chat_stream(&[
|
||||
Message { role: "user".into(), content: "Hello".into() }
|
||||
]).await.unwrap();
|
||||
|
||||
let mut chunks = Vec::new();
|
||||
while let Some(chunk) = stream.next().await {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
|
||||
assert!(!chunks.is_empty());
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 错误处理
|
||||
|
||||
#### TC-PROV-ERR-001: 网络错误
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_network_error_handling() {
|
||||
let provider = KiroProvider::new_with_endpoint("http://invalid-host:9999");
|
||||
|
||||
let result = provider.refresh_token("test").await;
|
||||
|
||||
assert!(result.is_err());
|
||||
assert!(matches!(result.unwrap_err(), ProviderError::NetworkError(_)));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-PROV-ERR-002: 认证错误
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_auth_error_handling() {
|
||||
let mock_server = setup_mock_oauth_error(401).await;
|
||||
let provider = KiroProvider::new_with_endpoint(&mock_server.uri());
|
||||
|
||||
let result = provider.refresh_token("invalid-token").await;
|
||||
|
||||
assert!(result.is_err());
|
||||
assert!(matches!(result.unwrap_err(), ProviderError::AuthError(_)));
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-PROV-ERR-003: 速率限制
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_rate_limit_handling() {
|
||||
let mock_server = setup_mock_rate_limit().await;
|
||||
let provider = OpenAIProvider::new_with_endpoint(&mock_server.uri());
|
||||
|
||||
let result = provider.chat(&[
|
||||
Message { role: "user".into(), content: "Hello".into() }
|
||||
]).await;
|
||||
|
||||
assert!(result.is_err());
|
||||
assert!(matches!(result.unwrap_err(), ProviderError::RateLimited(_)));
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 凭证池集成
|
||||
|
||||
#### TC-POOL-001: 凭证轮询
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_credential_rotation() {
|
||||
let pool = CredentialPool::new();
|
||||
|
||||
pool.add(create_credential("cred1")).await;
|
||||
pool.add(create_credential("cred2")).await;
|
||||
|
||||
let first = pool.get_next().await.unwrap();
|
||||
let second = pool.get_next().await.unwrap();
|
||||
let third = pool.get_next().await.unwrap();
|
||||
|
||||
assert_ne!(first.id, second.id);
|
||||
assert_eq!(first.id, third.id); // 回到第一个
|
||||
}
|
||||
```
|
||||
|
||||
#### TC-POOL-002: 健康检查
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn test_health_check() {
|
||||
let pool = CredentialPool::new();
|
||||
let cred = create_credential("test");
|
||||
|
||||
pool.add(cred.clone()).await;
|
||||
|
||||
// 标记为不健康
|
||||
pool.mark_unhealthy(&cred.id).await;
|
||||
|
||||
// 不应该返回不健康的凭证
|
||||
let result = pool.get_next().await;
|
||||
assert!(result.is_none());
|
||||
}
|
||||
```
|
||||
|
||||
## 测试矩阵
|
||||
|
||||
| 测试 ID | Provider | 场景 | 优先级 |
|
||||
|---------|----------|------|--------|
|
||||
| TC-KIRO-001 | Kiro | 凭证加载 | P0 |
|
||||
| TC-KIRO-002 | Kiro | Token 刷新 | P0 |
|
||||
| TC-GEMINI-001 | Gemini | OAuth 流程 | P0 |
|
||||
| TC-OPENAI-001 | OpenAI | API Key 验证 | P0 |
|
||||
| TC-PROV-ERR-001 | 通用 | 网络错误 | P1 |
|
||||
| TC-POOL-001 | 凭证池 | 轮询 | P0 |
|
||||
|
||||
## Mock 服务设置
|
||||
|
||||
```rust
|
||||
async fn setup_mock_oauth_server() -> MockServer {
|
||||
let server = MockServer::start().await;
|
||||
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/oauth/token"))
|
||||
.respond_with(ResponseTemplate::new(200)
|
||||
.set_body_json(json!({
|
||||
"access_token": "new-access-token",
|
||||
"refresh_token": "new-refresh-token",
|
||||
"expires_in": 3600
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
server
|
||||
}
|
||||
|
||||
async fn setup_mock_oauth_error(status: u16) -> MockServer {
|
||||
let server = MockServer::start().await;
|
||||
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/oauth/token"))
|
||||
.respond_with(ResponseTemplate::new(status)
|
||||
.set_body_json(json!({
|
||||
"error": "invalid_grant",
|
||||
"error_description": "Token expired"
|
||||
})))
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
server
|
||||
}
|
||||
```
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
cd src-tauri && cargo test provider::
|
||||
```
|
||||
@@ -0,0 +1,218 @@
|
||||
# ProxyCast 单元测试指南
|
||||
|
||||
> 针对独立模块的确定性测试
|
||||
|
||||
## 概述
|
||||
|
||||
单元测试是测试金字塔的基础,覆盖最小的可测试单元。ProxyCast 的单元测试主要针对:
|
||||
- 协议转换器
|
||||
- Provider 模块
|
||||
- 工具函数
|
||||
- 数据结构
|
||||
|
||||
## Rust 单元测试
|
||||
|
||||
### 运行命令
|
||||
|
||||
```bash
|
||||
# 运行所有测试
|
||||
cd src-tauri && cargo test
|
||||
|
||||
# 运行特定模块测试
|
||||
cargo test converter::
|
||||
cargo test provider::
|
||||
|
||||
# 显示详细输出
|
||||
cargo test -- --nocapture
|
||||
```
|
||||
|
||||
### 测试文件位置
|
||||
|
||||
```
|
||||
src-tauri/src/
|
||||
├── converter/
|
||||
│ ├── mod.rs
|
||||
│ └── tests.rs # 转换器测试
|
||||
├── providers/
|
||||
│ ├── kiro/
|
||||
│ │ └── tests.rs # Kiro Provider 测试
|
||||
│ └── gemini/
|
||||
│ └── tests.rs # Gemini Provider 测试
|
||||
└── services/
|
||||
└── tests.rs # 服务层测试
|
||||
```
|
||||
|
||||
### 测试模板
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_basic_conversion() {
|
||||
let input = OpenAIMessage {
|
||||
role: "user".to_string(),
|
||||
content: "Hello".to_string(),
|
||||
};
|
||||
|
||||
let result = convert_to_claude(&input);
|
||||
|
||||
assert_eq!(result.role, "user");
|
||||
assert!(result.content.contains("Hello"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_edge_case_empty_content() {
|
||||
let input = OpenAIMessage {
|
||||
role: "user".to_string(),
|
||||
content: "".to_string(),
|
||||
};
|
||||
|
||||
let result = convert_to_claude(&input);
|
||||
|
||||
// 空内容应该被正确处理
|
||||
assert!(result.content.is_empty());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 前端单元测试
|
||||
|
||||
### 运行命令
|
||||
|
||||
```bash
|
||||
# 运行所有测试
|
||||
npm test
|
||||
|
||||
# 运行特定文件
|
||||
npm test -- src/lib/utils.test.ts
|
||||
|
||||
# 监听模式
|
||||
npm test -- --watch
|
||||
```
|
||||
|
||||
### 测试文件位置
|
||||
|
||||
```
|
||||
src/
|
||||
├── lib/
|
||||
│ ├── utils.ts
|
||||
│ └── utils.test.ts # 工具函数测试
|
||||
├── hooks/
|
||||
│ ├── useCredentials.ts
|
||||
│ └── useCredentials.test.ts
|
||||
└── components/
|
||||
└── __tests__/ # 组件测试
|
||||
```
|
||||
|
||||
### 测试模板
|
||||
|
||||
```typescript
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { formatCredentialName, validateApiKey } from './utils';
|
||||
|
||||
describe('formatCredentialName', () => {
|
||||
it('should format kiro credential name', () => {
|
||||
const result = formatCredentialName('kiro', 'user@example.com');
|
||||
expect(result).toBe('Kiro (user@example.com)');
|
||||
});
|
||||
|
||||
it('should handle empty email', () => {
|
||||
const result = formatCredentialName('kiro', '');
|
||||
expect(result).toBe('Kiro');
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateApiKey', () => {
|
||||
it('should accept valid OpenAI key', () => {
|
||||
expect(validateApiKey('sk-1234567890abcdef')).toBe(true);
|
||||
});
|
||||
|
||||
it('should reject invalid key', () => {
|
||||
expect(validateApiKey('invalid')).toBe(false);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## 测试原则
|
||||
|
||||
### 1. 单一职责
|
||||
|
||||
每个测试只验证一个行为:
|
||||
|
||||
```rust
|
||||
// ✅ 好:单一职责
|
||||
#[test]
|
||||
fn test_token_refresh_updates_expiry() {
|
||||
// 只测试过期时间更新
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_token_refresh_preserves_scope() {
|
||||
// 只测试 scope 保留
|
||||
}
|
||||
|
||||
// ❌ 差:多个职责
|
||||
#[test]
|
||||
fn test_token_refresh() {
|
||||
// 测试过期时间、scope、错误处理...
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 独立性
|
||||
|
||||
测试之间不应该有依赖:
|
||||
|
||||
```rust
|
||||
// ✅ 好:每个测试独立
|
||||
#[test]
|
||||
fn test_a() {
|
||||
let state = TestState::new();
|
||||
// ...
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_b() {
|
||||
let state = TestState::new();
|
||||
// ...
|
||||
}
|
||||
|
||||
// ❌ 差:共享状态
|
||||
static mut SHARED_STATE: Option<TestState> = None;
|
||||
```
|
||||
|
||||
### 3. 可读性
|
||||
|
||||
测试名称应该描述行为:
|
||||
|
||||
```rust
|
||||
// ✅ 好:描述性名称
|
||||
#[test]
|
||||
fn test_expired_token_triggers_refresh()
|
||||
|
||||
#[test]
|
||||
fn test_invalid_credentials_returns_error()
|
||||
|
||||
// ❌ 差:模糊名称
|
||||
#[test]
|
||||
fn test_token()
|
||||
|
||||
#[test]
|
||||
fn test_error()
|
||||
```
|
||||
|
||||
## 覆盖率目标
|
||||
|
||||
| 模块 | 目标覆盖率 | 说明 |
|
||||
|------|-----------|------|
|
||||
| converter | 90%+ | 核心转换逻辑 |
|
||||
| providers | 80%+ | OAuth 流程 |
|
||||
| services | 70%+ | 业务逻辑 |
|
||||
| utils | 95%+ | 工具函数 |
|
||||
|
||||
## 下一步
|
||||
|
||||
- [集成测试指南](integration-tests.md)
|
||||
- [测试用例:转换器](test-cases/converter-tests.md)
|
||||
- [测试用例:Provider](test-cases/provider-tests.md)
|
||||
Reference in New Issue
Block a user