diff --git a/README.md b/README.md index fcb3a90..0e44c16 100644 --- a/README.md +++ b/README.md @@ -85,6 +85,7 @@ + -
+ • MiniCPM5
• Kimi-K2.5
• Step-3.5-Flash
• GLM-4.7-Flash
@@ -96,14 +97,14 @@ • Qwen3-VL
• SpatialLM
• Hunyuan3D-2
- • Qwen2-VL
+ • Qwen2-VL + 		• MiniCPM-o
• Qwen2.5-Coder
• DeepSeek-Coder-V2
• gpt-oss-20b
- • GLM-4.1-Thinking - 		+ • GLM-4.1-Thinking
• DeepSeek-R1
• InternLM3
• phi4
@@ -112,12 +113,12 @@ • DeepSeek
• Baichuan
• InternLM
- • Kimi
- • ERNIE-4.5
- • Llama4
- • Apple OpenELM + • Kimi 		+ • ERNIE-4.5
+ • Llama4
+ • Apple OpenELM
• Llama3.1
• Gemma-2
• Qwen2.5
@@ -128,10 +129,10 @@ • MiniCPM
• Yi 零一万物
• Yuan2.0
- • Yuan2.0-M32
- • 哔哩哔哩 Index + • Yuan2.0-M32 		+ • 哔哩哔哩 Index
• CharacterGLM
• BlueLM
• Qwen-Audio
diff --git a/README_en.md b/README_en.md index e69d92f..29dc338 100644 --- a/README_en.md +++ b/README_en.md @@ -85,21 +85,26 @@ + -
+ • MiniCPM5
+ • Kimi-K2.5
+ • Step-3.5-Flash
+ • GLM-4.7-Flash
• Gemma3
• MiniMax-M3
+ • MiniMax-M2.5
• MiniMax-M2
• Qwen3
• Qwen3-VL
• SpatialLM
• Hunyuan3D-2
- • Qwen2-VL
+ • Qwen2-VL + 		• MiniCPM-o
• Qwen2.5-Coder
• DeepSeek-Coder-V2
• gpt-oss-20b
- • GLM-4.1-Thinking - 		+ • GLM-4.1-Thinking
• DeepSeek-R1
• InternLM3
• phi4
@@ -108,12 +113,12 @@ • DeepSeek
• Baichuan
• InternLM
- • Kimi
- • ERNIE-4.5
- • Llama4
- • Apple OpenELM + • Kimi 		+ • ERNIE-4.5
+ • Llama4
+ • Apple OpenELM
• Llama3.1
• Gemma-2
• Qwen2.5
@@ -124,10 +129,10 @@ • MiniCPM
• Yi 零一万物
• Yuan2.0
- • Yuan2.0-M32
- • Bilibili Index + • Yuan2.0-M32 		+ • Bilibili Index
• CharacterGLM
• BlueLM
• Qwen-Audio
@@ -293,4 +298,4 @@
-
\ No newline at end of file + diff --git a/models/minicpm5/01-MiniCPM5-1B-vLLM 部署调用.md b/models/minicpm5/01-MiniCPM5-1B-vLLM 部署调用.md new file mode 100644 index 0000000..3d61d1a --- /dev/null +++ b/models/minicpm5/01-MiniCPM5-1B-vLLM 部署调用.md @@ -0,0 +1,227 @@ +# 01-MiniCPM5-1B vLLM 部署调用 + +## vLLM 简介 + +`vLLM` 框架是一个高效的大语言模型**推理和部署服务系统**，具备以下特性： + +- **高效的内存管理**：通过 `PagedAttention` 算法，`vLLM` 实现了对 `KV` 缓存的高效管理，减少了内存浪费，优化了模型的运行效率。 +- **高吞吐量**：`vLLM` 支持异步处理和连续批处理请求，显著提高了模型推理的吞吐量。 +- **易用性**：`vLLM` 与 `HuggingFace` 模型无缝集成，兼容 `OpenAI` 的 `API` 服务器。 +- **开源共享**：`vLLM` 开源，社区活跃。 + +> `MiniCPM5-1B` 采用**标准 `LlamaForCausalLM` 架构**，主流推理引擎可直接加载——无需自定义算子、无需模型代码 fork。本教程使用 `vLLM` 部署，**文中启动日志与接口返回均为实测真实输出**。 + +## 关于 MiniCPM5-1B + +`MiniCPM5-1B` 是面壁智能（ModelBest）/ OpenBMB 发布的 1B 稠密 Transformer，面向端侧、本地部署与资源受限场景，具备： + +- **同尺寸开源 SOTA**：在 Agentic 工具调用、代码生成、高难推理上优势明显。 +- **双模式推理（Hybrid Reasoning）**：内置 `` chat template，可通过 `enable_thinking` 在「思考」与「非思考」模式间切换，同一份权重既是快速助手也是深度推理器。 +- **原生长上下文**：支持最长 128K 上下文。 +- **架构**：`LlamaForCausalLM`，24 层，hidden_size 1536，GQA（16 注意力头 / 2 KV 头），rope_theta=5000000。 + +## 环境准备 + +本文实测基础环境如下： + +``` +---------------- +ubuntu 22.04 +python 3.12 +NVIDIA 驱动 580.105.08 +GPU: RTX 4090 D (24G) +torch 2.11.0+cu128 +vllm 0.23.0 +---------------- +``` + +> 本文默认学习者已配置好 `Pytorch (cuda)` 环境，如未配置请先自行安装。 + +```bash +python -m pip install --upgrade pip +pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple + +pip install modelscope +pip install "transformers>=5.6" +pip install "vllm>=0.21" +pip install openai +``` + +> 若启动时报 `ModuleNotFoundError: No module named 'flash_attn.ops'`，通常是环境里装了 `flash-attn-4`（会留下一个空的 `flash_attn` 命名空间包），而 vLLM 的 rotary 模块检测到 `flash_attn` 后会尝试导入其 `.ops` 子模块。解决：`pip uninstall flash-attn-4`，并删除残留的空目录 `rm -rf $(python -c "import site;print(site.getsitepackages()[0])")/flash_attn`，vLLM 会自动回退到自带实现。 + +## 模型下载 + +使用 modelscope 中的 `snapshot_download` 函数下载模型。 + +新建 `model_download.py`： + +```python +# model_download.py +from modelscope import snapshot_download + +model_dir = snapshot_download('OpenBMB/MiniCPM5-1B', cache_dir='/root/autodl-tmp') +print(f"模型下载完成，保存路径为：{model_dir}") +``` + +然后执行 `python model_download.py`。 + +> 注意：记得修改 `cache_dir` 为你的模型下载路径哦~ + +## 创建兼容 OpenAI API 接口的服务器 + +`MiniCPM5-1B` 兼容 `OpenAI API` 协议。常用启动参数： + +- `--host` / `--port`：地址与端口 +- `--model`：模型路径 +- `--served-model-name`：服务对外的模型名称 +- `--max-model-len`：最大上下文长度（1B 模型在 24G 显存上可设 `4096` 或更大） +- `--gpu-memory-utilization`：显存占用比例（1B 模型很小，0.6 即可） +- `--trust-remote-code`：信任远程代码 + +```bash +vllm serve /root/autodl-tmp/OpenBMB/MiniCPM5-1B \ + --served-model-name MiniCPM5-1B \ + --max-model-len 4096 \ + --gpu-memory-utilization 0.6 \ + --trust-remote-code \ + --host 0.0.0.0 --port 8000 +``` + +实测启动日志如下（vLLM 识别为 `LlamaForCausalLM`，1B 权重加载仅 0.52s）： + +![vLLM 启动日志](./images/01-vllm-startup.png) + +```bash +(APIServer) INFO [model.py:611] Resolved architecture: LlamaForCausalLM +(EngineCore) INFO [core.py:113] Initializing a V1 LLM engine (v0.23.0) ... +(EngineCore) INFO [default_loader.py:397] Loading weights took 0.52 seconds +(EngineCore) INFO [model_runner.py:319] Model loading took 2.09 GiB and 2.14 seconds +(EngineCore) INFO [gpu_worker.py:480] Available KV cache memory: 11.54 GiB +(EngineCore) INFO [kv_cache_utils.py:1744] GPU KV cache size: 504,192 tokens +(EngineCore) INFO [core.py:306] init engine (profile, create kv cache, warmup model) took 39.05 s (compilation: 18.97 s) +(APIServer) INFO: Application startup complete. +``` + +> 首次启动会触发 `torch.compile` 编译（约 19s），编译结果会缓存，后续启动更快。出现 `Application startup complete.` 即说明服务成功启动。 + +- 查看 `curl http://localhost:8000/v1/models`： + +```json +{ + "object": "list", + "data": [ + { + "id": "MiniCPM5-1B", + "object": "model", + "owned_by": "vllm", + "root": "/root/autodl-tmp/OpenBMB/MiniCPM5-1B", + "max_model_len": 4096 + } + ] +} +``` + +### 思考模式与非思考模式 + +`MiniCPM5-1B` 内置 `` 模板，可通过 `chat_template_kwargs.enable_thinking` 按**请求**级别控制： + +- **思考模式**（`enable_thinking=true`，推荐 `temperature=0.9, top_p=0.95`）：先输出 ` ... ` 推理过程，再给出答案 +- **非思考模式**（`enable_thinking=false`，推荐 `temperature=0.7, top_p=0.95`）：不强制思考，直接回答 + +| 模式 | 推荐参数 | enable_thinking | +| --- | --- | --- | +| Think | `temperature=0.9, top_p=0.95` | `True` | +| No Think | `temperature=0.7, top_p=0.95` | `False` | + +### 用 curl 测试 Chat Completions（非思考模式） + +```bash +curl http://localhost:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "MiniCPM5-1B", + "messages": [ + {"role": "user", "content": "你是谁？用一句话介绍自己。"} + ], + "temperature": 0.7, + "top_p": 0.95, + "max_tokens": 256, + "extra_body": {"chat_template_kwargs": {"enable_thinking": false}} + }' +``` + +实测返回值如下（`content` 中先是简短的 `` 思考，其后是最终回答，`finish_reason` 为 `stop`）： + +```json +{ + "id": "chatcmpl-9c66165de8661ff3", + "object": "chat.completion", + "model": "MiniCPM5-1B", + "choices": [ + { + "index": 0, + "message": { + "role": "assistant", + "content": "\n嗯，用户让我介绍自己，需要一句话说明身份。MiniCPM系列模型是由面壁智能和OpenBMB社区开发的，所以应该直接说明这一点。\n\n\n我是MiniCPM系列模型，由面壁智能（ModelBest）和OpenBMB开源社区开发。" + }, + "finish_reason": "stop" + } + ], + "usage": { + "prompt_tokens": 15, + "completion_tokens": 57, + "total_tokens": 72 + } +} +``` + +> 实测发现：`MiniCPM5-1B` 即便在非思考模式下，也常在 `content` 开头先输出一段简短的 ` ... ` 再给出回答（这是该模型后训练形成的习惯）。若需要纯粹的非思考输出，可适当调大 `max_tokens`。 + +### 用 Python 脚本请求（思考模式） + +```python +# vllm_openai_chat_completions.py +from openai import OpenAI + +client = OpenAI( + api_key="sk-xxx", # 随便填写，只是为了通过接口参数校验 + base_url="http://localhost:8000/v1", +) + +# 思考模式：模型会先输出推理过程 +chat_outputs = client.chat.completions.create( + model="MiniCPM5-1B", + messages=[{"role": "user", "content": "5的阶乘是多少？"}], + temperature=0.9, + top_p=0.95, + extra_body={"chat_template_kwargs": {"enable_thinking": True}}, +) +print(chat_outputs.choices[0].message.content) +``` + +输出包含 ` ... ` 思考过程与最终答案： + +``` + +5 的阶乘记作 5!，等于 5 × 4 × 3 × 2 × 1 ... + + +5 的阶乘（5!）= 5 × 4 × 3 × 2 × 1 = 120。 +``` + +### 运行时日志 + +在请求处理过程中，`API` 后端会持续打印日志与统计信息，便于观测服务状态。实测运行时日志如下： + +![vLLM 运行时日志](./images/01-vllm-runtime.png) + +```bash +(EngineCore) INFO [core.py:306] init engine (profile, create kv cache, warmup model) took 39.05 s (compilation: 18.97 s) +(APIServer) INFO: Application startup complete. +(APIServer) INFO: 127.0.0.1:34630 - "POST /v1/chat/completions HTTP/1.1" 200 OK +(APIServer) INFO: 127.0.0.1:34660 - "POST /v1/chat/completions HTTP/1.1" 200 OK +``` + +## 工具调用（Tool Calling） + +`MiniCPM5-1B` 原生支持 XML 风格的工具调用。在 vLLM 中可配合 `--tool-call-parser` 使用（vLLM 较新版本支持 `minicpm5` 解析器），将模型输出的 `` 转换为 OpenAI 兼容的 `tool_calls`。具体用法可参考 MiniCPM 官方 cookbook。 diff --git a/models/minicpm5/02-MiniCPM5-1B-SGLang 部署调用.md b/models/minicpm5/02-MiniCPM5-1B-SGLang 部署调用.md new file mode 100644 index 0000000..22020ef --- /dev/null +++ b/models/minicpm5/02-MiniCPM5-1B-SGLang 部署调用.md @@ -0,0 +1,255 @@ +# 02-MiniCPM5-1B SGLang 部署调用 + +## SGLang 简介 + +`SGLang` 是一款专为大语言模型/多模态模型设计的高性能推理与服务框架： + +- **后端一键启动**：一条命令完成环境适配与服务发布。 +- **前端无缝对接**：直接沿用 `OpenAI SDK` 或标准 `HTTP` 调用。 +- **高性能**：支持 `RadixAttention`（前缀复用）、连续批处理、CUDA Graph 等加速技术。 + +> `MiniCPM5-1B` 采用**标准 `LlamaForCausalLM` 架构**，SGLang 可直接加载，无需自定义算子。本教程使用 `SGLang` 部署，**文中启动日志与接口返回均为实测真实输出**。 + +> 官方提示：工具调用（Tool Calling）场景下，**SGLang 是推荐后端**——MiniCPM5-1B 输出 XML 风格工具调用，SGLang 内置的 `minicpm5` 解析器可将其原生转换为 OpenAI 兼容的 `tool_calls`。 + +## 关于 MiniCPM5-1B + +`MiniCPM5-1B` 是面壁智能 / OpenBMB 的 1B 稠密 Transformer，面向端侧与本地部署：标准 `LlamaForCausalLM` 架构（24 层，GQA，128K 上下文），内置 `` 模板支持「思考 / 非思考」双模式（通过 `enable_thinking` 切换）。 + +## 环境准备 + +本文实测基础环境如下： + +``` +---------------- +ubuntu 22.04 +python 3.12 +NVIDIA 驱动 580.105.08 +GPU: RTX 4090 D (24G, sm89) +torch 2.11.0+cu128 +sglang 0.5.13.post1 +---------------- +``` + +> 本文默认学习者已配置好 `Pytorch (cuda)` 环境，如未配置请先自行安装。 + +```bash +python -m pip install --upgrade pip +pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple + +pip install modelscope +pip install "transformers>=5.6" +pip install openai + +# 安装 sglang（官方建议 sglang[srt]>=0.5.12） +pip install "sglang[srt]>=0.5.12" +``` + +> 若在 RTX 4090（sm89）上启动报 `Could not load any common_ops library! Expected variant: SM89`，说明默认装的 `sglang-kernel` 是 CUDA 13 / sm90+ 构建，需要换成 sm89 兼容版本： +> ```bash +> pip install sglang-kernel --index-url https://docs.sglang.ai/whl/cu129/ +> ``` + +## 模型下载 + +新建 `model_download.py`： + +```python +# model_download.py +from modelscope import snapshot_download + +model_dir = snapshot_download('OpenBMB/MiniCPM5-1B', cache_dir='/root/autodl-tmp') +print(f"模型下载完成，保存路径为：{model_dir}") +``` + +执行 `python model_download.py`。 + +> 注意：记得修改 `cache_dir` 为你的模型下载路径哦~ + +## 启动 SGLang 服务 + +`MiniCPM5-1B` 为 1B 模型，单张 24G 显卡绰绰有余，无需张量并行。 + +### 命令行直接启动 + +```bash +python3 -m sglang.launch_server \ + --model-path /root/autodl-tmp/OpenBMB/MiniCPM5-1B \ + --served-model-name MiniCPM5-1B \ + --host 0.0.0.0 \ + --port 8000 \ + --mem-fraction-static 0.6 \ + --context-length 4096 \ + --trust-remote-code +``` + +> 新版 SGLang 推荐使用 `sglang serve ...` 入口（与 `python -m sglang.launch_server` 等价）。 +> 若需工具调用，加上 `--tool-call-parser minicpm5`（或 `--tool-call-parser auto`）。 + +常用参数： + +- `--model-path`：模型路径 +- `--served-model-name`：服务对外的模型名称 +- `--mem-fraction-static`：静态显存占用比例（1B 模型很小，0.6 即可） +- `--context-length`：最大上下文长度 +- `--tp-size`：张量并行数，单卡无需设置 +- `--trust-remote-code`：信任远程代码 + +实测启动日志如下（SGLang 识别为 `LlamaForCausalLM`，权重加载 0.95s）： + +![SGLang 启动日志](./images/02-sglang-startup.png) + +```bash +[22:24:31] Load weight end. elapsed=0.95 s, type=LlamaForCausalLM, avail mem=11.06 GB, mem usage=2.16 GB. +[22:24:31] KV Cache is allocated. dtype: torch.bfloat16, #tokens: 251788, K size: 2.88 GB, V size: 2.88 GB +[22:24:31] Memory pool end. avail mem=5.18 GB +[22:24:31] Capture cuda graph begin. This can take up to several minutes. avail mem=4.73 GB +[22:25:18] Capture cuda graph end. Time elapsed: 47.12 s. mem usage=3.97 GB. avail mem=0.76 GB. +[22:25:36] INFO: Application startup complete. +[22:25:37] The server is fired up and ready to roll! +``` + +> 首次启动会进行 CUDA graph 捕获（约 47s），完成后出现 `The server is fired up and ready to roll!` 即说明服务成功启动。 + +### Python 启动脚本 + +```python +# start_server.py +from sglang.utils import launch_server_cmd, wait_for_server + +cmd = ( + "python3 -m sglang.launch_server " + "--model-path /root/autodl-tmp/OpenBMB/MiniCPM5-1B " + "--served-model-name MiniCPM5-1B " + "--host 0.0.0.0 --port 8000 " + "--mem-fraction-static 0.6 --context-length 4096 " + "--trust-remote-code" +) + +server_process, port = launch_server_cmd(cmd, port=8000) +wait_for_server(f"http://127.0.0.1:{port}") +print(f"SGLang Server started: http://127.0.0.1:{port}") +``` + +## 调用示例 + +### 查看模型列表 + +```bash +curl http://localhost:8000/v1/models +``` + +实测返回值（`owned_by` 为 `sglang`）： + +```json +{ + "object": "list", + "data": [ + { + "id": "MiniCPM5-1B", + "object": "model", + "owned_by": "sglang", + "root": "MiniCPM5-1B", + "max_model_len": 4096 + } + ] +} +``` + +### 聊天对话（思考模式） + +`MiniCPM5-1B` 内置 `` 模板，通过 `chat_template_kwargs.enable_thinking` 控制模式： + +| 模式 | 推荐参数 | enable_thinking | +| --- | --- | --- | +| Think | `temperature=0.9, top_p=0.95` | `True` | +| No Think | `temperature=0.7, top_p=0.95` | `False` | + +```python +# test_chat.py +from openai import OpenAI + +client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1") + +# 思考模式：先输出 ... ，再给答案 +response = client.chat.completions.create( + model="MiniCPM5-1B", + messages=[{"role": "user", "content": "5的阶乘是多少？"}], + temperature=0.9, + top_p=0.95, + max_tokens=768, + extra_body={"chat_template_kwargs": {"enable_thinking": True}}, +) +print(response.choices[0].message.content) +``` + +实测输出包含完整推理与最终答案（`finish_reason: stop`）： + +``` + +5 的阶乘记作 5!，等于 5 × 4 × 3 × 2 × 1 = 120 ... + + +5 的阶乘（5!）等于 5 × 4 × 3 × 2 × 1 = 120。 +``` + +### 非思考模式 + +```python +response = client.chat.completions.create( + model="MiniCPM5-1B", + messages=[{"role": "user", "content": "你是谁？用一句话介绍自己。"}], + temperature=0.7, + top_p=0.95, + extra_body={"chat_template_kwargs": {"enable_thinking": False}}, +) +print(response.choices[0].message.content) +``` + +> 实测发现：`MiniCPM5-1B` 即便在非思考模式下，也常在 `content` 开头先输出一段简短的 ` ... ` 再给出回答，这是该模型后训练形成的习惯。 + +### 运行时日志 + +请求处理时，SGLang 后端会持续打印解码批次的统计信息。实测运行时日志如下： + +![SGLang 运行时日志](./images/02-sglang-runtime.png) + +```bash +[22:25:36] INFO: Application startup complete. +[22:25:37] The server is fired up and ready to roll! +[22:25:37] INFO: 127.0.0.1:xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK +``` + +### 工具调用（Tool Calling） + +SGLang 是 MiniCPM5-1B 工具调用的推荐后端。启动时加 `--tool-call-parser minicpm5`，即可把模型输出的 XML 风格 `` 原生转换为 OpenAI 兼容的 `tool_calls`： + +```bash +python3 -m sglang.launch_server --model-path /root/autodl-tmp/OpenBMB/MiniCPM5-1B \ + --served-model-name MiniCPM5-1B --port 8000 --tool-call-parser minicpm5 +``` + +```python +tools = [{ + "type": "function", + "function": { + "name": "get_weather", + "description": "获取指定城市的天气", + "parameters": { + "type": "object", + "properties": {"city": {"type": "string", "description": "城市名"}}, + "required": ["city"], + }, + }, +}] +response = client.chat.completions.create( + model="MiniCPM5-1B", + messages=[{"role": "user", "content": "北京今天天气怎么样？"}], + tools=tools, +) +print(response.choices[0].message.tool_calls) +``` + +## 小结 + +`MiniCPM5-1B` 作为标准 `LlamaForCausalLM` 架构的 1B 模型，在 `vLLM` 与 `SGLang` 中均可一键部署，无需任何特殊算子。结合其「思考/非思考」双模式与原生工具调用能力，非常适合端侧助手、coding agent 与工具调用场景。 diff --git a/models/minicpm5/03-MiniCPM5-1B-LoRA.ipynb b/models/minicpm5/03-MiniCPM5-1B-LoRA.ipynb new file mode 100644 index 0000000..a19cb2c --- /dev/null +++ b/models/minicpm5/03-MiniCPM5-1B-LoRA.ipynb @@ -0,0 +1,447 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "mcpm-00", + "metadata": {}, + "source": [ + "# MiniCPM5-1B LoRA 微调及 SwanLab 可视化记录\n", + "\n", + "本教程使用 LoRA 方法在「甄嬛」角色对话数据集上微调 **MiniCPM5-1B**，并使用 **SwanLab** 记录训练过程。\n", + "\n", + "## 环境配置\n", + "\n", + "> MiniCPM5-1B 采用标准 `LlamaForCausalLM` 架构，需要 `transformers>=5.6`。\n", + "\n", + "```bash\n", + "pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple\n", + "pip install \"transformers>=5.6\" accelerate datasets peft swanlab modelscope\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-01", + "metadata": {}, + "source": [ + "# 导入环境" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-02", + "metadata": {}, + "outputs": [], + "source": [ + "import json\n", + "import torch\n", + "from datasets import Dataset\n", + "from transformers import (\n", + " AutoTokenizer,\n", + " AutoModelForCausalLM,\n", + " TrainingArguments,\n", + " Trainer,\n", + " DataCollatorForSeq2Seq,\n", + ")\n", + "from peft import LoraConfig, TaskType, get_peft_model\n", + "import swanlab\n", + "from swanlab.integration.transformers import SwanLabCallback\n" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-03", + "metadata": {}, + "source": [ + "# 读取数据集\n", + "\n", + "本教程使用甄嬛对话数据集（Alpaca 格式：`instruction / input / output`）。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-04", + "metadata": {}, + "outputs": [], + "source": [ + "dataset_path = \"/root/autodl-tmp/huanhuan.json\" # 注意修改为你的数据集路径\n", + "with open(dataset_path, \"r\", encoding=\"utf-8\") as f:\n", + " data = json.load(f)\n", + "\n", + "ds = Dataset.from_list(data)\n", + "ds" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-05", + "metadata": {}, + "outputs": [], + "source": [ + "ds[:3]" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-06", + "metadata": {}, + "source": [ + "# 认识 MiniCPM5 的 Chat Template\n", + "\n", + "MiniCPM5-1B 采用 `<|im_start|>role\\n...<|im_end|>\\n` 格式，支持 `enable_thinking` 控制思考模式。角色扮演任务我们**关闭思考模式**。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-07", + "metadata": {}, + "outputs": [], + "source": [ + "model_id = \"/root/autodl-tmp/OpenBMB/MiniCPM5-1B\" # 注意修改为你的模型路径\n", + "tokenizer = AutoTokenizer.from_pretrained(model_id)\n", + "print(\"eos:\", tokenizer.eos_token, tokenizer.eos_token_id)\n", + "print(\"pad:\", tokenizer.pad_token, tokenizer.pad_token_id)\n", + "print(\"<|im_end|> id:\", tokenizer.convert_tokens_to_ids(\"<|im_end|>\"))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-08", + "metadata": {}, + "outputs": [], + "source": [ + "messages = [\n", + " {\"role\": \"system\", \"content\": \"现在你要扮演皇帝身边的女人--甄嬛\"},\n", + " {\"role\": \"user\", \"content\": \"你父亲是谁？\"},\n", + " {\"role\": \"assistant\", \"content\": \"家父是大理寺少卿甄远道。\"},\n", + "]\n", + "\n", + "text = tokenizer.apply_chat_template(\n", + " messages, tokenize=False, add_generation_prompt=False, enable_thinking=False\n", + ")\n", + "print(text)" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-09", + "metadata": {}, + "source": [ + "# 处理数据集\n", + "\n", + "**注意**：MiniCPM5 模板在「带 generation prompt」时会追加 `\\n\\n\\n\\n`（非思考占位），但「完整对话渲染」时助手回合**不**含这个 think 块。因此**不能用 token 级切片**，需分别对「前缀」和「回答」单独 tokenize 再拼接。`labels` 中只有回答部分参与 loss。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-10", + "metadata": {}, + "outputs": [], + "source": [ + "def process_func(example):\n", + " MAX_LENGTH = 1024\n", + " SYS = \"现在你要扮演皇帝身边的女人--甄嬛\"\n", + "\n", + " messages = [{\"role\": \"system\", \"content\": SYS},\n", + " {\"role\": \"user\", \"content\": example[\"instruction\"] + example[\"input\"]}]\n", + " # 前缀（system + user，带 generation prompt，含非思考 think 占位），不计算 loss\n", + " prompt_ids = tokenizer.apply_chat_template(\n", + " messages, tokenize=True, add_generation_prompt=True,\n", + " enable_thinking=False, return_dict=False,\n", + " )\n", + " # 回答部分：output + 结束符 <|im_end|>\n", + " response_ids = tokenizer(example[\"output\"], add_special_tokens=False).input_ids \\\n", + " + [tokenizer.convert_tokens_to_ids(\"<|im_end|>\")]\n", + "\n", + " input_ids = prompt_ids + response_ids\n", + " labels = [-100] * len(prompt_ids) + response_ids\n", + " attention_mask = [1] * len(input_ids)\n", + "\n", + " if len(input_ids) > MAX_LENGTH: # 超长截断\n", + " input_ids = input_ids[:MAX_LENGTH]\n", + " attention_mask = attention_mask[:MAX_LENGTH]\n", + " labels = labels[:MAX_LENGTH]\n", + "\n", + " return {\"input_ids\": input_ids, \"attention_mask\": attention_mask, \"labels\": labels}" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-11", + "metadata": {}, + "outputs": [], + "source": [ + "tokenized_id = ds.map(process_func, remove_columns=ds.column_names)\n", + "tokenized_id" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-12", + "metadata": {}, + "outputs": [], + "source": [ + "# 查看完整输入\n", + "print(tokenizer.decode(tokenized_id[0][\"input_ids\"]))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-13", + "metadata": {}, + "outputs": [], + "source": [ + "# 查看 labels（过滤掉 -100 后即为模型需要学习的回答）\n", + "print(tokenizer.decode(list(filter(lambda x: x != -100, tokenized_id[0][\"labels\"]))))" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-14", + "metadata": {}, + "source": [ + "# 加载模型\n", + "\n", + "MiniCPM5-1B 是标准 `LlamaForCausalLM`，直接用 `AutoModelForCausalLM` 加载。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-15", + "metadata": {}, + "outputs": [], + "source": [ + "model = AutoModelForCausalLM.from_pretrained(\n", + " model_id,\n", + " dtype=torch.bfloat16,\n", + " device_map=\"auto\",\n", + ")\n", + "# 开启梯度检查点时需要调用该方法\n", + "model.enable_input_require_grads()\n", + "model.dtype" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-16", + "metadata": {}, + "source": [ + "# LoRA 配置\n", + "\n", + "MiniCPM5-1B 是标准 Llama 架构，LoRA 目标模块与 Llama 一致。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-17", + "metadata": {}, + "outputs": [], + "source": [ + "config = LoraConfig(\n", + " task_type=TaskType.CAUSAL_LM,\n", + " target_modules=[\n", + " \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n", + " \"gate_proj\", \"up_proj\", \"down_proj\",\n", + " ],\n", + " inference_mode=False, # 训练模式\n", + " r=8, # Lora 秩\n", + " lora_alpha=32, # Lora alpha，缩放系数 = 32/8 = 4\n", + " lora_dropout=0.1, # Dropout 比例\n", + ")\n", + "config" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-18", + "metadata": {}, + "outputs": [], + "source": [ + "model = get_peft_model(model, config)\n", + "model.print_trainable_parameters()" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-19", + "metadata": {}, + "source": [ + "# 配置训练参数" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-20", + "metadata": {}, + "outputs": [], + "source": [ + "args = TrainingArguments(\n", + " output_dir=\"./output/MiniCPM5_1B_LoRA\",\n", + " per_device_train_batch_size=4,\n", + " gradient_accumulation_steps=4,\n", + " logging_steps=10,\n", + " num_train_epochs=3,\n", + " save_steps=100,\n", + " learning_rate=1e-4,\n", + " save_on_each_node=True,\n", + " gradient_checkpointing=True,\n", + " report_to=\"none\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-21", + "metadata": {}, + "source": [ + "# SwanLab 简介\n", + "\n", + "[SwanLab](https://github.com/swanhubx/swanlab) 是一个开源的模型训练记录工具，提供训练可视化、自动日志记录、超参数记录、实验对比、多人协同等功能。\n", + "\n", + "建议先在 [SwanLab 官网](https://swanlab.cn/) 注册账号，初始化时选择 `(2) Use an existing SwanLab account` 并使用 private API Key 登录。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-22", + "metadata": {}, + "outputs": [], + "source": [ + "# 实例化 SwanLabCallback\n", + "# 首次使用会提示登录，输入你在 SwanLab 官网获取的 API Key\n", + "swanlab_callback = SwanLabCallback(\n", + " project=\"MiniCPM5-Lora\",\n", + " experiment_name=\"MiniCPM5-1B-LoRA\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-23", + "metadata": {}, + "source": [ + "# 训练" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-24", + "metadata": {}, + "outputs": [], + "source": [ + "trainer = Trainer(\n", + " model=model,\n", + " args=args,\n", + " train_dataset=tokenized_id,\n", + " data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True),\n", + " callbacks=[swanlab_callback],\n", + ")\n", + "\n", + "trainer.train()" + ] + }, + { + "cell_type": "markdown", + "id": "mcpm-25", + "metadata": {}, + "source": [ + "# 加载 LoRA 权重推理\n", + "\n", + "得到 checkpoint 后，加载基础模型并挂载 LoRA 权重进行推理。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-26", + "metadata": {}, + "outputs": [], + "source": [ + "from peft import PeftModel\n", + "\n", + "# 基础模型路径 & 训练得到的 LoRA 权重路径（按实际 checkpoint 编号修改）\n", + "lora_path = \"./output/MiniCPM5_1B_LoRA/checkpoint-702\"\n", + "\n", + "# 加载基础模型\n", + "base_model = AutoModelForCausalLM.from_pretrained(\n", + " model_id, dtype=torch.bfloat16, device_map=\"auto\"\n", + ")\n", + "# 挂载 LoRA 权重\n", + "model = PeftModel.from_pretrained(base_model, model_id=lora_path)\n", + "model.eval()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "mcpm-27", + "metadata": {}, + "outputs": [], + "source": [ + "prompt = \"你是谁？\"\n", + "messages = [\n", + " {\"role\": \"system\", \"content\": \"现在你要扮演皇帝身边的女人--甄嬛\"},\n", + " {\"role\": \"user\", \"content\": prompt},\n", + "]\n", + "inputs = tokenizer.apply_chat_template(\n", + " messages,\n", + " add_generation_prompt=True,\n", + " enable_thinking=False, # 关闭思考模式，直接输出角色回答\n", + " tokenize=True,\n", + " return_dict=True,\n", + " return_tensors=\"pt\",\n", + ").to(model.device)\n", + "\n", + "gen_kwargs = {\"max_new_tokens\": 128, \"do_sample\": True, \"top_p\": 0.95, \"temperature\": 0.7}\n", + "with torch.no_grad():\n", + " outputs = model.generate(**inputs, **gen_kwargs)\n", + "outputs = outputs[:, inputs[\"input_ids\"].shape[1]:]\n", + "print(tokenizer.decode(outputs[0], skip_special_tokens=True))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e2cc414-c5fb-43ff-a4c1-9ab8c27bd03b", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/models/minicpm5/03-MiniCPM5-1B-LoRA及SwanLab可视化记录.md b/models/minicpm5/03-MiniCPM5-1B-LoRA及SwanLab可视化记录.md new file mode 100644 index 0000000..b5f3c18 --- /dev/null +++ b/models/minicpm5/03-MiniCPM5-1B-LoRA及SwanLab可视化记录.md @@ -0,0 +1,317 @@ +# MiniCPM5-1B-LoRA 及 SwanLab 可视化记录 + +> 本教程配套 notebook：[03-MiniCPM5-1B-LoRA.ipynb](./03-MiniCPM5-1B-LoRA.ipynb) + +## MiniCPM5-1B 简介 + +`MiniCPM5-1B` 是面壁智能（ModelBest）/ OpenBMB 发布的 1B 稠密 Transformer，采用**标准 `LlamaForCausalLM` 架构**（24 层，GQA，128K 上下文）。它内置 `` chat template，支持「思考 / 非思考」双模式（`enable_thinking` 切换），并原生支持工具调用。1B 的体量非常适合在单卡上做 LoRA 微调实验。 + +本教程使用官方推荐的纯 `transformers + peft` 方案完成 LoRA 微调，并使用 **SwanLab** 记录训练过程。 + +## 环境配置 + +```bash +# 换清华镜像源 +pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple + +# 核心依赖（MiniCPM5 需要 transformers>=5.6） +pip install "transformers>=5.6" +pip install accelerate datasets peft swanlab modelscope +``` + +> 考虑到部分同学配置环境可能会遇到一些问题，我们在 AutoDL 平台准备了环境镜像，点击下方链接并直接创建 Autodl 示例即可。 +> ***https://www.codewithgpu.com/i/datawhalechina/self-llm/MiniCPM5*** + +## 模型下载 + +```python +# model_download.py +from modelscope import snapshot_download + +model_dir = snapshot_download('OpenBMB/MiniCPM5-1B', cache_dir='/root/autodl-tmp') +print(f"模型下载完成，保存路径为：{model_dir}") +``` + +然后在终端中输入 `python model_download.py` 执行下载。 + +> 注意：记得修改 `cache_dir` 为你的模型下载路径哦~ + +## 数据集构建 + +对大语言模型进行 `supervised-finetuning`（`sft`，有监督微调）的数据格式如下： + +```json +{ + "instruction": "回答以下用户问题，仅输出答案。", + "input": "1+1等于几?", + "output": "2" +} +``` + +其中，`instruction` 是用户指令；`input` 是用户输入；`output` 是模型应该给出的输出。 + +我们的目标是通过大量人物对话数据微调得到一个能够 role-play 甄嬛对话风格的模型，数据示例如下： + +```json +{ + "instruction": "你父亲是谁？", + "input": "", + "output": "家父是大理寺少卿甄远道。" +} +``` + +本教程使用的甄嬛对话示例微调数据集位于 [/dataset/huanhuan.json](../../dataset/huanhuan.json)（共 3729 条），数据格式为 `instruction / input / output` 的 Alpaca 格式。 + +## 数据准备 + +LoRA 训练的数据需要经过格式化、编码之后再输入给模型。这里我们直接使用 tokenizer 自带的 `apply_chat_template` 构造对话模板。 + +### 认识 MiniCPM5 的 Chat Template + +`MiniCPM5-1B` 采用 `<|im_start|>role\n...<|im_end|>\n` 格式，并支持 `enable_thinking` 参数控制思考模式。对于「角色扮演」任务，我们关闭思考模式（`enable_thinking=False`）： + +```python +from transformers import AutoTokenizer + +model_id = '/root/autodl-tmp/OpenBMB/MiniCPM5-1B' +tokenizer = AutoTokenizer.from_pretrained(model_id) + +messages = [ + {"role": "system", "content": "现在你要扮演皇帝身边的女人--甄嬛"}, + {"role": "user", "content": "你父亲是谁？"}, + {"role": "assistant", "content": "家父是大理寺少卿甄远道。"}, +] + +text = tokenizer.apply_chat_template( + messages, tokenize=False, add_generation_prompt=False, enable_thinking=False +) +print(text) +``` + +输出如下： + +``` +<|im_start|>system +现在你要扮演皇帝身边的女人--甄嬛<|im_end|> +<|im_start|>user +你父亲是谁？<|im_end|> +<|im_start|>assistant +家父是大理寺少卿甄远道。<|im_end|> +``` + +### 构造处理函数 + +> **注意一个细节**：MiniCPM5 的模板在「带 generation prompt」时会追加 `\n\n\n\n`（非思考模式占位），但「完整对话渲染」时助手回合并**不**包含这个 think 块。因此这里**不能用 token 级切片**（`full[len(prompt):]`），而要分别对「前缀」和「回答」单独 tokenize 再拼接。 + +```python +def process_func(example): + MAX_LENGTH = 1024 + SYS = "现在你要扮演皇帝身边的女人--甄嬛" + + messages = [{"role": "system", "content": SYS}, + {"role": "user", "content": example["instruction"] + example["input"]}] + # 前缀（system + user，带 generation prompt，含非思考 think 占位），不计算 loss + prompt_ids = tokenizer.apply_chat_template( + messages, tokenize=True, add_generation_prompt=True, + enable_thinking=False, return_dict=False, + ) + # 回答部分：output + 结束符 <|im_end|> + response_ids = tokenizer(example["output"], add_special_tokens=False).input_ids \ + + [tokenizer.convert_tokens_to_ids("<|im_end|>")] + + input_ids = prompt_ids + response_ids + labels = [-100] * len(prompt_ids) + response_ids + attention_mask = [1] * len(input_ids) + + if len(input_ids) > MAX_LENGTH: # 超长截断 + input_ids = input_ids[:MAX_LENGTH] + attention_mask = attention_mask[:MAX_LENGTH] + labels = labels[:MAX_LENGTH] + + return {"input_ids": input_ids, "attention_mask": attention_mask, "labels": labels} +``` + +> 说明：MiniCPM5 的 `<|im_end|>` token id 为 `130073`，``（eos）为 `1`。这里用 `<|im_end|>` 作为回答的结束符，与模板一致。 + +读入数据集并应用处理函数： + +```python +import json +from datasets import Dataset + +with open("/root/autodl-tmp/huanhuan.json", "r", encoding="utf-8") as f: + data = json.load(f) + +ds = Dataset.from_list(data) +tokenized_id = ds.map(process_func, remove_columns=ds.column_names) +tokenized_id +``` + +可以解码查看处理后的样本： + +```python +print(tokenizer.decode(tokenized_id[0]["input_ids"])) +print(tokenizer.decode(list(filter(lambda x: x != -100, tokenized_id[0]["labels"])))) +``` + +``` +<|im_start|>system +现在你要扮演皇帝身边的女人--甄嬛<|im_end|> +<|im_start|>user +小姐，别的秀女都在求中选，唯有咱们小姐想被撂牌子，菩萨一定记得真真儿的——<|im_end|> +<|im_start|>assistant + + + + +嘘——都说许愿说破是不灵的。<|im_end|> +``` + +`labels`（过滤掉 -100）： + +``` +嘘——都说许愿说破是不灵的。<|im_end|> +``` + +## 加载模型和 tokenizer + +`MiniCPM5-1B` 是标准 `LlamaForCausalLM`，直接用 `AutoModelForCausalLM` 加载： + +```python +import torch +from transformers import AutoModelForCausalLM + +model = AutoModelForCausalLM.from_pretrained( + '/root/autodl-tmp/OpenBMB/MiniCPM5-1B', + dtype=torch.bfloat16, + device_map="auto", +) +model.enable_input_require_grads() # 开启梯度检查点时需要 +model.dtype # torch.bfloat16 +``` + +## LoRA Config + +`MiniCPM5-1B` 是标准 Llama 架构，LoRA 目标模块与 Llama 一致：`q_proj / k_proj / v_proj / o_proj / gate_proj / up_proj / down_proj`。 + +```python +from peft import LoraConfig, TaskType, get_peft_model + +config = LoraConfig( + task_type=TaskType.CAUSAL_LM, + target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], + inference_mode=False, # 训练模式 + r=8, # Lora 秩 + lora_alpha=32, # Lora alpha，缩放系数 = 32/8 = 4 + lora_dropout=0.1, # Dropout 比例 +) + +model = get_peft_model(model, config) +model.print_trainable_parameters() +``` + +输出（仅训练约 0.5% 的参数）： + +``` +trainable params: 5,603,328 || all params: 1,086,236,160 || trainable%: 0.5158 +``` + +## Training Arguments + +```python +from transformers import TrainingArguments, Trainer, DataCollatorForSeq2Seq + +args = TrainingArguments( + output_dir="./output/MiniCPM5_1B_LoRA", + per_device_train_batch_size=4, + gradient_accumulation_steps=4, + logging_steps=10, + num_train_epochs=3, + save_steps=100, + learning_rate=1e-4, + save_on_each_node=True, + gradient_checkpointing=True, + report_to="none", +) +``` + +## SwanLab 简介 + +![](./images/swanlab.png) + +[SwanLab](https://github.com/swanhubx/swanlab) 是一个开源的模型训练记录工具，提供训练可视化、自动日志记录、超参数记录、实验对比、多人协同等功能。 + +**为什么要记录训练**：模型训练更像一门实验科学，一个优秀模型背后往往是成千上万次实验。高效记录与对比对研究效率至关重要。 + +## 实例化 SwanLabCallback + +建议先在 [SwanLab 官网](https://swanlab.cn/) 注册账号，初始化时选择 `(2) Use an existing SwanLab account` 并使用 private API Key 登录。 + +```python +import swanlab +from swanlab.integration.transformers import SwanLabCallback + +swanlab_callback = SwanLabCallback( + project="MiniCPM5-Lora", + experiment_name="MiniCPM5-1B-LoRA", +) +``` + +## 使用 Trainer 训练 + +```python +trainer = Trainer( + model=model, + args=args, + train_dataset=tokenized_id, + data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True), + callbacks=[swanlab_callback], +) + +trainer.train() +``` + +训练完成后，打开 SwanLab 即可查看训练过程中记录的参数与 loss 曲线。 + +## 加载 LoRA 权重推理 + +```python +from transformers import AutoModelForCausalLM, AutoTokenizer +import torch +from peft import PeftModel + +model_id = '/root/autodl-tmp/OpenBMB/MiniCPM5-1B' +lora_path = './output/MiniCPM5_1B_LoRA/checkpoint-XXX' # 按实际 checkpoint 填写 + +tokenizer = AutoTokenizer.from_pretrained(model_id) +model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto") +model = PeftModel.from_pretrained(model, model_id=lora_path) +model.eval() + +messages = [ + {"role": "system", "content": "现在你要扮演皇帝身边的女人--甄嬛"}, + {"role": "user", "content": "你是谁？"}, +] +inputs = tokenizer.apply_chat_template( + messages, + add_generation_prompt=True, + enable_thinking=False, + tokenize=True, + return_dict=True, + return_tensors="pt", +).to(model.device) + +gen_kwargs = {"max_new_tokens": 128, "do_sample": True, "top_p": 0.95, "temperature": 0.7} +with torch.no_grad(): + outputs = model.generate(**inputs, **gen_kwargs) +outputs = outputs[:, inputs["input_ids"].shape[1]:] +print(tokenizer.decode(outputs[0], skip_special_tokens=True)) +``` + +输出示例： + +``` +我是甄嬛，家父是大理寺少卿甄远道。 +``` + +可以看到，经过 LoRA 微调后，模型已经学会了甄嬛的说话风格与人物设定。 diff --git a/models/minicpm5/images/01-vllm-runtime.png b/models/minicpm5/images/01-vllm-runtime.png new file mode 100644 index 0000000..dbaa0e7 Binary files /dev/null and b/models/minicpm5/images/01-vllm-runtime.png differ diff --git a/models/minicpm5/images/01-vllm-startup.png b/models/minicpm5/images/01-vllm-startup.png new file mode 100644 index 0000000..18782e2 Binary files /dev/null and b/models/minicpm5/images/01-vllm-startup.png differ diff --git a/models/minicpm5/images/02-sglang-runtime.png b/models/minicpm5/images/02-sglang-runtime.png new file mode 100644 index 0000000..77d2090 Binary files /dev/null and b/models/minicpm5/images/02-sglang-runtime.png differ diff --git a/models/minicpm5/images/02-sglang-startup.png b/models/minicpm5/images/02-sglang-startup.png new file mode 100644 index 0000000..831e193 Binary files /dev/null and b/models/minicpm5/images/02-sglang-startup.png differ diff --git a/support_model.md b/support_model.md index 8d0e78c..15429c9 100644 --- a/support_model.md +++ b/support_model.md @@ -4,6 +4,7 @@ ## 目录 +- [MiniCPM5](#minicpm5) - [Kimi-K2.5](#kimi-k25) - [Step-3.5-Flash](#step-35-flash) - [GLM-4.7-Flash](#glm-47-flash) @@ -62,6 +63,14 @@ ## 已支持模型列表 +### MiniCPM5 + +[MiniCPM5-1B](https://www.modelscope.cn/models/OpenBMB/MiniCPM5-1B) + - [x] [MiniCPM5-1B vLLM 部署调用](./models/minicpm5/01-MiniCPM5-1B-vLLM%20部署调用.md) + - [x] [MiniCPM5-1B SGLang 部署调用](./models/minicpm5/02-MiniCPM5-1B-SGLang%20部署调用.md) + - [x] [MiniCPM5-1B LoRA 及 SwanLab 可视化记录](./models/minicpm5/03-MiniCPM5-1B-LoRA及SwanLab可视化记录.md) + - [x] [MiniCPM5-1B LoRA Docker 镜像](https://www.codewithgpu.com/i/datawhalechina/self-llm/MiniCPM5) + ### Kimi-K2.5 [Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5)