mirror of
https://github.com/xming521/WeClone.git
synced 2026-08-28 18:07:28 +08:00
更新README文档，替换测试数据路径为新的评估数据文件。
This commit is contained in:
xming521
@@ -111,7 +111,7 @@ Please use [Telegram Desktop](https://desktop.telegram.org/) to export chat reco
|
||||
|
||||
|
||||
## Data Preprocessing
|
||||
|
||||
设语言、平台、类型别忘了
|
||||
- By default, the project uses Microsoft Presidio to remove `phone numbers, email addresses, credit card numbers, IP addresses, geographic location names, international bank account numbers, cryptocurrency wallet addresses, age information, and generic ID numbers` from the data, but it cannot guarantee 100% identification.
|
||||
- Therefore, a blocklist `blocked_words` is provided in `settings.jsonc`, allowing users to manually add words or phrases they want to filter (the entire sentence containing blocked words will be removed by default).
|
||||
|
||||
@@ -127,6 +127,7 @@ weclone-cli make-dataset
|
||||
- For **Image to Text**: Add `images` to `include_type` and configure `vision_api` parameters. The system will use external multimodal models to convert images to text, and the final generated dataset **is still used for training text-only LLM**.
|
||||
- You can enable the `enable_clean` option in `clean_dataset` to clean the data for better results (multimodal data is not currently supported). The current system supports using `llm judge` to score chat records, providing **vllm offline inference** and **API online inference** methods. By default, offline inference is enabled. To switch to API-based online inference mode, modify `"online_llm_clear": false` to `true` in the `settings.jsonc` file, and configure relevant parameters such as `base_url`, `llm_api_key`, and `model_name`. All models compatible with OpenAI interface can be accessed.
|
||||
- After obtaining the `llm scoring score distribution`, you can filter acceptable data by setting the `accept_score` parameter, and appropriately reduce the `lora_dropout` parameter in `train_sft_args` to improve the model's fitting effect.
|
||||
更多参数请查看文档[配置参数](https://docs.weclone.love/config/config.html)
|
||||
|
||||
## Configure Parameters and Fine-tune Model
|
||||
|
||||
@@ -146,51 +147,58 @@ uv pip install deepspeed
|
||||
deepspeed --num_gpus=number_of_gpus weclone/train/train_sft.py
|
||||
```
|
||||
|
||||
### 使用浏览器demo简单推理
|
||||
可以在这一步测试出合适的temperature、top_p值，修改settings.jsonc的`infer_args`后，供后续推理时使用。
|
||||
### Simple Inference with Browser Demo
|
||||
Test suitable temperature and top_p values, then modify `infer_args` in settings.jsonc for subsequent inference use.
|
||||
```bash
|
||||
weclone-cli webchat-demo
|
||||
```
|
||||
|
||||
### 使用接口进行推理
|
||||
### Inference Using API
|
||||
|
||||
```bash
|
||||
weclone-cli server
|
||||
```
|
||||
|
||||
### 使用常见聊天问题测试
|
||||
不包含询问个人信息的问题，仅有日常聊天。测试结果在test_result-my.txt。
|
||||
### Test with Common Chat Questions
|
||||
Does not include questions asking for personal information, only daily conversation. Test results are in test_result-my.txt.
|
||||
```bash
|
||||
weclone-cli server
|
||||
weclone-cli test-model
|
||||
```
|
||||
|
||||
## 🖼️ 微调效果
|
||||
## 🖼️ Fine-tuning Results
|
||||
> [!TIP]
|
||||
> **QQ群内有部署好的Qwen2.5VL 32B Bot，可以体验效果。更多案例可以关注[小红书](https://www.xiaohongshu.com/user/profile/628109730000000021029de4)**
|
||||
> **英文例子怎么发 More cases can be found on [XiaoHongShu](https://www.xiaohongshu.com/user/profile/628109730000000021029de4)**
|
||||
|
||||
使用Qwen2.5-14B-Instruct模型，大概3万条处理后的有效数据，loss降到了3.5左右的效果：
|
||||
Using the Qwen2.5VL 32B model with approximately 10,000 processed effective data samples, the loss was reduced to around 3.6:
|
||||
<details>
|
||||
<summary>截图</summary>
|
||||
<div style="display: flex; flex-wrap: wrap; gap: 10px;">
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<img src="https://github.com/user-attachments/assets/d129a81b-15e8-43bd-a8e8-1670c79428cf" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/5ae489b0-cb33-4868-8936-ff57c9d805da" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/62e58de8-1a73-44fc-a948-0d2e949e44a0" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/6bf6d0cc-7ff1-4748-a096-3850d924f954" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/b7d81f9b-ea56-4f7e-8ee5-7f4171bdc66d" alt="alt text" style="width: 52%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/62e58de8-1a73-44fc-a948-0d2e949e44a0" alt="alt text" style="width: 52%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/6bf6d0cc-7ff1-4748-a096-3850d924f954" alt="alt text" style="width: 52%; min-width: 150px;">
|
||||
</div>
|
||||
</details>
|
||||
|
||||
|
||||
## 🤖 部署到聊天机器人
|
||||
## 🤖 Deploy to Chat Bots
|
||||
### AstrBot
|
||||
[AstrBot](https://github.com/AstrBotDevs/AstrBot) is an easy-to-use multi-platform LLM chatbot and development framework ✨ Supports Discord, Telegram, Slack, QQ, WeChat, Enterprise WeChat, Feishu and other platforms.
|
||||
|
||||
Usage steps:
|
||||
1. Deploy AstrBot
|
||||
2. Deploy messaging platforms like Discord, Telegram, Slack in AstrBot
|
||||
3. Execute `weclone-cli server` to start the API service
|
||||
4. Add a new service provider in AstrBot, select OpenAI type, fill in the API Base URL according to AstrBot's deployment method (e.g., for docker deployment it might be http://172.17.0.1:8005/v1), fill in the model as gpt-3.5-turbo, and enter any API Key
|
||||
5. Tool calling is not supported after fine-tuning, please turn off the default tools first by sending the command: `/tool off all` on the messaging platform, otherwise the fine-tuned effect won't be visible.
|
||||
6. Set the system prompt in AstrBot according to the default_system used during fine-tuning.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Check the api_service logs to ensure that the large model service request parameters are consistent with those used during fine-tuning as much as possible, and turn off all tool plugin capabilities.
|
||||
|
||||
### LangBot
|
||||
|
||||
[LangBot](https://github.com/RockChinQ/LangBot) 是一个开源的接入全球多种即时通信平台的 LLM 机器人平台，支持Discord、Telegram、Slack等平台，适合各种场景使用。
|
||||
[LangBot](https://github.com/RockChinQ/LangBot) 是一个开源的接入全球多种即时通信平台的 LLM 机器人平台，适合各种场景使用。
|
||||
|
||||
1. [部署 LangBot](https://github.com/RockChinQ/LangBot/blob/master/README_EN.md#-getting-started)
|
||||
2. 在 LangBot 中添加一个机器人
|
||||
@@ -202,22 +210,7 @@ weclone-cli test-model
|
||||
|
||||
<img width="400px" alt="image" src="https://github.com/user-attachments/assets/dbb0fd0a-f760-42db-acd0-bb99c859b52e" />
|
||||
|
||||
### AstrBot
|
||||
|
||||
[AstrBot](https://github.com/AstrBotDevs/AstrBot) 是易上手的多平台 LLM 聊天机器人及开发框架 ✨ 平台支持 QQ、Telegram、微信、企微、飞书。
|
||||
|
||||
使用步骤：
|
||||
1. 部署 AstrBot
|
||||
2. 在 AstrBot 中部署消息平台
|
||||
3. 执行 `weclone-cli server` 启动api服务
|
||||
4. 在 AstrBot 中新增服务提供商，类型选择OpenAI，API Base URL 根据AstrBot部署方式填写（例如docker部署可能为http://172.17.0.1:8005/v1） ，模型填写gpt-3.5-turbo,API Key随意填写一个
|
||||
5. 微调后不支持工具调用，请先关掉默认的工具，消息平台发送指令： `/tool off all`，否则会没有微调后的效果。
|
||||
6. 根据微调时使用的default_system，在 AstrBot 中设置系统提示词。
|
||||
|
||||
> [!IMPORTANT]
|
||||
> 检查api_service的日志，尽量保证大模型服务请求的参数和微调时一致，tool插件能力都关掉。
|
||||
7. 调整采样参数，例如temperature、top_p、top_k等
|
||||
[配置自定义的模型参数](https://astrbot.app/config/model-config.html#%E9%85%8D%E7%BD%AE%E8%87%AA%E5%AE%9A%E4%B9%89%E7%9A%84%E6%A8%A1%E5%9E%8B%E5%8F%82%E6%95%B0)
|
||||
|
||||
|
||||
## 📌 路线图
|
||||
|
||||
+7
-9
@@ -153,7 +153,7 @@ deepspeed --num_gpus=使用显卡数量 weclone/train/train_sft.py
|
||||
```
|
||||
|
||||
### 使用浏览器demo简单推理
|
||||
可以在这一步测试出合适的temperature、top_p值，修改settings.jsonc的`infer_args`后，供后续推理时使用。
|
||||
测试出合适的temperature、top_p值，修改settings.jsonc的`infer_args`后，供后续推理时使用。
|
||||
```bash
|
||||
weclone-cli webchat-demo
|
||||
```
|
||||
@@ -173,16 +173,16 @@ weclone-cli test-model
|
||||
|
||||
## 🖼️ 微调效果
|
||||
> [!TIP]
|
||||
> **QQ群内有部署好的Qwen2.5VL 32B Bot，可以体验效果。**
|
||||
> **QQ群内有部署好的Qwen2.5VL 32B Bot，可以体验效果。更多案例可以关注[小红书](https://www.
|
||||
xiaohongshu.com/user/profile/628109730000000021029de4)**
|
||||
|
||||
使用Qwen2.5-14B-Instruct模型，大概3万条处理后的有效数据，loss降到了3.5左右的效果：
|
||||
使用Qwen2.5VL 32B模型，大概1万条处理后的有效数据，loss降到了3.6左右的效果：
|
||||
<details>
|
||||
<summary>截图</summary>
|
||||
<div style="display: flex; flex-wrap: wrap; gap: 10px;">
|
||||
<img src="https://github.com/user-attachments/assets/0775ec52-452b-485f-9785-c6eb7b277132" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/8c7628b5-da70-4c37-9e51-fdfb0eadd2df" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/523aa742-2aa3-40e9-bd67-b98b336e83a8" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/dabf0603-dcc4-4a47-b5c3-2bbc036820d9" alt="alt text" style="width: 48%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/b7d81f9b-ea56-4f7e-8ee5-7f4171bdc66d" alt="alt text" style="width: 52%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/62e58de8-1a73-44fc-a948-0d2e949e44a0" alt="alt text" style="width: 52%; min-width: 150px;">
|
||||
<img src="https://github.com/user-attachments/assets/6bf6d0cc-7ff1-4748-a096-3850d924f954" alt="alt text" style="width: 52%; min-width: 150px;">
|
||||
</div>
|
||||
</details>
|
||||
|
||||
@@ -203,8 +203,6 @@ weclone-cli test-model
|
||||
|
||||
> [!IMPORTANT]
|
||||
> 检查api_service的日志，尽量保证大模型服务请求的参数和微调时一致，tool插件能力都关掉。
|
||||
7. 调整采样参数，例如temperature、top_p、top_k等
|
||||
[配置自定义的模型参数](https://astrbot.app/config/model-config.html#%E9%85%8D%E7%BD%AE%E8%87%AA%E5%AE%9A%E4%B9%89%E7%9A%84%E6%A8%A1%E5%9E%8B%E5%8F%82%E6%95%B0)
|
||||
|
||||
### LangBot
|
||||
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
{
|
||||
"questions": [
|
||||
[
|
||||
"Have you eaten?",
|
||||
"What did you eat?",
|
||||
"Was it delicious?",
|
||||
"How much did it cost?",
|
||||
"Can you treat me to a meal?"
|
||||
],
|
||||
[
|
||||
"What are you doing?",
|
||||
"What are you planning to do later?"
|
||||
],
|
||||
[
|
||||
"What are you busy with?",
|
||||
"Do you have any special plans for today?",
|
||||
"How are you feeling?"
|
||||
],
|
||||
[
|
||||
"Anything new happening recently?",
|
||||
"Do you have any interesting stories to share?"
|
||||
],
|
||||
[
|
||||
"How was your weekend?",
|
||||
"What fun things did you do?"
|
||||
],
|
||||
[
|
||||
"Have you watched any good movies or TV shows recently?",
|
||||
"Any recommendations?",
|
||||
"What was it about?"
|
||||
],
|
||||
[
|
||||
"How's the weather today?",
|
||||
"How about on your end?"
|
||||
],
|
||||
[
|
||||
"Is work/study going well recently?",
|
||||
"Have you encountered any challenges?"
|
||||
],
|
||||
[
|
||||
"Hey, what are you busy with right now?",
|
||||
"Do you have any special plans for today?",
|
||||
"Everything going smoothly, I hope?"
|
||||
],
|
||||
[
|
||||
"How's the weather on your side?",
|
||||
"Is it sunny or a bit gloomy?",
|
||||
"Is it cold or hot?"
|
||||
],
|
||||
[
|
||||
"Is it mealtime yet?",
|
||||
"Planning to treat yourself to something delicious today?",
|
||||
"Anything special you want to eat, or any restaurant you want to try?"
|
||||
],
|
||||
[
|
||||
"Any fun news or memes online recently?",
|
||||
"Come across any interesting videos or jokes? Share them with me!"
|
||||
],
|
||||
[
|
||||
"What are your plans for later?",
|
||||
"How do you plan to spend the rest of the day?"
|
||||
],
|
||||
[
|
||||
"Did anything catch your eye today?",
|
||||
"Let's just chat casually, any light topics?"
|
||||
],
|
||||
[
|
||||
"Any new discoveries or insights today?",
|
||||
"Did today feel fast or slow? How was the pace?"
|
||||
],
|
||||
[
|
||||
"How's your surroundings right now, noisy or quiet?",
|
||||
"Did you go out for a walk today? Was it crowded outside?",
|
||||
"Look out the window, anything special to see?"
|
||||
],
|
||||
[
|
||||
"Have you eaten?",
|
||||
"What did you eat? Did you like it?"
|
||||
],
|
||||
[
|
||||
"How was your day? Are you tired?",
|
||||
"What's up?"
|
||||
],
|
||||
[
|
||||
"How's your health recently?",
|
||||
"Nothing bothering you, right?"
|
||||
],
|
||||
[
|
||||
"Are you busy today?",
|
||||
"What have you been up to?"
|
||||
],
|
||||
[
|
||||
"Everyone at home doing well?",
|
||||
"Need any help with anything?"
|
||||
],
|
||||
[
|
||||
"Did you go out today?",
|
||||
"Is it cold/hot outside? Dress warmly/stay cool."
|
||||
],
|
||||
[
|
||||
"Anything happy happening recently? Tell me about it!",
|
||||
"Or any troubles you want to talk about?"
|
||||
],
|
||||
[
|
||||
"Go to bed early tonight, don't stay up too late.",
|
||||
"How's your sleep been lately?"
|
||||
],
|
||||
[
|
||||
"Need anything? Just let me know.",
|
||||
"Do you have enough money?"
|
||||
],
|
||||
[
|
||||
"See anything interesting today?",
|
||||
"Or anything you want to share with me?"
|
||||
],
|
||||
[
|
||||
"Any weekend plans?",
|
||||
"Want to grab a meal together or go out?"
|
||||
],
|
||||
[
|
||||
"How are those friends you keep in touch with?",
|
||||
"You should get together more often."
|
||||
],
|
||||
[
|
||||
"Is work/study going smoothly?",
|
||||
"Don't put too much pressure on yourself."
|
||||
],
|
||||
[
|
||||
"What delicious food did you make today?",
|
||||
"Let me try some next time!"
|
||||
],
|
||||
[
|
||||
"Any news recently?",
|
||||
"Tell me about it."
|
||||
],
|
||||
[
|
||||
"How's so-and-so doing lately?",
|
||||
"Haven't heard from him/her in a while."
|
||||
],
|
||||
[
|
||||
"Are you in a good mood today?",
|
||||
"You look good/a bit tired."
|
||||
],
|
||||
[
|
||||
"Anything you want to eat? I'll make it/bring it for you next time.",
|
||||
"Or anywhere you want to go? I'll come with you."
|
||||
],
|
||||
[
|
||||
"Have you been watching any TV shows/movies recently?",
|
||||
"Any good recommendations for me?"
|
||||
],
|
||||
[
|
||||
"If there's nothing urgent, go home/rest early.",
|
||||
"Stay safe."
|
||||
]
|
||||
]
|
||||
}
|
||||
@@ -91,6 +91,6 @@
|
||||
"gpu_memory_utilization": 0.9
|
||||
},
|
||||
"test_model_args": {
|
||||
"test_data_path": "dataset/test_data.json"
|
||||
"test_data_path": "dataset/eval/test_data-en.json"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
"adapter_name_or_path": "./model_output", //同时做为train_sft_args的output_dir
|
||||
"template": "qwen2_vl",
|
||||
"default_system": "请你扮演一名人类，不要说自己是人工智能",
|
||||
"language": "en", // 聊天常用语言: zh(中文), en(英文)
|
||||
"finetuning_type": "lora",
|
||||
"media_dir": "dataset/media",
|
||||
"image_max_pixels": 409920, //720P
|
||||
@@ -20,7 +21,6 @@
|
||||
"telegram_args": {
|
||||
"my_id": "user1234567890"
|
||||
},
|
||||
"language": "en", // 聊天常用语言: zh(中文), en(英文)
|
||||
"include_type": [
|
||||
"text",
|
||||
"image"
|
||||
@@ -95,6 +95,6 @@
|
||||
"gpu_memory_utilization": 0.9
|
||||
},
|
||||
"test_model_args": {
|
||||
"test_data_path": "dataset/test_data.json"
|
||||
"test_data_path": "dataset/eval/test_data-en.json"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -93,6 +93,6 @@
|
||||
// "load_format": "bitsandbytes"
|
||||
},
|
||||
"test_model_args": {
|
||||
"test_data_path": "dataset/test_data.json"
|
||||
"test_data_path": "dataset/eval/test_data-zh.json"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -76,6 +76,8 @@ class FinetuningType(StrEnum):
|
||||
|
||||
|
||||
class CommonArgs(BaseConfigModel):
|
||||
"""NOTE that all parameters here will be parsed by `HfArgumentParser`. Non-HfArgumentParser parameters should be placed in make_dataset_args."""
|
||||
|
||||
model_name_or_path: str = Field(...)
|
||||
adapter_name_or_path: str = Field("./model_output", description="Also as output_dir of train_sft_args")
|
||||
template: str = Field(..., description="model template")
|
||||
@@ -189,7 +191,7 @@ class VllmArgs(BaseConfigModel):
|
||||
|
||||
|
||||
class TestModelArgs(BaseConfigModel):
|
||||
test_data_path: str = Field(default="dataset/test_data.json", description="测试数据路径")
|
||||
test_data_path: str = Field(default="dataset/eval/test_data-en.json", description="测试数据路径")
|
||||
|
||||
|
||||
class WcConfig(BaseModel):
|
||||
|
||||
Reference in New Issue
Block a user