opensource/self-llm
3
0
Fork 0
mirror of https://github.com/datawhalechina/self-llm.git synced 2026-09-01 15:39:45 +08:00
Code Issues Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Joe-2002
2024-11-14 17:51:39 +08:00
parent 6bd467a041
commit 406c1e4a32
4 changed files with 1177 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1 -1
View File
@@ -67,7 +67,7 @@ instance/
# Scrapy stuff:
.scrapy
.history
# Sphinx documentation
docs/_build/
README.md
+1 -1
View File
@@ -65,7 +65,7 @@
- [ ] [Qwen2-vl-2B FastApi 部署调用]()
- [ ] [Qwen2-vl-2B WebDemo 部署]()
- [ ] [Qwen2-vl-2B vLLM 部署]()
- [ ] [Qwen2-vl-2B Lora 微调]()
- [ ] [Qwen2-vl-2B Lora 微调]() @李柯辰
- [Qwen2.5](https://github.com/QwenLM/Qwen2.5)
- [x] [Qwen2.5-7B-Instruct FastApi 部署调用](./models/Qwen2.5/01-Qwen2.5-7B-Instruct%20FastApi%20部署调用.md) @娄天奥
models/Qwen2-vl-2B/04-Qwen2-vl-2B Lora .ipynb
+954
View File
@@ -0,0 +1,954 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "de53995b-32ed-4722-8cac-ba104c8efacb",
"metadata": {},
"source": [
"# 导入环境"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "52fac949-4150-4091-b0c3-2968ab5e385c",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from datasets import Dataset\n",
"import pandas as pd\n",
"from transformers import AutoTokenizer, AutoModelForCausalLM, DataCollatorForSeq2Seq, TrainingArguments, Trainer, GenerationConfig"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "e098d9eb",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# 将JSON文件转换为CSV文件\n",
"df = pd.read_json('dataset/huanhuan.json')\n",
"ds = Dataset.from_pandas(df)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "8ac92d42-efae-49b1-a00e-ccaa75b98938",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"{'instruction': ['小姐，别的秀女都在求中选，唯有咱们小姐想被撂牌子，菩萨一定记得真真儿的——',\n",
" '这个温太医啊，也是古怪，谁不知太医不得皇命不能为皇族以外的人请脉诊病，他倒好，十天半月便往咱们府里跑。',\n",
" '嬛妹妹，刚刚我去府上请脉，听甄伯母说你来这里进香了。'],\n",
" 'input': ['', '', ''],\n",
" 'output': ['嘘——都说许愿说破是不灵的。', '你们俩话太多了，我该和温太医要一剂药，好好治治你们。', '出来走走，也是散心。']}"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds[:3]"
]
},
{
"cell_type": "markdown",
"id": "51d05e5d-d14e-4f03-92be-9a9677d41918",
"metadata": {},
"source": [
"# 处理数据集"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "74ee5a67-2e55-4974-b90e-cbf492de500a",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
]
},
{
"data": {
"text/plain": [
"Qwen2Tokenizer(name_or_path='/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct', vocab_size=151643, model_max_length=131072, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'eos_token': '<|im_end|>', 'pad_token': '<|endoftext|>', 'additional_special_tokens': ['<|im_start|>', '<|im_end|>', '<|object_ref_start|>', '<|object_ref_end|>', '<|box_start|>', '<|box_end|>', '<|quad_start|>', '<|quad_end|>', '<|vision_start|>', '<|vision_end|>', '<|vision_pad|>', '<|image_pad|>', '<|video_pad|>']}, clean_up_tokenization_spaces=False), added_tokens_decoder={\n",
"\t151643: AddedToken(\"<|endoftext|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151644: AddedToken(\"<|im_start|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151645: AddedToken(\"<|im_end|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151646: AddedToken(\"<|object_ref_start|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151647: AddedToken(\"<|object_ref_end|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151648: AddedToken(\"<|box_start|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151649: AddedToken(\"<|box_end|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151650: AddedToken(\"<|quad_start|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151651: AddedToken(\"<|quad_end|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151652: AddedToken(\"<|vision_start|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151653: AddedToken(\"<|vision_end|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151654: AddedToken(\"<|vision_pad|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151655: AddedToken(\"<|image_pad|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151656: AddedToken(\"<|video_pad|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
"\t151657: AddedToken(\"<tool_call>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151658: AddedToken(\"</tool_call>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151659: AddedToken(\"<|fim_prefix|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151660: AddedToken(\"<|fim_middle|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151661: AddedToken(\"<|fim_suffix|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151662: AddedToken(\"<|fim_pad|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151663: AddedToken(\"<|repo_name|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"\t151664: AddedToken(\"<|file_sep|>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=False),\n",
"}"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tokenizer = AutoTokenizer.from_pretrained('/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct', use_fast=False, trust_remote_code=True)\n",
"tokenizer"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "2503a5fa-9621-4495-9035-8e7ef6525691",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"def process_func(example):\n",
" MAX_LENGTH = 384 # Llama分词器会将一个中文字切分为多个token，因此需要放开一些最大长度，保证数据的完整性\n",
" input_ids, attention_mask, labels = [], [], []\n",
" instruction = tokenizer(f\"<|im_start|>system\\n现在你要扮演皇帝身边的女人--甄嬛<|im_end|>\\n<|im_start|>user\\n{example['instruction'] + example['input']}<|im_end|>\\n<|im_start|>assistant\\n\", add_special_tokens=False) # add_special_tokens 不在开头加 special_tokens\n",
" response = tokenizer(f\"{example['output']}\", add_special_tokens=False)\n",
" input_ids = instruction[\"input_ids\"] + response[\"input_ids\"] + [tokenizer.pad_token_id]\n",
" attention_mask = instruction[\"attention_mask\"] + response[\"attention_mask\"] + [1] # 因为eos token咱们也是要关注的所以 补充为1\n",
" labels = [-100] * len(instruction[\"input_ids\"]) + response[\"input_ids\"] + [tokenizer.pad_token_id] \n",
" if len(input_ids) > MAX_LENGTH: # 做一个截断\n",
" input_ids = input_ids[:MAX_LENGTH]\n",
" attention_mask = attention_mask[:MAX_LENGTH]\n",
" labels = labels[:MAX_LENGTH]\n",
" return {\n",
" \"input_ids\": input_ids,\n",
" \"attention_mask\": attention_mask,\n",
" \"labels\": labels\n",
" }"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "84f870d6-73a9-4b0f-8abf-687b32224ad8",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Map: 0%| | 0/3729 [00:00<?, ? examples/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"Dataset({\n",
" features: ['input_ids', 'attention_mask', 'labels'],\n",
" num_rows: 3729\n",
"})"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tokenized_id = ds.map(process_func, remove_columns=ds.column_names)\n",
"tokenized_id"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "1f7e15a0-4d9a-4935-9861-00cc472654b1",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'<|im_start|>system\\n现在你要扮演皇帝身边的女人--甄嬛<|im_end|>\\n<|im_start|>user\\n小姐，别的秀女都在求中选，唯有咱们小姐想被撂牌子，菩萨一定记得真真儿的——<|im_end|>\\n<|im_start|>assistant\\n嘘——都说许愿说破是不灵的。<|endoftext|>'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tokenizer.decode(tokenized_id[0]['input_ids'])"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "97f16f66-324a-454f-8cc3-ef23b100ecff",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'你们俩话太多了，我该和温太医要一剂药，好好治治你们。<|endoftext|>'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tokenizer.decode(list(filter(lambda x: x != -100, tokenized_id[1][\"labels\"])))"
]
},
{
"cell_type": "markdown",
"id": "424823a8-ed0d-4309-83c8-3f6b1cdf274c",
"metadata": {},
"source": [
"# 创建模型"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "170764e5-d899-4ef4-8c53-36f6dec0d198",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "289ae6a90ec24c3c992405abbcf347c0",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"Qwen2ForCausalLM(\n",
" (model): Qwen2Model(\n",
" (embed_tokens): Embedding(152064, 3584)\n",
" (layers): ModuleList(\n",
" (0-27): 28 x Qwen2DecoderLayer(\n",
" (self_attn): Qwen2SdpaAttention(\n",
" (q_proj): Linear(in_features=3584, out_features=3584, bias=True)\n",
" (k_proj): Linear(in_features=3584, out_features=512, bias=True)\n",
" (v_proj): Linear(in_features=3584, out_features=512, bias=True)\n",
" (o_proj): Linear(in_features=3584, out_features=3584, bias=False)\n",
" (rotary_emb): Qwen2RotaryEmbedding()\n",
" )\n",
" (mlp): Qwen2MLP(\n",
" (gate_proj): Linear(in_features=3584, out_features=18944, bias=False)\n",
" (up_proj): Linear(in_features=3584, out_features=18944, bias=False)\n",
" (down_proj): Linear(in_features=18944, out_features=3584, bias=False)\n",
" (act_fn): SiLU()\n",
" )\n",
" (input_layernorm): Qwen2RMSNorm()\n",
" (post_attention_layernorm): Qwen2RMSNorm()\n",
" )\n",
" )\n",
" (norm): Qwen2RMSNorm()\n",
" )\n",
" (lm_head): Linear(in_features=3584, out_features=152064, bias=False)\n",
")"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import torch\n",
"\n",
"model = AutoModelForCausalLM.from_pretrained('/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct', device_map=\"auto\",torch_dtype=torch.bfloat16)\n",
"model"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "2323eac7-37d5-4288-8bc5-79fac7113402",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"model.enable_input_require_grads() # 开启梯度检查点时，要执行该方法"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "f808b05c-f2cb-48cf-a80d-0c42be6051c7",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"torch.bfloat16"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.dtype"
]
},
{
"cell_type": "markdown",
"id": "13d71257-3c1c-4303-8ff8-af161ebc2cf1",
"metadata": {},
"source": [
"# lora "
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "2d304ae2-ab60-4080-a80d-19cac2e3ade3",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"LoraConfig(peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, task_type=<TaskType.CAUSAL_LM: 'CAUSAL_LM'>, inference_mode=False, r=8, target_modules={'o_proj', 'gate_proj', 'q_proj', 'k_proj', 'up_proj', 'down_proj', 'v_proj'}, lora_alpha=32, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', loftq_config={}, use_dora=False, layer_replication=None)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from peft import LoraConfig, TaskType, get_peft_model\n",
"\n",
"config = LoraConfig(\n",
" task_type=TaskType.CAUSAL_LM, \n",
" target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"],\n",
" inference_mode=False, # 训练模式\n",
" r=8, # Lora 秩\n",
" lora_alpha=32, # Lora alaph，具体作用参见 Lora 原理\n",
" lora_dropout=0.1# Dropout 比例\n",
")\n",
"config"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "2c2489c5-eaab-4e1f-b06a-c3f914b4bf8e",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"LoraConfig(peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path='/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct', revision=None, task_type=<TaskType.CAUSAL_LM: 'CAUSAL_LM'>, inference_mode=False, r=8, target_modules={'o_proj', 'gate_proj', 'q_proj', 'k_proj', 'up_proj', 'down_proj', 'v_proj'}, lora_alpha=32, lora_dropout=0.1, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=None, init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', loftq_config={}, use_dora=False, layer_replication=None)"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model = get_peft_model(model, config)\n",
"config"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "ebf5482b-fab9-4eb3-ad88-c116def4be12",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"trainable params: 20,185,088 || all params: 7,635,801,600 || trainable%: 0.26434798934534914\n"
]
}
],
"source": [
"model.print_trainable_parameters()"
]
},
{
"cell_type": "markdown",
"id": "ca055683-837f-4865-9c57-9164ba60c00f",
"metadata": {},
"source": [
"# 配置训练参数"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "7e76bbff-15fd-4995-a61d-8364dc5e9ea0",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"args = TrainingArguments(\n",
" output_dir=\"./output/Qwen2.5_instruct_lora\",\n",
" per_device_train_batch_size=4,\n",
" gradient_accumulation_steps=4,\n",
" logging_steps=10,\n",
" num_train_epochs=3,\n",
" save_steps=100, \n",
" learning_rate=1e-4,\n",
" save_on_each_node=True,\n",
" gradient_checkpointing=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "f142cb9c-ad99-48e6-ba86-6df198f9ed96",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"trainer = Trainer(\n",
" model=model,\n",
" args=args,\n",
" train_dataset=tokenized_id,\n",
" data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "aec9bc36-b297-45af-99e1-d4c4d82be081",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n"
]
},
{
"data": {
"text/html": [
"\n",
" <div>\n",
" \n",
" <progress value='699' max='699' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
" [699/699 10:02, Epoch 2/3]\n",
" </div>\n",
" <table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>Step</th>\n",
" <th>Training Loss</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>10</td>\n",
" <td>3.759900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>20</td>\n",
" <td>3.096600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>30</td>\n",
" <td>3.063300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>40</td>\n",
" <td>2.917600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>50</td>\n",
" <td>2.900500</td>\n",
" </tr>\n",
" <tr>\n",
" <td>60</td>\n",
" <td>2.953600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>70</td>\n",
" <td>2.912600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>80</td>\n",
" <td>3.013900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>90</td>\n",
" <td>3.045900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>100</td>\n",
" <td>2.933000</td>\n",
" </tr>\n",
" <tr>\n",
" <td>110</td>\n",
" <td>2.927600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>120</td>\n",
" <td>2.942600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>130</td>\n",
" <td>2.921200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>140</td>\n",
" <td>2.876200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>150</td>\n",
" <td>2.984700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>160</td>\n",
" <td>2.968600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>170</td>\n",
" <td>2.955000</td>\n",
" </tr>\n",
" <tr>\n",
" <td>180</td>\n",
" <td>2.809300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>190</td>\n",
" <td>2.859800</td>\n",
" </tr>\n",
" <tr>\n",
" <td>200</td>\n",
" <td>2.913100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>210</td>\n",
" <td>2.815100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>220</td>\n",
" <td>2.853700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>230</td>\n",
" <td>2.863900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>240</td>\n",
" <td>2.807300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>250</td>\n",
" <td>2.647200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>260</td>\n",
" <td>2.501700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>270</td>\n",
" <td>2.542600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>280</td>\n",
" <td>2.585100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>290</td>\n",
" <td>2.608100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>300</td>\n",
" <td>2.556000</td>\n",
" </tr>\n",
" <tr>\n",
" <td>310</td>\n",
" <td>2.517500</td>\n",
" </tr>\n",
" <tr>\n",
" <td>320</td>\n",
" <td>2.585400</td>\n",
" </tr>\n",
" <tr>\n",
" <td>330</td>\n",
" <td>2.566300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>340</td>\n",
" <td>2.512500</td>\n",
" </tr>\n",
" <tr>\n",
" <td>350</td>\n",
" <td>2.631100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>360</td>\n",
" <td>2.518600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>370</td>\n",
" <td>2.568600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>380</td>\n",
" <td>2.528400</td>\n",
" </tr>\n",
" <tr>\n",
" <td>390</td>\n",
" <td>2.564100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>400</td>\n",
" <td>2.423300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>410</td>\n",
" <td>2.631800</td>\n",
" </tr>\n",
" <tr>\n",
" <td>420</td>\n",
" <td>2.402200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>430</td>\n",
" <td>2.598100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>440</td>\n",
" <td>2.533700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>450</td>\n",
" <td>2.479900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>460</td>\n",
" <td>2.486800</td>\n",
" </tr>\n",
" <tr>\n",
" <td>470</td>\n",
" <td>2.390800</td>\n",
" </tr>\n",
" <tr>\n",
" <td>480</td>\n",
" <td>2.299600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>490</td>\n",
" <td>2.225800</td>\n",
" </tr>\n",
" <tr>\n",
" <td>500</td>\n",
" <td>2.185200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>510</td>\n",
" <td>2.116700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>520</td>\n",
" <td>2.199900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>530</td>\n",
" <td>2.308900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>540</td>\n",
" <td>2.233000</td>\n",
" </tr>\n",
" <tr>\n",
" <td>550</td>\n",
" <td>2.086700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>560</td>\n",
" <td>2.243200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>570</td>\n",
" <td>2.195300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>580</td>\n",
" <td>2.232300</td>\n",
" </tr>\n",
" <tr>\n",
" <td>590</td>\n",
" <td>2.163100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>600</td>\n",
" <td>2.155500</td>\n",
" </tr>\n",
" <tr>\n",
" <td>610</td>\n",
" <td>2.266100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>620</td>\n",
" <td>2.220200</td>\n",
" </tr>\n",
" <tr>\n",
" <td>630</td>\n",
" <td>2.120700</td>\n",
" </tr>\n",
" <tr>\n",
" <td>640</td>\n",
" <td>1.994400</td>\n",
" </tr>\n",
" <tr>\n",
" <td>650</td>\n",
" <td>2.347600</td>\n",
" </tr>\n",
" <tr>\n",
" <td>660</td>\n",
" <td>2.120000</td>\n",
" </tr>\n",
" <tr>\n",
" <td>670</td>\n",
" <td>2.190900</td>\n",
" </tr>\n",
" <tr>\n",
" <td>680</td>\n",
" <td>2.190100</td>\n",
" </tr>\n",
" <tr>\n",
" <td>690</td>\n",
" <td>2.143500</td>\n",
" </tr>\n",
" </tbody>\n",
"</table><p>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/root/miniconda3/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /root/autodl-tmp/qwen/Qwen2.5-7B-Instruct - will assume that the vocabulary was not modified.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /root/autodl-tmp/qwen/Qwen2.5-7B-Instruct - will assume that the vocabulary was not modified.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /root/autodl-tmp/qwen/Qwen2.5-7B-Instruct - will assume that the vocabulary was not modified.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /root/autodl-tmp/qwen/Qwen2.5-7B-Instruct - will assume that the vocabulary was not modified.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /root/autodl-tmp/qwen/Qwen2.5-7B-Instruct - will assume that the vocabulary was not modified.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/peft/utils/save_and_load.py:154: UserWarning: Could not find a config file in /root/autodl-tmp/qwen/Qwen2.5-7B-Instruct - will assume that the vocabulary was not modified.\n",
" warnings.warn(\n",
"/root/miniconda3/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n"
]
},
{
"data": {
"text/plain": [
"TrainOutput(global_step=699, training_loss=2.5707558644176043, metrics={'train_runtime': 603.8093, 'train_samples_per_second': 18.527, 'train_steps_per_second': 1.158, 'total_flos': 4.578125275274342e+16, 'train_loss': 2.5707558644176043, 'epoch': 3.0})"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"trainer.train()"
]
},
{
"cell_type": "markdown",
"id": "8abb2327-458e-4e96-ac98-2141b5b97c8e",
"metadata": {},
"source": [
"# 合并加载模型"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "bd2a415a-a9ad-49ea-877f-243558a83bfc",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "e6dc6e1f8180464aa17ad958188e3aff",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu.\n",
"WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"我是甄嬛，一个在后宫中挣扎求存的女子。\n"
]
}
],
"source": [
"from transformers import AutoModelForCausalLM, AutoTokenizer\n",
"import torch\n",
"from peft import PeftModel\n",
"\n",
"mode_path = '/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct/'\n",
"lora_path = './output/Qwen2.5_instruct_lora/checkpoint-10' # 这里改称你的 lora 输出对应 checkpoint 地址\n",
"\n",
"# 加载tokenizer\n",
"tokenizer = AutoTokenizer.from_pretrained(mode_path, trust_remote_code=True)\n",
"\n",
"# 加载模型\n",
"model = AutoModelForCausalLM.from_pretrained(mode_path, device_map=\"auto\",torch_dtype=torch.bfloat16, trust_remote_code=True).eval()\n",
"\n",
"# 加载lora权重\n",
"model = PeftModel.from_pretrained(model, model_id=lora_path)\n",
"\n",
"prompt = \"你是谁？\"\n",
"inputs = tokenizer.apply_chat_template([{\"role\": \"user\", \"content\": \"假设你是皇帝身边的女人--甄嬛。\"},{\"role\": \"user\", \"content\": prompt}],\n",
" add_generation_prompt=True,\n",
" tokenize=True,\n",
" return_tensors=\"pt\",\n",
" return_dict=True\n",
" ).to('cuda')\n",
"\n",
"\n",
"gen_kwargs = {\"max_length\": 2500, \"do_sample\": True, \"top_k\": 1}\n",
"with torch.no_grad():\n",
" outputs = model.generate(**inputs, **gen_kwargs)\n",
" outputs = outputs[:, inputs['input_ids'].shape[1]:]\n",
" print(tokenizer.decode(outputs[0], skip_special_tokens=True))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
models/Qwen2-vl-2B/04-Qwen2-vl-2B Lora 微调.md
+221
View File
@@ -0,0 +1,221 @@
# Qwen2-vl 2B Lora 微调
本节我们简要介绍如何基于 transformers、peft 等框架，对 Qwen2-vl 2B Lora 模型进行 Lora 微调。Lora 是一种高效微调方法，深入了解其原理可参见博客：[知乎|深入浅出 Lora](https://zhuanlan.zhihu.com/p/650197598)。
这个教程会在同目录下给大家提供一个 [nodebook](./05-Qwen2-7B-Instruct%20Lora.ipynb) 文件，来让大家更好的学习。
## 环境配置
在完成基本环境配置和本地模型部署的情况下，你还需要安装一些第三方库，可以使用以下命令：
```bash
python -m pip install --upgrade pip
# 更换 pypi 源加速库的安装
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
pip install modelscope==1.18.0
pip install transformers==4.44.2
pip install streamlit==1.24.0
pip install sentencepiece==0.2.0
pip install accelerate==0.34.2
pip install datasets==2.20.0
pip install peft==0.11.1
```
<!-- > 考虑到部分同学配置环境可能会遇到一些问题，我们在 AutoDL 平台准备了 Qwen2.5 的环境镜像，该镜像适用于该仓库除 Qwen-GPTQ 和 vllm 外的所有部署环境。点击下方链接并直接创建 Autodl 示例即可。
> ***https://www.codewithgpu.com/i/datawhalechina/self-llm/Qwen2.5-self-llm*** -->
> 注意：flash-attn 安装会比较慢，大概需要十几分钟。
在本节教程里，我们将微调数据集放置在根目录 [/dataset](../dataset/huanhuan.json)。
## 模型下载
使用 `modelscope` 中的 `snapshot_download` 函数下载模型，第一个参数为模型名称，参数 `cache_dir` 为模型的下载路径。
新建 `model_download.py` 文件输入以下代码，并运行 `python model_download.py` 执行下载。
此处使用 `modelscope` 提供的 `snapshot_download` 函数进行下载，该方法对国内的用户十分友好。
在 /root/autodl-tmp 路径下新建 model_download.py 文件并在其中输入以下内容，粘贴代码后请及时保存文件，如下图所示。并运行 `python /root/autodl-tmp/model_download.py` 执行下载，模型大小为 15GB，下载模型大概需要 5 分钟。
```python
import torch
from modelscope import snapshot_download, AutoModel, AutoTokenizer
import os
model_dir = snapshot_download('qwen/Qwen2.5-7B-Instruct', cache_dir='/root/autodl-tmp', revision='master')
```
## 指令集构建
LLM 的微调一般指指令微调过程。所谓指令微调，是说我们使用的微调数据形如：
```json
{
"instruction": "回答以下用户问题，仅输出答案。",
"input": "1+1等于几?",
"output": "2"
}
```
其中，`instruction` 是用户指令，告知模型其需要完成的任务；`input` 是用户输入，是完成用户指令所必须的输入内容；`output` 是模型应该给出的输出。
即我们的核心训练目标是让模型具有理解并遵循用户指令的能力。因此，在指令集构建时，我们应针对我们的目标任务，针对性构建任务指令集。例如，在本节我们使用由笔者合作开源的 [Chat-甄嬛](https://github.com/KMnO4-zx/huanhuan-chat) 项目作为示例，我们的目标是构建一个能够模拟甄嬛对话风格的个性化 LLM，因此我们构造的指令形如：
```json
{
"instruction": "你是谁？",
"input": "",
"output": "家父是大理寺少卿甄远道。"
}
```
我们所构造的全部指令数据集在根目录下。
## 数据格式化
`Lora` 训练的数据是需要经过格式化、编码之后再输入给模型进行训练的，如果是熟悉 `Pytorch` 模型训练流程的同学会知道，我们一般需要将输入文本编码为 input_ids，将输出文本编码为 `labels`，编码之后的结果都是多维的向量。我们首先定义一个预处理函数，这个函数用于对每一个样本，编码其输入、输出文本并返回一个编码后的字典：
```python
def process_func(example):
MAX_LENGTH = 384 # Llama分词器会将一个中文字切分为多个token，因此需要放开一些最大长度，保证数据的完整性
input_ids, attention_mask, labels = [], [], []
instruction = tokenizer(f"<|im_start|>system\n现在你要扮演皇帝身边的女人--甄嬛<|im_end|>\n<|im_start|>user\n{example['instruction'] + example['input']}<|im_end|>\n<|im_start|>assistant\n", add_special_tokens=False) # add_special_tokens 不在开头加 special_tokens
response = tokenizer(f"{example['output']}", add_special_tokens=False)
input_ids = instruction["input_ids"] + response["input_ids"] + [tokenizer.pad_token_id]
attention_mask = instruction["attention_mask"] + response["attention_mask"] + [1] # 因为eos token咱们也是要关注的所以 补充为1
labels = [-100] * len(instruction["input_ids"]) + response["input_ids"] + [tokenizer.pad_token_id]
if len(input_ids) > MAX_LENGTH: # 做一个截断
input_ids = input_ids[:MAX_LENGTH]
attention_mask = attention_mask[:MAX_LENGTH]
labels = labels[:MAX_LENGTH]
return {
"input_ids": input_ids,
"attention_mask": attention_mask,
"labels": labels
}
```
`Qwen2` 采用的 `Prompt Template`格式如下：
```text
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
你是谁？<|im_end|>
<|im_start|>assistant
我是一个有用的助手。<|im_end|>
```
## 加载 tokenizer 和半精度模型
模型以半精度形式加载，如果你的显卡比较新的话，可以用 `torch.bfolat`形式加载。对于自定义的模型一定要指定 `trust_remote_code`参数为 `True`。
```python
tokenizer = AutoTokenizer.from_pretrained('/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct/', use_fast=False, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained('/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct/', device_map="auto",torch_dtype=torch.bfloat16)
```
## 定义 LoraConfig
`LoraConfig`这个类中可以设置很多参数，但主要的参数没多少，简单讲一讲，感兴趣的同学可以直接看源码。
- `task_type`：模型类型
- `target_modules`：需要训练的模型层的名字，主要就是 `attention`部分的层，不同的模型对应的层的名字不同，可以传入数组，也可以字符串，也可以正则表达式。
- `r`：`lora`的秩，具体可以看 `Lora`原理
- `lora_alpha`：`Lora alaph`，具体作用参见 `Lora` 原理
`Lora`的缩放是啥嘞？当然不是 `r`（秩），这个缩放就是 `lora_alpha/r`, 在这个 `LoraConfig`中缩放就是 4 倍。
```python
config = LoraConfig(
task_type=TaskType.CAUSAL_LM,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
inference_mode=False, # 训练模式
r=8, # Lora 秩
lora_alpha=32, # Lora alaph，具体作用参见 Lora 原理
lora_dropout=0.1# Dropout 比例
)
```
## 自定义 TrainingArguments 参数
`TrainingArguments`这个类的源码也介绍了每个参数的具体作用，当然大家可以来自行探索，这里就简单说几个常用的。
- `output_dir`：模型的输出路径
- `per_device_train_batch_size`：顾名思义 `batch_size`
- `gradient_accumulation_steps`: 梯度累加，如果你的显存比较小，那可以把 `batch_size` 设置小一点，梯度累加增大一些。
- `logging_steps`：多少步，输出一次 `log`
- `num_train_epochs`：顾名思义 `epoch`
- `gradient_checkpointing`：梯度检查，这个一旦开启，模型就必须执行 `model.enable_input_require_grads()`，这个原理大家可以自行探索，这里就不细说了。
```python
args = TrainingArguments(
output_dir="./output/Qwen2.5_instruct_lora",
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
logging_steps=10,
num_train_epochs=3,
save_steps=100,
learning_rate=1e-4,
save_on_each_node=True,
gradient_checkpointing=True
)
```
## 使用 Trainer 训练
```python
trainer = Trainer(
model=model,
args=args,
train_dataset=tokenized_id,
data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True),
)
trainer.train()
```
## 加载 lora 权重推理
训练好了之后可以使用如下方式加载 `lora`权重进行推理：
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
from peft import PeftModel
model_path = '/root/autodl-tmp/qwen/Qwen2.5-7B-Instruct/'
lora_path = 'lora_path'
# 加载tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path)
# 加载模型
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto",torch_dtype=torch.bfloat16)
# 加载lora权重
model = PeftModel.from_pretrained(model, model_id=lora_path, config=config)
prompt = "你是谁？"
messages = [
{"role": "system", "content": "现在你要扮演皇帝身边的女人--甄嬛"},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to('cuda')
generated_ids = model.generate(
model_inputs.input_ids,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
```