opensource/self-llm
3
0
Fork 0
mirror of https://github.com/datawhalechina/self-llm.git synced 2026-08-28 15:11:50 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update 02-OpenELM-3B-Instruct Lora微调.md

Browse Source 
Fix doc error.
This commit is contained in:
Cyzsd
2024-12-15 17:51:50 +08:00
committed by GitHub
parent a24cd04181
commit 5b74093f94
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
models/OpenELM/02-OpenELM-3B-Instruct Lora微调.md
+1 -1
View File
@@ -134,7 +134,7 @@ model = AutoModelForCausalLM.from_pretrained(
- `r`：`lora`的秩，具体可以看`Lora`原理
- `lora_alpha`：`Lora alaph`，具体作用参见 `Lora` 原理
`Lora`的缩放是啥嘞？当然不是`r`（秩），这个缩放就是`lora_alpha/r`, 在这个`LoraConfig`中缩放就是4倍。
`Lora`的缩放是啥嘞？当然不是`r`（秩），这个缩放就是`lora_alpha/r`, 在这个`LoraConfig`中缩放就是1倍。
```python
config = LoraConfig(