Merge pull request #13 from xming521/dependencies1

Dependencies1
2026-08-30 21:29:24 +08:00 · 2025-04-03 20:41:38 +08:00
parent 6e13da99e8 cf62b5e37e
commit 4361a0954d
20 changed files with 555 additions and 95 deletions
@@ -150,6 +150,9 @@ data/test
 .vscode
 *-my.*
 *.csv
-test.py
+test.*
 *users.json   
 WeClone-audio/src/output*.wav
+WeClone-audio/uv.lock
+Spark-TTS-0.5B/
+uv.lock
@@ -2,7 +2,7 @@

 ## 核心功能✨
 - 💬 使用微信聊天记录微调LLM
- 🎙️ 使用微信语音消息结合大模型实现高质量声音克隆 👉[WeClone-audio](https://github.com/xming521/WeClone/tree/master/WeClone-audio)
+- 🎙️ 使用微信语音消息➕0.5B大模型实现高质量声音克隆 👉[WeClone-audio](https://github.com/xming521/WeClone/tree/master/WeClone-audio)
 - 🔗 绑定到微信机器人，实现自己的数字分身

 ## 特性与说明📋
@@ -21,44 +21,29 @@
 目前项目默认使用chatglm3-6b模型，LoRA方法对sft阶段微调，大约需要16GB显存。也可以使用[LLaMA Factory](https://github.com/hiyouga/LLaMA-Factory/blob/main/README_zh.md#%E6%A8%A1%E5%9E%8B)支持的其他模型和方法，占用显存更少，需要自行修改模板的system提示词等相关配置。

 需要显存的估算值：
-| 训练方法 | 精度 |   7B  |  13B  |  30B  |   65B  |   8x7B |
-| ------- | ---- | ----- | ----- | ----- | ------ | ------ |
-| 全参数   |  16  | 160GB | 320GB | 600GB | 1200GB |  900GB |
-| 部分参数 |  16  |  20GB |  40GB | 120GB |  240GB |  200GB |
-| LoRA    |  16  |  **16GB** |  32GB |  80GB |  160GB |  120GB |
-| QLoRA   |   8  |  10GB |  16GB |  40GB |   80GB |   80GB |
-| QLoRA   |   4  |   6GB |  12GB |  24GB |   48GB |   32GB |
+| 方法                             | 精度 |   7B  |  14B  |  30B  |   70B  |   `x`B  |
+| ------------------------------- | ---- | ----- | ----- | ----- | ------ | ------- |
+| Full (`bf16` or `fp16`)         |  32  | 120GB | 240GB | 600GB | 1200GB | `18x`GB |
+| Full (`pure_bf16`)              |  16  |  60GB | 120GB | 300GB |  600GB |  `8x`GB |
+| Freeze/LoRA/GaLore/APOLLO/BAdam |  16  |  16GB |  32GB |  64GB |  160GB |  `2x`GB |
+| QLoRA                           |   8  |  10GB |  20GB |  40GB |   80GB |   `x`GB |
+| QLoRA                           |   4  |   6GB |  12GB |  24GB |   48GB | `x/2`GB |
+| QLoRA                           |   2  |   4GB |   8GB |  16GB |   24GB | `x/4`GB |

-### 软件要求
-
-| 必需项       | 至少     | 推荐      |
-| ------------ | ------- | --------- |
-| python       | 3.8     | 3.10      |
-| torch        | 1.13.1  | 2.2.1     |
-| transformers | 4.37.2  | 4.38.1    |
-| datasets     | 2.14.3  | 2.17.1    |
-| accelerate   | 0.27.2  | 0.27.2    |
-| peft         | 0.9.0   | 0.9.0     |
-| trl          | 0.7.11  | 0.7.11    |
-
-| 可选项       | 至少     | 推荐      |
-| ------------ | ------- | --------- |
-| CUDA         | 11.6    | 12.2      |
-| deepspeed    | 0.10.0  | 0.13.4    |
-| bitsandbytes | 0.39.0  | 0.41.3    |
-| flash-attn   | 2.3.0   | 2.5.5     |

 ### 环境搭建
-
+建议使用 [uv](https://docs.astral.sh/uv/)，这是一个非常快速的 Python 环境管理器。安装uv后，您可以使用以下命令创建一个新的Python环境并安装依赖项，注意这不包含xcodec（音频克隆）功能的依赖：
 ```bash
 git clone https://github.com/xming521/WeClone.git
-conda create -n weclone python=3.10
-conda activate weclone
 cd WeClone
-pip install -r requirements.txt
+uv venv .venv --python=3.9
+source .venv/bin/activate
+uv pip install --group main -e . 
 ```

-训练以及推理相关配置统一在文件[settings.json](settings.json)
+> [!NOTE]
+> 训练以及推理相关配置统一在文件[settings.json](settings.json)
+

 ### 数据准备

@@ -86,6 +71,7 @@ export USE_MODELSCOPE_HUB=1 # Windows 使用 `set USE_MODELSCOPE_HUB=1`
 git lfs install
 git clone https://www.modelscope.cn/ZhipuAI/chatglm3-6b.git
 ```
+魔搭社区的`modeling_chatglm.py`文件需要更换为Hugging Face的

 ### 配置参数并微调模型

@@ -1,9 +1,16 @@
 # WeClone-audio 模块

-WeClone-audio 是一个使用微信语音消息克隆声音的模块，使用 Llasa 模型实现高质量语音合成。
+WeClone-audio 是一个使用微信语音消息克隆声音的模块，使用模型实现高质量语音合成。
 ### 显存需求
+**Spark-TTS** 推荐
+- **0.5B 模型**: 约 4GB 显存
+
+**Llasa**
 - **3B 模型**: 约 16GB 显存
- **1B 模型**: 约 9GB 显存
+- **1B 模型**: 约 9GB 显存  
+
+
+

 ## 1. 导出微信语音数据

@@ -14,16 +21,14 @@ WeClone-audio 是一个使用微信语音消息克隆声音的模块，使用 Ll

 ### 1.2 环境配置
 语音导出仅支持Windows环境
-
-WeClone Audio使用uv作为包管理器，暂时独立于WeClone项目。请确保已安装uv。 
+WeClone Audio使用uv作为包管理器。 
 ```bash
 # 为 PyWxDump 创建 Python 环境和安装依赖
 # 
-cd ./WeClone-audio
 uv venv .venv-wx --python=3.9
 source .venv-wx/bin/activate
 # 安装 wx 依赖组
-uv pip install -e '.[wx]'
+uv pip install --group wx -e .
 ```

 ### 1.3 导出语音文件
@@ -32,14 +37,66 @@ python ./WeClone-audio/get_sample_audio.py --db-path "导出数据库路径" --M
 ```

 ## 2. 语音合成推理
+### Spark-TTS模型

+**环境安装**
+可不创建新环境，直接安装依赖组到WeClone共主环境
+
+```bash
+uv venv .venv-sparktts --python=3.9
+source .venv-sparktts/bin/activate
+uv pip install --group sparktts -e .
+
+cd WeClone-audio/src
+git clone https://github.com/SparkAudio/Spark-TTS.git
+```
+
+**模型下载**
+
+通过python下载:
+```python
+from huggingface_hub import snapshot_download
+
+snapshot_download("SparkAudio/Spark-TTS-0.5B", local_dir="pretrained_models/Spark-TTS-0.5B")
+```
+
+或通过git下载:
+```sh
+cd WeClone-audio
+mkdir -p pretrained_models
+
+# Make sure you have git-lfs installed (https://git-lfs.com)
+git lfs install
+git clone https://huggingface.co/SparkAudio/Spark-TTS-0.5B pretrained_models/Spark-TTS-0.5B
+```
+使用代码推理
+```python
+import os
+import SparkTTS
+import soundfile as sf
+import torch
+
+from SparkTTS import SparkTTS
+
+model = SparkTTS("WeClone-audio/pretrained_models/Spark-TTS-0.5B", "cuda")
+
+
+with torch.no_grad():
+    wav = model.inference(
+        text="晚上好啊,小可爱们，该睡觉了哦",
+        prompt_speech_path=os.path.join(os.path.dirname(__file__), "sample.wav"),
+        prompt_text="对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。",
+    )
+    sf.write(os.path.join(os.path.dirname(__file__), "output.wav"), wav, samplerate=16000)
+```
+### Llasa模型
 ### 2.1 环境配置
 ```bash
-# 创建并配置推理环境
+# 创建并配置推理环境 
+## 可不创建新环境，与LLaMA-Factory环境共用
 uv venv .venv-xcodec --python=3.9
 source .venv-xcodec/bin/activate
-uv pip install -e '.[xcodec]'
-
+uv pip install --group xcodec -e .
 # 退出环境
 deactivate

@@ -1,30 +0,0 @@
-[project]
-name = "WeClone-audio"
-version = "0.1.0"
-description = ""
-authors = [
-    {name = "xming521"}
-]
-readme = "README.md"
-requires-python = ">=3.9,<3.10"
-dependencies = []
-
-[project.optional-dependencies]
-xcodec = ["xcodec2==0.1.3"]
-wx = ["pywxdump"]
-
-[dependency-groups]
-xcodec = ["xcodec2==0.1.3"]
-wx = ["pywxdump"]
-
-[tool.uv]
-conflicts = [
-    [
-      { group = "xcodec" },
-      { group = "wx" },
-    ],
-]
-
-[[tool.uv.index]]
-url = "https://pypi.tuna.tsinghua.edu.cn/simple/"
-default = true
@@ -0,0 +1,12 @@
+import os
+import soundfile as sf
+from text_to_speech import TextToSpeech
+
+
+sample_audio_text = "对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。"  # 示例音频文本
+sample_audio_path = os.path.join(os.path.dirname(__file__), "sample.wav")  # 示例音频路径
+tts = TextToSpeech(sample_audio_path, sample_audio_text)
+target_text = "晚上好啊"  # 生成目标文本
+result = tts.infer(target_text)
+sf.write(os.path.join(os.path.dirname(__file__), "output.wav"), result[1], result[0])  # 保存生成音频
+
@@ -0,0 +1,131 @@
+import os
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+import soundfile as sf
+from xcodec2.modeling_xcodec2 import XCodec2Model
+import torchaudio
+
+
+class TextToSpeech:
+    def __init__(self, sample_audio_path, sample_audio_text):
+        self.sample_audio_text = sample_audio_text
+        # 初始化模型
+        llasa_3b = "HKUSTAudio/Llasa-3B"
+        xcodec2 = "HKUSTAudio/xcodec2"
+
+        self.tokenizer = AutoTokenizer.from_pretrained(llasa_3b)
+        self.llasa_3b_model = AutoModelForCausalLM.from_pretrained(
+            llasa_3b,
+            trust_remote_code=True,
+            device_map="auto",
+        )
+        self.llasa_3b_model.eval()
+
+        self.xcodec_model = XCodec2Model.from_pretrained(xcodec2)
+        self.xcodec_model.eval().cuda()
+
+        # 处理音频
+        waveform, sample_rate = torchaudio.load(sample_audio_path)
+        if len(waveform[0]) / sample_rate > 15:
+            print("已将音频裁剪至前15秒。")
+            waveform = waveform[:, : sample_rate * 15]
+
+        # 检查音频是否为立体声
+        if waveform.size(0) > 1:
+            waveform_mono = torch.mean(waveform, dim=0, keepdim=True)
+        else:
+            waveform_mono = waveform
+
+        self.prompt_wav = torchaudio.transforms.Resample(
+            orig_freq=sample_rate, new_freq=16000
+        )(waveform_mono)
+
+        # Encode the prompt wav
+        vq_code_prompt = self.xcodec_model.encode_code(input_waveform=self.prompt_wav)
+        vq_code_prompt = vq_code_prompt[0, 0, :]
+        self.speech_ids_prefix = self.ids_to_speech_tokens(vq_code_prompt)
+        self.speech_end_id = self.tokenizer.convert_tokens_to_ids("<|SPEECH_GENERATION_END|>")
+
+    def ids_to_speech_tokens(self, speech_ids):
+        speech_tokens_str = []
+        for speech_id in speech_ids:
+            speech_tokens_str.append(f"<|s_{speech_id}|>")
+        return speech_tokens_str
+
+    def extract_speech_ids(self, speech_tokens_str):
+        speech_ids = []
+        for token_str in speech_tokens_str:
+            if token_str.startswith("<|s_") and token_str.endswith("|>"):
+                num_str = token_str[4:-2]
+                num = int(num_str)
+                speech_ids.append(num)
+            else:
+                print(f"Unexpected token: {token_str}")
+        return speech_ids
+
+    @torch.inference_mode()
+    def infer(self, target_text):
+        if len(target_text) == 0:
+            return None
+        elif len(target_text) > 300:
+            print("文本过长，请保持在300字符以内。")
+            target_text = target_text[:300]
+
+        input_text = self.sample_audio_text + " " + target_text
+
+        formatted_text = (
+            f"<|TEXT_UNDERSTANDING_START|>{input_text}<|TEXT_UNDERSTANDING_END|>"
+        )
+
+        chat = [
+            {
+                "role": "user",
+                "content": "Convert the text to speech:" + formatted_text,
+            },
+            {
+                "role": "assistant",
+                "content": "<|SPEECH_GENERATION_START|>"
+                + "".join(self.speech_ids_prefix),
+            },
+        ]
+
+        input_ids = self.tokenizer.apply_chat_template(
+            chat, tokenize=True, return_tensors="pt", continue_final_message=True
+        )
+        input_ids = input_ids.to("cuda")
+
+        outputs = self.llasa_3b_model.generate(
+            input_ids,
+            max_length=2048,
+            eos_token_id=self.speech_end_id,
+            do_sample=True,
+            top_p=1,
+            temperature=0.8,
+        )
+        generated_ids = outputs[0][input_ids.shape[1] - len(self.speech_ids_prefix): -1]
+
+        speech_tokens = self.tokenizer.batch_decode(
+            generated_ids, skip_special_tokens=True
+        )
+
+        speech_tokens = self.extract_speech_ids(speech_tokens)
+        speech_tokens = torch.tensor(speech_tokens).cuda().unsqueeze(0).unsqueeze(0)
+
+        gen_wav = self.xcodec_model.decode_code(speech_tokens)
+        gen_wav = gen_wav[:, :, self.prompt_wav.shape[1]:]
+
+        return (16000, gen_wav[0, 0, :].cpu().numpy())
+
+
+if __name__ == "__main__":
+    # 如果遇到问题，请尝试将参考音频转换为WAV或MP3格式，将其裁剪至15秒以内，并缩短提示文本。
+    sample_audio_text = "对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。"
+    sample_audio_path = os.path.join(os.path.dirname(__file__), "sample.wav")
+
+    tts = TextToSpeech(sample_audio_path, sample_audio_text)
+    target_text = "晚上好啊，吃了吗您"
+    result = tts.infer(target_text)
+    sf.write(os.path.join(os.path.dirname(__file__), "output.wav"), result[1], result[0])
+    target_text = "我是老北京正黄旗！"
+    result = tts.infer(target_text)
+    sf.write(os.path.join(os.path.dirname(__file__), "output1.wav"), result[1], result[0])
@@ -0,0 +1,223 @@
+import re
+import torch
+from typing import Tuple
+from pathlib import Path
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import os
+import sys
+sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "./Spark-TTS")))
+from sparktts.utils.file import load_config
+from sparktts.models.audio_tokenizer import BiCodecTokenizer
+from sparktts.utils.token_parser import LEVELS_MAP, GENDER_MAP, TASK_TOKEN_MAP
+
+
+class SparkTTS:
+    """
+    Spark-TTS for text-to-speech generation.
+    """
+
+    def __init__(self, model_dir: Path, device: torch.device = torch.device("cuda:0")):
+        """
+        Initializes the SparkTTS model with the provided configurations and device.
+
+        Args:
+            model_dir (Path): Directory containing the model and config files.
+            device (torch.device): The device (CPU/GPU) to run the model on.
+        """
+        self.device = device
+        self.model_dir = model_dir
+        self.configs = load_config(f"{model_dir}/config.yaml")
+        self.sample_rate = self.configs["sample_rate"]
+        self._initialize_inference()
+
+    def _initialize_inference(self):
+        """Initializes the tokenizer, model, and audio tokenizer for inference."""
+        self.tokenizer = AutoTokenizer.from_pretrained(f"{self.model_dir}/LLM")
+        self.model = AutoModelForCausalLM.from_pretrained(f"{self.model_dir}/LLM")
+        self.audio_tokenizer = BiCodecTokenizer(self.model_dir, device=self.device)
+        self.model.to(self.device)
+
+    def process_prompt(
+        self,
+        text: str,
+        prompt_speech_path: Path,
+        prompt_text: str = None,
+    ) -> Tuple[str, torch.Tensor]:
+        """
+        Process input for voice cloning.
+
+        Args:
+            text (str): The text input to be converted to speech.
+            prompt_speech_path (Path): Path to the audio file used as a prompt.
+            prompt_text (str, optional): Transcript of the prompt audio.
+
+        Return:
+            Tuple[str, torch.Tensor]: Input prompt; global tokens
+        """
+
+        global_token_ids, semantic_token_ids = self.audio_tokenizer.tokenize(
+            prompt_speech_path
+        )
+        global_tokens = "".join(
+            [f"<|bicodec_global_{i}|>" for i in global_token_ids.squeeze()]
+        )
+
+        # Prepare the input tokens for the model
+        if prompt_text is not None:
+            semantic_tokens = "".join(
+                [f"<|bicodec_semantic_{i}|>" for i in semantic_token_ids.squeeze()]
+            )
+            inputs = [
+                TASK_TOKEN_MAP["tts"],
+                "<|start_content|>",
+                prompt_text,
+                text,
+                "<|end_content|>",
+                "<|start_global_token|>",
+                global_tokens,
+                "<|end_global_token|>",
+                "<|start_semantic_token|>",
+                semantic_tokens,
+            ]
+        else:
+            inputs = [
+                TASK_TOKEN_MAP["tts"],
+                "<|start_content|>",
+                text,
+                "<|end_content|>",
+                "<|start_global_token|>",
+                global_tokens,
+                "<|end_global_token|>",
+            ]
+
+        inputs = "".join(inputs)
+
+        return inputs, global_token_ids
+
+    def process_prompt_control(
+        self,
+        gender: str,
+        pitch: str,
+        speed: str,
+        text: str,
+    ):
+        """
+        Process input for voice creation.
+
+        Args:
+            gender (str): female | male.
+            pitch (str): very_low | low | moderate | high | very_high
+            speed (str): very_low | low | moderate | high | very_high
+            text (str): The text input to be converted to speech.
+
+        Return:
+            str: Input prompt
+        """
+        assert gender in GENDER_MAP.keys()
+        assert pitch in LEVELS_MAP.keys()
+        assert speed in LEVELS_MAP.keys()
+
+        gender_id = GENDER_MAP[gender]
+        pitch_level_id = LEVELS_MAP[pitch]
+        speed_level_id = LEVELS_MAP[speed]
+
+        pitch_label_tokens = f"<|pitch_label_{pitch_level_id}|>"
+        speed_label_tokens = f"<|speed_label_{speed_level_id}|>"
+        gender_tokens = f"<|gender_{gender_id}|>"
+
+        attribte_tokens = "".join(
+            [gender_tokens, pitch_label_tokens, speed_label_tokens]
+        )
+
+        control_tts_inputs = [
+            TASK_TOKEN_MAP["controllable_tts"],
+            "<|start_content|>",
+            text,
+            "<|end_content|>",
+            "<|start_style_label|>",
+            attribte_tokens,
+            "<|end_style_label|>",
+        ]
+
+        return "".join(control_tts_inputs)
+
+    @torch.no_grad()
+    def inference(
+        self,
+        text: str,
+        prompt_speech_path: Path = None,
+        prompt_text: str = None,
+        gender: str = None,
+        pitch: str = None,
+        speed: str = None,
+        temperature: float = 0.8,
+        top_k: float = 50,
+        top_p: float = 0.95,
+    ) -> torch.Tensor:
+        """
+        Performs inference to generate speech from text, incorporating prompt audio and/or text.
+
+        Args:
+            text (str): The text input to be converted to speech.
+            prompt_speech_path (Path): Path to the audio file used as a prompt.
+            prompt_text (str, optional): Transcript of the prompt audio.
+            gender (str): female | male.
+            pitch (str): very_low | low | moderate | high | very_high
+            speed (str): very_low | low | moderate | high | very_high
+            temperature (float, optional): Sampling temperature for controlling randomness. Default is 0.8.
+            top_k (float, optional): Top-k sampling parameter. Default is 50.
+            top_p (float, optional): Top-p (nucleus) sampling parameter. Default is 0.95.
+
+        Returns:
+            torch.Tensor: Generated waveform as a tensor.
+        """
+        if gender is not None:
+            prompt = self.process_prompt_control(gender, pitch, speed, text)
+
+        else:
+            prompt, global_token_ids = self.process_prompt(
+                text, prompt_speech_path, prompt_text
+            )
+        model_inputs = self.tokenizer([prompt], return_tensors="pt").to(self.device)
+
+        # Generate speech using the model
+        generated_ids = self.model.generate(
+            **model_inputs,
+            max_new_tokens=3000,
+            do_sample=True,
+            top_k=top_k,
+            top_p=top_p,
+            temperature=temperature,
+        )
+
+        # Trim the output tokens to remove the input tokens
+        generated_ids = [
+            output_ids[len(input_ids):]
+            for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+        ]
+
+        # Decode the generated tokens into text
+        predicts = self.tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+
+        # Extract semantic token IDs from the generated text
+        pred_semantic_ids = (
+            torch.tensor([int(token) for token in re.findall(r"bicodec_semantic_(\d+)", predicts)])
+            .long()
+            .unsqueeze(0)
+        )
+
+        if gender is not None:
+            global_token_ids = (
+                torch.tensor([int(token) for token in re.findall(r"bicodec_global_(\d+)", predicts)])
+                .long()
+                .unsqueeze(0)
+                .unsqueeze(0)
+            )
+
+        # Convert semantic tokens back to waveform
+        wav = self.audio_tokenizer.detokenize(
+            global_token_ids.to(self.device).squeeze(0),
+            pred_semantic_ids.to(self.device),
+        )
+
+        return wav
@@ -1,11 +1,18 @@
 import os
+import SparkTTS
 import soundfile as sf
-from text_to_speech import TextToSpeech
+import torch
+
+from SparkTTS import SparkTTS
+
+model = SparkTTS("WeClone-audio/pretrained_models/Spark-TTS-0.5B", "cuda")


-sample_audio_text = "对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。"  # 示例音频文本
-sample_audio_path = os.path.join(os.path.dirname(__file__), "sample.wav")  # 示例音频路径
-tts = TextToSpeech(sample_audio_path, sample_audio_text)
-target_text = "晚上好啊"  # 生成目标文本
-result = tts.infer(target_text)
-sf.write(os.path.join(os.path.dirname(__file__), "output.wav"), result[1], result[0])  # 保存生成音频
+with torch.no_grad():
+    wav = model.inference(
+        text="晚上好啊,小可爱们，该睡觉了哦",
+        prompt_speech_path=os.path.join(os.path.dirname(__file__), "sample.wav"),
+        prompt_text="对，这就是我万人敬仰的太乙真人，虽然有点婴儿肥，但也掩不住我逼人的帅气。",
+    )
+    sf.write(os.path.join(os.path.dirname(__file__), "output.wav"), wav, samplerate=16000)
+    print("生成成功！")
@@ -0,0 +1,65 @@
+[project]
+name = "WeClone"
+version = "0.1.2"
+description = ""
+authors = [{ name = "xming521" }]
+readme = "README.md"
+requires-python = ">=3.9,<3.10"
+dependencies = ["pandas", "pydantic==2.10.6"]
+
+# [project.optional-dependencies]
+# xcodec = ["xcodec2==0.1.3"]
+# wx = ["pywxdump"]
+# sparktts = [
+#     "einops>=0.8.1",
+#     "einx>=0.3.0",
+#     "numpy==1.26.4",
+#     "omegaconf>=2.3.0",
+#     "packaging>=24.2",
+#     "safetensors>=0.5.2",
+#     "soundfile>=0.12.1",
+#     "soxr>=0.5.0.post1",
+#     "torch>=2.5.1",
+#     "torchaudio>=2.5.1",
+#     "tqdm>=4.66.5",
+#     "transformers==4.45.2"
+# ]
+
+[dependency-groups]
+xcodec = ["xcodec2==0.1.3"]
+wx = ["pywxdump"]
+sparktts = [
+    "einops>=0.8.1",
+    "einx>=0.3.0",
+    "numpy==1.26.4",
+    "omegaconf>=2.3.0",
+    "packaging>=24.2",
+    "safetensors>=0.5.2",
+    "soundfile>=0.12.1",
+    "soxr>=0.5.0.post1",
+    "torch>=2.5.1",
+    "torchaudio>=2.5.1",
+    "tqdm>=4.66.5",
+    "transformers==4.45.2",
+]
+main = ["transformers==4.45.2", "llamafactory>=0.9.2", "openai==0.28.0"]
+
+[tool.uv]
+conflicts = [
+    [
+        { group = "wx" },
+        { group = "sparktts" },
+    ],
+    [
+        { group = "wx" },
+        { group = "main" },
+    ],
+    [
+        { group = "wx" },
+        { group = "xcodec" },
+    ],
+]
+
+[[tool.uv.index]]
+url = "https://pypi.tuna.tsinghua.edu.cn/simple/"
+default = true
@@ -1,5 +1,5 @@
 # LLaMA-Factory
-llmtuner==0.5.3
+llmtuner
 # wechat
 itchat-uos==1.5.0.dev0
 # others
@@ -46,7 +46,8 @@
        "model_name_or_path": "./chatglm3-6b",
        "adapter_name_or_path": "./model_output",
        "template": "chatglm3-weclone",
-        "finetuning_type": "lora"
+        "finetuning_type": "lora",
+        "trust_remote_code": true
    },
    "_comment": "adapter_name_or_path同时做为train_sft_args的output_dir "
 }
@@ -1,6 +1,7 @@
 import os
 import uvicorn
-from llmtuner import ChatModel, create_app
+from llamafactory.chat import ChatModel
+from llamafactory.api.app import create_app
 from template import template_register
 from utils.config import load_config

@@ -1,5 +1,5 @@
-from llmtuner import ChatModel
-from llmtuner.extras.misc import torch_gc
+from llamafactory.chat import ChatModel
+from llamafactory.extras.misc import torch_gc


 try:
@@ -1,4 +1,4 @@
-from llmtuner import Evaluator
+from llamafactory.eval.evaluator import Evaluator


 def main():
@@ -1,4 +1,4 @@
-from llmtuner import export_model
+from llamafactory.train.tuner import export_model


 def main():
@@ -1,21 +1,24 @@
-from llmtuner.data.formatter import FunctionFormatter, StringFormatter
-from llmtuner.data.template import _register_template
+from llamafactory.data.formatter import FunctionFormatter, StringFormatter, ToolFormatter, EmptyFormatter
+from llamafactory.data.template import register_template

 default_prompt = "请你扮演一名人类，不要说自己是人工智能"


 def template_register():
-    _register_template(
+    register_template(
        name="chatglm3-weclone",
        default_system=(
            default_prompt
        ),
        format_user=StringFormatter(slots=[{"token": "<|user|>"}, "\n", "{{content}}", {"token": "<|assistant|>"}]),
        format_assistant=StringFormatter(slots=["\n", "{{content}}"]),
-        format_system=StringFormatter(slots=[{"token": "[gMASK]"}, {"token": "sop"}, {"token": "<|system|>"}, "\n", "{{content}}"]),
-        format_function=FunctionFormatter(slots=["{{name}}\n{{arguments}}"]),
-        format_observation=StringFormatter(slots=[{"token": "<|observation|>"}, "\n", "{{content}}"]),
+        format_system=StringFormatter(slots=[{"token": "<|system|>"}, "\n", "{{content}}"]),
+        format_function=FunctionFormatter(slots=["{{content}}"], tool_format="glm4"),
+        format_observation=StringFormatter(
+            slots=[{"token": "<|observation|>"}, "\n", "{{content}}", {"token": "<|assistant|>"}]
+        ),
+        format_tools=ToolFormatter(tool_format="glm4"),
+        format_prefix=EmptyFormatter(slots=[{"token": "[gMASK]"}, {"token": "sop"}]),
        stop_words=["<|user|>", "<|observation|>"],
        efficient_eos=True,
-        force_system=True
    )
@@ -1,4 +1,4 @@
-from llmtuner import run_exp
+from llamafactory.train.tuner import run_exp
 from utils.config import load_config

 config = load_config('train_pt')
@@ -1,4 +1,4 @@
-from llmtuner import run_exp
+from llamafactory.train.tuner import run_exp
 from template import template_register
 from utils.config import load_config

@@ -1,4 +1,4 @@
-from llmtuner import create_web_demo
+from llamafactory.webui.interface import create_web_demo
 from template import template_register
 from utils.config import load_config