diff --git a/TERM_SCROLLBACK_TOOL.md b/TERM_SCROLLBACK_TOOL.md deleted file mode 100644 index b6b1c8ace..000000000 --- a/TERM_SCROLLBACK_TOOL.md +++ /dev/null @@ -1,218 +0,0 @@ -# Terminal Scrollback Tool 实现文档 - -## 概述 - -`term_get_scrollback` 工具允许 AI 读取终端的输出历史，而不是直接执行命令。这是一个**只读工具**，完全避免了 AI 重复执行命令的问题。 - -## 架构设计 - -### 后端（Rust） - -1. **TermScrollbackTool** (`src-tauri/src/agent/tools/term_scrollback.rs`) - - 实现 `Tool` trait - - 使用响应通道机制（类似 TerminalTool） - - 通过 Tauri 事件系统与前端通信 - -2. **全局实例管理** - - 使用 `once_cell::sync::Lazy` 创建全局单例 - - 在 `setup.rs` 和 `runner.rs` 中初始化 AppHandle - - 提供 `get_term_scrollback_tool()` 获取全局实例 - -3. **Tauri 命令** - - `agent_term_scrollback_response`: 接收前端的响应 - -### 前端（TypeScript/React） - -1. **API 层** (`src/lib/api/agent.ts`) - - `TermScrollbackRequest`: 请求类型 - - `TermScrollbackResponse`: 响应类型 - - `sendTermScrollbackResponse()`: 发送响应函数 - -2. **Hook 层** (`src/components/terminal/ai/useTerminalAI.ts`) - - 监听 `term_get_scrollback_request` 事件 - - 读取终端输出历史 - - 发送响应给后端 - -## 工作流程 - -``` -AI Agent - ↓ (调用 term_get_scrollback 工具) -TermScrollbackTool - ↓ (发送 term_get_scrollback_request 事件) -前端 useTerminalAI - ↓ (读取终端输出) - ↓ (调用 sendTermScrollbackResponse) -Tauri 命令 agent_term_scrollback_response - ↓ (调用 handle_term_scrollback_response) -TermScrollbackTool - ↓ (通过响应通道返回结果) -AI Agent (收到终端输出) -``` - -## 工具参数 - -```json -{ - "session_id": "终端会话 ID", - "line_start": 0, // 可选，起始行号（从 0 开始） - "count": 50 // 可选，读取行数 -} -``` - -## 工具描述（给 AI 的说明） - -``` -Read terminal output history without executing commands. - -This tool allows you to view the terminal's scrollback buffer (output history) -without executing any commands. Use this to: -- Check the results of previously executed commands -- Review terminal output before suggesting next steps -- Understand the current state of the terminal session - -Parameters: -- session_id: Terminal session ID (required) -- line_start: Starting line number (optional, default: 0) -- count: Number of lines to read (optional, default: all lines) - -Returns the terminal output as plain text. - -IMPORTANT: This is a READ-ONLY tool. It does NOT execute commands. -``` - -## 使用场景 - -### 场景 1：纯只读模式（推荐） - -**配置：** -- 移除 `terminal` 工具 -- 只保留 `term_get_scrollback` 工具 - -**优势：** -- AI 只能读取输出，不能执行命令 -- 完全避免重复执行问题 -- 用户完全控制命令执行 - -**工作流程：** -1. 用户手动在终端执行命令 -2. AI 使用 `term_get_scrollback` 读取输出 -3. AI 根据输出提供建议 -4. 用户决定是否执行 AI 的建议 - -### 场景 2：混合模式（当前实现） - -**配置：** -- 保留 `terminal` 工具（需要审批） -- 添加 `term_get_scrollback` 工具 - -**优势：** -- AI 可以建议命令（需要审批） -- AI 也可以读取历史输出 -- 灵活性更高 - -**工作流程：** -1. AI 建议命令（通过 `terminal` 工具） -2. 用户审批并执行 -3. AI 使用 `term_get_scrollback` 读取输出 -4. AI 根据输出继续工作 - -## 测试步骤 - -### 1. 编译项目 - -```bash -cargo build --manifest-path src-tauri/Cargo.toml -``` - -### 2. 启动应用 - -```bash -npm run tauri dev -``` - -### 3. 测试工具 - -1. 打开终端 -2. 执行一些命令（例如：`ls`, `pwd`, `echo hello`） -3. 打开 AI 面板 -4. 发送消息：`请读取终端的输出历史` -5. AI 应该使用 `term_get_scrollback` 工具读取输出 -6. 检查 AI 是否正确显示了终端输出 - -### 4. 验证日志 - -**后端日志：** -``` -[TermScrollbackTool] 创建全局实例 -[TermScrollbackTool] 设置全局 AppHandle -[TermScrollbackTool] AppHandle 设置成功，已验证 -[TermScrollbackTool] 请求获取滚动缓冲区: session_id=xxx, request_id=xxx -[TermScrollbackTool] 已发送请求到前端: xxx -[TermScrollbackTool] 收到响应: request_id=xxx, success=true -``` - -**前端日志：** -``` -[useTerminalAI] 收到终端滚动缓冲区请求: {...} -[useTerminalAI] 已发送滚动缓冲区响应: 0-50/100 行 -``` - -## 故障排查 - -### 问题 1：AppHandle 未设置 - -**症状：** -``` -[TermScrollbackTool] 警告：AppHandle 未设置！工具将无法正常工作 -``` - -**解决方案：** -- 检查 `setup.rs` 和 `runner.rs` 中是否调用了 `set_term_scrollback_tool_app_handle()` -- 确保在工具注册之前设置 AppHandle - -### 问题 2：前端未收到事件 - -**症状：** -- 后端发送了事件，但前端没有日志 - -**解决方案：** -- 检查 `useTerminalAI` 是否正确监听 `term_get_scrollback_request` 事件 -- 确保 `terminalSessionId` 不为空 -- 检查浏览器控制台是否有错误 - -### 问题 3：响应超时 - -**症状：** -``` -[TermScrollbackTool] 请求超时: xxx -``` - -**解决方案：** -- 检查前端是否正确发送响应 -- 增加超时时间：`TermScrollbackTool::new().with_timeout(60)` -- 检查 `getTerminalOutput()` 函数是否正常工作 - -## 下一步改进 - -1. **添加过滤功能** - - 支持正则表达式过滤 - - 支持关键词搜索 - -2. **添加格式化选项** - - 支持 ANSI 颜色代码 - - 支持纯文本输出 - -3. **添加缓存机制** - - 缓存最近的输出 - - 减少重复读取 - -4. **添加增量读取** - - 只读取新增的输出 - - 支持实时监控 - -## 参考资料 - -- [Waveterm Terminal AI](https://github.com/wavetermdev/waveterm) -- [Tauri Event System](https://tauri.app/v1/guides/features/events/) -- [Tokio Oneshot Channel](https://docs.rs/tokio/latest/tokio/sync/oneshot/) diff --git a/artifacts/ref_model_selector.png b/artifacts/ref_model_selector.png deleted file mode 100644 index 593e216d2..000000000 Binary files a/artifacts/ref_model_selector.png and /dev/null differ diff --git a/aster-rust/crates/aster-cli/static/img/logo_dark.png b/aster-rust/crates/aster-cli/static/img/logo_dark.png new file mode 100644 index 000000000..7b64fd2ca Binary files /dev/null and b/aster-rust/crates/aster-cli/static/img/logo_dark.png differ diff --git a/aster-rust/crates/aster-cli/static/img/logo_light.png b/aster-rust/crates/aster-cli/static/img/logo_light.png new file mode 100644 index 000000000..b4e3c3d8b Binary files /dev/null and b/aster-rust/crates/aster-cli/static/img/logo_light.png differ diff --git a/package.json b/package.json index 0921af82c..91d627af1 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "proxycast", "private": true, - "version": "0.42.0", + "version": "0.43.0", "type": "module", "repository": { "type": "git", @@ -45,6 +45,7 @@ "@tauri-apps/plugin-dialog": "^2.4.2", "@tauri-apps/plugin-global-shortcut": "^2", "@tauri-apps/plugin-shell": "^2.0.0", + "@tonejs/midi": "^2.0.28", "@types/lodash-es": "^4.17.12", "@types/styled-components": "^5.1.36", "@xterm/addon-fit": "^0.11.0", @@ -75,7 +76,8 @@ "remark-math": "^6.0.0", "sonner": "^2.0.7", "styled-components": "^6.1.19", - "tailwind-merge": "^2.6.0" + "tailwind-merge": "^2.6.0", + "tone": "^15.1.22" }, "devDependencies": { "@babel/plugin-transform-react-jsx-source": "^7.27.1", diff --git a/src-tauri/resources/music/chord-progressions.json b/src-tauri/resources/music/chord-progressions.json new file mode 100644 index 000000000..a6d98cbc6 --- /dev/null +++ b/src-tauri/resources/music/chord-progressions.json @@ -0,0 +1,251 @@ +{ + "meta": { + "version": "1.0", + "description": "和弦进行数据库 - 覆盖多种音乐风格的和弦进行模式", + "created_for": "Musicify Music Theory Skill" + }, + + "basic_progressions": { + "pop_progressions": [ + { + "name": "流行四和弦", + "pattern": "vi-IV-I-V", + "chords": ["Am", "F", "C", "G"], + "roman_numeral": ["vi", "IV", "I", "V"], + "emotion": "感人、朗朗上口", + "difficulty": 1, + "usage": "最常用的流行歌曲进行，适合主歌和副歌", + "examples": ["《Someone Like You》", "《Let It Be》"], + "variations": [ + {"pattern": "vi-V-IV-I", "description": "更加流畅的解决"}, + {"pattern": "vi-IV-I-V-vi", "description": "添加循环"} + ] + }, + { + "name": "卡农进行", + "pattern": "I-V-vi-IV", + "chords": ["C", "G", "Am", "F"], + "roman_numeral": ["I", "V", "vi", "IV"], + "emotion": "优美、经典、温暖", + "difficulty": 1, + "usage": "经典抒情歌曲，主歌部分特别适用", + "examples": ["《Canon in D》", "《Five Hundred Miles》"], + "variations": [ + {"pattern": "I-V-vi-iii-IV", "description": "加入三级和弦增加色彩"}, + {"pattern": "I-V-vi-IV-V", "description": "强调属功能"} + ] + }, + { + "name": "循环进行", + "pattern": "I-V-vi-iii-IV-I-IV-V", + "chords": ["C", "G", "Am", "Em", "F", "C", "F", "G"], + "roman_numeral": ["I", "V", "vi", "iii", "IV", "I", "IV", "V"], + "emotion": "流动、持续、丰富", + "difficulty": 2, + "usage": "适合较长的歌曲段落，创造持续的动力", + "examples": ["《Autumn Leaves》", "《Fly Me to the Moon》"] + } + ], + + "rock_progressions": [ + { + "name": "力量三和弦", + "pattern": "I-♭VII-IV", + "chords": ["C", "Bb", "F"], + "roman_numeral": ["I", "♭VII", "IV"], + "emotion": "有力、激昂、叛逆", + "difficulty": 2, + "usage": "摇滚歌曲副歌，营造强烈的推动力", + "examples": ["《Sweet Caroline》", "《Free Fallin'》"], + "guitar_tips": "使用强力和弦(power chords)，强调根音和五度音" + }, + { + "name": "蓝调进行", + "pattern": "I-I-I-I-IV-IV-I-I-V-IV-I-I", + "chords": ["C7", "C7", "C7", "C7", "F7", "F7", "C7", "C7", "G7", "F7", "C7", "C7"], + "roman_numeral": ["I7", "I7", "I7", "I7", "IV7", "IV7", "I7", "I7", "V7", "IV7", "I7", "I7"], + "emotion": "忧郁、深沉、表达性强", + "difficulty": 2, + "usage": "12小节蓝调，摇滚和布鲁斯的基础", + "examples": ["《Johnny B. Goode》", "《Stormy Monday》"] + } + ], + + "jazz_progressions": [ + { + "name": "ii-V-I 进行", + "pattern": "ii7-V7-Imaj7", + "chords": ["Dm7", "G7", "Cmaj7"], + "roman_numeral": ["ii7", "V7", "Imaj7"], + "emotion": "成熟、精致、和谐", + "difficulty": 3, + "usage": "爵士乐最基本的进行，用于歌曲的解决", + "examples": ["《All The Things You Are》", "《Autumn Leaves》"], + "extensions": [ + {"pattern": "ii7(b5)-V7alt-i7", "description": "小调ii-V-i"}, + {"pattern": "IImaj7-V7-Imaj7", "description": "大二级替代"} + ] + }, + { + "name": "Circle of Fifths", + "pattern": "I-vi-ii-V", + "chords": ["Cmaj7", "Am7", "Dm7", "G7"], + "roman_numeral": ["Imaj7", "vi7", "ii7", "V7"], + "emotion": "流动、自然、渐进", + "difficulty": 3, + "usage": "爵士标准曲常用，创造平滑的和声运动", + "examples": ["《I Got Rhythm》", "《All of Me》"] + } + ], + + "chinese_style": [ + { + "name": "五声音阶进行", + "pattern": "I-III-vi-IV", + "chords": ["C", "E", "Am", "F"], + "roman_numeral": ["I", "III", "vi", "IV"], + "emotion": "古典、东方韵味、祥和", + "difficulty": 2, + "usage": "中国风歌曲，古风音乐", + "examples": ["《青花瓷》", "《菊花台》"], + "scales": ["C-D-E-G-A (宫商角徵羽)"], + "instruments": ["古筝", "二胡", "笛子", "琵琶"] + }, + { + "name": "宫调式进行", + "pattern": "I-V-vi-IV-ii-V-I", + "chords": ["C", "G", "Am", "F", "Dm", "G", "C"], + "roman_numeral": ["I", "V", "vi", "IV", "ii", "V", "I"], + "emotion": "庄重、典雅、传统", + "difficulty": 2, + "usage": "古风歌曲的主歌部分，营造古典氛围", + "traditional_harmony": "以宫音为主，强调五度圈运动" + } + ] + }, + + "advanced_techniques": { + "chord_substitutions": [ + { + "name": "三全音替代", + "original": "V7", + "substitute": "♭II7", + "example": {"original": "G7", "substitute": "D♭7"}, + "effect": "增加色彩和张力", + "usage": "爵士乐中常用，特别在ii-V-I进行中" + }, + { + "name": "相对和弦替代", + "original": "I", + "substitute": "vi", + "example": {"original": "C", "substitute": "Am"}, + "effect": "从大调转向小调色彩", + "usage": "营造忧郁或内省的情绪" + }, + { + "name": "二级和弦", + "technique": "目标和弦前加入其二级和弦", + "example": "C-Dm-G7-C (在G7前加入Dm)", + "effect": "增强和声运动感", + "usage": "增加进行的丰富性" + } + ], + + "modal_progressions": [ + { + "mode": "Dorian", + "characteristic": "自然六度，♭七度", + "progression": "i-IV-♭VII-i", + "chords": ["Dm", "G", "C", "Dm"], + "emotion": "神秘、中性、民族色彩", + "examples": ["《Scarborough Fair》", "《Eleanor Rigby》"] + }, + { + "mode": "Mixolydian", + "characteristic": "♭七度", + "progression": "I-♭VII-IV-I", + "chords": ["G", "F", "C", "G"], + "emotion": "开朗中带有忧郁", + "examples": ["《Sweet Caroline》", "《Norwegian Wood》"] + }, + { + "mode": "Aeolian (Natural Minor)", + "characteristic": "♭三度，♭六度，♭七度", + "progression": "i-♭VII-♭VI-♭VII", + "chords": ["Am", "G", "F", "G"], + "emotion": "忧郁、深沉、戏剧性", + "examples": ["《House of the Rising Sun》", "《Stairway to Heaven》"] + } + ] + }, + + "rhythm_patterns": { + "basic_strumming": [ + { + "name": "基础4/4拍型", + "pattern": "D-D-U-U-D-U", + "notation": "下-下-上-上-下-上", + "tempo": "适中速度 (120 BPM)", + "usage": "最基础的吉他扫弦模式", + "songs": ["《Hotel California》", "《Wonderwall》"] + }, + { + "name": "民谣分解", + "pattern": "1-3-2-3-1-3-2-3", + "fingers": "拇指-食指-中指-食指", + "usage": "指弹民谣，抒情歌曲", + "songs": ["《Dust in the Wind》", "《Blackbird》"] + } + ], + + "advanced_patterns": [ + { + "name": "放克节奏", + "pattern": "重音在16分音符的切分位置", + "characteristic": "强调反拍，使用切音技巧", + "instruments": ["电吉他", "贝斯", "鼓"], + "examples": ["《Superstition》", "《I Want Your Love》"] + }, + { + "name": "雷鬼节奏", + "pattern": "强调2、4拍的后半拍", + "characteristic": "轻松摇摆，强调上拍", + "tempo": "中慢速度 (70-90 BPM)", + "examples": ["《No Woman No Cry》", "《Three Little Birds》"] + } + ] + }, + + "song_structure_templates": { + "pop_structure": { + "sections": ["Intro", "Verse 1", "Pre-Chorus", "Chorus", "Verse 2", "Pre-Chorus", "Chorus", "Bridge", "Chorus", "Outro"], + "chord_suggestions": { + "Verse": "较为平静的进行，如 vi-IV-I-V", + "Pre-Chorus": "建立张力，如 ii-V 或 IV-V", + "Chorus": "强有力的进行，如 I-V-vi-IV", + "Bridge": "对比性进行，可尝试不同调性" + } + }, + + "ballad_structure": { + "sections": ["Intro", "Verse 1", "Chorus", "Verse 2", "Chorus", "Bridge", "Chorus", "Outro"], + "dynamic": "从安静开始，逐渐建立到高潮", + "chord_suggestions": { + "Verse": "简单温柔的进行", + "Chorus": "情感饱满的和弦" + } + } + }, + + "key_relationships": { + "circle_of_fifths": { + "major_keys": ["C", "G", "D", "A", "E", "B", "F#", "Db", "Ab", "Eb", "Bb", "F"], + "relative_minors": ["Am", "Em", "Bm", "F#m", "C#m", "G#m", "D#m", "Bbm", "Fm", "Cm", "Gm", "Dm"], + "modulation_techniques": [ + "共同和弦转调", + "属和弦转调", + "半音阶下行转调" + ] + } + } +} \ No newline at end of file diff --git a/src-tauri/resources/music/guofeng-patterns.json b/src-tauri/resources/music/guofeng-patterns.json new file mode 100644 index 000000000..6cce684b8 --- /dev/null +++ b/src-tauri/resources/music/guofeng-patterns.json @@ -0,0 +1,266 @@ +{ + "meta": { + "version": "1.0", + "description": "国风旋律模式库 - 定义情绪映射、结构差异和常用旋律模式", + "created_for": "Musicify 国风旋律生成 Skill" + }, + + "emotions": { + "sorrowful": { + "name": "忧伤", + "preferredMode": "yu", + "contour": "descending", + "intervalRange": 3, + "tempo": "慢板 (60-72 BPM)", + "characteristics": [ + "多用下行旋律线", + "羽调式为主，偶用商调式", + "音程以级进为主，偶有小跳", + "多用滑音和颤音装饰", + "句尾常落在羽音(6)或商音(2)" + ], + "typicalPatterns": ["weeping", "sighing"] + }, + "joyful": { + "name": "欢快", + "preferredMode": "gong", + "contour": "ascending", + "intervalRange": 5, + "tempo": "快板 (120-140 BPM)", + "characteristics": [ + "多用上行旋律线", + "宫调式为主，偶用徵调式", + "音程跳进较多，节奏活泼", + "装饰音轻快，多用波音", + "句尾常落在宫音(1)或徵音(5)" + ], + "typicalPatterns": ["celebration", "dance"] + }, + "peaceful": { + "name": "平静", + "preferredMode": "jue", + "contour": "stable", + "intervalRange": 2, + "tempo": "中板 (80-96 BPM)", + "characteristics": [ + "旋律走向平稳", + "角调式为主，音色空灵", + "以级进为主，避免大跳", + "装饰音少而精", + "音域范围较窄" + ], + "typicalPatterns": ["flowing", "meditation"] + }, + "passionate": { + "name": "激昂", + "preferredMode": "zhi", + "contour": "wave", + "intervalRange": 6, + "tempo": "快板 (116-132 BPM)", + "characteristics": [ + "旋律起伏大，波浪式进行", + "徵调式为主，热情奔放", + "音程跳进多，张力强", + "高音区使用频繁", + "句尾常有力度强调" + ], + "typicalPatterns": ["heroic", "climax"] + }, + "nostalgic": { + "name": "思念", + "preferredMode": "shang", + "contour": "wave", + "intervalRange": 4, + "tempo": "慢板 (66-80 BPM)", + "characteristics": [ + "旋律婉转起伏", + "商调式为主，深沉内敛", + "上行后下行，欲言又止", + "多用滑音表达情感", + "句尾常有延长音" + ], + "typicalPatterns": ["longing", "sighing"] + }, + "ethereal": { + "name": "空灵", + "preferredMode": "jue", + "contour": "ascending", + "intervalRange": 5, + "tempo": "自由节拍", + "characteristics": [ + "旋律飘逸，多用高音区", + "角调式为主，清新脱俗", + "音符稀疏，留白较多", + "颤音装饰增加空灵感", + "节奏自由，不拘一格" + ], + "typicalPatterns": ["floating", "meditation"] + } + }, + + "structures": { + "verse": { + "name": "主歌", + "range": 6, + "contour": "stable", + "characteristics": [ + "音域较窄，一般在六度以内", + "旋律平稳，以叙述为主", + "节奏规整，便于歌词表达", + "情感内敛，为副歌做铺垫" + ], + "typicalStartNotes": [5, 3, 1], + "typicalEndNotes": [1, 5, 6] + }, + "chorus": { + "name": "副歌", + "range": 10, + "contour": "wave", + "characteristics": [ + "音域扩展，可达十度", + "旋律起伏大，情感爆发", + "常有高潮点设计", + "节奏可更自由或更强烈" + ], + "typicalStartNotes": [1, 5, 6], + "typicalEndNotes": [1, 5] + }, + "bridge": { + "name": "桥段", + "range": 8, + "contour": "ascending", + "characteristics": [ + "音域介于主歌和副歌之间", + "旋律上行为主，推向高潮", + "可转调或变化调式", + "为副歌再现做准备" + ], + "typicalStartNotes": [6, 3, 2], + "typicalEndNotes": [5, 1] + } + }, + + "patterns": { + "weeping": { + "name": "哭腔模式", + "sequence": [6, 5, 3, 2, 1, 6], + "contour": "descending", + "usage": "表达悲伤、哀怨情绪", + "examples": ["《青花瓷》副歌", "《烟花易冷》"], + "ornaments": ["滑音", "颤音"] + }, + "sighing": { + "name": "叹息模式", + "sequence": [5, 6, 5, 3, 2], + "contour": "wave", + "usage": "表达思念、无奈情绪", + "examples": ["《千里之外》", "《菊花台》"], + "ornaments": ["滑音"] + }, + "celebration": { + "name": "欢庆模式", + "sequence": [1, 3, 5, 6, 5, 3, 1], + "contour": "ascending", + "usage": "表达喜悦、庆祝情绪", + "examples": ["《好日子》", "《恭喜发财》"], + "ornaments": ["波音"] + }, + "dance": { + "name": "舞曲模式", + "sequence": [5, 1, 3, 5, 6, 5], + "contour": "wave", + "usage": "活泼的舞蹈节奏", + "examples": ["《最炫民族风》"], + "ornaments": ["波音", "倚音"] + }, + "flowing": { + "name": "流水模式", + "sequence": [3, 2, 1, 2, 3, 5], + "contour": "stable", + "usage": "平静、流畅的叙述", + "examples": ["《高山流水》", "《渔舟唱晚》"], + "ornaments": ["滑音"] + }, + "meditation": { + "name": "禅意模式", + "sequence": [5, 3, 5, 6, 5], + "contour": "stable", + "usage": "空灵、冥想的意境", + "examples": ["《大悲咒》", "《心经》"], + "ornaments": ["颤音"] + }, + "heroic": { + "name": "英雄模式", + "sequence": [5, 1, 5, 6, 1, 5], + "contour": "ascending", + "usage": "激昂、豪迈的情绪", + "examples": ["《精忠报国》", "《男儿当自强》"], + "ornaments": ["倚音"] + }, + "climax": { + "name": "高潮模式", + "sequence": [1, 3, 5, 6, 1, 6, 5], + "contour": "wave", + "usage": "歌曲高潮部分", + "examples": ["副歌高潮段落"], + "ornaments": ["颤音", "滑音"] + }, + "longing": { + "name": "思念模式", + "sequence": [2, 3, 5, 3, 2, 1, 2], + "contour": "wave", + "usage": "表达思念、期盼情绪", + "examples": ["《但愿人长久》", "《明月几时有》"], + "ornaments": ["滑音", "颤音"] + }, + "floating": { + "name": "飘逸模式", + "sequence": [3, 5, 6, 5, 3], + "contour": "ascending", + "usage": "空灵、超脱的意境", + "examples": ["《沧海一声笑》"], + "ornaments": ["颤音"] + } + }, + + "references": { + "classicSongs": [ + { + "name": "青花瓷", + "mode": "yu", + "emotion": "nostalgic", + "features": "羽调式为主，旋律婉转，多用下行和滑音" + }, + { + "name": "菊花台", + "mode": "yu", + "emotion": "sorrowful", + "features": "羽调式，忧伤婉转，句尾多用延长音" + }, + { + "name": "千里之外", + "mode": "shang", + "emotion": "nostalgic", + "features": "商调式，深沉内敛，旋律起伏适中" + }, + { + "name": "沧海一声笑", + "mode": "zhi", + "emotion": "passionate", + "features": "徵调式，豪迈奔放，音域宽广" + }, + { + "name": "高山流水", + "mode": "gong", + "emotion": "peaceful", + "features": "宫调式，典雅庄重，旋律流畅" + }, + { + "name": "茉莉花", + "mode": "gong", + "emotion": "peaceful", + "features": "宫调式，清新优美，级进为主" + } + ] + } +} diff --git a/src-tauri/resources/music/midi-parser-rules.json b/src-tauri/resources/music/midi-parser-rules.json new file mode 100644 index 000000000..d7d9a5fa2 --- /dev/null +++ b/src-tauri/resources/music/midi-parser-rules.json @@ -0,0 +1,251 @@ +{ + "meta": { + "version": "1.0", + "description": "MIDI 解析规则库 - 定义音符映射、时值解析、节奏型识别和调式推断规则", + "created_for": "Musicify 旋律风格学习 Skill" + }, + + "noteMapping": { + "midiToName": { + "36": "C2", "37": "C#2", "38": "D2", "39": "D#2", "40": "E2", "41": "F2", + "42": "F#2", "43": "G2", "44": "G#2", "45": "A2", "46": "A#2", "47": "B2", + "48": "C3", "49": "C#3", "50": "D3", "51": "D#3", "52": "E3", "53": "F3", + "54": "F#3", "55": "G3", "56": "G#3", "57": "A3", "58": "A#3", "59": "B3", + "60": "C4", "61": "C#4", "62": "D4", "63": "D#4", "64": "E4", "65": "F4", + "66": "F#4", "67": "G4", "68": "G#4", "69": "A4", "70": "A#4", "71": "B4", + "72": "C5", "73": "C#5", "74": "D5", "75": "D#5", "76": "E5", "77": "F5", + "78": "F#5", "79": "G5", "80": "G#5", "81": "A5", "82": "A#5", "83": "B5", + "84": "C6", "85": "C#6", "86": "D6", "87": "D#6", "88": "E6", "89": "F6", + "90": "F#6", "91": "G6", "92": "G#6", "93": "A6", "94": "A#6", "95": "B6" + }, + "midiToJianpu": { + "description": "基于 C 大调的简谱映射，实际使用时需根据调号偏移", + "baseKey": "C", + "mapping": { + "0": "1", "2": "2", "4": "3", "5": "4", "7": "5", "9": "6", "11": "7" + }, + "octaveMarkers": { + "-2": ",,", "-1": ",", "0": "", "1": "'", "2": "''" + } + } + }, + + "durationMapping": { + "ticksPerBeat": 480, + "durationNames": { + "1920": { "name": "全音符", "symbol": "○", "beats": 4 }, + "1440": { "name": "附点二分音符", "symbol": "●.", "beats": 3 }, + "960": { "name": "二分音符", "symbol": "●", "beats": 2 }, + "720": { "name": "附点四分音符", "symbol": "♩.", "beats": 1.5 }, + "480": { "name": "四分音符", "symbol": "♩", "beats": 1 }, + "360": { "name": "附点八分音符", "symbol": "♪.", "beats": 0.75 }, + "240": { "name": "八分音符", "symbol": "♪", "beats": 0.5 }, + "180": { "name": "附点十六分音符", "symbol": "♬.", "beats": 0.375 }, + "120": { "name": "十六分音符", "symbol": "♬", "beats": 0.25 }, + "160": { "name": "三连音（四分）", "symbol": "♩³", "beats": 0.333 }, + "80": { "name": "三连音（八分）", "symbol": "♪³", "beats": 0.167 } + }, + "tolerancePercent": 10 + }, + + "rhythmPatterns": { + "quarter": { + "name": "四分音符型", + "durations": [480], + "description": "稳定的四分音符节奏，常用于叙述性段落", + "category": "basic" + }, + "eighth": { + "name": "八分音符型", + "durations": [240, 240], + "description": "连续八分音符，增加流动感", + "category": "basic" + }, + "dotted_quarter_eighth": { + "name": "附点四分+八分", + "durations": [720, 240], + "description": "附点节奏，增加推动力", + "category": "dotted" + }, + "eighth_dotted_quarter": { + "name": "八分+附点四分", + "durations": [240, 720], + "description": "切分感的附点节奏", + "category": "dotted" + }, + "syncopation_basic": { + "name": "基本切分", + "durations": [240, 480, 240], + "description": "基本切分节奏，强拍弱化", + "category": "syncopation" + }, + "syncopation_offbeat": { + "name": "后半拍切分", + "durations": [240, 240, 480], + "description": "后半拍强调的切分", + "category": "syncopation" + }, + "triplet_quarter": { + "name": "四分三连音", + "durations": [160, 160, 160], + "description": "三连音节奏，增加流畅感", + "category": "triplet" + }, + "long_short": { + "name": "长短型", + "durations": [960, 480], + "description": "二分+四分，舒缓的节奏", + "category": "basic" + }, + "sixteenth_group": { + "name": "十六分音符组", + "durations": [120, 120, 120, 120], + "description": "快速的十六分音符，增加紧张感", + "category": "fast" + } + }, + + "modeDetection": { + "description": "基于五声音阶特征音检测调式", + "pentatonic": { + "gong": { + "name": "宫调式", + "characteristicDegrees": [0, 2, 4, 7, 9], + "rootDegree": 0, + "endingNotes": [0, 7], + "weight": { "root": 3, "fifth": 2, "others": 1 } + }, + "shang": { + "name": "商调式", + "characteristicDegrees": [0, 2, 4, 7, 9], + "rootDegree": 2, + "endingNotes": [2, 0], + "weight": { "root": 3, "fifth": 2, "others": 1 } + }, + "jue": { + "name": "角调式", + "characteristicDegrees": [0, 2, 4, 7, 9], + "rootDegree": 4, + "endingNotes": [4, 2], + "weight": { "root": 3, "fifth": 2, "others": 1 } + }, + "zhi": { + "name": "徵调式", + "characteristicDegrees": [0, 2, 4, 7, 9], + "rootDegree": 7, + "endingNotes": [7, 9], + "weight": { "root": 3, "fifth": 2, "others": 1 } + }, + "yu": { + "name": "羽调式", + "characteristicDegrees": [0, 2, 4, 7, 9], + "rootDegree": 9, + "endingNotes": [9, 7], + "weight": { "root": 3, "fifth": 2, "others": 1 } + } + }, + "keySignatures": { + "C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3, + "E": 4, "F": 5, "F#": 6, "Gb": 6, "G": 7, "G#": 8, + "Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11 + } + }, + + "trackMatching": { + "vocalRangeMin": 48, + "vocalRangeMax": 84, + "vocalRangeDescription": "人声音域范围 C3(48) 到 C6(84)", + "vocalRangeZones": { + "belowVocal": { "min": 0, "max": 47, "description": "低于人声范围，可能是贝斯" }, + "vocalLow": { "min": 48, "max": 59, "description": "C3-B3，男声常用区" }, + "vocalMid": { "min": 60, "max": 71, "description": "C4-B4，男女声共用区" }, + "vocalHigh": { "min": 72, "max": 84, "description": "C5-C6，女声常用区" }, + "aboveVocal": { "min": 85, "max": 127, "description": "高于人声范围，可能是装饰音" } + }, + "tolerancePercent": 15, + "minVocalRangeOverlap": 0.5, + "priorityKeywords": ["vocal", "melody", "voice", "lead", "主旋律", "人声"], + "matchingRules": [ + { + "rule": "keyword_match", + "description": "音轨名称包含人声关键词时优先选择", + "priority": 1, + "scoreBonus": 30 + }, + { + "rule": "note_count_match", + "description": "音符数量与歌词字数最接近的音轨", + "priority": 2, + "maxScore": 40, + "toleranceLevels": { + "exact": { "tolerance": 0, "score": 40 }, + "close": { "tolerance": 0.05, "score": 35 }, + "acceptable": { "tolerance": 0.15, "score": 28 } + } + }, + { + "rule": "pitch_range_filter", + "description": "过滤音域超出人声范围的音轨", + "priority": 3, + "maxScore": 30, + "overlapScoring": { + "full": { "minOverlap": 1.0, "score": 30 }, + "high": { "minOverlap": 0.75, "score": 22 }, + "medium": { "minOverlap": 0.5, "score": 15 }, + "low": { "minOverlap": 0, "score": 0 } + } + } + ], + "confidenceThresholds": { + "high": 90, + "medium": 70, + "low": 50 + }, + "confidenceDescriptions": { + "high": "自动选择，无需确认", + "medium": "建议选择，请求确认", + "low": "需要用户手动确认", + "noMatch": "不推荐，列出供参考" + }, + "conflictResolution": { + "scoreDifferenceThreshold": 5, + "priorityOrder": ["keyword_match", "note_count_match", "pitch_range_filter"] + } + }, + + "intervalClassification": { + "stepwise": { + "name": "级进", + "semitones": [1, 2], + "description": "相邻音级的进行，旋律流畅" + }, + "smallLeap": { + "name": "小跳", + "semitones": [3, 4], + "description": "三度或四度跳进，增加起伏" + }, + "largeLeap": { + "name": "大跳", + "semitones": [5, 6, 7, 8, 9, 10, 11, 12], + "description": "五度及以上跳进，戏剧性强" + } + }, + + "contourAnalysis": { + "ascending": { + "name": "上行", + "condition": "后一音高于前一音", + "emotion": "积极、上升、期待" + }, + "descending": { + "name": "下行", + "condition": "后一音低于前一音", + "emotion": "忧伤、下沉、释放" + }, + "stable": { + "name": "平稳", + "condition": "音高变化在二度以内", + "emotion": "平静、叙述、稳定" + } + } +} diff --git a/src-tauri/resources/music/pentatonic-rules.json b/src-tauri/resources/music/pentatonic-rules.json new file mode 100644 index 000000000..26dc9c61f --- /dev/null +++ b/src-tauri/resources/music/pentatonic-rules.json @@ -0,0 +1,104 @@ +{ + "meta": { + "version": "1.0", + "description": "五声音阶规则库 - 定义中国传统五声音阶的调式结构、装饰音和音程规则", + "created_for": "Musicify 国风旋律生成 Skill" + }, + + "scales": { + "gong": { + "name": "宫调式", + "notes": [1, 2, 3, 5, 6], + "root": 1, + "characteristic": "以宫音(1)为主音，音阶明亮开阔，具有庄重典雅的特点", + "emotion": "庄重、明亮、欢快、积极向上", + "typicalCadence": [5, 1], + "avoidNotes": [4, 7] + }, + "shang": { + "name": "商调式", + "notes": [1, 2, 3, 5, 6], + "root": 2, + "characteristic": "以商音(2)为主音，音阶略带忧郁，具有深沉内敛的特点", + "emotion": "深沉、内敛、略带忧郁、思念", + "typicalCadence": [1, 2], + "avoidNotes": [4, 7] + }, + "jue": { + "name": "角调式", + "notes": [1, 2, 3, 5, 6], + "root": 3, + "characteristic": "以角音(3)为主音，音阶清新脱俗，具有空灵飘逸的特点", + "emotion": "清新、空灵、飘逸、超脱", + "typicalCadence": [2, 3], + "avoidNotes": [4, 7] + }, + "zhi": { + "name": "徵调式", + "notes": [1, 2, 3, 5, 6], + "root": 5, + "characteristic": "以徵音(5)为主音，音阶热情奔放，具有激昂豪迈的特点", + "emotion": "热情、奔放、激昂、豪迈", + "typicalCadence": [6, 5], + "avoidNotes": [4, 7] + }, + "yu": { + "name": "羽调式", + "notes": [1, 2, 3, 5, 6], + "root": 6, + "characteristic": "以羽音(6)为主音，音阶柔和婉转，具有忧伤哀怨的特点", + "emotion": "忧伤、婉转、哀怨、柔美", + "typicalCadence": [5, 6], + "avoidNotes": [4, 7] + } + }, + + "ornaments": { + "huayin": { + "name": "滑音", + "notation": "↗ 或 ↘", + "description": "从一个音滑向另一个音，常用于表达情感的流动", + "usage": "句尾延长音、情感转折处、模仿人声哭腔", + "examples": ["5↗6", "3↘2"] + }, + "chanyin": { + "name": "颤音", + "notation": "~", + "description": "在主音上快速交替相邻音，增加音色的丰富性", + "usage": "长音装饰、情感强调、模仿弦乐器效果", + "examples": ["5~", "6~"] + }, + "yiyin": { + "name": "倚音", + "notation": "小音符标记", + "description": "在主音前快速演奏的装饰音，增加旋律的流畅性", + "usage": "乐句开头、强拍装饰、增加韵味", + "examples": ["(3)5", "(6)1"] + }, + "boyin": { + "name": "波音", + "notation": "∿", + "description": "主音与上方或下方相邻音快速交替一次", + "usage": "轻快段落、活泼情绪、增加灵动感", + "examples": ["5∿", "3∿"] + } + }, + + "intervals": { + "allowed": [1, 2, 3, 4, 5], + "preferred": [1, 2], + "descriptions": { + "1": "同度/八度 - 稳定、强调", + "2": "二度 - 级进，最常用，流畅自然", + "3": "三度 - 小跳进，增加起伏", + "4": "四度 - 中跳进，增加张力", + "5": "五度 - 大跳进，戏剧性强，慎用" + }, + "rules": [ + "优先使用级进（二度）保持旋律流畅", + "跳进后宜用级进反向进行", + "避免连续大跳进", + "句尾常用下行级进解决到主音" + ] + } +} diff --git a/src-tauri/resources/music/rhyme-patterns.json b/src-tauri/resources/music/rhyme-patterns.json new file mode 100644 index 000000000..ccaefe07a --- /dev/null +++ b/src-tauri/resources/music/rhyme-patterns.json @@ -0,0 +1,183 @@ +{ + "meta": { + "version": "1.0", + "description": "中文歌词押韵数据库 - 基于拼音和声调的押韵分析", + "created_for": "Musicify Skill System" + }, + + "rhyme_patterns": { + "AABB": { + "description": "两行一韵，连续押韵", + "difficulty": 1, + "usage": "适合流行歌曲，容易上口" + }, + "ABAB": { + "description": "交错押韵", + "difficulty": 2, + "usage": "增加节奏变化，适合抒情歌曲" + }, + "ABCB": { + "description": "隔行押韵", + "difficulty": 2, + "usage": "常用于民谣和说唱" + }, + "AAAA": { + "description": "通韵到底", + "difficulty": 3, + "usage": "适合短小精悍的段落" + } + }, + + "common_rhymes": { + "爱情主题": [ + {"group": "ai", "words": ["爱", "在", "来", "开", "怀", "猜", "陪", "等待"]}, + {"group": "ing", "words": ["情", "心", "真", "深", "亲", "信", "认", "永恒"]}, + {"group": "ou", "words": ["走", "久", "守", "有", "后", "手", "温柔", "拥有"]}, + {"group": "an", "words": ["伴", "暖", "看", "汗", "伞", "岸", "陪伴", "温暖"]} + ], + + "励志主题": [ + {"group": "eng", "words": ["梦", "能", "成", "风", "空", "勇", "冲", "成功"]}, + {"group": "iang", "words": ["想", "强", "光", "方", "向", "长", "希望", "力量"]}, + {"group": "u", "words": ["路", "步", "住", "哭", "努", "苦", "付出", "坚持"]}, + {"group": "i", "words": ["力", "立", "起", "地", "意", "义", "坚毅", "奇迹"]} + ], + + "青春回忆": [ + {"group": "ian", "words": ["年", "天", "前", "甜", "变", "见", "青春", "遇见"]}, + {"group": "ao", "words": ["好", "老", "少", "跑", "闹", "笑", "美好", "年少"]}, + {"group": "ei", "words": ["美", "回", "累", "醉", "泪", "岁", "珍贵", "无悔"]}, + {"group": "ong", "words": ["梦", "中", "空", "痛", "重", "懂", "朦胧", "感动"]} + ], + + "离别思念": [ + {"group": "ie", "words": ["别", "夜", "雪", "月", "切", "说", "离别", "永别"]}, + {"group": "iao", "words": ["远", "想", "飘", "桥", "料", "瞧", "思念", "遥远"]}, + {"group": "iu", "words": ["留", "久", "流", "愁", "求", "收", "停留", "不朽"]}, + {"group": "eng", "words": ["等", "朋", "冷", "疼", "能", "层", "等候", "心疼"]} + ], + + "家乡故土": [ + {"group": "ang", "words": ["乡", "长", "方", "香", "窗", "望", "故乡", "远方"]}, + {"group": "ou", "words": ["家", "花", "话", "画", "挂", "牵挂", "变化"]}, + {"group": "i", "words": ["地", "里", "起", "记", "意", "立", "土地", "回忆"]}, + {"group": "an", "words": ["山", "田", "甘", "看", "暖", "伴", "青山", "温暖"]} + ] + }, + + "emotion_vocabulary": { + "欢快": { + "adjectives": ["明亮", "轻快", "绚烂", "灿烂", "活泼", "欢乐", "愉悦", "畅快"], + "verbs": ["跳跃", "飞扬", "奔跑", "舞蹈", "歌唱", "欢笑", "庆祝", "绽放"], + "nouns": ["阳光", "彩虹", "花朵", "蝴蝶", "鸟儿", "春风", "笑声", "节拍"] + }, + + "忧伤": { + "adjectives": ["黯然", "凄凉", "孤独", "冷清", "沉重", "苦涩", "惆怅", "迷茫"], + "verbs": ["凋零", "飘零", "消散", "哭泣", "叹息", "怀念", "失去", "离开"], + "nouns": ["雨滴", "落叶", "寒风", "夜晚", "眼泪", "回忆", "阴霾", "孤影"] + }, + + "温暖": { + "adjectives": ["温柔", "暖和", "亲切", "慈爱", "安详", "舒适", "贴心", "甜蜜"], + "verbs": ["拥抱", "守护", "陪伴", "关怀", "温暖", "照亮", "安慰", "包容"], + "nouns": ["怀抱", "家", "母亲", "暖阳", "火炉", "热茶", "羽毛", "港湾"] + }, + + "励志": { + "adjectives": ["坚强", "勇敢", "坚定", "不屈", "执着", "顽强", "无畏", "坚毅"], + "verbs": ["奋斗", "追求", "坚持", "突破", "攀登", "拼搏", "冲刺", "征服"], + "nouns": ["梦想", "目标", "理想", "信念", "勇气", "力量", "意志", "希望"] + }, + + "浪漫": { + "adjectives": ["浪漫", "梦幻", "迷人", "优雅", "柔美", "诗意", "唯美", "动人"], + "verbs": ["邂逅", "心动", "倾心", "眷恋", "凝视", "等候", "思念", "相拥"], + "nouns": ["月光", "星空", "玫瑰", "诗歌", "约定", "信物", "回音", "倩影"] + } + }, + + "rhyme_quality_metrics": { + "perfect_match": { + "score": 95, + "description": "完全押韵，音调和韵母都匹配" + }, + "near_rhyme": { + "score": 80, + "description": "近似押韵，韵母相同音调略不同" + }, + "assonance": { + "score": 65, + "description": "元音押韵，主要元音相同" + }, + "consonance": { + "score": 50, + "description": "辅音押韵，结尾辅音相同" + }, + "weak_rhyme": { + "score": 30, + "description": "弱押韵，仅部分音素相似" + }, + "no_rhyme": { + "score": 0, + "description": "无押韵关系" + } + }, + + "songwriting_tips": { + "rhyme_techniques": [ + { + "name": "内部押韵", + "description": "在同一行或相邻行的内部创造押韵效果", + "example": "心中的梦想如星光闪亮" + }, + { + "name": "重复押韵", + "description": "使用相同的韵脚增强记忆点", + "example": "爱你的心永不改变，爱你到永远" + }, + { + "name": "多重押韵", + "description": "在一行中使用多个押韵点", + "example": "阳光灿烂照人间，温暖如春风拂面" + } + ], + + "rhythm_patterns": [ + { + "name": "七字句", + "pattern": "2-2-3", + "example": "青春/如梦/多美好", + "usage": "经典中文歌词节奏" + }, + { + "name": "五字句", + "pattern": "2-3", + "example": "思君/不见君", + "usage": "古风歌曲常用" + }, + { + "name": "九字句", + "pattern": "3-3-3", + "example": "走过了/春夏秋冬/多少年", + "usage": "适合叙事性歌曲" + } + ] + }, + + "advanced_features": { + "tone_analysis": { + "first_tone": {"description": "阴平，高平调", "compatibility": ["first_tone", "second_tone"]}, + "second_tone": {"description": "阳平，中升调", "compatibility": ["first_tone", "second_tone"]}, + "third_tone": {"description": "上声，低降升调", "compatibility": ["third_tone", "fourth_tone"]}, + "fourth_tone": {"description": "去声，高降调", "compatibility": ["third_tone", "fourth_tone"]} + }, + + "syllable_structure": { + "monosyllabic": {"description": "单音节词", "usage": "适合快节奏部分"}, + "disyllabic": {"description": "双音节词", "usage": "最常用的词汇结构"}, + "trisyllabic": {"description": "三音节词", "usage": "适合慢节奏抒情"}, + "polysyllabic": {"description": "多音节词", "usage": "用于特殊效果"} + } + } +} \ No newline at end of file diff --git a/src-tauri/resources/scripts/audio_to_midi.py b/src-tauri/resources/scripts/audio_to_midi.py new file mode 100644 index 000000000..28bbf3e73 --- /dev/null +++ b/src-tauri/resources/scripts/audio_to_midi.py @@ -0,0 +1,367 @@ +#!/usr/bin/env python3 +""" +MP3 转 MIDI 工具 +使用 Demucs 分离人声 + Basic Pitch 转换 MIDI + +用法: + python audio_to_midi.py [output_dir] + python audio_to_midi.py --check # 检查依赖和硬件 + +输出: + JSON 格式的处理结果 +""" + +import sys +import os +import json +import subprocess +import shutil +from pathlib import Path +from datetime import datetime + + +def output_json(data): + """输出 JSON 格式结果""" + print(json.dumps(data, ensure_ascii=False, indent=2)) + + +def detect_hardware(): + """检测可用硬件加速""" + try: + import torch + if torch.cuda.is_available(): + device_name = torch.cuda.get_device_name(0) + return { + "device": "cuda", + "name": device_name, + "description": f"NVIDIA GPU 加速 ({device_name})", + "estimated_time": "1-2 分钟" + } + elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available(): + return { + "device": "mps", + "name": "Apple Silicon", + "description": "Apple Silicon 加速 (MPS)", + "estimated_time": "2-3 分钟" + } + except ImportError: + pass + + return { + "device": "cpu", + "name": "CPU", + "description": "CPU 模式 (较慢)", + "estimated_time": "8-15 分钟" + } + + +def check_dependencies(): + """检查依赖是否安装""" + dependencies = { + "demucs": {"installed": False, "version": None}, + "basic_pitch": {"installed": False, "version": None}, + "torch": {"installed": False, "version": None}, + } + + try: + import demucs + dependencies["demucs"]["installed"] = True + dependencies["demucs"]["version"] = getattr(demucs, '__version__', 'unknown') + except ImportError: + pass + + try: + import basic_pitch + dependencies["basic_pitch"]["installed"] = True + dependencies["basic_pitch"]["version"] = getattr(basic_pitch, '__version__', 'unknown') + except ImportError: + pass + + try: + import torch + dependencies["torch"]["installed"] = True + dependencies["torch"]["version"] = torch.__version__ + except ImportError: + pass + + return dependencies + + +def check_command_available(cmd): + """检查命令行工具是否可用""" + return shutil.which(cmd) is not None + + +def separate_vocals(input_mp3, output_dir, device="cpu"): + """ + 使用 Demucs 分离人声 + + Args: + input_mp3: 输入 MP3 文件路径 + output_dir: 输出目录 + device: 使用的设备 (cuda/mps/cpu) + + Returns: + vocals_path: 人声文件路径 + """ + input_path = Path(input_mp3) + output_path = Path(output_dir) + + # 构建 demucs 命令 + cmd = [ + sys.executable, "-m", "demucs", + "--two-stems=vocals", # 只分离人声和伴奏 + "-o", str(output_path), + "--device", device if device != "mps" else "mps", + ] + + # 添加输入文件 + cmd.append(str(input_path)) + + # 执行命令 + try: + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=1800 # 30 分钟超时 + ) + + if result.returncode != 0: + return None, f"Demucs 执行失败: {result.stderr}" + + # 查找输出的人声文件 + # Demucs 输出格式: output_dir/htdemucs/song_name/vocals.wav + song_name = input_path.stem + vocals_path = output_path / "htdemucs" / song_name / "vocals.wav" + + if not vocals_path.exists(): + # 尝试其他可能的路径 + for model_dir in output_path.iterdir(): + if model_dir.is_dir(): + possible_path = model_dir / song_name / "vocals.wav" + if possible_path.exists(): + vocals_path = possible_path + break + + if vocals_path.exists(): + return str(vocals_path), None + else: + return None, f"未找到人声文件，请检查 {output_path} 目录" + + except subprocess.TimeoutExpired: + return None, "Demucs 处理超时 (超过 30 分钟)" + except Exception as e: + return None, f"Demucs 执行异常: {str(e)}" + + +def convert_to_midi(vocals_wav, output_dir): + """ + 使用 Basic Pitch 将人声转换为 MIDI + + Args: + vocals_wav: 人声 WAV 文件路径 + output_dir: 输出目录 + + Returns: + midi_path: MIDI 文件路径 + """ + vocals_path = Path(vocals_wav) + output_path = Path(output_dir) + + # 构建 basic-pitch 命令 + cmd = [ + sys.executable, "-m", "basic_pitch", + str(output_path), + str(vocals_path) + ] + + try: + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=300 # 5 分钟超时 + ) + + if result.returncode != 0: + return None, f"Basic Pitch 执行失败: {result.stderr}" + + # 查找输出的 MIDI 文件 + # Basic Pitch 输出格式: output_dir/vocals_basic_pitch.mid + midi_name = vocals_path.stem + "_basic_pitch.mid" + midi_path = output_path / midi_name + + if midi_path.exists(): + return str(midi_path), None + else: + # 尝试查找任何 .mid 文件 + for f in output_path.glob("*.mid"): + return str(f), None + return None, f"未找到 MIDI 文件，请检查 {output_path} 目录" + + except subprocess.TimeoutExpired: + return None, "Basic Pitch 处理超时 (超过 5 分钟)" + except Exception as e: + return None, f"Basic Pitch 执行异常: {str(e)}" + + +def process_audio(input_mp3, output_dir=None): + """ + 完整的音频处理流程 + + Args: + input_mp3: 输入 MP3 文件路径 + output_dir: 输出目录 (默认为输入文件所在目录) + + Returns: + 处理结果字典 + """ + input_path = Path(input_mp3) + + if not input_path.exists(): + return { + "status": "error", + "error": f"输入文件不存在: {input_mp3}" + } + + if output_dir is None: + output_dir = input_path.parent + + output_path = Path(output_dir) + output_path.mkdir(parents=True, exist_ok=True) + + # 检测硬件 + hardware = detect_hardware() + + # 检查依赖 + deps = check_dependencies() + missing_deps = [name for name, info in deps.items() + if not info["installed"] and name != "torch"] + + if missing_deps: + return { + "status": "error", + "error": "缺少必要依赖", + "missing_dependencies": missing_deps, + "install_command": f"pip install {' '.join(missing_deps).replace('_', '-')}", + "alternative": "或使用在线工具: https://basicpitch.spotify.com" + } + + result = { + "status": "processing", + "input_file": str(input_path), + "output_dir": str(output_path), + "hardware": hardware, + "steps": [] + } + + # Step 1: 分离人声 + result["steps"].append({ + "step": 1, + "name": "分离人声", + "status": "in_progress", + "tool": "Demucs" + }) + + vocals_path, error = separate_vocals( + input_mp3, + output_path, + hardware["device"] + ) + + if error: + result["status"] = "error" + result["steps"][-1]["status"] = "failed" + result["steps"][-1]["error"] = error + return result + + result["steps"][-1]["status"] = "completed" + result["steps"][-1]["output"] = vocals_path + result["vocals_file"] = vocals_path + + # Step 2: 转换为 MIDI + result["steps"].append({ + "step": 2, + "name": "转换 MIDI", + "status": "in_progress", + "tool": "Basic Pitch" + }) + + midi_path, error = convert_to_midi(vocals_path, output_path) + + if error: + result["status"] = "error" + result["steps"][-1]["status"] = "failed" + result["steps"][-1]["error"] = error + return result + + result["steps"][-1]["status"] = "completed" + result["steps"][-1]["output"] = midi_path + result["midi_file"] = midi_path + + # 重命名 MIDI 文件为更友好的名称 + final_midi_name = input_path.stem + ".mid" + final_midi_path = output_path / final_midi_name + + if str(midi_path) != str(final_midi_path): + try: + shutil.move(midi_path, final_midi_path) + result["midi_file"] = str(final_midi_path) + except Exception: + pass # 保持原文件名 + + result["status"] = "success" + result["message"] = "MP3 转 MIDI 完成" + result["completed_at"] = datetime.now().isoformat() + + return result + + +def main(): + """主函数""" + if len(sys.argv) < 2: + output_json({ + "status": "error", + "error": "缺少参数", + "usage": "python audio_to_midi.py [output_dir]", + "examples": [ + "python audio_to_midi.py song.mp3", + "python audio_to_midi.py song.mp3 ./output", + "python audio_to_midi.py --check" + ] + }) + sys.exit(1) + + # 检查模式 + if sys.argv[1] == "--check": + deps = check_dependencies() + hardware = detect_hardware() + + all_installed = all( + info["installed"] + for name, info in deps.items() + if name != "torch" + ) + + output_json({ + "status": "ready" if all_installed else "missing_dependencies", + "dependencies": deps, + "hardware": hardware, + "install_command": "pip install demucs basic-pitch" if not all_installed else None, + "online_alternative": "https://basicpitch.spotify.com" + }) + sys.exit(0 if all_installed else 1) + + # 处理模式 + input_mp3 = sys.argv[1] + output_dir = sys.argv[2] if len(sys.argv) > 2 else None + + result = process_audio(input_mp3, output_dir) + output_json(result) + + sys.exit(0 if result["status"] == "success" else 1) + + +if __name__ == "__main__": + main() diff --git a/src-tauri/resources/scripts/midi_analyzer.py b/src-tauri/resources/scripts/midi_analyzer.py new file mode 100644 index 000000000..f5ac878b3 --- /dev/null +++ b/src-tauri/resources/scripts/midi_analyzer.py @@ -0,0 +1,677 @@ +#!/usr/bin/env python3 +""" +MIDI 音乐分析器 - 专业级旋律风格分析 +从"太简单"的文件检查升级为专业 MIDI 分析和特征提取 + +支持功能： +- 智能人声音轨识别 +- 深度旋律特征分析（节奏型、音程、调式） +- 音乐理论分析（五声音阶、调式推断） +- AI 风格学习准备 +""" + +import sys +import json +import argparse +from pathlib import Path +from typing import Dict, List, Any, Optional, Tuple +from dataclasses import dataclass, asdict +import traceback + +# 检查并导入依赖 +try: + import mido + import music21 + import numpy as np +except ImportError as e: + print(json.dumps({ + "status": "error", + "error_type": "missing_dependency", + "message": f"缺少必需的 Python 库: {str(e)}", + "solution": "请安装依赖: pip install mido music21 numpy", + "dependencies": { + "mido": "MIDI 文件解析", + "music21": "音乐理论分析", + "numpy": "数值计算" + } + }, ensure_ascii=False, indent=2)) + sys.exit(1) + +@dataclass +class VocalTrackCandidate: + """人声音轨候选""" + track_index: int + track_name: str + note_count: int + note_range: Tuple[int, int] # (min_pitch, max_pitch) + confidence_score: float + reasons: List[str] + +@dataclass +class MelodyFeatures: + """旋律特征分析结果""" + # 基本信息 + total_notes: int + note_range: Tuple[int, int] + duration_beats: float + + # 节奏特征 + rhythm_complexity: float + rhythm_patterns: Dict[str, float] # 节奏型分布 + syncopation_level: float + + # 音程特征 + interval_distribution: Dict[str, float] + stepwise_ratio: float + leap_ratio: float + + # 调式特征 + key_signature: str + mode_analysis: Dict[str, float] + scale_notes: List[str] + + # 旋律轮廓 + contour_vector: List[int] + phrase_structure: List[Tuple[int, int]] + +class ProfessionalMidiAnalyzer: + """专业级 MIDI 分析器""" + + def __init__(self): + # 人声音域范围 (MIDI note numbers) + self.vocal_range = (48, 84) # C3 to C6 + + # 五声音阶映射 + self.pentatonic_scales = { + 'C': [0, 2, 4, 7, 9], # C D E G A + 'G': [7, 9, 11, 2, 4], # G A B D E + 'D': [2, 4, 6, 9, 11], # D E F# A B + 'A': [9, 11, 1, 4, 6], # A B C# E F# + 'E': [4, 6, 8, 11, 1], # E F# G# B C# + 'B': [11, 1, 3, 6, 8], # B C# D# F# G# + 'F#': [6, 8, 10, 1, 3], # F# G# A# C# D# + 'Db': [1, 3, 5, 8, 10], # Db Eb F Ab Bb + 'Ab': [8, 10, 0, 3, 5], # Ab Bb C Eb F + 'Eb': [3, 5, 7, 10, 0], # Eb F G Bb C + 'Bb': [10, 0, 2, 5, 7], # Bb C D F G + 'F': [5, 7, 9, 0, 2] # F G A C D + } + + # 节奏模式识别 + self.rhythm_patterns = { + 'quarter': 480, # 四分音符 + 'eighth': 240, # 八分音符 + 'dotted_quarter': 720, # 附点四分音符 + 'sixteenth': 120, # 十六分音符 + 'triplet': 160 # 三连音 + } + + def analyze_midi_file(self, midi_path: str, lyrics_path: Optional[str] = None) -> Dict[str, Any]: + """分析 MIDI 文件的主入口""" + try: + # 基本文件检查 + if not Path(midi_path).exists(): + raise FileNotFoundError(f"MIDI 文件不存在: {midi_path}") + + # 加载 MIDI 文件 + midi_file = mido.MidiFile(midi_path) + + # 分析歌词信息 + lyrics_info = self._analyze_lyrics(lyrics_path) if lyrics_path else None + + # 识别人声音轨 + vocal_candidates = self._identify_vocal_tracks(midi_file, lyrics_info) + + if not vocal_candidates: + return self._create_error_result("no_vocal_track", "未找到合适的人声音轨") + + # 选择最佳人声音轨 + best_vocal = max(vocal_candidates, key=lambda x: x.confidence_score) + + # 提取音轨的音符数据 + notes = self._extract_notes_from_track(midi_file, best_vocal.track_index) + + if not notes: + return self._create_error_result("no_notes", "人声音轨中未找到音符数据") + + # 深度旋律特征分析 + melody_features = self._extract_melody_features(notes, midi_file) + + # 生成创作模式推荐 + mode_recommendation = self.recommend_creation_mode(melody_features, lyrics_info) + + # 构建分析结果 + result = { + "status": "success", + "analysis_type": "professional", + "file_info": { + "midi_path": midi_path, + "lyrics_path": lyrics_path, + "file_size": Path(midi_path).stat().st_size, + "track_count": len(midi_file.tracks) + }, + "vocal_track_analysis": { + "selected_track": asdict(best_vocal), + "all_candidates": [asdict(c) for c in vocal_candidates], + "selection_confidence": best_vocal.confidence_score + }, + "melody_features": asdict(melody_features), + "lyrics_analysis": lyrics_info, + "mode_recommendation": mode_recommendation, # NEW: 模式推荐信息 + "technical_info": { + "ticks_per_beat": midi_file.ticks_per_beat, + "total_time": sum(msg.time for track in midi_file.tracks for msg in track), + "format_type": midi_file.type + } + } + + return result + + except Exception as e: + return self._create_error_result( + "analysis_error", + f"分析过程中发生错误: {str(e)}", + {"traceback": traceback.format_exc()} + ) + + def _analyze_lyrics(self, lyrics_path: str) -> Optional[Dict[str, Any]]: + """分析歌词文件""" + try: + with open(lyrics_path, 'r', encoding='utf-8') as f: + content = f.read() + + # 统计字数（排除标点符号） + clean_text = ''.join(char for char in content if char.isalpha()) + + # 检测段落结构 + sections = [] + current_section = None + + for line in content.split('\n'): + line = line.strip() + if line.startswith('[') and line.endswith(']'): + if current_section: + sections.append(current_section) + current_section = { + "name": line[1:-1], + "lines": [], + "char_count": 0 + } + elif line and current_section: + current_section["lines"].append(line) + current_section["char_count"] += len([c for c in line if c.isalpha()]) + + if current_section: + sections.append(current_section) + + return { + "total_chars": len(clean_text), + "total_lines": len([line for line in content.split('\n') if line.strip() and not line.strip().startswith('[')]), + "sections": sections, + "has_structure_markers": any(line.startswith('[') for line in content.split('\n')) + } + + except Exception as e: + return {"error": f"歌词分析失败: {str(e)}"} + + def _identify_vocal_tracks(self, midi_file: mido.MidiFile, lyrics_info: Optional[Dict]) -> List[VocalTrackCandidate]: + """智能识别人声音轨""" + candidates = [] + + for track_idx, track in enumerate(midi_file.tracks): + notes = self._extract_notes_from_track(midi_file, track_idx) + + if not notes: + continue + + # 计算基本信息 + pitches = [note['pitch'] for note in notes] + min_pitch, max_pitch = min(pitches), max(pitches) + note_count = len(notes) + + # 评分系统 + score = 0.0 + reasons = [] + + # 1. 音轨名称匹配（30分） + track_name = getattr(track, 'name', f'Track {track_idx}') + vocal_keywords = ['vocal', 'voice', 'melody', 'lead', '主旋律', '人声'] + if any(keyword.lower() in track_name.lower() for keyword in vocal_keywords): + score += 30 + reasons.append(f"音轨名包含人声关键词: {track_name}") + + # 2. 音域匹配（25分） + vocal_range_overlap = self._calculate_range_overlap( + (min_pitch, max_pitch), self.vocal_range + ) + if vocal_range_overlap > 0.7: + score += 25 + reasons.append(f"音域高度匹配人声范围: {vocal_range_overlap:.1%}") + elif vocal_range_overlap > 0.5: + score += 15 + reasons.append(f"音域部分匹配人声范围: {vocal_range_overlap:.1%}") + + # 3. 歌词字数匹配（20分） + if lyrics_info and 'total_chars' in lyrics_info: + lyrics_chars = lyrics_info['total_chars'] + if lyrics_chars > 0: + ratio = abs(1 - note_count / lyrics_chars) + if ratio < 0.1: # 10%内匹配 + score += 20 + reasons.append(f"音符数与歌词字数高度匹配: {note_count}≈{lyrics_chars}") + elif ratio < 0.3: # 30%内匹配 + score += 10 + reasons.append(f"音符数与歌词字数基本匹配: {note_count}vs{lyrics_chars}") + + # 4. 音符密度合理性（15分） + if 20 <= note_count <= 200: # 合理的旋律长度 + score += 15 + reasons.append(f"音符数量合理: {note_count}") + elif note_count > 10: + score += 5 + reasons.append(f"音符数量可接受: {note_count}") + + # 5. 旋律特征（10分） + interval_variety = self._calculate_interval_variety(notes) + if interval_variety > 0.3: # 有合理的音程变化 + score += 10 + reasons.append(f"音程变化丰富: {interval_variety:.2f}") + + candidates.append(VocalTrackCandidate( + track_index=track_idx, + track_name=track_name, + note_count=note_count, + note_range=(min_pitch, max_pitch), + confidence_score=score, + reasons=reasons + )) + + # 按置信度排序 + return sorted(candidates, key=lambda x: x.confidence_score, reverse=True) + + def _extract_notes_from_track(self, midi_file: mido.MidiFile, track_idx: int) -> List[Dict]: + """从指定音轨提取音符信息""" + track = midi_file.tracks[track_idx] + notes = [] + current_time = 0 + active_notes = {} # pitch -> start_time + + for msg in track: + current_time += msg.time + + if msg.type == 'note_on' and msg.velocity > 0: + active_notes[msg.note] = current_time + elif msg.type == 'note_off' or (msg.type == 'note_on' and msg.velocity == 0): + if msg.note in active_notes: + start_time = active_notes.pop(msg.note) + duration = current_time - start_time + + notes.append({ + 'pitch': msg.note, + 'start_time': start_time, + 'duration': duration, + 'velocity': getattr(msg, 'velocity', 64) + }) + + # 按开始时间排序 + return sorted(notes, key=lambda x: x['start_time']) + + def _extract_melody_features(self, notes: List[Dict], midi_file: mido.MidiFile) -> MelodyFeatures: + """深度旋律特征提取""" + ticks_per_beat = midi_file.ticks_per_beat + + # 基本信息 + pitches = [note['pitch'] for note in notes] + durations = [note['duration'] for note in notes] + + # 节奏分析 + rhythm_analysis = self._analyze_rhythm_patterns(durations, ticks_per_beat) + + # 音程分析 + interval_analysis = self._analyze_intervals(pitches) + + # 调式分析 + key_analysis = self._analyze_key_and_mode(pitches) + + # 旋律轮廓 + contour = self._extract_melody_contour(pitches) + + # 乐句结构 + phrases = self._identify_phrases(notes, ticks_per_beat) + + return MelodyFeatures( + total_notes=len(notes), + note_range=(min(pitches), max(pitches)), + duration_beats=sum(durations) / ticks_per_beat, + rhythm_complexity=rhythm_analysis['complexity'], + rhythm_patterns=rhythm_analysis['patterns'], + syncopation_level=rhythm_analysis['syncopation'], + interval_distribution=interval_analysis['distribution'], + stepwise_ratio=interval_analysis['stepwise_ratio'], + leap_ratio=interval_analysis['leap_ratio'], + key_signature=key_analysis['key'], + mode_analysis=key_analysis['modes'], + scale_notes=key_analysis['scale_notes'], + contour_vector=contour, + phrase_structure=phrases + ) + + def _analyze_rhythm_patterns(self, durations: List[int], ticks_per_beat: int) -> Dict[str, Any]: + """分析节奏型模式""" + if not durations: + return {'complexity': 0, 'patterns': {}, 'syncopation': 0} + + # 标准化时值到节拍单位 + beat_durations = [d / ticks_per_beat for d in durations] + + # 计算节奏模式分布 + patterns = { + 'whole': 0, # 全音符 + 'half': 0, # 二分音符 + 'quarter': 0, # 四分音符 + 'eighth': 0, # 八分音符 + 'sixteenth': 0, # 十六分音符 + 'dotted': 0, # 附点节奏 + 'triplet': 0 # 三连音 + } + + for duration in beat_durations: + if abs(duration - 4.0) < 0.1: + patterns['whole'] += 1 + elif abs(duration - 2.0) < 0.1: + patterns['half'] += 1 + elif abs(duration - 1.0) < 0.1: + patterns['quarter'] += 1 + elif abs(duration - 0.5) < 0.1: + patterns['eighth'] += 1 + elif abs(duration - 0.25) < 0.1: + patterns['sixteenth'] += 1 + elif abs(duration - 1.5) < 0.1: + patterns['dotted'] += 1 + elif abs(duration - 0.33) < 0.1: + patterns['triplet'] += 1 + + total = len(durations) + pattern_ratios = {k: v/total for k, v in patterns.items()} if total > 0 else patterns + + # 计算节奏复杂度 + complexity = len([v for v in pattern_ratios.values() if v > 0.05]) # 超过5%的模式 + + # 简单的切分检测 + syncopation = sum(1 for d in beat_durations if 0.3 < d < 0.7 or 1.3 < d < 1.7) / total if total > 0 else 0 + + return { + 'complexity': complexity, + 'patterns': pattern_ratios, + 'syncopation': syncopation + } + + def _analyze_intervals(self, pitches: List[int]) -> Dict[str, Any]: + """分析音程分布""" + if len(pitches) < 2: + return {'distribution': {}, 'stepwise_ratio': 0, 'leap_ratio': 0} + + intervals = [pitches[i+1] - pitches[i] for i in range(len(pitches)-1)] + + # 音程分类 + interval_types = { + 'unison': 0, # 同度 (0) + 'step': 0, # 级进 (1-2) + 'small_leap': 0, # 小跳 (3-4) + 'large_leap': 0, # 大跳 (5+) + 'octave': 0 # 八度 (12) + } + + for interval in intervals: + abs_interval = abs(interval) + if abs_interval == 0: + interval_types['unison'] += 1 + elif abs_interval <= 2: + interval_types['step'] += 1 + elif abs_interval <= 4: + interval_types['small_leap'] += 1 + elif abs_interval == 12: + interval_types['octave'] += 1 + else: + interval_types['large_leap'] += 1 + + total = len(intervals) + distribution = {k: v/total for k, v in interval_types.items()} if total > 0 else interval_types + + return { + 'distribution': distribution, + 'stepwise_ratio': distribution['step'], + 'leap_ratio': distribution['small_leap'] + distribution['large_leap'] + } + + def _analyze_key_and_mode(self, pitches: List[int]) -> Dict[str, Any]: + """分析调性和调式""" + if not pitches: + return {'key': 'Unknown', 'modes': {}, 'scale_notes': []} + + # 统计音高类别 + pitch_classes = [p % 12 for p in pitches] + pc_counts = {} + for pc in pitch_classes: + pc_counts[pc] = pc_counts.get(pc, 0) + 1 + + # 尝试匹配五声音阶 + best_key = 'C' + best_score = 0 + + for key, scale in self.pentatonic_scales.items(): + score = sum(pc_counts.get(pc, 0) for pc in scale) + if score > best_score: + best_score = score + best_key = key + + # 生成调式信息 + note_names = ['C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B'] + scale_notes = [note_names[pc] for pc in self.pentatonic_scales[best_key]] + + # 简化的调式检测 + modes = { + 'pentatonic': best_score / len(pitches) if pitches else 0, + 'major': 0.5, # 占位符 + 'minor': 0.3 # 占位符 + } + + return { + 'key': best_key, + 'modes': modes, + 'scale_notes': scale_notes + } + + def _extract_melody_contour(self, pitches: List[int]) -> List[int]: + """提取旋律轮廓""" + if len(pitches) < 2: + return [] + + contour = [] + for i in range(1, len(pitches)): + diff = pitches[i] - pitches[i-1] + if diff > 0: + contour.append(1) # 上行 + elif diff < 0: + contour.append(-1) # 下行 + else: + contour.append(0) # 平行 + + return contour + + def _identify_phrases(self, notes: List[Dict], ticks_per_beat: int) -> List[Tuple[int, int]]: + """识别乐句结构""" + if not notes: + return [] + + # 简单的乐句分割：基于较长的休止或时间间隔 + phrases = [] + phrase_start = 0 + + for i in range(1, len(notes)): + # 检测乐句间隔（如果两个音符间隔超过一拍） + gap = notes[i]['start_time'] - (notes[i-1]['start_time'] + notes[i-1]['duration']) + if gap > ticks_per_beat: # 超过一拍的间隔 + phrases.append((phrase_start, i-1)) + phrase_start = i + + # 添加最后一个乐句 + phrases.append((phrase_start, len(notes)-1)) + + return phrases + + def _calculate_range_overlap(self, range1: Tuple[int, int], range2: Tuple[int, int]) -> float: + """计算两个音域的重叠度""" + overlap_start = max(range1[0], range2[0]) + overlap_end = min(range1[1], range2[1]) + + if overlap_start >= overlap_end: + return 0.0 + + overlap_size = overlap_end - overlap_start + range1_size = range1[1] - range1[0] + + return overlap_size / range1_size if range1_size > 0 else 0.0 + + def _calculate_interval_variety(self, notes: List[Dict]) -> float: + """计算音程变化丰富度""" + if len(notes) < 2: + return 0.0 + + pitches = [note['pitch'] for note in notes] + intervals = [abs(pitches[i+1] - pitches[i]) for i in range(len(pitches)-1)] + unique_intervals = len(set(intervals)) + + return unique_intervals / len(intervals) if intervals else 0.0 + + def calculate_complexity(self, melody_features: MelodyFeatures, lyrics_info: Dict = None) -> float: + """计算旋律复杂度（0-100分）""" + try: + # 节奏复杂度 (0-40分) + rhythm_score = 0 + if hasattr(melody_features, 'rhythm_patterns'): + syncopation = melody_features.rhythm_patterns.get('syncopation', 0) + sixteenth = melody_features.rhythm_patterns.get('sixteenth', 0) + rhythm_score = min(40, (syncopation + sixteenth) * 0.4) + + # 音程复杂度 (0-30分) + interval_score = 0 + if hasattr(melody_features, 'interval_distribution'): + large_leap = melody_features.interval_distribution.get('large_leap', 0) + interval_score = min(30, large_leap * 0.3) + + # 调式不确定性 (0-30分) + modal_score = 0 + if hasattr(melody_features, 'mode_analysis'): + # 如果调式分析有置信度信息 + max_confidence = max(melody_features.mode_analysis.values()) if melody_features.mode_analysis else 0 + modal_uncertainty = 1 - max_confidence + modal_score = modal_uncertainty * 30 + + total_complexity = rhythm_score + interval_score + modal_score + return min(100, total_complexity) + + except Exception: + return 30.0 # 默认中等复杂度 + + def recommend_creation_mode(self, melody_features: MelodyFeatures, lyrics_info: Dict = None) -> Dict[str, Any]: + """基于旋律特征推荐创作模式""" + try: + # 计算整体复杂度 + complexity = self.calculate_complexity(melody_features, lyrics_info) + + # 段落数量（从歌词信息获取） + section_count = 0 + if lyrics_info and 'structure' in lyrics_info: + sections = lyrics_info['structure'].get('sections', []) + section_count = len(sections) + + # 推荐逻辑 + if complexity >= 60: + recommended = "expert" + reason = f"旋律复杂度高 ({complexity:.1f}/100)，建议使用专家模式进行精细控制" + elif complexity <= 25: + recommended = "express" + reason = f"旋律相对简单 ({complexity:.1f}/100)，适合快速模式自动生成" + elif section_count >= 4: + recommended = "coach" + reason = f"歌曲结构复杂 ({section_count}个段落)，建议教练模式逐步创作" + else: + recommended = "professional" + reason = f"旋律复杂度适中 ({complexity:.1f}/100)，推荐专业模式平衡效率与质量" + + # 备选方案 + alternatives = [] + if recommended != "express": + alternatives.append({"mode": "express", "reason": "需要快速原型或demo时使用"}) + if recommended != "professional": + alternatives.append({"mode": "professional", "reason": "平衡创作质量与效率的通用选择"}) + if recommended != "coach": + alternatives.append({"mode": "coach", "reason": "学习创作技巧或深度个性化表达时使用"}) + if recommended != "expert": + alternatives.append({"mode": "expert", "reason": "需要完全控制创作过程的专业制作"}) + + return { + "recommended": recommended, + "complexity_score": complexity, + "reasoning": reason, + "section_count": section_count, + "alternatives": alternatives + } + + except Exception as e: + # 错误时返回默认推荐 + return { + "recommended": "professional", + "complexity_score": 50.0, + "reasoning": "分析过程中出现问题，推荐使用通用的专业模式", + "section_count": 0, + "alternatives": [], + "error": f"推荐逻辑错误: {str(e)}" + } + + def _create_error_result(self, error_type: str, message: str, details: Dict = None) -> Dict[str, Any]: + """创建错误结果""" + result = { + "status": "error", + "error_type": error_type, + "message": message, + "timestamp": __import__('datetime').datetime.now().isoformat() + } + + if details: + result["details"] = details + + return result + +def main(): + """命令行入口""" + parser = argparse.ArgumentParser(description="专业级 MIDI 音乐分析器") + parser.add_argument("midi_file", help="MIDI 文件路径") + parser.add_argument("--lyrics", help="歌词文件路径（可选）") + parser.add_argument("--output", help="输出 JSON 文件路径（可选）") + parser.add_argument("--pretty", action="store_true", help="格式化 JSON 输出") + + args = parser.parse_args() + + # 创建分析器 + analyzer = ProfessionalMidiAnalyzer() + + # 执行分析 + result = analyzer.analyze_midi_file(args.midi_file, args.lyrics) + + # 输出结果 + if args.pretty: + output = json.dumps(result, ensure_ascii=False, indent=2) + else: + output = json.dumps(result, ensure_ascii=False) + + if args.output: + with open(args.output, 'w', encoding='utf-8') as f: + f.write(output) + print(f"分析结果已保存到: {args.output}") + else: + print(output) + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/src-tauri/src/app/runner.rs b/src-tauri/src/app/runner.rs index 9a9d00edd..743f99344 100644 --- a/src-tauri/src/app/runner.rs +++ b/src-tauri/src/app/runner.rs @@ -1180,6 +1180,12 @@ pub fn run() { commands::update_cmd::update_last_check_timestamp, commands::update_cmd::close_update_window, commands::update_cmd::test_update_window, + // Music commands + commands::music_cmd::check_python_env, + commands::music_cmd::analyze_midi, + commands::music_cmd::convert_mp3_to_midi, + commands::music_cmd::load_music_resource, + commands::music_cmd::install_python_dependencies, ]) .run(tauri::generate_context!()) .expect("error while running tauri application"); diff --git a/src-tauri/src/commands/mod.rs b/src-tauri/src/commands/mod.rs index 4fcd1b055..d979ecccf 100644 --- a/src-tauri/src/commands/mod.rs +++ b/src-tauri/src/commands/mod.rs @@ -12,6 +12,7 @@ pub mod machine_id_cmd; pub mod mcp_cmd; pub mod model_registry_cmd; pub mod models_cmd; +pub mod music_cmd; pub mod native_agent_cmd; pub mod network_cmd; pub mod oauth_cmd; diff --git a/src-tauri/src/commands/music_cmd.rs b/src-tauri/src/commands/music_cmd.rs new file mode 100644 index 000000000..7b25b8833 --- /dev/null +++ b/src-tauri/src/commands/music_cmd.rs @@ -0,0 +1,212 @@ +use serde::{Deserialize, Serialize}; +use std::path::PathBuf; +use std::process::Command; +use tauri::State; + +/// MIDI 分析结果 +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct MidiAnalysisResult { + /// 调式信息 + pub mode: String, + /// BPM (每分钟节拍数) + pub bpm: f64, + /// 拍号 + pub time_signature: String, + /// 音轨信息 + pub tracks: Vec, + /// 旋律特征 + pub melody_features: MelodyFeatures, +} + +/// 音轨信息 +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TrackInfo { + /// 音轨索引 + pub index: usize, + /// 音轨名称 + pub name: String, + /// 乐器名称 + pub instrument: String, + /// 音符数量 + pub note_count: usize, + /// 是否为人声音轨 + pub is_vocal: bool, +} + +/// 旋律特征 +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct MelodyFeatures { + /// 音域范围 (半音数) + pub range: i32, + /// 平均音高 + pub avg_pitch: f64, + /// 音程跳跃频率 + pub interval_jumps: f64, + /// 节奏复杂度 + pub rhythm_complexity: f64, +} + +/// Python 环境检测结果 +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct PythonEnvInfo { + /// 是否已安装 Python + pub python_installed: bool, + /// Python 版本 + pub python_version: Option, + /// 缺失的依赖包 + pub missing_packages: Vec, +} + +/// 检查 Python 环境 +#[tauri::command] +pub async fn check_python_env() -> Result { + // 检查 Python 是否安装 + let python_check = Command::new("python3").arg("--version").output(); + + let (python_installed, python_version) = match python_check { + Ok(output) => { + let version = String::from_utf8_lossy(&output.stdout).trim().to_string(); + (true, Some(version)) + } + Err(_) => (false, None), + }; + + if !python_installed { + return Ok(PythonEnvInfo { + python_installed: false, + python_version: None, + missing_packages: vec![], + }); + } + + // 检查必需的 Python 包 + let required_packages = vec!["mido", "music21", "numpy", "demucs", "basic-pitch"]; + let mut missing_packages = Vec::new(); + + for package in required_packages { + let check = Command::new("python3") + .arg("-c") + .arg(format!("import {}", package.replace("-", "_"))) + .output(); + + if check.is_err() || !check.unwrap().status.success() { + missing_packages.push(package.to_string()); + } + } + + Ok(PythonEnvInfo { + python_installed, + python_version, + missing_packages, + }) +} + +/// 分析 MIDI 文件 +#[tauri::command] +pub async fn analyze_midi(midi_path: String) -> Result { + // 获取 Python 脚本路径 + let script_path = get_resource_path("scripts/midi_analyzer.py")?; + + // 调用 Python 脚本 + let output = Command::new("python3") + .arg(&script_path) + .arg(&midi_path) + .output() + .map_err(|e| format!("Failed to execute Python script: {}", e))?; + + if !output.status.success() { + let error = String::from_utf8_lossy(&output.stderr); + return Err(format!("MIDI analysis failed: {}", error)); + } + + // 解析 JSON 输出 + let result_json = String::from_utf8_lossy(&output.stdout); + serde_json::from_str(&result_json) + .map_err(|e| format!("Failed to parse analysis result: {}", e)) +} + +/// 将 MP3 转换为 MIDI +#[tauri::command] +pub async fn convert_mp3_to_midi(mp3_path: String, output_path: String) -> Result { + // 获取 Python 脚本路径 + let script_path = get_resource_path("scripts/audio_to_midi.py")?; + + // 调用 Python 脚本 + let output = Command::new("python3") + .arg(&script_path) + .arg(&mp3_path) + .arg(&output_path) + .output() + .map_err(|e| format!("Failed to execute Python script: {}", e))?; + + if !output.status.success() { + let error = String::from_utf8_lossy(&output.stderr); + return Err(format!("MP3 to MIDI conversion failed: {}", error)); + } + + Ok(output_path) +} + +/// 加载资源文件 +#[tauri::command] +pub async fn load_music_resource(resource_name: String) -> Result { + let resource_path = get_resource_path(&format!("music/{}", resource_name))?; + + std::fs::read_to_string(&resource_path) + .map_err(|e| format!("Failed to read resource file: {}", e)) +} + +/// 获取资源文件路径 +fn get_resource_path(relative_path: &str) -> Result { + // 在开发环境中，资源文件在 src-tauri/resources/ + // 在生产环境中，资源文件会被打包到应用程序包中 + let mut path = + std::env::current_exe().map_err(|e| format!("Failed to get executable path: {}", e))?; + + path.pop(); // 移除可执行文件名 + + #[cfg(target_os = "macos")] + { + // macOS: 资源在 .app/Contents/Resources/ + path.pop(); // 移除 MacOS + path.push("Resources"); + } + + #[cfg(not(target_os = "macos"))] + { + // Windows/Linux: 资源在可执行文件同级目录 + path.push("resources"); + } + + path.push(relative_path); + + if !path.exists() { + // 尝试开发环境路径 + let dev_path = PathBuf::from("src-tauri/resources").join(relative_path); + if dev_path.exists() { + return Ok(dev_path); + } + return Err(format!("Resource not found: {}", relative_path)); + } + + Ok(path) +} + +/// 安装 Python 依赖 +#[tauri::command] +pub async fn install_python_dependencies() -> Result { + let packages = vec!["mido", "music21", "numpy", "demucs", "basic-pitch"]; + + let output = Command::new("pip3") + .arg("install") + .args(&packages) + .output() + .map_err(|e| format!("Failed to install packages: {}", e))?; + + if !output.status.success() { + let error = String::from_utf8_lossy(&output.stderr); + return Err(format!("Installation failed: {}", error)); + } + + Ok("Dependencies installed successfully".to_string()) +} diff --git a/src-tauri/tauri.conf.headless.json b/src-tauri/tauri.conf.headless.json index af878070e..2b57b8cf4 100644 --- a/src-tauri/tauri.conf.headless.json +++ b/src-tauri/tauri.conf.headless.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "ProxyCast", - "version": "0.42.0", + "version": "0.43.0", "identifier": "com.proxycast.app", "build": { "beforeDevCommand": "npm run dev", diff --git a/src-tauri/tauri.conf.json b/src-tauri/tauri.conf.json index 355829b02..4280d91df 100644 --- a/src-tauri/tauri.conf.json +++ b/src-tauri/tauri.conf.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "ProxyCast", - "version": "0.42.0", + "version": "0.43.0", "identifier": "com.proxycast.app", "build": { "beforeDevCommand": "npm run dev", diff --git a/src/components/agent/chat/components/EmptyState.tsx b/src/components/agent/chat/components/EmptyState.tsx index 35f2e0016..45c73e463 100644 --- a/src/components/agent/chat/components/EmptyState.tsx +++ b/src/components/agent/chat/components/EmptyState.tsx @@ -16,6 +16,7 @@ import { Zap, RefreshCw, LayoutTemplate, + Music, } from "lucide-react"; /** @@ -354,6 +355,9 @@ const CATEGORIES = [ label: "通用对话", icon: , }, + { id: "social", label: "社媒内容", icon: }, + { id: "image", label: "图文海报", icon: }, + { id: "music", label: "歌词曲谱", icon: }, { id: "knowledge", label: "知识探索", @@ -364,14 +368,12 @@ const CATEGORIES = [ label: "计划规划", icon: , }, - { id: "social", label: "社媒内容", icon: }, - { id: "image", label: "图文海报", icon: }, { id: "office", label: "办公文档", icon: }, { id: "video", label: "短视频", icon: