Handed the guide to a fresh agent twice with nothing but the file and "get this
running", and told it to verify every claim against the code. Both runs found
real defects; this fixes what they found.
Order of operations was wrong in two places. Step 0 gave the clone command before
the "don't clone over an existing checkout" caveat, so a literal reader clones
first -- and since the directory may be named anything, the trailing `cd
index-tts` then lands somewhere else. Now it tests for a checkout first. Step 4's
cache-reuse subsection said "do this before any auxiliary download" while its
command needs the step 3 environment; it now says plainly that it runs after step
3, and leads with a no-venv check for whether the subsection applies at all --
on a machine where `hf_cache/` is already populated the whole section is moot and
nothing said so.
Step 3 contradicted itself: after establishing that Apple Silicon must use
`--extra webui --extra test`, its verification block handed back `uv sync
--all-extras --dry-run` with "expect Would make no changes", which cannot succeed
there. It now says to repeat whichever extras you installed with.
Claims that did not hold up:
- The disk figures were measured on Linux/CUDA and presented as universal. The
same venv on Apple Silicon is 1.7 GB, not 9.5 GB, because the bulk is CUDA
wheels -- so ~10 GB standing rather than ~18 GB.
- The Small GPUs numbers are dedicated VRAM and do not describe unified memory.
Worse, `detect_vram_gb()` returns None whenever CUDA is unavailable, so the
low-VRAM adaptation never engages off-CUDA at all: an 8 GB Mac silently gets
the full default configuration. Both now stated.
- The `preview-features` trap entry was backwards. That key is not in
pyproject.toml, while the `extra-build-dependencies` experimental warning fires
on every single uv invocation and was undocumented. An agent consulting the
table on seeing that warning was told to remove a key whose presence would
actually make uv discard the whole `[tool.uv]` table.
- "Serves on localhost:7860 (local access)" sat one clause away from "binds
0.0.0.0"; webui.py:26 confirms the latter.
- Naming a single extra prunes the others -- measured at 3 packages removed for
`--extra webui`, up to 30 elsewhere -- which matters for a guide whose premise
is not redoing work.
- `--lang` is not validated: tokenizer.py:173-177 falls back silently, so a typo
degrades audio instead of erroring.
Robustness of the guide's own commands: `ls -d /usr/local/cuda*` aborts the line
under zsh on an unmatched glob (the shell fails before `ls` runs, and
`2>/dev/null` does not help), so it is now `find`. The manifest check was a
pipeline whose curl status is only reachable through `$PIPESTATUS`/`$pipestatus`
depending on shell -- split into two commands with `curl -sf`. It also flagged
`.gitattributes`, `LICENSE` and `README.md` as needing re-download on a complete
set of weights, which reads as a 5 GB failure; those are repo metadata and are
now skipped, verified still catching a simulated truncation. The vendor probes'
exit 127 was indistinguishable from "ran, found nothing", and `torch.version.cuda
== None` was unexplained; both now are.
Smaller: the Python snippet used `IndexTTS2` without importing it; the example
summary counted the always-untracked `checkpoints/config.yaml` (a .gitignore
negation, not a user edit) as an uncommitted file, which is the exact conflation
the surrounding text warns against; migration output has three wordings, not one.
a0bb942 flagged that infer_v2_5.py sets HF_HUB_CACHE internally. That is true --
line 4 does `os.environ['HF_HUB_CACHE'] = './checkpoints/hf_cache'` -- and it
makes the exported variable useless, so the guide was telling agents to do
something that silently finds nothing and re-downloads 2.8 GB.
Verified the three paths:
import model_download only -> keeps the exported value, migration works
import infer_v2_5 -> overwritten to ./checkpoints/hf_cache
webui.py -> same, it imports infer_v2_5 too
So the note's scoping was the wrong way round: it said webui.py was affected but
the smoke test was not, when in fact both are, and only a standalone import is
not. Replaced with a migration step that imports just the download helper, plus
the output to expect. Once it has run, checkpoints/hf_cache/ is populated and
nothing consults HF_HUB_CACHE again.
The closing section was four lines of prose asking for "what you reused vs
installed" and little else, which leaves the shape of the output to the agent.
The parts most worth having are the ones most easily dropped: which
configuration is actually in effect, hence which features exist, and what was
skipped rather than done.
Replace it with an ordered spec -- verdict first, then the commands to run it
again, the configuration, reused vs installed vs downloaded, what needs the
user's attention, and what went unverified -- followed by a worked example.
The example matters more than the list: an agent imitates a sample report far
more reliably than it follows a description of one.
Step 0 already collected memory.total and step 1 never said what to do with it,
so an agent on a small card would install for twenty minutes and then meet an
OOM with no idea which knob to turn.
Records what was measured rather than a guess. Peak reserved memory for one
short utterance on a 4090:
full precision + QwenEmotion 8.15 GB
half precision + QwenEmotion 6.54 GB
half precision, no QwenEmotion 5.48 GB
and under a hard per-process cap, which is what actually decides whether a card
works: at 8 GB all three complete, but the full configuration only by peaking at
exactly 8.00 GB, leaving nothing for the display output a real 8 GB card also
funds. At 6 GB only half-precision-without-QwenEmotion survives; at 5 GB even
that OOMs. The two failing 6 GB runs shared a box with ~6.2 GB physically free,
so the cap was not the sole constraint there -- noted in the doc, and immaterial
since those configurations need 7.70 GB and 6.13 GB just to load.
Also corrects the requirements table: it listed CUDA Toolkit 12.8+ as
non-negotiable, but the torch wheels bundle their own CUDA runtime. Inference
ran with CUDA_HOME unset on a box whose driver reports 12.4; the toolkit is
needed only to compile.
Ran the guide's own probe block and the cache-reuse path on a Linux GPU box.
Three things needed correcting.
The CUDA probe was actively misleading. `nvcc` on PATH was 11.5 while
/usr/local/cuda pointed at 12.8, so an agent reading only the PATH one would
conclude the machine is below the documented 12.8 minimum and either abandon a
working box or install a toolkit it already has. Probe both, and say which
value actually governs what: torch.version.cuda for wheel selection, CUDA_HOME
for compilation. Also record the visible symptom of getting this wrong -- the
BigVGAN kernel build failing on compute_89, which falls back to torch and is
survivable, but shares the root cause.
The cache-reuse saving was an estimate and it was wrong: 2.77 GiB, not the
~4.5 GB claimed, migrated in 2.1s. Inference against the migrated directory
runs (RTF 0.90), so the path is real -- only the figure was off.
Step 2 said little more than `git pull --ff-only`, which is not enough
guidance for an agent operating on someone's working tree. A user upgrading
from IndexTTS-2 may have been running patched inference code for months.
Verified the actual behaviour before writing it down: pull --ff-only aborts and
applies nothing when a tracked edit overlaps, so git does not silently clobber
anything -- the real hazard is an agent "fixing" that abort with reset --hard.
Name the forbidden commands, require showing the user their diff first, and
note that a conflicted `stash pop` is recoverable (the entry is kept, untracked
files return) but is the user's merge to resolve, not the agent's.
`hf-xet` is an unconditional base dependency of huggingface-hub on x86_64,
arm64 and aarch64, so `[hf_xet]` added nothing the plain package does not
already pull in.
Users increasingly hand a repo URL to an agent and ask it to "get this
running". The README is written for a person on a fresh machine, so an agent
following it re-downloads ~10 GB of wheels and ~5 GB of weights that an
existing IndexTTS-2 install already has, and it has no way to tell which
verification commands actually prove anything.
This guide is written for that reader: probe commands to run before touching
anything, the decision points for reusing an existing .venv or HuggingFace
cache, how to check local weights against the remote manifest instead of
re-downloading, and a table of the failure signatures seen while running the
README end-to-end on a clean Linux box.
Notable points it records, since none are obvious from the code:
- `uv lock --check` compares pyproject.toml against uv.lock and says nothing
about what is installed; `uv sync --dry-run` is the environment check.
- `ensure_models_available()` migrates an old HF cache layout instead of
downloading, saving ~4.5 GB — but it copies rather than symlinks.
- Auxiliary models are not in the model repo; they arrive in hf_cache/ on
first run, and the ModelScope 404 for BigVGAN is benign fallback noise.
- The WebUI binds 0.0.0.0 with no authentication.
* webui: adapt precision and QwenEmotion loading to available VRAM
The WebUI hardcoded use_qwen_emo=True and tied precision to --fp16, which
defaults off, so a small card got the most expensive combination available.
Measured on a 4090, peak reserved memory for a short utterance:
fp32 + QwenEmotion 8.15 GB (the previous default)
bf16 + QwenEmotion 6.54 GB
bf16, no QwenEmotion 5.48 GB
Re-run under a hard per-process cap via set_per_process_memory_fraction, the
old default does complete on an 8 GB budget -- but only by peaking at exactly
8.00 GB, i.e. with no headroom for the display output and driver overhead a
real 8 GB card also has to fund. The new path leaves ~2.5 GB spare and still
runs under a 6 GB cap (5.44 GB peak).
So below 10 GB, enable half precision and skip QwenEmotion, reusing the
threshold infer_v2_5.py already applies for text chunking. --qwen_emo forces it
back on for someone who knows their card can take it.
Skipping that model removes emotion-control-from-text, so gate the UI on it
too: mode 3 disappears from the experimental choices, its examples are filtered
out, and a preset saved with it resets to the default with a warning. Without
this gating, selecting mode 3 raises "use_emo_text=True requires QwenEmotion"
from infer_v2_5.py at inference time.
Verified all three configurations start and report the intended decision, with
process memory matching the table above.
* Gate use_bf16 on torch.cuda.is_bf16_supported() and reset is_experimental on fallback
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
---------
Co-authored-by: nanaoto <10inspiral@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
The recipe has landed, so the links no longer need to reference the pull
request. Verified https://recipes.vllm.ai/IndexTeam/IndexTTS-2.5 returns 200
and titles itself "IndexTeam/IndexTTS-2.5 | vLLM Recipes".
Seven occurrences across the English, Chinese, Japanese, Spanish and Arabic
READMEs.
Co-authored-by: nanaoto <10inspiral@gmail.com>
The 2.5 row's only Demos entry was index-tts2-5.github.io, which serves the
technical report rather than something you can talk to. Add the hosted studio
next to it so a reader can try the model without installing anything.
Verified https://modelscope.cn/studios/IndexTeam/IndexTTS-2.5 returns 200 and
titles itself "IndexTTS-2.5". English, Chinese, Japanese, Spanish and Arabic.
Co-authored-by: nanaoto <10inspiral@gmail.com>
* fix: repair Linux installs and README commands that fail on a clean machine
Verified on a Linux GPU box using a fresh .venv and a separate UV_CACHE_DIR,
since a warm cache hides every one of these.
pyproject.toml:
- triton-windows had no platform marker, so `uv sync --extra accel` (or
--all-extras) aborted on Linux: the package only publishes win_amd64 wheels.
Linux already gets `triton` as a torch dependency.
- Replace `no-build-isolation-package` with `extra-build-dependencies`. That
mode uses `.venv` itself as the build environment, and a fresh `.venv` has no
setuptools yet, so deepspeed failed with `ModuleNotFoundError: No module
named 'setuptools'` on any clean machine. Injecting torch with
`match-runtime` keeps the CUDA-enabled build inputs while leaving isolation
(and its setuptools) in place. It also lets flash-attn resolve a prebuilt
wheel from its own torch version and C++ ABI flag instead of invoking nvcc.
README (and the zh/ja/es/ar translations):
- `examples/09.wav` does not exist in the demo space and is not in
cases.jsonl; the on-demand download fetches `voice_09.wav`.
- `huggingface-hub` dropped its `cli` extra, so the install line warned.
- Example audio is fetched by the WebUI only, not by `infer_v2_5.py`, whose
default --prompt_wav is `examples/voice_01.wav`. Say so and give a one-liner
to fetch it.
- Drop `uv run indextts/infer_v2.py`: it is a benchmark loop hardcoded to
`checkpoints/`, so with the documented `checkpoints_2` layout it dies on
`vocab_file checkpoints/bpe.model does not exist`.
- `use_emo_text=True` needs `use_qwen_emo=True` at construction, which the
init example omits.
webui.py: title and arXiv badge were hardcoded to IndexTTS2, so the 2.5 UI
announced itself as 2 and linked the 2.0 paper.
* docs: install huggingface-hub without extras
`hf-xet` is an unconditional base dependency of huggingface-hub on x86_64,
arm64 and aarch64, not just an extra, so `[hf_xet]` selected nothing that the
plain package does not already pull in. The plain install still provides the
`hf`, `huggingface-cli` and `tiny-agents` executables.
Applies to the remaining seven translations and archived READMEs too, which
still carried the `[cli,hf_xet]` form whose `cli` extra no longer exists.
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: nanaoto <10inspiral@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* feat: default webui to IndexTTS-2.5
- webui.py: --version default 2 -> 2.5. The v2 default existed only
because the empty-checkpoints auto-download fallback was implemented
for v2 alone; 2.5 just exited. Now both versions auto-download their
own repo (IndexTTS-2 / IndexTTS-2.5) when checkpoints are incomplete.
- webui.py: version-aware required_files. v2 checks bpe.model etc.;
2.5 checks codec.pth + multilingual_zh_ja_yue_char_del.tiktoken
(bpe.model is v2-only, 2.5 uses tiktoken).
- model_download.py: ensure_config_available takes version, downloads
config.yaml from the matching repo instead of hardcoded IndexTTS-2.
- README: 2.5 launch is now 'uv run webui.py' (zero flags).
* Fix ensure_config_available version validation and sync localized READMEs
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
---------
Co-authored-by: nanaoto <10inspiral@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
- Add translated READMEs: docs/README_ja.md, docs/README_es.md,
docs/README_ar.md (structure, code blocks and tables kept identical
to the English README)
- Add WebUI locales: ja_JP.json, es_ES.json, ar_SA.json (110 keys
each, covering every i18n key used in webui.py)
- Fill 3 missing keys in en_US.json (删除/情感向量/语言)
- Add language switcher links to the README headers
Co-authored-by: nanaoto <10inspiral@gmail.com>
- Add light/dark transparent logo variants, switch via <picture>
prefers-color-scheme so the header no longer shows a white box in
GitHub dark mode
- Add stars / arXiv / Discord badges under the language links
- Reorganize both READMEs into a clear section order: header intro,
model zoo, news, demos, getting started, usage, evaluation,
community, citation, acknowledgements, license
- Fix BibTeX author separators (use 'and') and complete the 2.5
author list in the zh README
- Add missing duration_factor and vLLM sections to the zh README
- Drop git-lfs steps (examples download on demand from HF/ModelScope)
- Remove uv advocacy block
Three checks on split_text_by_tokens, driven through a stub that bypasses
__init__ so they need no checkpoints and run in milliseconds:
- annotations stay paired across segment boundaries
- every segment fits text_pos_embedding's capacity
- short text is returned untouched
The annotation case is the one with teeth. Run against the pre-53dad14 splitter
it reports 4 unpaired markers; with the fix, 0. Worth recording why the earlier
manual cases were weak: single-annotation inputs pass either way, because the
splitter breaks on punctuation and then repacks chunks up to the budget, which
usually rejoins the two halves. Only when an annotation lands on a boundary does
the break survive -- so the fixture carries 10 annotations across 5 repeated
blocks to hit it.
That also corrects the trigger described in 53dad14 and e8d4383: it is not "any
annotated input long enough to segment", it is "an annotation landing on a
segment boundary", which is rarer.
Ports two changes from the internal tree (git.bilibili.co main, f8d2304) and
fixes an interaction between them and the token splitter from e8d4383.
From the internal tree, in front.py: TextNormalizer.normalize now swaps
<word|pronunciation> annotations for alphabetic placeholders before running the
zh/en normalizers and restores them afterwards, so the normalizer no longer
expands digits inside an annotation (XING2 -> XING二). Also picks up its
--text_normalization argparse fix (type=bool never produces False).
The splitter needed a matching fix. apply_pronunciation_annotations runs before
segmentation and rewrites <going|G OW1 . IH0 NG> as
<|SPECIAL_TOKEN_1|>G OW1 . IH0 NG<|SPECIAL_TOKEN_1|>, whose payload contains a
period -- one of the splitter's break characters -- so a boundary could land
inside the pair:
in: 他要<|SPECIAL_TOKEN_1|>G OW1 . IH0 NG<|SPECIAL_TOKEN_1|>去上学。
out: ['他要<|SPECIAL_TOKEN_1|>G OW1 .', ' IH0 NG<|SPECIAL_TOKEN_1|>去上学。']
leaving each half with an unpaired marker. Each
<|SPECIAL_TOKEN_n|>...<|SPECIAL_TOKEN_n|> span is now an atomic piece that
punctuation and per-character splitting skip over. This only bit inputs long
enough to segment, which is why the long-text testing in e8d4383 missed it --
those samples had no annotations.
Checked against annotated English (period inside the payload), annotated Chinese,
multiple annotations in one input, and unannotated text: markers stay paired in
all four, and unannotated input segments as before. Repo tests: 7 passed.
2.5 never segmented text. infer_generator encoded the whole input into one
tensor of shape (1, N) and then looped `for seg_idx, sent in enumerate(text_tokens)`,
which iterates the batch dimension -- so it ran exactly once no matter how long
the input was. `split_segments` appears zero times in infer_v2_5.py, versus once
in infer_v2.py.
text_pos_embedding holds max_text_tokens + 2 = 602 entries, so anything past
~600 tokens indexed out of bounds and surfaced as
`CUDA error: device-side assert triggered`. Measured on an RTX 4090 with the 2.5
weights: 450 chars worked (450 tokens), 750 and 1200 chars both crashed. 2.0 took
the same 450-char input fine because it segments.
webui's "分句最大Token数" slider (max_text_tokens_per_segment) reached 2.5 only as
a print statement.
Adds split_text_by_tokens, which counts real tokens rather than characters --
max_chars is unreliable for the multilingual tokenizer, where the character/token
ratio varies by language. It breaks on punctuation, falls back to per-character
accumulation for a clause that alone exceeds the budget, then packs clauses back
up to the limit. The budget is min(max_text_tokens_per_segment, capacity - 2)
minus the language prefix, which is re-applied to every segment.
The loop now iterates the segments, and reuses infer_generator's existing
multi-segment machinery (wavs accumulation, insert_interval_silence) plus the
per-segment progress callback 2.0 already had.
Verified on an RTX 4090: 750 chars -> 132.6s audio, 1200 chars -> 213.3s, both
previously a hard crash; short text unchanged at 2.00s. Splitter unit-checked at
1200 chars -> 11 segments, max 117 tokens each, all within the 602 capacity.
The existing low_vram path (split_text_by_punctuation, max_chars=40) is untouched.
Note it only engages below 10 GB VRAM, so larger cards were the ones hitting this.
nanaoto