mirror of
https://github.com/index-tts/index-tts.git
synced 2026-08-28 23:01:17 +08:00
main
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Souptik Chakraborty
|
ee40fa7d6c |
fix: normalize waveforms before torchaudio.save to avoid saturated WAVs (#773)
torchaudio 2.9 moved save() onto TorchCodec, whose compatibility shim converts non-float32 input with a bare src.float() and no rescaling, then treats the result as [-1, 1] audio. Every save site passed a PCM-scale int16 tensor, so on 2.9+ roughly 99.99% of frames clipped to full scale with no exception and no warning. Add save_pcm_wav() in indextts/utils/common.py, which normalizes PCM-scale input to float32 [-1, 1] and pins encoding="PCM_S"/bits_per_sample=16 on the torchaudio versions that still honour them. Pinning matters: below 2.9 the WAV subtype is derived from the input dtype, so normalizing alone would silently switch output from 16-bit PCM to 32-bit float. Above 2.9 the two arguments are ignored and warn, so they are omitted there. Route all six save sites in indextts/ and backends/trt/ through it. Verified on torchaudio 2.7.1, 2.8.0, 2.9.0 and 2.11.0: identical 16-bit PCM output, 0% clipped frames. Before the change 2.9.0 and 2.11.0 produced only 3 distinct sample values with 99.9875% of frames at full scale. Fixes #724 |
||
|
nanaoto
|
583d6d4c83 | feat: IndexTTS-2.5 release (#738) | ||
|
nanaoto
|
90ca4d6082 |
feat: add TensorRT inference backend under backends/trt/ (#733)
* feat: add TensorRT inference backend under backends/trt/
Adds a TensorRT/TensorRT-LLM inference backend for IndexTTS2, taken from
MuyangDu/index-tts deploy/ (Faster IndexTTS-2, arXiv:2607.21042) and placed
under backends/ so future inference backends can sit alongside it.
The GPT stage runs on TensorRT-LLM; the other nine components (BigVGAN, DiT,
speech semantic encoder, semantic codec, speaker/emotion perceiver
conditioners, latent projector, CAMPPlus, length regulator) run on plain
TensorRT engines exported via ONNX.
Directory is backends/trt/ rather than backends/tensorrt/ on purpose:
scripts/setup_env.sh puts the project root on PYTHONPATH, and 16 files in this
tree do `import tensorrt`. A tensorrt/ package anywhere on that path shadows
the real TensorRT library.
Self-contained: ships its own pyproject.toml and venv (Python 3.12,
torch>=2.7.1,<2.8, transformers~=4.51.1, numpy<2), which TensorRT-LLM 0.21.0
requires and which conflict with the root lockfile. Nothing outside backends/
changes, so the existing PyTorch path (indextts2 synth / webui.py, with
--accel / --torch-compile / --deepspeed / --fp16) is untouched.
* refactor: drop untested Docker and native-Triton serving paths
Removes the parts of the upstream backend that were never run here, so
everything shipped has been exercised end-to-end on real hardware:
- Dockerfile.triton / Dockerfile.pytriton (+ dockerignores)
- scripts/build_image.sh, scripts/entrypoint.sh
- serving/model_repository/ (native Triton Python-backend models + config.pbtxt)
The PyTriton path (serving/triton_server.py + triton_client.py) stays: it needs
no container, since nvidia-pytriton bundles the server binary, and both
non-streaming and streaming modes were verified on an RTX 4090.
These two serving paths are independent implementations — model_repository's
model.py does not import pytriton — so nothing that remains references the
deleted files. README's Docker and config.pbtxt sections are replaced with the
PyTriton usage that was actually tested, including a note that the server binds
0.0.0.0 with no authentication.
pyproject.toml is unchanged: nvidia-pytriton and tritonclient serve the
remaining path, and onnxruntime-gpu is used by two export scripts for
verification.
* docs: make upstream attribution explicit in backend README
The README was copied from MuyangDu/index-tts and is written in that project's
first person ("our paper", "we"). Adds a leading attribution block naming the
authors, linking the source tree and paper, stating what was changed here, and
clarifying that "we"/"our" refers to them rather than the IndexTTS team.
The upstream Citation and License sections were already preserved verbatim.
* docs: document the host OpenMPI requirement
Upstream never mentions MPI anywhere in deploy/ — no docs, no dependency in
pyproject.toml. It doesn't need to: its Dockerfiles build on
nvcr.io/nvidia/tritonserver, which bundles HPC-X OpenMPI (verified in the
24.11-py3 image: libmpi.so.40 at /opt/hpcx/ompi/lib, orted at
/usr/local/mpi/bin, Open MPI 4.1.7rc1). Anyone following the container path
inherits it for free.
Since this PR keeps only the manual path, that dependency is now the user's to
satisfy, and hitting it looks like an unrelated failure
(RuntimeError: cannot load MPI library from `import tensorrt_llm`). Adds it to
Prerequisites with the install command and a note that Intel MPI does not
substitute.
* feat: add preflight check and one-shot entry point for the TRT backend
Nothing in the backend validated its environment: setup_env.sh only exported
variables, and the build scripts only checked for files. Every missing system
dependency surfaced as a crash partway through a long step, which is how the
OpenMPI requirement cost hours to diagnose.
scripts/preflight.sh checks venv, python 3.12, PYTHONPATH, libmpi.so.40, orted,
libpython, tensorrt/torch/tensorrt_llm imports, CUDA availability, free VRAM,
and which build artifacts exist. Each failure prints the command that fixes it.
Exits 1 on blocking problems, 0 when only artifacts are missing (i.e. ready to
build).
scripts/run.sh activates the venv, sources setup_env.sh, locates OpenMPI
(honouring OPENMPI_PREFIX, then searching ~/local-mpi/root/usr, the distro
paths, and /opt/hpcx/ompi so it also works inside the NVIDIA Triton images),
runs preflight, then dispatches to infer/serve/client/check/python.
Verified on an RTX 4090 from a bare `env -i` shell: `run.sh check` passes all
checks, and `run.sh infer` produces audio (RTF 0.2067, peak 16327) with nothing
sourced beforehand.
One bug found while testing this: judging the tensorrt_llm import by whether
stderr was empty reported a false failure, since a healthy import warns about
TORCH_CUDA_ARCH_LIST. It now checks the exit status.
* refactor: collapse the backend scripts into one entry point
Replaces scripts/preflight.sh and scripts/run.sh with a single
backends/trt/run.sh exposing four commands: check, build, infer, serve.
`--help` documents them, and `infer --help` / `serve --help` forward to the
underlying scripts.
`build` chains export_models.sh, convert_checkpoint.sh and build_engines.sh, so
the first-time setup is one command instead of three. The per-step scripts stay
for redoing a single stage.
Verified on an RTX 4090 from a bare `env -i` shell: --help works without any
environment, check reports ready, an unknown command exits 1 with usage, and
infer produces audio (RTF 0.2134).
* docs: record the verified environment and the single-venv finding
Adds a table of what was actually run (RTX 4090 24GB, python 3.12.11,
tensorrt 10.11.0.33 / tensorrt-llm 0.21.0, torch 2.7.1+cu128, OpenMPI 4.1.2,
fp16 with MAX_BATCH_SIZE=1, RTF 0.19-0.21, both PyTriton modes) and states what
was not: MAX_BATCH_SIZE > 1, int8/int4, multi-GPU.
Also records the outcome of testing whether this backend could share the
project's main venv instead of an isolated one. It resolves (228 packages,
python 3.10.14) but downgrades python, torch, numpy and transformers, and would
revert #720 and #721. Python can only be 3.10 there, since TensorRT-LLM 0.21.0
ships cp310/cp312 wheels only while the project caps python at <3.12 for
llvmlite. Only resolution was tested -- the merged environment was never
installed or run, and flash-attn (the accel extra, built for torch 2.8) was
outside that test.
No behaviour change; the backend keeps its own venv.
* fix: address three code-correctness issues flagged in PR #733 review (#734)
* chore: stage PR #733 files for review fixes
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
* fix: address PR #733 review comments in 3 files
- convert_gpt_checkpoint.py: derive config path from --model_dir
instead of hardcoding "checkpoints/config.yaml"
- triton_client.py: skip all-zero padded audio chunks in streaming mode
- text_processing.py: convert for-loop to while-loop so i += 1
actually skips the consumed lookahead token
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
* Fix world_size, stride validation, and speaker_audio size limit per review comments
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
---------
Co-authored-by: nanaoto <10inspiral@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: nanaoto <19526637+nanaoto@users.noreply.github.com>
|