Commit Graph
7 Commits
Author SHA1 Message Date
Siddharth e5462f1ac8 captions: run Whisper in a Web Worker, move auto-captions button to timeline
Move model load and transcription off the renderer main thread into a
dedicated worker (transcribe.worker.ts + transcribeCore.ts) so the editor
UI no longer freezes during captioning. Relocate the auto-captions action
from the editor header into the timeline toolbar, and add the missing
it/pt-BR/ru/vi autoCaptions translations.
2026-06-01 20:42:41 -07:00
parse-nipandCursor 2ee00a7cfa fix(captioning): narrow ORT node shim, trim retries, and abort checks
Scope withoutNodeVersion to Transformers import/pipeline only; reapply trim-region filtering after ignore-trims retries; check AbortSignal after each slice inference before chunk processing.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-15 12:38:10 +08:00
parse-nipandCursor a0bcd9fe90 feat(editor): polish auto-caption loading toasts
Use one Sonner id for the full caption flow, show a distinct transcribing step, yield before Whisper so updates paint, match editor dark chrome on toasts, and keep pointer-events only on toast bodies. Add transcribing strings across locales.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-15 12:28:18 +08:00
parse-nipandCursor 3a48e54db1 fix: time-slice single-group phrase captions into lines
Phrase mode with one merged span now splits wrapCaptionTextByWordBounds
lines into separate CaptionSegments with even time allocation across
the phrase span (fallback when duration is too short for min spans).
Update unit test accordingly.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-14 10:51:31 +08:00
parse-nipandCursor 2b7e6f1677 fix: address PR review for auto captions and locales
- Gate trimSec segment shift when transcription used full buffer retry
- Restore UTF-8 autoCaptions strings (ar, es, fr, ja-JP, ko-KR, tr, zh-CN, zh-TW)
- Dedupe caption segments after grouping only; stricter chunk dedupe in transcribe
- Post-truncate duration, explicit consume merge for web demuxer, trim region shift cleanup
- Tests for caption annotation pipeline

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-14 10:39:08 +08:00
parse-nip 1c431fe473 fix: improve auto-caption timing and pause handling 2026-05-14 09:41:39 +08:00
parse-nipandCursor 7216f255a0 feat: add video transcriptions (in-browser auto captions)
Decode mono 16k audio from the editor video, run Whisper via Transformers.js, and insert linked text annotations with timing and layout helpers.

Adds Vite resolve shims for Node-only imports used by the model stack, optional leading-silence trim for the caption buffer, timeline gap reconciliation for auto-caption regions, and editor i18n. Raises Select content z-index so caption controls stay usable over the video surface.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-13 22:33:10 +08:00