We edited three talking clips in one pass: cut, word-timed captions, loudness, burn-in
One script, four free tools: cut the quiet parts, get word times, group the words into three-word captions, level the loudness and burn the captions in. We ran it on three 60-second clips of Tears of Steel.
Tested by us on Nov 6, 2026, on the clips and machine listed below. Every number on this page was measured in that run.
The result
It works in one script, with free tools, on a laptop CPU. A whole run took a median of 68.4, 77.8 and 72.8 seconds for the three 60-second clips, with peak memory of 947 MB or less. All 78 of 78 captions were in the picture at their time. The three-word grouping fixed the fault from our earlier test: the longest caption is now 6.4 s, where the earlier version held one for 12.3 s, and none stayed up longer than 7 s.
Two things need a caution. The single-pass loudness step did not land on its target: the final files measured -16, -13.9 and -16.8 LUFS against a target of -16. We reran only that step with two passes of loudnorm (measure first, then apply the measured values), and the finals then measured -15.9, -16 and -16. And the words are the same as before, with a word error of 21.5%, 17.2% and 18.9% against the film's subtitles. Read the failure box before you use this on your own footage.
What it made
Footage: Tears of Steel, (c) Blender Foundation, mango.blender.org, CC-BY 3.0. These are files we made from it; they are shown under the same licence terms.
Six-second excerpts of the finished files, from the run with the two-pass loudness step. Press play for the sound. Nothing loads until you do.
What failed or surprised us
- Loudness missed its target on two of three clips with one pass. We asked for -16 LUFS with one pass of FFmpeg's loudnorm. The inputs measured -16.8, -12.4 and -12.7 LUFS and the final files -16, -13.9 and -16.8. We then ran the usual remedy, a measuring pass first and the measured values in a second pass with linear=true, on the same three cuts. The finals measured -15.9, -16 and -16 LUFS, all within 0.2 of the target. For 2 of 3 clips loudnorm applied one fixed gain; for the other it reported falling back to its dynamic mode and we did not investigate why. The measuring pass took 2.9 to 3 s per clip (one run each, outside the timed runs). The earlier output files had been deleted, so we rebuilt each cut with the same auto-editor command (same lengths) and reran only the loudness and burn-in step.
- A word's timing can be badly wrong. On tos_a the transcript gives the word "robots" the span 23.4 to 28.8 s, which is 5.5 s, so the caption "done robots memory" stays up for 6.4 s. That word is also a mishearing. The caption grouping cannot fix timing the transcriber got wrong, and this is why the longest caption is still 6.4 s.
- The words are no better than before. We transcribe the cut clip, as in the earlier test, and the word error is unchanged. Short captions made the errors easier to see on screen: "Who'd that lip?" and "done robots memory" are in the frames we checked.
- The cut step removed little: 2.1, 5.4 and 0.7 s of 60, because the film has music under its dialogue. A clean talking-head recording would cut more, and we have not measured that.
What we tested and how
- Tool: auto-editor, faster-whisper and FFmpeg, one pass version auto-editor 29.3.1, faster-whisper 1.2.1, FFmpeg 8.1.2, Unlicense (auto-editor), MIT (faster-whisper), LGPL/GPL (FFmpeg build).
- Settings compared: auto-editor (threshold 8%), faster-whisper small.en with word times, FFmpeg loudnorm and burn-in, 3-word captions in the Shorts style.
- Input clips: Tears of Steel, (CC) Blender Foundation, mango.blender.org, CC-BY 3.0. 60 s excerpts re-encoded to H.264 720p.
tos_a: 60 s starting at 20 s of the film, SHA-256945262dcbe106f1e...tos_b: 60 s starting at 180 s of the film, SHA-2564097773b615cddfa...tos_c: 60 s starting at 360 s of the film, SHA-2563d3a8cd015310349...
- Machine: AMD Ryzen 7 7735U (8 cores, 16 threads), integrated Radeon 680M, not used (CPU-only runs), 31.2 GB RAM, Windows 11 10.0.26300. The laptop was shared with other work, so timings are what a busy machine gives, not a clean benchmark.
- Runs: one cold run, then three timed runs per clip and setting. Run on 2026-11-06.
The job: take a short clip, remove the quiet parts, get word times, show the words three at a time, level the loudness and end with one file you can upload. We chained four free tools in one Python script, listed under Reproduce it: auto-editor at its stricter 8% threshold, faster-whisper small.en with word times on the cut clip, captions grouped to at most three words (a new group starts after a half-second pause or a sentence end), then FFmpeg's loudnorm (-16 LUFS) and subtitles filter in the Shorts style. The script below is the single-pass version we timed; the two-pass loudness step we added afterwards is under Reproduce it.
We ran the whole script as one process tree: one cold run, then three timed runs per clip, measuring wall time, peak memory and the CPU load just before each run. The script prints the time of each step. We measured integrated loudness (EBU R128) of the input, of the cut file and of the final file, checked every caption in the picture, measured word error against the official subtitles that survive the cut, and looked at frames of the finished videos ourselves.
Measurements by clip
The whole job, per clip
| Clip | Input / cut / final length (s) | Cold run (s) | Median run (s) | Peak memory (MB) | Output | SHA-256 of the output (first 16) |
|---|---|---|---|---|---|---|
| tos_a | 60 / 57.92 / 58 | 59.7 | 68.4 | 779 | h264 yuv420p 1280x534 | 4e892566dc08fdc8 |
| tos_b | 60 / 54.58 / 54.6 | 78.9 | 77.8 | 947 | h264 yuv420p 1280x534 | 5f35915b9318e217 |
| tos_c | 60 / 59.29 / 59.4 | 66.1 | 72.8 | 767 | h264 yuv420p 1280x534 | a47b3f2d63fb085d |
Where the time went (median of the three timed runs)
| Clip | 1. Cut silence (s) | 2. Transcribe with word times (s) | 3. Loudness and burn-in (s) | CPU load just before the four runs (%) |
|---|---|---|---|---|
| tos_a | 22.8 | 19.7 | 23.6 | 5, 1, 1, 12 |
| tos_b | 25.1 | 24.7 | 25.5 | 9, 12, 1, 14 |
| tos_c | 26.1 | 19.6 | 27.8 | 6, 28, 25, 19 |
Loudness before and after (integrated LUFS, EBU R128)
| Clip | Input | After the cut | Final file |
|---|---|---|---|
| tos_a | -16.8 | -16.8 | -16 |
| tos_b | -12.4 | -12.5 | -13.9 |
| tos_c | -12.7 | -12.8 | -16.8 |
Did the captions come out right?
| Clip | Captions in the picture / checked | Word error vs the film subtitles | Words (subtitles / ours) | Longest caption on screen (s) | Median caption (s) | Captions over 7 s |
|---|---|---|---|---|---|---|
| tos_a | 26 / 26 | 21.5% | 79 / 74 | 6.42 | 0.86 | 0 of 26 |
| tos_b | 28 / 28 | 17.2% | 64 / 64 | 1.38 | 0.84 | 0 of 28 |
| tos_c | 24 / 24 | 18.9% | 53 / 52 | 1.94 | 0.58 | 0 of 24 |
Reproduce it
"""One pass over a talking clip: cut the quiet parts, caption in short word groups, level the loudness, burn in. Four free tools in a row.
usage: python pipeline_one_pass.py clip.mp4 workdir
Needs auto-editor installed next to this Python, faster-whisper installed, and ffmpeg (set FFMPEG, default: ffmpeg).
Prints one JSON line with the seconds each stage took.
"""
import json
import os
import subprocess
import sys
import time
import numpy as np
from faster_whisper import WhisperModel
FFMPEG = os.environ.get("FFMPEG", "ffmpeg")
AUTO_EDITOR = os.path.join(os.path.dirname(sys.executable), "auto-editor.exe")
clip, work = sys.argv[1], sys.argv[2]
os.makedirs(work, exist_ok=True)
cut, srt, final = os.path.join(work, "cut.mp4"), os.path.join(work, "cut.srt"), os.path.join(work, "final.mp4")
times = {}
MAX_WORDS, MAX_GAP, MIN_HOLD = 3, 0.5, 0.35
def stamp(k, t0):
times[k] = round(time.perf_counter() - t0, 2)
def srt_time(t):
ms = int(round(t * 1000))
return f"{ms // 3600000:02d}:{ms // 60000 % 60:02d}:{ms // 1000 % 60:02d},{ms % 1000:03d}"
# 1. cut the quiet parts by loudness
t0 = time.perf_counter()
subprocess.run([AUTO_EDITOR, clip, "--edit", "audio:threshold=8%", "-o", cut, "--no-open"], check=True, capture_output=True)
stamp("cut_s", t0)
# 2. transcribe the cut clip with word times
t0 = time.perf_counter()
raw = subprocess.run([FFMPEG, "-v", "error", "-i", cut, "-ac", "1", "-ar", "16000", "-f", "f32le", "-"], capture_output=True, check=True).stdout
audio = np.frombuffer(raw, dtype=np.float32)
model = WhisperModel("small.en", device="cpu", compute_type="int8", cpu_threads=8)
segments, _ = model.transcribe(audio, language="en", word_timestamps=True, vad_filter=False, beam_size=5)
words = [{"start": w.start, "end": w.end, "text": w.word.strip()} for s in segments for w in s.words]
stamp("transcribe_s", t0)
# 3. group words into captions of at most 3 words; break on a pause or sentence end
t0 = time.perf_counter()
groups, cur = [], []
for w in words:
if cur and (len(cur) >= MAX_WORDS or w["start"] - cur[-1]["end"] > MAX_GAP or cur[-1]["text"][-1:] in ".?!"):
groups.append(cur)
cur = []
cur.append(w)
if cur:
groups.append(cur)
caps = []
for i, g in enumerate(groups):
a, b = g[0]["start"], max(g[-1]["end"], g[0]["start"] + MIN_HOLD)
if i + 1 < len(groups):
b = min(b, groups[i + 1][0]["start"])
caps.append({"start": a, "end": b, "text": " ".join(w["text"] for w in g)})
with open(srt, "w", encoding="utf-8") as f:
for i, c in enumerate(caps, 1):
f.write(f"{i}\n{srt_time(c['start'])} --> {srt_time(c['end'])}\n{c['text']}\n\n")
json.dump({"words": words, "captions": caps}, open(os.path.join(work, "cut.json"), "w", encoding="utf-8"), ensure_ascii=False)
stamp("group_s", t0)
# 4. level the loudness (EBU R128, one pass) and burn the captions in
t0 = time.perf_counter()
style = "FontName=Arial,FontSize=22,Bold=1,Outline=2,Shadow=0,MarginV=60"
subprocess.run([FFMPEG, "-v", "error", "-y", "-i", "cut.mp4", "-vf", f"subtitles=cut.srt:force_style='{style}'", "-af", "loudnorm=I=-16:TP=-1.5:LRA=11",
"-c:v", "libx264", "-crf", "20", "-preset", "medium", "-c:a", "aac", "-b:a", "160k", "final.mp4"], check=True, cwd=work)
stamp("polish_and_burn_in_s", t0)
print(json.dumps(times))
# run: python pipeline_one_pass.py clip.mp4 workdirThis is the exact script we ran for the timings and the caption checks. It needs auto-editor, faster-whisper and numpy in one Python environment and ffmpeg on your path (or the FFMPEG variable set). Run it once per clip. The first run downloads the small.en model.
The two-pass loudness step replaces the last FFmpeg call. First measure the cut file (audio only), then apply the measured values. Replace the placeholders with what the commands print: measured_I, measured_LRA and measured_TP from the ebur128 line, measured_thresh and offset from the loudnorm JSON.
ffmpeg -nostats -i cut.mp4 -vn -af "loudnorm=I=-16:TP=-1.5:LRA=11:print_format=json" -f null -
ffmpeg -nostats -i cut.mp4 -vn -af ebur128=peak=true -f null -ffmpeg -y -i cut.mp4 -vf "subtitles=cut.srt:force_style='FontName=Arial,FontSize=22,Bold=1,Outline=2,Shadow=0,MarginV=60'" -af "loudnorm=I=-16:TP=-1.5:LRA=11:measured_I=<input_i>:measured_LRA=<input_lra>:measured_TP=<input_tp>:measured_thresh=<input_thresh>:offset=<target_offset>:linear=true" -c:v libx264 -crf 20 -preset medium -c:a aac -b:a 160k final.mp4One thing we changed: the integrated loudness we passed as measured_I is the reading of FFmpeg's ebur128 filter, not loudnorm's own. On the podcast test loudnorm's own reading differed from ebur128 by up to 1 LU and the result missed by that much; see that test. LRA is raised to the measured value when it is above 11, because loudnorm only uses linear mode when the measured range is not above the target range.
Method and limits
- Three 60-second clips of one film, with a music score and effects under the dialogue. A clean talking-head recording would cut and transcribe differently.
- Word error is measured against the film's official English subtitles, which shorten and reword speech, so part of every error rate is that difference (see the faster-whisper test).
- Captions are checked as present in the picture, not as well timed against the speech, except through the word-timing failure above.
- 720p H.264 at CRF 20, CPU only, on a laptop shared with other work. The CPU load before each run is in the table, and the runs were not run at the same time as other tests of ours. Another machine will give other times.
- Footage: Tears of Steel, (CC) Blender Foundation, mango.blender.org, used under CC-BY 3.0.
Page checked on Nov 4, 2026.