AI Video Editing Flows

We cut the silence and added captions to three clips, start to finish

This is the whole job in one go: cut the quiet parts of a video, caption what is left, and burn the captions in. We chained auto-editor, faster-whisper and FFmpeg on three 60-second clips from the open film Tears of Steel, timed every step on a laptop CPU and checked the finished video.

Tested by us on Nov 6, 2026, on the clips and machine listed below. Every number on this page was measured in that run.

The result

It works end to end, with free tools, on a laptop CPU. We cut the quiet parts of three 60-second clips, transcribed what was left, burned the captions into the picture and checked the finished files. All 29 captions across the three clips were in the picture at their time, the burned-in file had exactly the length of the cut file, and the audio and video streams ended within 0.001 s of each other.

A whole run took a median of 91.8, 118.4 and 100.2 seconds for the three 60-second clips, roughly a third of it in each step: cutting, transcribing and burning in. Peak memory was 720 to 947 MB. The film has a music score under its dialogue, so the cut was small: 2.1, 5.4 and 0.7 seconds removed. On a clean talking-head recording the cut step would remove more, and we have not measured that yet.

The captions were the weak link, not the pipeline. Word error against the official subtitles was 21.5%, 17.2% and 18.9%, higher than the same model gave on the uncut clips (15.2%, 10.9% and 17.0% on the faster-whisper test page), and the captions came out as long blocks. Read the failure box before you use this on your own footage.

What failed or surprised us

  • Captions stayed on screen far too long. The model returned segments, and one segment became one caption: the longest stayed up 12.3 s on clip tos_a, 8.9 s on tos_b, and 4 of the 29 captions stayed up longer than 7 s in total. Several captions broke in the middle of a sentence (for example "We should" and "The" on tos_b). For Shorts-style captions you want two to four words at a time, so this pipeline needs a step that splits segments by word timing, which the faster-whisper output allows but our script did not do.
  • Transcribing after the cut gave worse words than transcribing before it, on all three clips. On tos_a the transcript read "giant robotic glass" where the subtitles have "giant robotic claws of death", "luck" for "locked", and it dropped the line "Whatever, Thom". We did not test a cleaner order (transcribe first, cut the transcript to match), so we cannot say that is better.
  • Two captions sat where the subtitles show nobody speaking ("you" on tos_a, "Not my fault!" on tos_c). The second may be real speech the subtitle file leaves out; we did not listen to settle it, so we count it as stray.
  • The cut step was the least useful on this footage: on a film with a music score, loudness cutting removed 0.7 to 5.4 s of 60 s. That matches the auto-editor test page and is a limit of the footage, not a failure of the tool.

What we tested and how

  • Tool: auto-editor, faster-whisper and FFmpeg in a row version auto-editor 29.3.1, faster-whisper 1.2.1, FFmpeg 8.1.2, Unlicense (auto-editor), MIT (faster-whisper), LGPL/GPL (FFmpeg build).
  • Settings compared: auto-editor (stricter threshold), then faster-whisper small.en, then FFmpeg burn-in (Shorts style).
  • Input clips: Tears of Steel, (CC) Blender Foundation, mango.blender.org, CC-BY 3.0. 60 s excerpts re-encoded to H.264 720p.
  • tos_a: 60 s starting at 20 s of the film, SHA-256 945262dcbe106f1e...
  • tos_b: 60 s starting at 180 s of the film, SHA-256 4097773b615cddfa...
  • tos_c: 60 s starting at 360 s of the film, SHA-256 3d3a8cd015310349...
  • Machine: AMD Ryzen 7 7735U (8 cores, 16 threads), integrated Radeon 680M, not used (CPU-only runs), 31.2 GB RAM, Windows 11 10.0.26300. The laptop was shared with other work, so timings are what a busy machine gives, not a clean benchmark.
  • Runs: one cold run, then three timed runs per clip and setting. Run on 2026-11-06.

The job: take a short talking clip, remove the quiet parts, add captions that match the speech, and end with one video file you can upload. We chained three free tools in one small Python script, listed under Reproduce it: auto-editor with its stricter threshold (audio:threshold=8%), faster-whisper with the small.en model on the cut clip, then FFmpeg burning the captions in with the Shorts style from our burn-in test. Each captioned cue is one transcribed segment.

We ran the whole script as one process tree: one cold run, then three timed runs per clip, measuring wall time, peak memory of the tree and the CPU load just before each run. The script prints how long each step took, and we report the median of the three timed runs for each. Quality was measured on the finished file of the last run: the length against the cut file, every caption checked in the picture, word error against the film's official subtitles, and the longest caption on screen. We also looked at frames of the finished videos ourselves.

Measurements by clip

The whole job, per clip

ClipInput / cut / final length (s)Cold run (s)Median run (s)Peak memory (MB)Output
tos_a60 / 57.92 / 57.9291.691.8728h264 yuv420p 1280x534
tos_b60 / 54.58 / 54.58116.6118.4947h264 yuv420p 1280x534
tos_c60 / 59.29 / 59.29100.5100.2720h264 yuv420p 1280x534

Where the time went (median of the three timed runs)

Clip1. Cut silence (s)2. Transcribe (s)3. Burn in (s)
tos_a31.525.133.2
tos_b37.438.140
tos_c35.925.637.9

Did the captions come out right?

ClipCaptions in the picture / checkedWord error vs the film subtitlesWords (subtitles / ours)Longest caption on screen (s)Captions over 7 sCaptions where nobody speaks
tos_a7 / 721.5%79 / 7412.31 of 71
tos_b6 / 617.2%64 / 648.93 of 60
tos_c16 / 1618.9%53 / 5230 of 161

Reproduce it

auto-editor (stricter threshold), then faster-whisper small.en, then FFmpeg burn-in (Shorts style)
"""Cut the silence from a clip, caption what is left, and burn the captions in. Three free command-line tools in a row.

usage: python pipeline_cut_caption.py clip.mp4 workdir
Needs auto-editor installed next to this Python, faster-whisper installed, and ffmpeg (set FFMPEG, default: ffmpeg).
Prints one JSON line with the seconds each stage took.
"""
import json
import os
import subprocess
import sys
import time

import numpy as np
from faster_whisper import WhisperModel

FFMPEG = os.environ.get("FFMPEG", "ffmpeg")
AUTO_EDITOR = os.path.join(os.path.dirname(sys.executable), "auto-editor.exe")
clip, work = sys.argv[1], sys.argv[2]
os.makedirs(work, exist_ok=True)
cut, srt, final = os.path.join(work, "cut.mp4"), os.path.join(work, "cut.srt"), os.path.join(work, "final.mp4")
times = {}


def stamp(k, t0):
    times[k] = round(time.perf_counter() - t0, 2)


def srt_time(t):
    ms = int(round(t * 1000))
    return f"{ms // 3600000:02d}:{ms // 60000 % 60:02d}:{ms // 1000 % 60:02d},{ms % 1000:03d}"


# 1. cut the quiet parts by loudness
t0 = time.perf_counter()
subprocess.run([AUTO_EDITOR, clip, "--edit", "audio:threshold=8%", "-o", cut, "--no-open"], check=True, capture_output=True)
stamp("cut_s", t0)

# 2. transcribe the cut clip (audio decoded to 16 kHz mono first, see the faster-whisper test page)
t0 = time.perf_counter()
raw = subprocess.run([FFMPEG, "-v", "error", "-i", cut, "-ac", "1", "-ar", "16000", "-f", "f32le", "-"], capture_output=True, check=True).stdout
audio = np.frombuffer(raw, dtype=np.float32)
model = WhisperModel("small.en", device="cpu", compute_type="int8", cpu_threads=8)
segments, _ = model.transcribe(audio, language="en", word_timestamps=True, vad_filter=False, beam_size=5)
segs = [{"start": s.start, "end": s.end, "text": s.text.strip()} for s in segments]
with open(srt, "w", encoding="utf-8") as f:
    for i, s in enumerate(segs, 1):
        f.write(f"{i}\n{srt_time(s['start'])} --> {srt_time(s['end'])}\n{s['text']}\n\n")
json.dump(segs, open(os.path.join(work, "cut.json"), "w", encoding="utf-8"), ensure_ascii=False)
stamp("transcribe_s", t0)

# 3. burn the captions in (Shorts style)
t0 = time.perf_counter()
style = "FontName=Arial,FontSize=22,Bold=1,Outline=2,Shadow=0,MarginV=60"
subprocess.run([FFMPEG, "-v", "error", "-y", "-i", "cut.mp4", "-vf", f"subtitles=cut.srt:force_style='{style}'", "-c:v", "libx264", "-crf", "20",
                "-preset", "medium", "-c:a", "copy", "final.mp4"], check=True, cwd=work)
stamp("burn_in_s", t0)
print(json.dumps(times))

# run: python pipeline_cut_caption.py clip.mp4 workdir

This is the exact script we ran. It needs auto-editor, faster-whisper and numpy installed in one Python environment, and ffmpeg on your path (or the FFMPEG environment variable set). Run it once per clip. The first run downloads the small.en model.

Method and limits

  • Three 60-second clips of one film, with a music score and effects under the dialogue. A clean talking-head recording would cut and transcribe differently; that is on our list.
  • The cut step uses a stricter threshold than auto-editor's default, because the default cut almost nothing on this footage (see the auto-editor test).
  • Word error is measured against the official English subtitles of the film, which shorten and reword speech; part of every error rate is that difference, as explained on the faster-whisper test page. All subtitle lines were inside the kept parts of the clips, so no line was dropped from the reference.
  • Captions are checked as present in the picture, not as well timed: we did not measure caption timing against speech here.
  • 720p H.264 at CRF 20, preset medium, CPU only, on a laptop that was shared with other work. Another machine will give other times.
  • Footage: Tears of Steel, (CC) Blender Foundation, mango.blender.org, used under CC-BY 3.0.

Page checked on Nov 4, 2026.