We cut a vertical short out of three 60-second clips and measured the result
Turning a longer video into one vertical short with free tools: faster-whisper picks the stretch with the most speech, FFmpeg makes it 1080x1920, burns in three-word captions and levels the loudness. We ran it on three 60-second clips of Tears of Steel.
Tested by us on Nov 6, 2026, on the clips and machine listed below. Every number on this page was measured in that run.
The result
It makes a usable vertical file from each clip, and the pick is a rule, not an editor. From each 60-second clip the script took a stretch of 15.5, 17.9 and 15.5 seconds and wrote a 1080x1920 H.264 file. A whole run took a median of 21.9, 28.4 and 24.6 seconds, with peak memory under 1.5 GB. All 44 of 44 captions were in the picture.
The stretch it picks is the one with the most words per second (3.4, 1.9 and 1.6, against 1.3, 1.1 and 0.8 for the whole clip). That finds the densest talk. It does not judge whether it is interesting, and it cannot find a hook.
What it made
Footage: Tears of Steel, (c) Blender Foundation, mango.blender.org, CC-BY 3.0. These are files we made from it; they are shown under the same licence terms.
The first eight seconds of each vertical short, scaled down for the page, from the run with the two-pass loudness step. Press play for the sound. Nothing loads until you do.
What failed or surprised us
- One pick starts mid-sentence. On tos_c the best stretch began at 0.0 s, the start of our clip, and its first word is lower case ("and"). We cut whole transcript segments, so the edge is only as clean as the segments are. 2 of 3 picks start on a capital letter and end on a full stop, question mark or exclamation mark.
- Loudness missed its target with one pass. We asked for -14 LUFS with one pass of loudnorm. The picked stretches measured -16.3, -13.5 and -12.1 LUFS before and the shorts -13.5, -12.7 and -12.4 after. All three ended louder than the target, and the third, which started at -12.1, barely moved.
- Two passes helped, but not at first. We reran only the loudness step with a measuring pass and linear=true. Using loudnorm's own reading as the measured value, the shorts measured -14, -15 and -14.6 LUFS, up to 1 LU under the target, because loudnorm read the stretches at -16.1, -12.5 and -11.6 where the ebur128 filter we check with read -16.3, -13.5 and -12.1. We do not know why the two meters differ. Passing the ebur128 reading as the measured value gave -14, -13.6 and -14 LUFS: two within 0.1 of the target and one 0.4 above it, where loudnorm reported falling back to its dynamic mode (1 of 3 clips ran in linear mode). The measuring pass took under a second per clip and the second pass about 8 s (one run each, outside the timed runs).
- Centre-cropping a wide film frame to 9:16 would throw away about three quarters of the picture, so we used a different layout: the clip at full width over a blurred, enlarged copy of itself. The picture ends up a thin strip in the middle of the vertical frame, which suits wide footage and wastes space on 16:9 footage with a face in the middle. We did not test a face-tracking crop.
- The words are as good as the model: word error against the film's subtitles inside the stretch was 13.7%, 12.1% and 12.5%, on 50, 33 and 24 words.
What we tested and how
- Tool: faster-whisper and FFmpeg: long video to a vertical short version faster-whisper 1.2.1 (small.en), FFmpeg 8.1.2, MIT (faster-whisper), LGPL/GPL (FFmpeg build) licence.
- Settings compared: faster-whisper word times on the whole clip, pick the densest 15 to 25 s of whole segments, blurred-background 9:16 layout, 3-word captions, loudnorm to -14 LUFS, FFmpeg encode.
- Input clips: Tears of Steel, (CC) Blender Foundation, mango.blender.org, CC-BY 3.0. 60 s excerpts re-encoded to H.264 720p.
tos_a: 60 s starting at 20 s of the film, SHA-256945262dcbe106f1e...tos_b: 60 s starting at 180 s of the film, SHA-2564097773b615cddfa...tos_c: 60 s starting at 360 s of the film, SHA-2563d3a8cd015310349...
- Machine: AMD Ryzen 7 7735U (8 cores, 16 threads), integrated Radeon 680M, not used (CPU-only runs), 31.2 GB RAM, Windows 11 10.0.26300. The laptop was shared with other work, so timings are what a busy machine gives, not a clean benchmark.
- Runs: one cold run, then three timed runs per clip and setting. Run on 2026-11-06.
The job: take a longer video and make one vertical short. Four free steps in one script, listed under Reproduce it: faster-whisper small.en transcribes the whole clip with word times; the script takes the stretch of whole transcript segments between 15 and 25 seconds with the most words per second; FFmpeg cuts it, builds the 1080x1920 layout, burns in captions of at most three words and levels the loudness to -14 LUFS in one command. The timed script is single-pass; the two-pass loudness step is under Reproduce it.
We ran the script as one process tree: one cold run and three timed runs per clip, with peak memory and the CPU load before each. We checked every caption in the picture against the same layout without captions, measured loudness before and after, measured word error against the official subtitles that fall inside the stretch, and looked at frames from every short ourselves. Our clips are one-minute excerpts, so the pick is only a choice within a minute. A long podcast would need more transcription time, in rough proportion to its length, which we did not measure.
Measurements by clip
The whole job, per clip
| Clip | Cold run (s) | Median run (s) | Peak memory (MB) | Output | Short length (s) | SHA-256 of the output (first 16) |
|---|---|---|---|---|---|---|
| tos_a | 20.7 | 21.9 | 1451 | h264 yuv420p 1080x1920 | 15.6 | 5b8a44f754b707ad |
| tos_b | 27 | 28.4 | 1478 | h264 yuv420p 1080x1920 | 17.96 | ff8a6bca6e1e75cd |
| tos_c | 25.7 | 24.6 | 1476 | h264 yuv420p 1080x1920 | 15.5 | b1bcb228973d239e |
Where the time went (median of the three timed runs)
| Clip | 1. Transcribe the whole clip (s) | 2. Pick the stretch (s) | 3. Vertical layout, captions, loudness, encode (s) |
|---|---|---|---|
| tos_a | 11.8 | 0 | 9.1 |
| tos_b | 14.1 | 0 | 13.4 |
| tos_c | 13.1 | 0 | 10.8 |
The pick
| Clip | Stretch taken (s in the clip) | Words per second, pick / whole clip | Starts on a capital letter | Ends on a full stop, ? or ! |
|---|---|---|---|---|
| tos_a | 7.2 to 22.7 | 3.4 / 1.28 | yes | yes |
| tos_b | 10.8 to 28.7 | 1.9 / 1.05 | yes | yes |
| tos_c | 0 to 15.5 | 1.59 / 0.85 | no | yes |
The short itself
| Clip | Captions in the picture / checked | Word error vs the film subtitles | Longest caption (s) | Loudness before / after (LUFS) |
|---|---|---|---|---|
| tos_a | 19 / 19 | 13.7% | 1.48 | -16.3 / -13.5 |
| tos_b | 14 / 14 | 12.1% | 1.26 | -13.5 / -12.7 |
| tos_c | 11 / 11 | 12.5% | 1.02 | -12.1 / -12.4 |
Reproduce it
"""Long video to one vertical short: transcribe, pick the best 20-second stretch, make it 9:16, caption it, level the sound.
usage: python pipeline_podcast_clip.py clip.mp4 workdir
Needs faster-whisper, numpy and ffmpeg (set FFMPEG, default: ffmpeg). Prints one JSON line with the seconds each stage took.
The pick is a rule, not a judgement of interest: among windows made of whole transcript segments, 15 to 25 seconds long, take
the one with the most words per second of speech (silence inside the window counts against it).
"""
import json
import os
import subprocess
import sys
import time
import numpy as np
from faster_whisper import WhisperModel
FFMPEG = os.environ.get("FFMPEG", "ffmpeg")
clip, work = sys.argv[1], sys.argv[2]
os.makedirs(work, exist_ok=True)
times = {}
MIN_LEN, MAX_LEN = 15.0, 25.0
MAX_WORDS, MAX_GAP, MIN_HOLD = 3, 0.5, 0.35
def stamp(k, t0):
times[k] = round(time.perf_counter() - t0, 2)
def srt_time(t):
ms = int(round(t * 1000))
return f"{ms // 3600000:02d}:{ms // 60000 % 60:02d}:{ms // 1000 % 60:02d},{ms % 1000:03d}"
# 1. transcribe the whole clip with word times
t0 = time.perf_counter()
raw = subprocess.run([FFMPEG, "-v", "error", "-i", clip, "-vn", "-ac", "1", "-ar", "16000", "-f", "f32le", "-"], capture_output=True, check=True).stdout
model = WhisperModel("small.en", device="cpu", compute_type="int8", cpu_threads=8)
segments, _ = model.transcribe(np.frombuffer(raw, dtype=np.float32), language="en", word_timestamps=True, vad_filter=False, beam_size=5)
segs = [{"start": s.start, "end": s.end, "words": [{"start": w.start, "end": w.end, "text": w.word.strip()} for w in s.words]} for s in segments]
segs = [s for s in segs if s["words"]]
stamp("transcribe_s", t0)
# 2. pick the window: consecutive whole segments, 15 to 25 s, most words per second
t0 = time.perf_counter()
best = None
for i in range(len(segs)):
for j in range(i, len(segs)):
a, b = segs[i]["start"], segs[j]["end"]
if b - a > MAX_LEN:
break
if b - a < MIN_LEN:
continue
n = sum(len(s["words"]) for s in segs[i:j + 1])
score = n / (b - a)
if best is None or score > best[0]:
best = (score, a, b, i, j)
if best is None: # no window long enough: take everything
best = (0, segs[0]["start"], segs[-1]["end"], 0, len(segs) - 1)
score, a, b, i, j = best
a = max(0.0, a - 0.15)
b = b + 0.35
words = [dict(w, start=w["start"] - a, end=w["end"] - a) for s in segs[i:j + 1] for w in s["words"]]
groups, cur = [], []
for w in words:
if cur and (len(cur) >= MAX_WORDS or w["start"] - cur[-1]["end"] > MAX_GAP or cur[-1]["text"][-1:] in ".?!"):
groups.append(cur)
cur = []
cur.append(w)
if cur:
groups.append(cur)
caps = []
for k, g in enumerate(groups):
s0, e0 = g[0]["start"], max(g[-1]["end"], g[0]["start"] + MIN_HOLD)
if k + 1 < len(groups):
e0 = min(e0, groups[k + 1][0]["start"])
caps.append({"start": s0, "end": e0, "text": " ".join(w["text"] for w in g)})
with open(os.path.join(work, "short.srt"), "w", encoding="utf-8") as f:
for k, c in enumerate(caps, 1):
f.write(f"{k}\n{srt_time(c['start'])} --> {srt_time(c['end'])}\n{c['text']}\n\n")
json.dump({"pick": {"start": a, "end": b, "words_per_s": score, "segments": [i, j]}, "words": words, "captions": caps, "clip_words": sum(len(s["words"]) for s in segs), "clip_segments": len(segs)},
open(os.path.join(work, "pick.json"), "w", encoding="utf-8"), ensure_ascii=False)
stamp("pick_s", t0)
# 3. cut the stretch, make it 1080x1920 (the clip over a blurred, enlarged copy of itself), level the loudness, burn the captions in
t0 = time.perf_counter()
vf = ("split[a][b];[a]scale=1080:1920:force_original_aspect_ratio=increase,crop=1080:1920,gblur=sigma=40[bg];[b]scale=1080:-2[fg];[bg][fg]overlay=(W-w)/2:(H-h)/2,setsar=1,"
"subtitles=short.srt:force_style='FontName=Arial,FontSize=11,Bold=1,Outline=2,Shadow=0,MarginV=100'")
subprocess.run([FFMPEG, "-v", "error", "-y", "-ss", f"{a:.3f}", "-t", f"{b - a:.3f}", "-i", os.path.abspath(clip), "-vf", vf, "-af", "loudnorm=I=-14:TP=-1.5:LRA=11",
"-c:v", "libx264", "-crf", "20", "-preset", "medium", "-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "160k", "short.mp4"], check=True, cwd=work)
stamp("vertical_caption_loudness_s", t0)
print(json.dumps(times))
# run: python pipeline_podcast_clip.py clip.mp4 workdirThe exact script we ran for the timings. It needs faster-whisper, numpy and ffmpeg (set FFMPEG, or have it on your path). Run it once per clip.
For the two-pass loudness step, measure the picked window first, then use the printed values in the final command in place of the single-pass loudnorm. A and B are the pick start and length printed in pick.json. The measured_I we passed is the ebur128 reading, not loudnorm's, for the reason in the failure box.
ffmpeg -nostats -ss A -t B -i clip.mp4 -vn -af "loudnorm=I=-14:TP=-1.5:LRA=11:print_format=json" -f null -
ffmpeg -nostats -ss A -t B -i clip.mp4 -vn -af ebur128=peak=true -f null -loudnorm=I=-14:TP=-1.5:LRA=11:measured_I=<ebur128 I>:measured_LRA=<ebur128 LRA>:measured_TP=<ebur128 Peak>:measured_thresh=<input_thresh>:offset=<target_offset>:linear=trueMethod and limits
- Three 60-second excerpts of one film, not a podcast with a few people at a table. A real long podcast is a different test, and we did not have one we are allowed to use.
- The pick is a density rule. Whether the stretch is worth watching is a human call, and we did not rate it.
- Captions are checked as present, not as well timed. CPU only, on a laptop shared with other work.
- Footage: Tears of Steel, (CC) Blender Foundation, mango.blender.org, CC-BY 3.0.
Page checked on Nov 4, 2026.