We turned a wide film clip into a vertical Short with free tools and checked the safe zone
A wide clip becomes a 1080x1920 Short in one script: auto-editor tightens it, FFmpeg finds the shots and crops each one to 9:16, faster-whisper words become captions inside Google's vertical safe zone, and loudnorm levels the sound. We ran it on three 60-second clips of Tears of Steel and looked at the frames ourselves.
Tested by us on Nov 7, 2026, on the clips and machine listed below. Every number on this page was measured in that run.
The result
It makes a 1080x1920 Short from a wide clip with free tools, and every part but the crop choice behaved. From each 60-second clip the script wrote a 58.1, 54.7 and 59.5 second H.264 file of 20.2, 25.3 and 20.9 MB. A whole run took a median of 188.4, 212 and 205.4 seconds on a CPU-only laptop, with peak memory of 2 GB or less.
Captions: 78 of 78 were in the picture, and 78 of 78 sat inside Google's published vertical safe zone (288 px from the top, 672 px from the bottom, 48 px left, 192 px right on a 1080x1920 canvas). The lowest caption edge on any clip was at 1236 px from the top, against a limit of 1248. Sound: the loudness target was -14 LUFS and the Shorts measured -14.1, -13.4 and -14.4.
What it made




Footage: Tears of Steel, (c) Blender Foundation, mango.blender.org, CC-BY 3.0. These are files we made from it; they are shown under the same licence terms.
The first eight seconds of the finished Short from the clip tos_a, scaled down for the page, and four frames from the three Shorts: one crop that works and three that miss. Press play for the sound. Nothing loads until you do.
What failed or surprised us
- The crop rule finds faces in close-ups and misses them in medium shots. It puts the 9:16 window where the picture has the most edge detail, one frame per shot. In the frames we looked at, the close-ups of two men (tos_b, tos_c) are framed well. Elsewhere the window sat on the back of a head and bright light (tos_a), on a robot arm with the face out of frame (tos_c), and a robotic hand was cut by the left edge of the window (tos_a). A mechanical arm has more edge detail than a face. The script found 12, 15 and 23 shots, so a person has 50 crops to check.
- The picture is soft. The source is 2.4:1 (1280x534), so the 9:16 window is 300 px wide and was enlarged 3.6 times to 1080 px. Wide 16:9 footage at 720p would give a window 405 px wide (720 x 9 / 16), enlarged 2.7 times.
- Our first version put one caption past the safe zone. We centred the captions in the frame. On the first clip, before the timed runs, 25 of 26 captions were inside Google's zone, and the widest reached 908 px against a limit of 888. The zone leaves 192 px on the right and 48 px on the left, so a centred caption wider than 840 px crosses it. We added side margins of about 48 and 192 px to the caption style, which centres the captions in the zone. After that, 78 of 78 captions on the three clips were inside. The lowest caption edge was 1236 px from the top, against a limit of 1248.
- Loudness landed near the target, and the peak did not. We asked for -14 LUFS with a true peak of -1.5 dBFS. The Shorts measured -14.1, -13.4 and -14.4 LUFS, and their true peaks after the AAC encode were -1.3, -0.4 and -0.7 dBFS, all above -1.5. Encoding to AAC after the level step can raise the peak; we did not try a limiter after the encode.
- Word timing failed the same way as in the one-pass test. The longest caption stayed up 6.42 s on tos_a (1.38 and 1.94 s on the others) because the transcript gives one word a very long span.
- The files are not small. 20.2, 25.3 and 20.9 MB for 58 to 60 seconds, about 2.9 to 3.9 Mbit/s at CRF 23 and 30 fps. A higher CRF would shrink them; we did not test how the picture holds up.
What we tested and how
- Tool: shorts-reel version auto-editor 29.3.1, faster-whisper 1.2.1 (small.en), FFmpeg 8.1.2, Unlicense (auto-editor), MIT (faster-whisper), LGPL/GPL (FFmpeg build).
- Settings compared: {"steps":"auto-editor tighten (8%), FFmpeg scene detect (0.30), edge-detail 9:16 crop per shot, faster-whisper 3-word captions, two-pass loudnorm to -14 LUFS, one FFmpeg encode with captions inside the safe zone"}.
- Input clips: Tears of Steel, (CC) Blender Foundation, mango.blender.org, CC-BY 3.0. 60 s excerpts re-encoded to H.264 720p.
tos_a: 60 s starting at 20 s of the film, SHA-256945262dcbe106f1e...tos_b: 60 s starting at 180 s of the film, SHA-2564097773b615cddfa...tos_c: 60 s starting at 360 s of the film, SHA-2563d3a8cd015310349...
- Machine: AMD Ryzen 7 7735U (8 cores, 16 threads), integrated Radeon 680M, not used (CPU-only runs), 31.2 GB RAM, Windows 11 10.0.26300. The laptop was shared with other work, so timings are what a busy machine gives, not a clean benchmark.
- Runs: one cold run, then three timed runs per clip and setting. Run on 2026-11-07.
The job: edit footage you already have into vertical video. Six free steps in one script, listed under Reproduce it: auto-editor tightens the clip; FFmpeg's scene score finds the shots (above 0.30, shots under 0.8 s merged); for each shot the script looks at one middle frame and places a 9:16 window where the picture has the most edge detail; faster-whisper gives word times for three-word captions; loudnorm is measured in a first pass and applied in a second to -14 LUFS; and one FFmpeg encode crops each shot, joins them at 1080x1920, burns the captions in after the crop and writes the file.
We ran the script as one process tree: one cold run and three timed runs per clip, measuring wall time, peak memory and the CPU load before and during each run. A run during which another program outside our process tree used more than half a core is discarded and redone. We checked every caption in the picture against the same render without captions, measured where the changed pixels sit against the safe zone, measured loudness with the same meter before and after, and looked at frames from every clip ourselves.
Measurements by clip
The whole job, per clip
| Clip | Length: clip / tightened / Short (s) | Cold run (s) | Median run (s) | Peak memory (MB) | Output | File size (MB) | SHA-256 of the output (first 16) |
|---|---|---|---|---|---|---|---|
| tos_a | 60 / 57.92 / 58.1 | 174.4 | 188.4 | 1868 | h264 yuv420p 1080x1920 30/1 | 20.2 | 4bc0285af8c7e301 |
| tos_b | 60 / 54.58 / 54.7 | 212.3 | 212 | 1878 | h264 yuv420p 1080x1920 30/1 | 25.3 | b2c88ec311a35f29 |
| tos_c | 60 / 59.29 / 59.5 | 204.4 | 205.4 | 2084 | h264 yuv420p 1080x1920 30/1 | 20.9 | 356432f3a94a1ace |
Where the time went (median of the three timed runs, seconds)
| Clip | 1. Tighten | 2. Find shots | 3. Crop plan | 4. Transcribe | 5. Loudness pass 1 | 6. Crop, caption and encode | CPU load just before the four runs (%) |
|---|---|---|---|---|---|---|---|
| tos_a | 31.9 | 5.1 | 5.4 | 25.8 | 3.9 | 113.6 | 16, 1, 14, 1 |
| tos_b | 37.7 | 5.2 | 7.4 | 34 | 3.7 | 123.3 | 15, 12, 13, 11 |
| tos_c | 35.9 | 5.3 | 9.8 | 25.8 | 3.7 | 122.4 | 2, 16, 25, 21 |
The crops
| Clip | Shots found | Shot lengths, shortest to longest (s) | Crop window in the source (px) | Enlarged to 1080 wide by | Crop left edge per shot (px) |
|---|---|---|---|---|---|
| tos_a | 12 | 1.33 to 13.67 | 300 x 534 of 1280 x 534 | 3.6x | 44, 584, 316, 748, 700, 820, 928, 320, 340, 464, 976, 812 |
| tos_b | 15 | 0.88 to 6.88 | 300 x 534 of 1280 x 534 | 3.6x | 504, 748, 188, 532, 268, 536, 324, 332, 444, 204, 976, 404, 444, 532, 204 |
| tos_c | 23 | 0.92 to 6.5 | 300 x 534 of 1280 x 534 | 3.6x | 504, 156, 44, 972, 360, 568, 428, 232, 0, 632, 212, 512, 192, 480, 248, 480, 520, 560, 468, 324, 220, 240, 668 |
Captions and sound
| Clip | Captions in the picture / checked | Inside the safe zone | Lowest caption edge (px from top; zone ends at 1248) | Widest caption (px from left / to right; zone 48 to 888) | Loudness before / after (LUFS) | True peak after (dBFS) |
|---|---|---|---|---|---|---|
| tos_a | 26 / 26 | 26 of 26 | 1236 | 144 / 876 | -16.8 / -14.1 | -1.3 |
| tos_b | 28 / 28 | 28 of 28 | 1236 | 196 / 816 | -12.5 / -13.4 | -0.4 |
| tos_c | 24 / 24 | 24 of 24 | 1236 | 232 / 788 | -12.8 / -14.4 | -0.7 |
Reproduce it
"""Wide clip to a vertical Short: tighten, find the shots, crop each shot to 9:16, caption inside the safe zone, level the sound. Free tools.
usage: python pipeline_shorts_reel.py clip.mp4 workdir
Needs auto-editor installed next to this Python, faster-whisper, numpy and ffmpeg (set FFMPEG, default: ffmpeg).
Prints one JSON line with the seconds each stage took. Files written in workdir: tight.mp4, scenes.txt, plan.json, short.srt, short.mp4.
The crop rule is deliberately simple: for each shot, take one frame from the middle, and put the 9:16 window where the picture has the
most edge detail. A person checks every crop afterwards. Captions are burned in AFTER the crop, inside Google's published vertical safe
zone (288 px top, 672 px bottom, 48 px left, 192 px right on a 1080x1920 canvas), with the bottom of the caption at least 700 px from the bottom.
"""
import json
import math
import os
import re
import subprocess
import sys
import time
import numpy as np
from faster_whisper import WhisperModel
FFMPEG = os.environ.get("FFMPEG", "ffmpeg")
FFPROBE = os.path.join(os.path.dirname(FFMPEG), "ffprobe") if os.path.dirname(FFMPEG) else "ffprobe"
AUTO_EDITOR = os.path.join(os.path.dirname(sys.executable), "auto-editor.exe")
clip, work = os.path.abspath(sys.argv[1]), os.path.abspath(sys.argv[2])
os.makedirs(work, exist_ok=True)
tight, srt, final = os.path.join(work, "tight.mp4"), os.path.join(work, "short.srt"), os.path.join(work, "short.mp4")
times = {}
SCENE, MIN_SHOT = 0.30, 0.8
MAX_WORDS, MAX_GAP, MIN_HOLD = 3, 0.5, 0.35
def stamp(k, t0):
times[k] = round(time.perf_counter() - t0, 2)
def srt_time(t):
ms = int(round(t * 1000))
return f"{ms // 3600000:02d}:{ms // 60000 % 60:02d}:{ms // 1000 % 60:02d},{ms % 1000:03d}"
def run(args, **kw):
return subprocess.run(args, capture_output=True, text=True, check=True, **kw)
# 1. tighten: cut the quiet parts by loudness
t0 = time.perf_counter()
subprocess.run([AUTO_EDITOR, clip, "--edit", "audio:threshold=8%", "-o", tight, "--no-open"], check=True, capture_output=True)
stamp("tighten_s", t0)
# 2. find the shots in the tightened clip (FFmpeg scene score above 0.30; shots under 0.8 s are merged into the one before)
t0 = time.perf_counter()
dur = float(run([FFPROBE, "-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", tight]).stdout)
scenes = os.path.join(work, "scenes.txt")
run([FFMPEG, "-v", "error", "-i", tight, "-an", "-vf", f"select='gt(scene,{SCENE})',metadata=print:file=scenes.txt", "-f", "null", "-"], cwd=work)
cuts = [float(m) for m in re.findall(r"pts_time:([\d.]+)", open(scenes).read())]
bounds = [0.0]
for c in cuts:
if c - bounds[-1] >= MIN_SHOT and dur - c >= MIN_SHOT:
bounds.append(c)
bounds.append(dur)
shots = [(bounds[i], bounds[i + 1]) for i in range(len(bounds) - 1)]
stamp("shots_s", t0)
# 3. choose the 9:16 window for each shot: the most edge detail in one middle frame
t0 = time.perf_counter()
info = json.loads(run([FFPROBE, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height", "-of", "json", tight]).stdout)["streams"][0]
W, H = info["width"], info["height"]
cw = int(H * 9 / 16) // 2 * 2
plan = []
for a, b in shots:
mid = (a + b) / 2
raw = subprocess.run([FFMPEG, "-v", "error", "-ss", f"{mid:.3f}", "-i", tight, "-frames:v", "1", "-vf", "scale=320:-2", "-pix_fmt", "gray", "-f", "rawvideo", "-"], capture_output=True, check=True).stdout
h = len(raw) // 320
img = np.frombuffer(raw, dtype=np.uint8)[: 320 * h].reshape(h, 320).astype(float)
g = np.abs(np.diff(img, axis=1)).sum(axis=0)
win = max(2, int(round(h * 9 / 16)))
sums = np.convolve(g, np.ones(win), mode="valid")
best = int(np.argmax(sums))
x = int(round(best / 320 * W)) // 2 * 2
x = max(0, min(W - cw, x))
plan.append({"start": round(a, 3), "end": round(b, 3), "crop_x": x, "crop_w": cw, "src_w": W, "src_h": H})
json.dump({"duration": dur, "shots": plan}, open(os.path.join(work, "plan.json"), "w"))
stamp("crop_plan_s", t0)
# 4. transcribe the tightened clip with word times, 3-word captions
t0 = time.perf_counter()
raw = subprocess.run([FFMPEG, "-v", "error", "-i", tight, "-vn", "-ac", "1", "-ar", "16000", "-f", "f32le", "-"], capture_output=True, check=True).stdout
model = WhisperModel("small.en", device="cpu", compute_type="int8", cpu_threads=8)
segments, _ = model.transcribe(np.frombuffer(raw, dtype=np.float32), language="en", word_timestamps=True, vad_filter=False, beam_size=5)
words = [{"start": w.start, "end": w.end, "text": w.word.strip()} for s in segments for w in s.words]
groups, cur = [], []
for w in words:
if cur and (len(cur) >= MAX_WORDS or w["start"] - cur[-1]["end"] > MAX_GAP or cur[-1]["text"][-1:] in ".?!"):
groups.append(cur)
cur = []
cur.append(w)
if cur:
groups.append(cur)
caps = []
for i, g in enumerate(groups):
a, b = g[0]["start"], max(g[-1]["end"], g[0]["start"] + MIN_HOLD)
if i + 1 < len(groups):
b = min(b, groups[i + 1][0]["start"])
caps.append({"start": a, "end": b, "text": " ".join(w["text"] for w in g)})
with open(srt, "w", encoding="utf-8") as f:
for i, c in enumerate(caps, 1):
f.write(f"{i}\n{srt_time(c['start'])} --> {srt_time(c['end'])}\n{c['text']}\n\n")
json.dump({"words": words, "captions": caps}, open(os.path.join(work, "short.json"), "w", encoding="utf-8"), ensure_ascii=False)
stamp("transcribe_s", t0)
# 5. loudness, pass 1: measure the tightened audio (target -14 LUFS, true peak -1.5)
t0 = time.perf_counter()
e = subprocess.run([FFMPEG, "-nostats", "-hide_banner", "-i", tight, "-vn", "-af", "ebur128=peak=true", "-f", "null", "-"], capture_output=True, text=True).stderr.rsplit("Integrated loudness", 1)[1]
mi, ml, mt = (re.search(p, e).group(1) for p in (r"I:\s+(-?[\d.]+) LUFS", r"LRA:\s+(-?[\d.]+) LU", r"Peak:\s+(-?[\d.]+) dBFS"))
ln = subprocess.run([FFMPEG, "-nostats", "-hide_banner", "-i", tight, "-vn", "-af", "loudnorm=I=-14:TP=-1.5:LRA=11:print_format=json", "-f", "null", "-"], capture_output=True, text=True).stderr
j = json.loads(re.findall(r"\{[^{}]*\"input_i\"[^{}]*\}", ln, re.S)[-1])
lra = max(11, math.ceil(float(ml)))
af = (f"loudnorm=I=-14:TP=-1.5:LRA={lra}:measured_I={mi}:measured_LRA={ml}:measured_TP={mt}:measured_thresh={j['input_thresh']}:offset={j['target_offset']}:linear=true")
stamp("loudness_measure_s", t0)
# 6. one encode from the tightened file: crop each shot, join, 1080x1920, captions inside the safe zone, levelled audio
t0 = time.perf_counter()
parts, labels = [], ""
for i, s in enumerate(plan):
parts.append(f"[0:v]trim=start={s['start']}:end={s['end']},setpts=PTS-STARTPTS,crop={s['crop_w']}:{s['src_h']}:{s['crop_x']}:0,scale=1080:1920,setsar=1[v{i}]")
parts.append(f"[0:a]atrim=start={s['start']}:end={s['end']},asetpts=PTS-STARTPTS[a{i}]")
labels += f"[v{i}][a{i}]"
style = "FontName=Arial,FontSize=11,Bold=1,Outline=2,Shadow=0,MarginV=105,MarginL=7,MarginR=29" # side margins of about 48 px and 192 px put the caption in the middle of the safe zone, not of the frame
fc = ";".join(parts) + f";{labels}concat=n={len(plan)}:v=1:a=1[vc][ac];[vc]subtitles=short.srt:force_style='{style}'[v];[ac]{af}[a]"
open(os.path.join(work, "filter.txt"), "w").write(fc)
subprocess.run([FFMPEG, "-v", "error", "-y", "-i", "tight.mp4", "-filter_complex_script", "filter.txt", "-map", "[v]", "-map", "[a]", "-c:v", "libx264", "-crf", "23", "-preset", "medium", "-pix_fmt", "yuv420p",
"-r", "30", "-c:a", "aac", "-b:a", "128k", "-ar", "48000", "-movflags", "+faststart", "short.mp4"], check=True, cwd=work)
stamp("crop_caption_encode_s", t0)
print(json.dumps(times))
# run: python pipeline_shorts_reel.py clip.mp4 workdirThe exact script we ran for the timings. It needs auto-editor, faster-whisper and numpy in one Python environment, and ffmpeg and ffprobe (set FFMPEG to the ffmpeg path, or have both on your path). Run it once per clip.
Method and limits
- Three 60-second excerpts of one film, cut from a 2.4:1 picture (1280x534), so a 9:16 window is only 300 px wide. Wide 16:9 footage with a face in the middle would give a larger window and a different result.
- The crop rule is edge detail in one frame per shot. It does not know where a face or the subject is, and we did not test face tracking. A person has to check every crop.
- Safe-zone figures are Google's published vertical ad zone. YouTube publishes no separate figure for organic Shorts, Meta's figures come from third-party guides, and TikTok publishes none, so the check shows the captions sit inside one published zone, not inside all of them.
- CPU only, on a laptop shared with other work. The CPU load before and during each run is in the result file. 0 runs were discarded and redone because another encoder, browser or script of ours was busy during them. Windows background processes (the antivirus, the screensaver, search) did use CPU during the runs; they are listed in the result file, and the run times carry that noise.
- Footage: Tears of Steel, (CC) Blender Foundation, mango.blender.org, CC-BY 3.0.
Page checked on Nov 4, 2026.