Scarlet Pictures
All journal entries

AI Short Film Character Consistency Workflow: A Reference-First Guide

  • AI
  • Filmmaking
  • Character consistency

Generating a striking three-second clip is now easy. Holding the same person, coat, and room across sixty cuts without watching your lead actor slowly turn into a cousin is where most projects stall.

In the U.S. Copyright Office’s January 2025 report on copyrightability, the office concluded that, given current generally available technology, prompts alone do not provide sufficient human control to make users the authors of the output, because prompts function as instructions rather than direct authorship. Human authorship enters when a filmmaker selects, arranges, modifies, and edits material into a coherent work. That legal distinction mirrors the creative reality on the timeline. If you rely on text prompts alone, the model makes the choices, and the character drifts from shot to shot. When you build a reference-first workflow and direct the coverage pass by pass, you keep control of the frame.

Earlier entries in this journal covered how to plan an AI short film before the first generation and broke down the craft behind two standout shorts: Robert Gaudette’s A Face Only A Mother Could Love and Chris Capel’s Dino Demolition Squad. This guide focuses on the operational bottleneck that sits between your script and your edit: an AI short film character consistency workflow that holds up across an entire scene.

Why text prompts alone lose the face

A text prompt describes a category of person, not an individual. Even when you repeat the exact same thirty-word description of a character’s jawline, hair, and jacket in every prompt, a video model treats each generation as a new casting call unless an image anchor locks the identity.

Drift happens in three predictable places:

  1. Lens and angle jumps. Moving from a wide establishing shot to a tight close-up asks the model to invent facial detail that was not visible in the wide frame.
  2. Lighting shifts. Changing from warm interior lamp light to cold exterior street light often alters skin tone, hair color, and fabric texture at the same time.
  3. Combined camera and acting instructions. Asking a model to execute a complex camera move and a subtle facial reaction in the same pass forces the generation to compromise on facial structure.

The solution is to treat generative video the way a physical production treats continuity: lock the wardrobe and casting in pre-production, separate layout from performance, and use editing and sound to protect the cut.

Step 1: Build a three-still bible before any video

In the production notes for A Face Only A Mother Could Love, Robert Gaudette says character consistency came from generating character sheets first. In our review, a scene-detection pass counted roughly 150 cuts, and Marcel stays recognizable as a boy and as a man throughout.

Before opening a video timeline, approve three static reference files for every scene:

  • The neutral character sheet. Three angles of your protagonist (front, three-quarter, and profile) in flat, even daylight against a plain background. Avoid dramatic shadows in this master sheet. If a shadow hides the left cheek in your reference image, the video model will guess what the left cheek looks like every time the character turns.
  • The silhouette and wardrobe anchor. Three unmistakable visual markers that read from across a room: a specific haircut, one outer garment in a distinct color, and one recurring accessory or prop. If a clip blurs a facial feature during fast motion, the audience tracks the silhouette and the coat.
  • The master room plate. One clean wide still of the location lit the way the scene will play, with no characters inside it. Every wide and medium setup draws its lighting and geometry from this plate.

Never update your character sheet using a frame from a generated video clip that drifted slightly. Chaining each new shot to the last frame of the previous shot is a game of visual telephone; by shot twelve, your lead is wearing a different collar and has aged ten years. Always pull back to the original approved stills.

Step 2: Separate staging from performance

One of the clearest lessons in Chris Capel’s behind-the-scenes guide for Dino Demolition Squad is how he divides the labor inside the model: “someone finds the frame, someone stages the scene, someone gets the performance. I use Seedance for all three jobs, but never in the same pass.”

When a director asks a video generator to stage two characters across a room, track the camera left, and deliver a deadpan reaction all at once, the identity lock breaks. Splitting the workflow into four distinct passes keeps the face intact.

Pass Primary goal Reference inputs What to lock What to ignore for now
1. Layout and staging Camera angle, screen direction, blocking Room plate + graybox or rough layout sketch Horizon line, character position, lens size Fine facial detail and lip movement
2. Identity anchor still Clean start frame for the specific shot Approved character sheet + layout frame Face, hair, coat, lighting direction Motion and timing
3. Motion and performance Single physical action or single reaction beat Locked start frame (Image-to-Video) One timed gesture or expression (2 to 4 seconds) Multi-sentence dialogue or camera acrobatics
4. Voice, foley, and grade Emotional continuity across the cut Picture-locked timeline + voice and sound stems Room tone, consistent vocal timbre, scene LUT Micro-variations in background texture

By locking a start frame in Pass 2 before generating motion in Pass 3, you turn an unpredictable text-to-video roll into a controlled image-to-video take. If the acting misses the mark, you regenerate from the same start frame until the expression lands.

Step 3: Write physical camera and acting briefs

Adjectives like “emotional,” “cinematic,” or “tense” invite a video model to exaggerate facial features, which immediately breaks character consistency. In his Dino Demolition Squad notes, Capel points out that early takes of his imperial officer looked sarcastic or knowing until he rewrote the prompt as a purely physical instruction: “a single flat head movement with a completely blank face.”

Translate every emotional note into a physical action and a clock time:

  • Replace mood words with anatomy. Instead of “she looks worried and exhausted,” write “her shoulders drop, she looks down at the table at 1.0s, and her jaw stays still.”
  • Give the camera a physical rig. Instead of “dynamic cinematic camera,” specify “locked tripod, 50mm lens at eye level” or “handheld operator walking backward at a slow pace.”
  • Phrase instructions positively. Tell the model what occupies the frame and what stays still, rather than listing negatives. Saying “her face remains still and unblinking” works far better than “do not grimace or smile.”

Step 4: Use inserts, masks, and audio to bridge hard cuts

Even with strict image references, two back-to-back close-ups of the same character turning in different directions can reveal a slight shift in cheekbone or eye shape. Traditional film editing solved this problem a century ago, and the same grammar applies to generative video.

First, cut away to a tactile insert at the turn of a beat. In A Face Only A Mother Could Love, pivotal story turns happen on hands and objects: scissors cutting a paper bag, an iron pressing a shirt, or a needle dropping onto a record. An insert gives the audience’s eye a rest from the face while advancing the story, and when you cut back to a medium shot, the viewer accepts the character without comparing pixel-to-pixel geometry.

Second, let the audio track carry the continuity. When the same voice (generated in ElevenLabs or recorded by an actor, then tuned for consistent pacing), the same continuous room tone, and the same physical foley run uninterrupted across a cut, the audience perceives a continuous person in a continuous room.

A worked example (hypothetical)

Consider a hypothetical two-minute dramatic scene called Night Shift: an archivist named Mara discovers a missing ledger page in a basement vault at 2:00 a.m. Here is how a solo filmmaker applies the consistency workflow across a six-shot sequence.

The locked bible:

  • Character sheet: Mara, mid-forties, dark chin-length bob with a gray streak on the left, charcoal wool cardigan with rolled sleeves, brass key lanyard around her wrist. Three neutral-light angles approved.
  • Hero prop: A burgundy leather ledger with a torn cream ribbon marker and “1974” stamped in gold on the spine.
  • Room plate: A windowless concrete vault lit by a single overhead fluorescent tube with a warm desk lamp on the wooden sorting table.

The six-shot execution:

  1. Shot 1 (Wide master, 3.5s): Built from the room plate and Mara’s full-body reference. Mara enters from screen left and sets the burgundy ledger on the sorting table under the desk lamp. Camera is locked.
  2. Shot 2 (Medium close-up, Mara, 2.5s): Instead of prompting from scratch, the director crops into the sorting table area, composites Mara’s three-quarter reference into the warm desk-lamp light to create a clean start frame, and animates a single action: she looks down at the ledger and her brow furrows at 1.0s.
  3. Shot 3 (Insert, hands and prop, 2.0s): Close on the burgundy ledger opening to a ragged paper stub where a page was torn out. We see the charcoal cardigan cuff and the brass lanyard at the edge of the frame. No face is needed, and the lanyard confirms it is Mara’s hand.
  4. Shot 4 (Close-up, Mara’s reaction, 3.0s): Start frame locked from the front-angle character sheet matched to the warm lamp key light. Direction: “Mara’s eyes lift from the book toward the vault door at 0.8s; her mouth stays closed and her head does not tilt.”
  5. Shot 5 (Insert, vault door handle, 1.5s): A static close-up of the heavy steel door handle from inside the room. Over this cut, the audio track plays the low hum of the fluorescent tube and the distant click of a corridor latch.
  6. Shot 6 (Wide master, returning to Shot 1’s angle, 3.0s): Using the exact camera setup from Shot 1, Mara closes the ledger, pulls it against her charcoal cardigan, and switches off the desk lamp.

Because Shots 1 and 6 share the same room plate and camera angle, and Shots 3 and 5 use inserts between the close-ups in Shots 2 and 4, Mara’s face never has to morph through an impossible 180-degree camera turn. The fluorescent room tone and the sound of the ledger pages bind the six clips into one physical space.

Keep a clean production log

Good workflow hygiene also protects the finished film when you submit to festivals or distribute the work. The U.S. Copyright Office’s registration guidance asks applicants to disclose AI-generated material and describe the human contribution, so a record of how the piece was made is worth keeping. Keep a simple folder alongside your edit containing your approved character sheets, your start frames, your shot list, and your project files from DaVinci Resolve or Premiere. That folder is both your continuity safety net during reshoots and your record of human authorship when the film is done.

You can see how these visual and editing experiments come together on the AI page, or watch our live-action and upscaled work on the films page.

Sources

Drafted with AI assistance; reviewed and edited by David May.