Animate a character from one image using any driving video

SCAIL-2 does end-to-end character animation. You provide a reference character and a driving video, and it transfers that motion onto your character directly, without first converting the video into a skeleton or pose map.

Dropping that intermediate step is the interesting part. Skeleton and pose maps get ambiguous under fast or unusual motion and usually assume a human driver, so most tools break outside that lane. Because SCAIL-2 reads the driving video directly, it also handles jobs those pipelines struggle with: swapping one character's identity onto another's performance, driving a character with non-human or animal motion, and animating scenes with more than one character at once.

Start with the public Space by fffiloni. Upload a character image and a short driving clip, and watch whether the motion transfers cleanly and the character keeps its identity across the shot. A good first test is a clip with fast or non-human motion, since that is exactly where pose-based tools drift.

For local use, the weights are MIT-licensed and the GitHub repo has the inference code and setup. It runs at 512p and 704p (704p is sharper for pose-driven and replacement work), but it is a diffusion video model, so expect a real GPU rather than a laptop.

It is a diffusion video model, so local runs need a capable GPU and are not quick. Results also lean on the driving clip: very low-quality or heavily occluded motion can still confuse the transfer, so judge it on a clean reference first.

• Animate a drawn or photographed character to match a reference performance.
• Replace a character in existing footage with a different one while keeping the motion.
• Drive a character with animal or non-human motion that pose-based tools cannot represent.
• Animate multi-character scenes without rigging each one separately.
• Prototype a motion idea in the browser before committing to a local render.

Try the demo

View model page