A television network commissioned a daily 90-second news capsule pilot featuring photorealistic AI anchors. I developed custom generative workflows to make this pipeline scalable and production-ready. The challenge lay in orchestrating multiple diffusion models, custom LoRAs, and diverse input sources—such as text prompts, reference images, audio speech tracks, and video-based pose control. I designed a node-based generative workflow (powered by ComfyUI) leveraging a local GPU-accelerated environment to manage the entire pipeline from initial concept stills to the final animated video.
Backgrounds
For the environment, I established an Image-to-Image pipeline using real location photographs as reference. This allowed me to control the spatial composition while leveraging the diffusion model to experiment with diverse lighting setups, seasons, and art styles rapidly.
Canny
Characters
The client initially provided diverse visual references, leading to explore non-realistic styles, including papercut puppets and 3D cartoon aesthetics. To ensure production consistency, I developed a master character sheet with multi-angle turnarounds. By designing a custom Flux workflow that combines text and image inputs, I established a template that allows for modular adjustments, making future clothing swaps and character variations seamless
Layout
To prepare the final plates for animation, I integrated the posed characters into the corrected backgrounds. I combined traditional Photoshop editing with a Flux inpainting workflow to resolve perspective issues, blend the lighting, and fix visual seams. This hybrid process allowed me to deliver a clean, unified still frame ready for the motion pipeline.
Animation
I tested several animation pipelines to match the unique aesthetic of each character style. For the papercut puppets, a pose-driven, speech-to-video workflow delivered the ideal stylized motion. For the photorealistic anchors, an LTX 2.3 image-to-video workflow—driven by audio and image inputs—proved to be the most viable solution, yielding consistent facial movements and natural pacing with minimal input data requirements.
Tech Notes
Success in this pipeline relies on balancing prompt guidance, managing upscaling, and using custom LoRAs to control style and character acting. Crucially, I used sound design and audio editing as a structural guide, leveraging the audio track to dictate the timing and direct the flow of the AI-generated video clips.
