NEW PODCAST! "The Creator" DP Oren Soffer's BRUTAL take on AI video →🎙️ WATCH/LISTEN Now
The Creator" DP Oren Soffer's BRUTAL take on AI🎙️ NEW PODCAST: Watch/Listen Now
Education for Filmmakers
Language
The CineD Channels
Info
New to CineD?
You are logged in as
We will send you notifications in your browser, every time a new article is published in this category.
You can change which notifications you are subscribed to in your notification settings.
Annoying AI slop is all over the Internet, and video generators are a topic of huge controversy for many reasons. Lots of professionals are on the fence. They can’t imagine how generative models could be useful in their work. Why would I want a machine to create a random, generic clip for me – synthetic, uncanny, questionable, and completely devoid of my voice? If we put it this way, I don’t have an answer. Yet what if it could keep your voice? And what if there were indeed workflows for which AI video generators could be of real use? Seasoned filmmaker Drew Geraci offers a different perspective and shows us his approach to hybrid video production with AI.
Drew Geraci is a renowned cinematographer and photographer with over 20 years of experience in the industry, creating captivating time-lapse and hyper-lapse works that you’ve probably seen. (For instance, the “House of Cards” opening credits.) Recently, Drew has launched his latest MZed course, “Directing the Future: Ethical AI Video for Filmmakers.” This article utilizes materials from the course and offers a sneak peek at his workflow for transforming still images into videos.
My colleagues and I have written numerous news articles on AI video generators over the last few years. Sora, Google Veo, Dream Machine, Veo 3, Sora 2… Some of them are already super outdated, because the generative tech is developing at a breakneck speed. Yet, I’ve never used AI video generators in real life, apart from testing them out to report on their current capabilities. Why? Different reasons, with the question of ethics at the forefront. But also, I just couldn’t imagine how a generated AI video could be part of my filmmaking routine. Especially, because I love concepting and creating impactful cinematic images, working with real people, and having full control of the result. Frankly, none of that applies to AI video. Or am I wrong?
In his course, Drew Geraci demonstrates various use cases that offer a different approach to these deep-learning tools. Let’s call it a hybrid video production. A production that seamlessly blends AI-generated elements with live footage and keeps your artistic voice intact. It could be, for instance, the creation of assets, based on your real photographs, and then carefully animated and post-processed in After Effects to put together a compelling shot:
Another approach would be to bring motion to your still photography and combine generated clips with live-action footage in post-production. Let’s have a closer look at Drew’s workflow here.
So, for this demonstration, Drew went on a sunny shoot in a pool with a model and an assistant. Most of the time, it was just like any other professional photo and videography gig. High-resolution cameras (Sony a7R V and Sony a1 II, in this case), working with lighting, looking for the best image compositions, giving acting directions, and capturing various angles.
However, they knew from the start that this would be a hybrid video production, so they also planned for it:
After the shoot, Drew Geraci went through his usual selection process and edited the photos in Adobe Lightroom, giving them a soft, almost ethereal look he was striving for.
So, now he had a collection of beautiful still images that he could already deliver to the client. Yet apart from that, Drew decided to use some of them for a video edit. For that, he needed to give the images a little motion to match them seamlessly with other footage. And this is where AI video generation comes in.
In his course demonstrations, Drew Geraci uses Google Flow. This is a platform for creatives, with an easy-to-understand interface that utilizes Google’s latest generative models, including Veo 3. (Please note that a trial version is available, but for further creations, a subscription is required.) Google Flow is one of the most generative tools we know of, utilizing text prompts. However, it also includes a feature called “Frames-to-Video.”
Using Frames-to-Video, Drew uploads one of his still images and then writes a text description of the motion. For the feet close-up above, the prompt was: “Stationary shot; a woman’s feet walk forward completely into a pool with both feet, one after another, in a very delicate manner; cinematic look.” As you can see, it is quite an elaborate text with numerous exact details. The reason behind this is that you’ll need several iterations to achieve the desired result. After the first try, you most likely will get a result where something is off. This should guide you in altering the text prompt and what you need to specify. Generally, in Drew’s experience, the more specific your command, the better result you get.
So, what was the result? Google Flow took Drew’s photo, kept the image composition, the lighting, the colors, and the model’s feet, and only guessed the correct motion based on the text prompt. It didn’t invent anything, and it kept the creative vision intact.
Another tip for using Google Flow, as shared by Drew Geraci, is to try using different AI models for your shots. In a roll-down menu, you can enable older versions of Veo, as shown in the screenshot:
Why? Some would say: the newer, the more advanced. Yet Drew finds Veo 3’s results sometimes too plastic-looking, as if they try to smooth out the image’s details. Also, this neural network can go a bit astray from your established color grading look. Thus, for the mermaid transformation idea, he used Veo 2.
It took several iterations to achieve a transition from legs to a mermaid tail that looked decent. However, because the effect had been planned in advance and they had chosen a good angle, it wasn’t a difficult process.
Before proceeding, Drew uses Topaz Labs to upscale AI generations to 4K. After all shots are completed, he combines them into a single timeline in DaVinci Resolve and edits them into a single video. His side tip for working with Google Flow’s results is to play around with speed. For instance, he prompted the sequences in the water to be in slow motion. Yet, when he increases their speed, say, to 150%, all of a sudden, they feel much more realistic.
The final clip, demonstrated in the course, appears consistent, although it incorporates both AI-enhanced footage and live-action shots. Well, that’s because both use the same talent, style, lighting, color palette, and vision. As Drew points out, this could be a short, fun, social media clip. Alternatively, if you want to have a more elaborate VFX transition to a mermaid, done by hand by an experienced artist, this clip could serve as a great stylized previz for a pitch.
So, this was Drew’s take on hybrid video production and how it can streamline some of your workflow while still preserving your voice and vision. While a lot of what is said about AI video generators is negative (for instance, that they are used for scams and in other harmful ways), this could become a different, more ethical, and sustainable approach. Drew is currently working on additional lessons, so stay tuned!
As an MZed Pro member, you can access hundreds of hours of filmmaking education. Additionally, we continually add new courses (several are currently in production).
Starting at just $29/month (billed at $349 in the first year, $199 from the second year, or billed monthly at $49), here’s what you’ll get:
Full disclosure: MZed is owned by CineD.Join MZed Pro now and start watching today!
What are your thoughts on this approach to AI video generators? Would you consider a hybrid video production for some of your projects? How else could it become a part of your real-life workflow? Share your ideas with us in the comments below!
Image credit: Directing the Future course / MZed / Drew Geraci.
Δ
Stay current with regular CineD updates about news, reviews, how-to’s and more.
You can unsubscribe at any time via an unsubscribe link included in every newsletter. For further details, see our Privacy Policy
Want regular CineD updates about news, reviews, how-to’s and more?Sign up to our newsletter and we will give you just that.
You can unsubscribe at any time via an unsubscribe link included in every newsletter. The data provided and the newsletter opening statistics will be stored on a personal data basis until you unsubscribe. For further details, see our Privacy Policy
Mascha Deikova is a freelance director and writer based in Salzburg, Austria. She creates concepts for and works on commercials, music videos, corporate films, and documentaries. Mascha’s huge passion lies in exploring all the varieties of cinematic and narrative techniques to tell her stories.