Adobe opens up its generative AI video models in Premiere for the first time
(Image credit: Future)
It's Adobe Max 2024! Which means Adobe is divulging all the details on the big projects it has been working on since the last Max – and unsurprisingly, for anyone following recent years, the focus is again all about Generative AI.
Since its launch, Adobe’s Firefly AI models have become an immensely popular part of Photoshop, Lightroom, and Express, (over 4 billion images generated!) and have rapidly expanded into touching every part of the Adobe ecosystem. AI has certainly changed the photo editing game forever – now Adobe aims to inspire a similar shift in video editing with the launch of the first public beta for generative AI for video.
So, what’s new in the beta? The big news is Adobe has added its first publically available AI video models to extend video or audio to cover gaps in footage, smooth out jarring transitions, and hold shots for longer. I have just seen a demonstration of some of the features on offer and it looks mind-blowing.
All these work in a similar way, you click and drag out the AI generator tool to fill in gaps in your timeline, and the AI works to analyze the clip you are editing and generate new frames that perfectly match that clip. The AI works out if the shot is best held static or if it should continue the current camera movement in the clip. It is really that simple, and in Adobe's demonstrations, the results are really impressive.
I am notoriously bad at holding shots for not quite long enough or not remembering to get enough b-roll footage, and Generative Extend is definitely set to save me a lot of stress in trying to fill the inevitable gaps I always find in my footage when I come to edit it.
Quick side note – Generative Extend is not to be confused with Generative Expand from apps like Photoshop, which can be used to expand the background of an image and add in more information, which Premiere doesn’t (currently) offer.
Check out a very quick demo of the software in action below:
Get the Digital Camera World Newsletter
The best camera deals, reviews, product advice, and unmissable photography news, direct to your inbox!
(Image credit: Adobe)
Adobe is quick to point out that its AI is commercially safe by design and is trained on Adobe licensed stock and public domain content, with the model adhering to Adobe AI ethics principles, and is connected to the Adobe-led Content Credentials initiative to label any content generated using AI models.
This isn't the OpenAI Sora video generator equivalent that many were hoping for just yet. But while you also can’t yet generate new footage directly in Premiere Pro from a text or video prompt – it’s not far away. Adobe has teased a new text-to-video or image-to-video generator which will be available in Firefly soon (you can join the waitlist now).
If you have an Adobe Premiere Pro or All Apps subscription you can sign up for the Adobe Premiere Pro (Beta) right now in the Apps panel on Creative Cloud, and then select the Beta tab at the top.
Gareth is a photographer based in London, working as a freelance photographer and videographer for the past several years, having the privilege to shoot for some household names. With work focusing on fashion, portrait and lifestyle content creation, he has developed a range of skills covering everything from editorial shoots to social media videos. Outside of work, he has a personal passion for travel and nature photography, with a devotion to sustainability and environmental causes.
