Adobe has been talking up its AI video models for a long time, with the company teasing the first integrations into its extensive suite of software at last year's Adobe Max. Now Adobe is finally ready to go public, launching a brand new Firefly web application putting all of Adobe’s generative image, audio, and video powers all in one place.
At launch, the Firefly Video Model will offer the ability to generate video clips from a simple text prompt – or import images or 3D models that can be AI-animated into a video. There will also be the ability to add an image as the starting and ending frame, and Adobe’s video model will generate all the movement between the two shots. Videos generated by the model can then be refined based on camera angles, movement, and framing to more closely fit a specific project or follow a particular artistic vision.
On the audio side of audio-visual – the Firefly Beta web app will also offer an AI translation feature that will translate speech into another language and match the voice of the original speaker, with 20 languages currently supported. Although, yes, this does sound very similar to what sparked the accent-altering controversy surrounding the Oscar-nominated movie The Brutalist – which creators have some pretty strong feelings about.
If you were hoping this might be a part of your existing Adobe subscription to offset all those recent price rises – think again – prices start at $9.99/month for the Firefly Standard Plan which offers 2000 credits for up to 20 1080p video generations, or $29.99 for the Firefly Pro plan and 7000 credits, or up to 70 1080p video generations.
These plans cover unlimited image generations as well as video and audio generations. Firefly AI credits granted through app subscriptions for existing features like image generation will continue as normal, for example, a single app plan comes with between 100-500 credits, but their use on video generation is limited without a Firefly subscription as well. If you are understandably confused, then you can find out more about Adobe's Firefly pricing here, and the complex credits and exclusions on its help pages.
Currently, videos generated by Adobe’s Video Model are limited to 1080p, although Adobe promises a Firefly Premium 4K version will be coming soon, as well as a lower-resolution version for ideating that will be faster to generate – although no word on when these might launch. With Premiere Pro alone starting from $22.99/month and the All Apps Plan at $59.99/month for individuals, this is going to get expensive fast.
And speaking of Premiere Pro – the Firefly Video Model will also be integrated into Adobe’s Premiere Pro application through the already-previewed Generative Extend feature. Within the Adobe Premiere Pro app, the Video Model will allow you to extend any clips on your timeline for up to 5 seconds, matching both the subject and the camera movement from the original clip to predict the next frames. This also stretches to any ambient audio in the clip, so crashing waves or birdsong will also be continued into the newly generated content.
I have seen a demo of both the Video Model in the Adobe Firefly web app and Generative Extend in Adobe Premiere Pro in action, and they look spectacular – although I will say few of the demonstrations I have seen have managed to shift that uncanny AI-generated look – so luckily still we might not be duped by AI-content just yet.
Adobe will tout its work on the Content Authenticity Initiative (CAI), which attaches a hidden watermark, like a digital label, to all AI outputs – known as Content Credentials. These Content Credentials will also apply to all videos generated by Adobe’s Firefly Video Model. However, these endeavors haven’t stopped AI-generated content from doing the rounds on the wild west of social media where people are less inclined to check the authenticity of what they are seeing – and recent policy changes at Meta aren’t going to slow this down.
Adobe is also very keen to push its AI Ethics, stressing that it only trained its video model on Adobe Stock and public domain content, and not from any Adobe customer data – although I doubt that will help soothe any creatives such as animators, foreign-language voice artists, or B-roll crews whose futures are under threat from this AI model. But as Adobe has rights to all the training content – this does mean that video generated through Firefly is commercially safe to use in your projects without worrying about copyright.
If you want to try out Adobe's new video model right now, all you need is an Adobe account to sign up for a limited free trial.
You might also like...
For more on all things AI – check out our guide to the best AI image generators or the best AI culling software.
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
