Adobe's generative AI in Premiere Pro looks like a total game-changer for video editing

Ahead of NAB last year, Adobe teased us with a little glimpse into future of Firefly in its video editing products. Since then, Adobe's focus has seemingly been on AI in photography and content creation, with it all going a little quiet on the video front. 

This has enabled technologies like the mind-blowing Sora text-to-video model from OpenAI to grab headlines in the AI video space. However, just ahead of Adobe Max taking place in London next week, Adobe is taking the wraps off some of the latest AI capabilities it has been cooking up for Premiere Pro.

Now, as you might expect, some of the generative AI capabilities of Firefly in Premiere Pro closely follow what we have already seen in Photoshop – although there is a pretty substantial difference between altering one static frame to changing 30 or 60 frames per second while also factoring in movement, and looks set to be a gamechanger in video editing.

So what's new? One of the most substantial new features teased is Generative Extend, which enables editors to add generative frames to clips to increase their overall length. This is incredibly useful if you are like me, and forget to linger that second or two extra on a shot, which makes clips harder to blend into a transition when you make it to editing. 

Adobe's example of holding the shot on a person's solemn and static face is neatly done, but it would be interesting to see how this works with more complicated shots with additional motion.

Object Addition & Removal will also be a very welcome feature for editors who are weary of tediously removing unwanted objects that have strayed into shots. 

Firefly can recognize and highlight objects in the scene, like in Photoshop, or you can manually select areas in the footage, and these objects or selections can then be tracked across the whole clip. Objects can be completely removed and replaced with a seamless background, or replaced with an alternate generative object that better fits the scene.

Image 1 of 2

Text to Video is also coming to Premiere Pro, enabling entirely new footage to be generated from within the software based on text prompts. Adobe is a little less bombastic on this one and suggests that it might be a good way to ideate, storyboard or augment B-roll footage rather than as a significant content creation tool (in its current form, anyway!).

In a surprising move, Adobe isn't just pushing its own Firefly AI model in this reveal; it is also working on integrating third-party AI models directly into Premiere Pro's interface. In its showcase video, Adobe shows how it can create content using Sora, Pika and Runway just as easily as Firefly – so even if Adobe loses the AI race, its software might still be the best place to use other AI models in the future.

Finally, as a founding member of the Content Authenticity Initiative, Adobe is keen to reaffirm its commitment to Content Credentials, which is currently the most widely recognized way to identify AI-altered content. Adobe will be integrating Credentials into Premiere Pro as well as exports that use generative content. 

For the full lowdown on all the latest AI tools Adobe has been working on, check out the video below.

For more on video editing check out our guide to the best video editing software, or for stills the best photo editing software.