Stability AI has released a new “camera” of sorts – the Stable Virtual Camera is a generative AI that takes 2D still images and creates immersive videos, including user-defined camera movements. Announced on March 18, Stable Virtual Camera is still in early testing but available to try out for non-commercial purposes.
The new generative AI program takes flat images and creates videos with what Stability AI describes as “realistic depth and perception.” The tool is a “camera” only in the virtual camera sense, similar to the virtual cameras used in animation. The AI allows creators to take a virtual step inside their still images, generating a video moving around the scene as if it were captured with a complex 3D camera setup.
The resulting “3D” video creates a video with depth and movement and not the 3D-wear-funky-glasses-to-the-movie-theatres definition of 3D. The generative AI includes Dynamic Camera Control tools, which allow the creator to choose how to move through the scene. The list of tools includes classic cinematic movements like pan, roll, zoom, and dolly zoom, along with custom camera movements on a path the creator defines. A 360-degree camera trajectory will circle the object, which means the AI is generating the entire view behind the object that was not originally captured in the still photo.
While the software can generate such videos from a single image, users can also upload as many as 32 still photos to give the system more real data to work with. The resulting video can be a 1:1, a 9:16, a 16:9 or even a custom aspect ratio.
Currently, Stable Virtual Camera works for videos up to 1,000 frames in length – which in 24p would mean a video roughly 41 seconds long.
StabilityAI says that the tool is trained as a multi-view diffusion model. A “two-pass” sampling helps create more consistent results by breaking the generation down into two parts, first creating the anchor views and then creating the target views for smooth movement through the video.
While the Stable Virtual Camera is in early testing, Stability AI has already noted a handful of limitations of the AI. Photos that have humans or animals in the original shot won’t turn out as well as scenes lacking that life. What the company refers to as dynamic textures such as water is also a challenge for the AI, along with irregular shapes and camera movements that intersect objects.
Stable Virtual Camera is currently in early testing, which means the generative AI is free to use but only for research under a non-commercial license. The AI is currently available to download on Hugging Face.
You may also like
Before you go, learn how to spot an AI-generated image or browse the best photo editing software.
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
