While technology enthusiasts (and not only) await the extended launch of Sora, artificial intelligence capable of generating photorealistic videos has already arrived and is available through two direct “competitors” of the OpenAI tool: in June 2024, in fact, Two AI models have already been released that can be used by users who challenge that of Sam Altman and associates. We are referring above all to Luma Dream Machinewhich can already be used in our area via login with a Google account, but also to Kling AI.
While waiting to experience Sora’s potential firsthand, here is what can already be done with Artificial Intelligence in the field of video generation.
From Luma Dream Machine to Kling AI
Luma Dream Machine presents itself as a service that promises to generate 5 second video with fluid movements and cinematic scenes. The aim is for photorealism, although there are still limitations in this sense.
Net of the fact that OpenAI’s Sora will be able to generate videos lasting up to one minute, therefore much longer than Dream Machine, it is worth noting that we are still in an experimental phase of these text-to-video solutions.
The Dream Machine developers themselves warn of potential problems relating, for example, to the generation of text elements in the video, as well as movement errors that can be encountered with certain prompts. This is not to mention the hallucinations, for example those relating to the “on the fly” transformation of an object into an unsolicited one. Despite the presence of limitations of this type, however, what is surprising is the fact that Dream Machine is a free service that allows you to generate up to 30 watermarked videos per month.
The service also offers the possibility of paying a subscription to remove limitations, so this is an important step forward compared to already known solutions such as Kaiber.aiwhich instead only offer a quick initial free trial.
Furthermore, Dream Machine stands just like a Sora’s rivalconsidering that the intended objective is to generate photorealistic videos.
All interesting, but how does it work in practice? After logging in with your Google account, the bar where you can enter your text command immediately appears on the screen: in this field you can type text input or upload a reference image to animate. The advice is to leave the option enabled Enhance promptso as to allow the service to improve the prompt in order to have a better chance of obtaining a result that meets expectations.
During our test, carried out in the early moments when word of the release of the AI model was spreading, we had to wait a while in line for the videos to be generated. This is due to high demand, but you may be interested to know that the actual generation of an AI video using Dream Machine takes 120 seconds.
In any case, for our first test we used a prompt in English, asking the AI to imagine a video of a blogger typing on the computer in front of the Colosseum while drinking a cold drink. The result, once the phase is over Dreaming, it was interesting, but there was some inaccuracy in terms of movements and especially reproduction of the hands. After carrying out some generic tests, we wanted to go further by trying to type a prompt in Italian: “a panda parading on the red carpet greeting the photographers“. Once the Dreaming phase was completed, the result correctly responded to our requests. You can find the video generated by Luma Dream Machine at this link. There are quite a few imperfections, starting from the fact that, instead of a simple greeting, what happens in the video is that the panda’s hand becomes a sort of camera with which the animal starts taking photos. This is just one of the problems: for example, when the arm with the camera is lowered, the paw becomes a hand, while looking behind the panda you notice a morphing problem, that is, the unwanted transformation of the people in the background. As previously mentioned, the service is still in the experimental phase.
Although the AI model does not appear to “think” directly in Italian, thus carrying out a “behind the scenes” translation from Italian to English, as you can clearly see it is already possible, in fact, to use prompts in our language to obtain potentially interesting results. You may, therefore, be interested in using your creativity to generate the most disparate scenarios, given that the model is already open and accessible to all.
However, if you want to better understand the extent to which these models can hallucinate, especially with prompts in Italian, you might be interested in taking a look at the failed tests relating to the gameplay of a soulslike set in Milan and the dog with the Italian flag running in a cyberpunk city.
The cartoon of the rabbit in a forest is more interesting, but you understand that there is still a way to go (although a lot is also related to the ability to provide effective prompts to the service, as happens with any self-respecting AI service).
Going beyond Dream Machine, this period also saw the launch of Kling AI, another artificial intelligence model which aims to create photorealistic videos.
The most interesting aspect of this other tool is that it can generate up to 2 minutes of 1080p video at 30 fps. The Chinese technology giant is behind the project Kuaishou Technology and the goal is to accurately simulate real-world physics. This model is currently available on the waitlist in China and we don’t know if it will also be made available in our parts, but certainly the month of June 2024 brought with it some notable news in terms of generating photorealistic AI videos.
