Captions, an AI-powered video modifying startup, has launched a brand new app referred to as Lipdub for translating clips into 28 languages.
Lipdub is offered within the App Retailer without spending a dime and helps a number of languages, together with French, Hindi, Spanish, Italian, Portuguese, Japanese and extra. The app even lets customers translate movies into Texas slang, Gen Z, pirate and child discuss. The demo video reveals that the app may also change lip motion in line with the chosen goal language. Nonetheless, at instances there’s a sure lag between the audio and the lip motion.
Customers can translate a video of a single individual speaking throughout as much as one minute after which share it on different social media platforms.
On its web site, Captions says that greater than 3 million creators have used its eponymous video modifying app. The startup claims that it has greater than 100,000 each day customers. The Captions app presents a number of AI-powered options round video modifying, similar to eradicating “ums” and “ahs,” lowering background noise and enhancing speech. The app additionally has an “AI Lipdub” characteristic that may change lip motion in post-production modifying in case you change the transcript.
Captions was based in 2021 by Gaurav Misra, who was head of design engineering at Snap. In June, the corporate secured $25 million in a Series B round led by Kleiner Perkins with participation from Sequoia Capital, Andreessen Horowitz (a16z) and SV Angel. Up to now, Captions has raised $40 million in funding.
Utilizing translation and AI-dubbing to succeed in a wider viewers is a rising pattern. In June, YouTube introduced that it’s testing an AI-powered software to let customers robotically dub their movies in different languages. The corporate mentioned it’s even engaged on higher lip-syncing. Final month, the corporate mentioned that it’s integrating AI-powered dubbing instantly into YouTube Studio for simpler entry for folks trying to convert movies into different languages.
Earlier this month, AI-powered voice-generating platform ElevenLabs released its dubbing tool with assist for 29 languages. Rest of World beforehand reported that dubbing service supplier firms are producing hundreds of thousands of {dollars} by translating content material for standard YouTubers like MrBeast.
AI-powered dubbing startups have generated quite a lot of investor curiosity, with startups like U.Okay.-based Papercup and Isreal-based Deepdub elevating hundreds of thousands of {dollars}.