Speak Any Language with Lipdub

Captions first entered the translation space with our release of AI Dubbing earlier this year. We saw immediate adoption of the feature from both English and non-English speaking parts of the world. We could have never predicted all of the use cases that began to emerge and so we knew we had to take it further…

Since then the machine learning team at Captions has been heads down on the latest iteration of our model, called Lipdub. A first of its kind in terms of quality, performance, and use case. We first started testing it ourselves [dogfooding is good!!] in markets like France, India, Spain, and more. Once we were satisfied with the performance, the team expanded the test and brought together a few UGC creators.

UGC creators quickly realized they could now add a multi-lingual option to their brand contracts. As a result, the brands they work with could take that video, with no additional work, to all of their markets [regardless of the language spoken] – it was a complete game-changer.

Not only did it increase income for the creators, but the results were nothing short of incredible for the advertiser. I’m excited to share some of the results we saw from our own campaigns:

1. Our ad content, leveraging Lipdub, significantly outperformed our original content (that was in English). On average, the cost was 54% less per subscription in a test conducted in over 5 languages (French, Hindi, Spanish, Italian, and Portuguese).
2. The highest performing example was our ad in French. It used Lipdub and resulted in a 67% lower cost per subscription compared to the original ad. 

To give you an idea of what this can look like, Josh was nice enough to let us use his video to show an example of Lipdub:

‍

‍

The entire team at Captions could not be more excited about what Lipdub will unlock for everyone across the globe. To make that even easier, we're releasing a standalone app that you can now pre-order right now. We will be releasing the app next month and rolling out new languages as well: