Over the weekend, social network Facebook announced a new artificial intelligence (AI) project, “Learning From Videos”. It said the project was designed to “automatically learn audio, textual, & visual representations from the data in publicly available videos uploaded to Facebook.”
FB explained further saying by learning from videos spanning nearly every country & hundreds of languages, this project will not just help it improve its core AI systems for applications like Content recommendation & policy enforcement — it would also enable entirely new experiences.”
This project is also being looked as as the 1st step in unsupervised learning in machine learning (ML), helping AI researchers break away from the reliance on labeled data.
Although we’ve just scratched the surface, using semi- and self-supervised learning on the videos uploaded to Facebook has already improved our computer vision and speech recognition systems. Within six months of developing a state-of-the-art, self-supervised framework for video understanding, we’ve built and deployed an AI model in Instagram Reels’ recommendation system. And this is just the beginning of our Learning from Videos project. Early experiments in applying self-supervised learning to real-world videos also show a 20 percent reduction in speech recognition errors, which could improve a wide range of applications like auto-captioning and tasks that help flag harmful content like hate speech.
And we’re researching ways to apply new capabilities, like multimodal video retrieval, in order to make it easier for people to surface key moments in time from their trove of digital memories.
–Facebook
Facebook said the “Learning from Videos” project signals a paradigm shift in the way machines are able to understand videos. The work will allow it to move away from AI that required people to look at & label videos by hand, & would make it possible to build AI systems that used the most advanced techniques, such as self-supervision, to improve recommendations, search, & retrieval, & other important applications for everyone on Facebook.”
Image credit: Facebook