The results of the Facebook-run (& partnered with other industry leaders & academic experts) Deepfake Detection Challenge (DFDC) are out.
Announcing them on its official blog, the artificial intelligence (AI) team of FB said the aim of the program was “to accelerate development of new ways to detect deepfake videos.”
The competition drew over 2,000 participants, who trained & tested their models using a unique new data set created for the challenge. The winning model achieved a precision rate of 65.18%.
Each of the entries were tested against a black box data set with challenging real world examples that were not shared with entrants, said the FB Team.
By creating and sharing a unique new data set of more than 100,000 videos, the DFDC has enabled experts from around the world to come together, benchmark their deepfake detection models, try new approaches, & learn from each others’ work. This open, collaborative effort will help the industry and society at large meet the challenge presented by deepfake technology & help everyone better assess the legitimacy of Content they see Online.
– Facebook AI
The top-performing model by Selim Seferbekovon running on a public data set achieved 82.56% average precision, a common accuracy measure for computer vision tasks. But when evaluating the entrants against the black box data set, the ranking of top-performing models changed significantly, said the blog post.
The highest-performing entrant achieved an average precision of 65.18% against the black box data set. Using the public data set, this model had been ranked 4th. Similarly, the other winning models, which were 2nd to 5th when tested against the black box environment, also ranked lower on the public leaderboard, said FB.
For more on the competition click here.
Image credit: Facebook
