This article was 1st published on our sister Site, The Internet Of All Things.
Those in the world of movies and television are well-aware of actors muffing their dialogues, or fumbling. This often necessitates re-takes or some clever editing.
But now, thanks to technology, a video editor can modify the visuals using a text transcript. Which means, he/she can add new words or delete unwanted ones from the dialogues?
That’s exactly what researchers have now developed. A team of researchers from Stanford University, Max Planck Institute for Informatics, Princeton University and Adobe Research have come up with an algorithm for editing talking-head videos – videos showing speakers from the shoulders up. All of it means the use of AI in video editing.
Here`s how it works:
The new app uses the transcript to extract speech motions from various video pieces.
Then, with machine learning, it converts those into a final video that appears natural to the viewer – lip-synched and all.
So, essentially, when an actor flubs a word or mins-pronounces, the new technique allows the editor to edit the transcript, and the application will assemble the right word from various words or portions of words spoken elsewhere in the video. It’s the equivalent of rewriting with video, much like a writer retypes a misspelled or unfit word.
What is required, though is at least 40 minutes of original video as input.
