With little to amuse ourselves over the past year of a global pandemic deepfakes have in part filled the void, social media streams are full of the state of the art ‘photoshopped’ videos. Some of them are hilarious and intended to be quite obviously a fake with the outgoing President the subject of many.
However, the malicious intent of some state-of-the-art deepfakes may not be obvious and if you are a rebel looking for a cause these videos will do as intended, draw you in, convince you of their intent and influence your decisions on matters such as voting, the fight against Corona virus or tarnishing a reputation.
It’s imperative these deepfakes are detected as soon published and flagged for what they are, and online media outlets are investing heavily in solutions to detect these as accurately and quickly as possible.
The overall winner of this year’s BT Young Scientist and Technology competition in Ireland was a project using AI to detect state of the art deep fakes. Seventeen-year-old student Gregory Tarr presented his project online for this year’s virtual version of the annual competition.
He had taken the top five performers from last summer’s Deepfake Detection Challenge, a global competition with a prize fund of $1m and set himself the task of improving on their offerings.
“I reimplemented and analysed the top five entries, it became clear from my analysis that the methods, while effective, were inefficient and so computationally expensive it would be impractical to employ any of them at the scale required to solve,” said Tarr at the beginning of his pitch.
Initially Tarr ran the video frames in batches, rewrote the media loader, and refactored the code ‘to make more efficient use of the hardware which together resulted in a threefold improvement.’
Gregory Tarr and his parents
The winning process used an ensemble method so in Tarr’s method he first uses a single model and only implements the ensemble if the single model is uncertain. “This gives us the best of both worlds in a fraction of the time. This faster classification stage meant that the initial face detection phase was now the slowest component,” he explains.
To speed this process up he developed a ‘novel high speed face detector’ which apparently took some time to train but those hundreds of hours were worth it as his method is twice as fast as methods gone before this and coupled with accuracy, he improved the system speed overall delivering a system that will run at a tenth of the cost.
With his winning project, which is 150,000 lines of code, Tarr now goes on to represent Ireland in the EU Contest for Young Scientists in the autumn.
Previous winners of the BTYSTE are Shane Curran CEO of Evervault and Patrick Collison CEO of Stripe, and while Tarr is keen to follow them into entrepreneurship this might not be the exact line he tows.
“I might take a different approach to the same problem,” says Tarr. I am a specialist in computer vision as a whole so I won’t limit myself purely to Deepfake detection, but it is definitely one of the problems I will be addressing in the future.” – watch this space.
