Startup Can Identify Deepfake Video In Real Time (wired.com) 28
An anonymous reader quotes a report from Wired: Real-time video deepfakes are a growing threat for governments, businesses, and individuals. Recently, the chairman of the US Senate Committee on Foreign Relations mistakenly took a video call with someone pretending to be a Ukrainian official. An international engineering company lost millions of dollars earlier in 2024 when one employee was tricked by a deepfake video call. Also, romance scams targeting everyday individuals have employed similar techniques. "It's probably only a matter of months before we're going to start seeing an explosion of deepfake video, face-to-face fraud," says Ben Colman, CEO and cofounder at Reality Defender. When it comes to video calls, especially in high-stakes situations, seeing should not be believing.
The startup is laser-focused on partnering with business and government clients to help thwart AI-powered deepfakes. Even with this core mission, Colman doesn't want his company to be seen as more broadly standing against artificial intelligence developments. "We're very pro-AI," he says. "We think that 99.999 percent of use cases are transformational -- for medicine, for productivity, for creativity -- but in these kinds of very, very small edge cases the risks are disproportionately bad." Reality Defender's plan for the real-time detector is to start with a plug-in for Zoom that can make active predictions about whether others on a video call are real or AI-powered impersonations. The company is currently working on benchmarking the tool to determine how accurately it discerns real video participants from fake ones. Unfortunately, it's not something you'll likely be able to try out soon. The new software feature will only be available in beta for some of the startup's clients.
As Reality Defender works to improve the detection accuracy of its models, Colman says that access to more data is a critical challenge to overcome -- a common refrain from the current batch of AI-focused startups. He's hopeful more partnerships will fill in these gaps, and without specifics, hints at multiple new deals likely coming next year. After ElevenLabs was tied to a deepfake voice call of US president Joe Biden, the AI-audio startup struck a deal with Reality Defender to mitigate potential misuse. [...] "We don't ask my 80-year-old mother to flag ransomware in an email," says Colman. "Because she's not a computer science expert." In the future, it's possible real-time video authentication, if AI detection continues to improve and shows to be reliably accurate, will be as taken for granted as that malware scanner quietly humming along in the background of your email inbox.
The startup is laser-focused on partnering with business and government clients to help thwart AI-powered deepfakes. Even with this core mission, Colman doesn't want his company to be seen as more broadly standing against artificial intelligence developments. "We're very pro-AI," he says. "We think that 99.999 percent of use cases are transformational -- for medicine, for productivity, for creativity -- but in these kinds of very, very small edge cases the risks are disproportionately bad." Reality Defender's plan for the real-time detector is to start with a plug-in for Zoom that can make active predictions about whether others on a video call are real or AI-powered impersonations. The company is currently working on benchmarking the tool to determine how accurately it discerns real video participants from fake ones. Unfortunately, it's not something you'll likely be able to try out soon. The new software feature will only be available in beta for some of the startup's clients.
As Reality Defender works to improve the detection accuracy of its models, Colman says that access to more data is a critical challenge to overcome -- a common refrain from the current batch of AI-focused startups. He's hopeful more partnerships will fill in these gaps, and without specifics, hints at multiple new deals likely coming next year. After ElevenLabs was tied to a deepfake voice call of US president Joe Biden, the AI-audio startup struck a deal with Reality Defender to mitigate potential misuse. [...] "We don't ask my 80-year-old mother to flag ransomware in an email," says Colman. "Because she's not a computer science expert." In the future, it's possible real-time video authentication, if AI detection continues to improve and shows to be reliably accurate, will be as taken for granted as that malware scanner quietly humming along in the background of your email inbox.
Re: your mom (Score:2)
I hope they make their detector as accurate as possible. Because then it can be weaponized against them. In fact just get the AI neural network to keep changing the image/video until it "looks less fake". In fact, this is kind of how you train neural networks in a way.
Re:Laser focused? (Score:5, Insightful)
Cool! Do you have a Dockerfile available so I can deploy you into our K8s cluster?
Or are you not scaleable enough to help us meet our compliance requirements?
Just a headsup, this isn't about you. Its about the other 8.2bil-1 people in the world
Re: Laser focused? (Score:1)
Re:Laser focused? (Score:4, Interesting)
No, lasers aren't focused.
Yeah, I've seen people claim that too. Doesn't mean it's correct.
https://www.edmundoptics.com/f/laser-focusing-singlet-lenses/39590/ [edmundoptics.com]
Re:Laser focused? (Score:4, Insightful)
Those who use that phrase are selling something.
Well, no sh!t... People usually make startups to sell something.
It can detect deepfakes? So can I.
Good for you. I, however, can not, or at least I'm not at all sure that I can. And the majority of the human population is just like me concerning this.
With all that said, I highly doubt that what this startup has can detect deepfakes, either. It's trying to play an unwinnable game of cat and mouse. It might at best be able to detect yesterday's deepfakes, while today's ones are flooding our lives, and tomorrow it will be the same, and the day after tomorrow too. The ship has sailed. Now the only thing we can do is educate people to NOT trust anything they see or hear in an electronic medium. See something on the street, and you may trust it. See some video in YT or on TV, then assume that it's a lie. Yes, it will be immensely painful to do this, and it will be accompanied by a severe economic hit and a societal turmoil, but we might have no other choice. Either this, or perhaps start a Butlerian Jihad and start training mentats.
Re: (Score:2)
At worst it's like we are back in the 1900s before we had easy access to video and audio recordings. People managed pretty well then. I think it will be less disruptive than you suggest even if -- just like now -- older folks who aren't used to the new dangers are vulnerable to scams.
But I think we can solve this by just having cameras and audio recording devices sign their output using hardware keys.
Re: (Score:2)
I thought the same.
If it can detect deep fakes, then whatever it's using to detect them will be turned into making them better in future.
However, I guess for now they can make a tool that can tell if the person you're interviewing for a job actually is a person or not - that would likely make some money for the next few years while they figure out a better business model.
Re: (Score:2)
See something on the street, and you may trust it. See some video in YT or on TV, then assume that it's a lie. Yes, it will be immensely painful to do this, and it will be accompanied by a severe economic hit and a societal turmoil, but we might have no other choice. Either this, or perhaps start a Butlerian Jihad and start training mentats.
I think you are exactly right. I would have used the word "entertainment" instead of "lie" but that is probably just semantics. I think Thufir would agree: Multiple independent sources are a better way to getting ground truth. Not all sources are reliable, but ones that have orthogonal interests that are saying the same thing are more likely to be presenting the the real picture.
Re: (Score:2)
My take as well. Bombastic language, not credible claims. Looks like a scam to me.
Re: (Score:2)
A laser beam is still just light, and no beam of light exists that doesn't diverge. Just because it's all one wavelength doesn't mean it isn't still composed of photons acting like photons.
When last I checked, all optical drives -- from CD to Blu-Ray, and even old M-O and floptical formats -- have movable lenses, and actually DO focus their lasers onto the media, since the beam has to be tight enough to fit within the width of one track[*] at the point w
Do they sell the countermeasures too? (Score:2)
You know, introduce the detection mechanism, introduce the counter measures, cash in twice.
More Harm Than Good? (Score:5, Interesting)
The problem with any technology like this that can be run cheaply by the end user is that the more advanced attackers can just take that software and train models to specifically trick it. Sure, maybe it catches the low effort attacks but at the cost of potentially helping the more advanced attacks seem more legitimate when they don't trigger the fake detection.
The real solution is the same one we used back before photography, audio and video were common and people could pretend to be anyone they wanted in a letter. People need to be skeptical and authenticate interactions in other ways -- be it via shared knowledge or cryptography.
---
Yes, if you only run the detection on a server and limit reporting -- for instance only report a fake/non-fake determination after several minutes of video -- and don't share information about how the technology works adversarial training might be difficult but that has it's own problems. If researchers can't put the reliability of the security to the test there is every incentive just to half-ass it and eventually attackers will figure out the vulnerabilities like most security through obscurity.
Better Solution (Score:2)
Actually, it occurs to me that there is a technological solution to this problem. Simply have camera device makers sign their output using some kind of secure hardware key so the receiver can verify that the video was the input as seen by the camera on a X laptop or whatever. Of course, you still need to guard against attacks that stick a screen in front of a camera but that's doable if the camera has any focusing information or uses IR lines to reconstruct 3d information.
I'm sure there will be all sorts
Re: (Score:2)
Forget about iot. Seriously. Cameras would need to be secure devices for this. They are not.
Re: (Score:2)
Actually, it occurs to me that there is a technological solution to this problem. Simply have camera device makers sign their output
Simply? Oh no, definitely not simply. When you have a trusted device, that's the device people will want to attack, and the entities with an interest in it will be nations so they will have The People's money to spend attacking it. And since we're talking about commercial devices, they will be able to buy their own and mount attacks on them.
Sounds like a scam (Score:2)
Well, they want to sell something in the "AI" space, so "scam" is the normal approach.
Arms race (Score:1)
Re: (Score:2)
Yet another arms race, then.
Like trying to build a better battleship when the aircraft carrier will make them obselete.
I think what will happen is that people will learn to distrust videos from the internet the same as young people have learned not to believe what's in the papers, doubly so if said paper is owned by Murdoch.
Why this is BS (Score:4, Interesting)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
"There should never be seat belts because some people will die even if they wear it."
Standard moron pablum. Yes, some deepfakes will be so sophisticated they avoid detection. But not all.
Automated suppression (Score:2)
Case in point, the Hunter Biden laptop story, suppressed at a crucial moment before an election, across social media, that later turned out to be true. They may appologise later, but the effects are done, it can't be undone.
China probably also has such an automated system for ensuring "social harmony".
Easy solution (Score:2)
I'm giving up on modern communications. I'm going back to traveling criers, bards and paintings.
Coming soon, the Butt-lerian Jihad (Score:2)
As the people who run the chatbots that generate deepfakes are attacked, and the datacenters destroyed, by people who have had their lives screwed by them.
Re: (Score:2)
What do you mean coming soon? Someone already started it.
https://slashdot.org/comments.... [slashdot.org]
https://slashdot.org/comments.... [slashdot.org]