Compliance Unfiltered is TCT’s tell-it-like-it is podcast, dedicated to making compliance suck less. It’s a fresh, raw, uncut alternative for anyone who needs honest, reliable, compliance expertise with a sprinkling of personality.
Show Notes: AI Fraud, Deepfakes & the Death of Trust
Quick Take
On this week’s Compliance Unfiltered, AI-driven fraud is escalating, with deepfake voices and synthetic identities posing new threats.
This episode reveals how traditional security measures fall short against sophisticated scams, like a $25 million fraud in Hong Kong. Discover innovative detection techniques and why real-time monitoring is crucial.
Perfect for security professionals and leaders, this episode is a wake-up call to adapt and protect your organization from AI-enabled deception.
Read The Transcript
So let’s face it, managing compliance sucks. It’s complicated, it’s hard to keep organized, and it requires a ton of expertise in order to survive the entire process.
Welcome to Compliance Unfiltered, a podcast dedicated to making compliance suck less. Now, here’s your host, Todd Coshow with Adam Goslin.
Todd Coshow:
Welcome in to another edition of Compliance Unfiltered. I’m Todd Coshow, alongside the enchilada sauce to your compliance chimichanga, Mr. Adam Goslin. How the heck are you, sir?
Adam Goslin:
I’m doing good. It is today, Cinco de Mayo. So nicely done.
Todd Coshow:
Thank you, sir. Today we’re going to have a conversation up and down about fraud, but before we get there, I want to make sure that we say thank you to our listeners.
We would love to hear from you. If you’ve got something that you’d like to share, if you have some suggestions or show ideas, please do not hesitate to reach out to us at [email protected]. We’re excited to hear what you have to say.
As I teased just a moment ago, today we are talking about AI fraud, deepfakes, and the death of trust. Let me ask you a kind of off-the-wall question, Adam: if your CEO called asking for a wire transfer, would you trust it? Obviously, in this instance, you are a CEO. In your past life, if you were to receive a call from your CEO, would you trust it?
Adam Goslin:
It’s becoming more and more difficult these days. I was going to say, if I got a call from me, then the cat would be out of the bag.
Looking at it philosophically, it used to be that the bigger concern was somebody taking over an email account and passing you weird messages, or they’ve ghosted out the number and they’re sending texts to your phone from the appropriate number, or phone calls that would come in with somebody trying to mimic their voice or whatever. But nowadays, it’s not a safe assumption.
AI can clone voices. You can clone identities well enough to trick. Let’s say you got a real strong training program and people are going through training not once a year, but twice a year, three times a year, monthly. It’s tough because when you’ve got the capability to mirror off faces, voices, etc., it becomes astronomically difficult to blindly trust.
It’s the same mantra that we’ve had in this arena for quite some period of time. One needs to ask themselves, does what is happening right now feel right? Trust your gut and be a little skeptical because I would much rather have somebody coming in and saying, “Hey, I got this request. I’m coming through a back channel just to validate that this is actually legitimate.” I would rather answer those types of inquiries from my people a hundred times than take one opportunity for somebody to not bother or drop the ball or trust what they were hearing. It’s what these people need to do.
There was a recent thing that happened out of Hong Kong, and this was just fascinating. It was a finance worker in Hong Kong where the fraudsters were using AI deepfake technology. They impersonated the company’s CFO and other colleagues on a video conference call. Initially, the employee was doubting the request, but was convinced when all these different participants in the video call appeared to be familiar staff members. They basically set up this multi-person video call with everybody where everybody except the finance guy was fake.
They basically told them they needed to execute a confidential transaction. That’s why you hadn’t heard about this previously, etc. They convinced this dude on this call to go in, and they did 15 different transfers to five different local bank accounts totaling $25.6 million. They didn’t even figure out what the heck happened until later on, the employee checked the validation with the company’s UK-based headquarters. That’s when they ended up discovering, “No, you’ve just been had.”
It’s wild what’s going on out there right now. It’s extremely difficult for folks to just take whatever they’re seeing at face value anymore.
Todd Coshow:
Speaking of, how real are deepfakes today? How does synthetic identity fraud fit in here?
Adam Goslin:
It’s real enough to be capable of bypassing human intuition. You’ve got voice, videos, messages, all of which can be faked in real time. That’s one of the biggest problems.
You go back to that Hong Kong company. They literally had them completely convinced eventually that, “Oh, yeah, this is legitimate and let’s go about getting this done.” It’s getting good. I don’t know, man. I can still go in and look at these videos that they’ve made and say, it’s not perfect, but it’s good enough to pass a high-level sniff test. If you’re not really paying attention, then again, just trust your gut. We should make t-shirts that say that or something.
The synthetic identity fraud is when attackers are basically combining real data, SSNs, emails, or identifying information about the person with AI-generated details so that they can create identities that are passing sniff checks, etc. It’s definitively getting harder and harder for the folks out there to be able to tell the difference. That’s for sure.
Todd Coshow:
Why is AI such a force multiplier for fraud? Is it not just like phishing 2.0?
Adam Goslin:
What AI allows is very personalized attacks at scale. It doesn’t cost them a lot. It’s cheap to do. It’s quick to accomplish, globally distributed.
I sit here and think about if somebody wanted to do this with you and I, we have how many podcasts out there, etc. We’ve got pictures on the internet that they could leverage and feed into the AI. I guarantee you, somebody that had a directed effort to want to go do it, they could probably pull it off with you and I, probably easier than most people, just because we have so much stuff out there. We’re going to be prime targets.
We’ll have to bring that up on the team meeting. Just a friendly reminder.
It’s not just like phishing 2.0. This is taking a step up. It’s targeted. It’s intelligent impersonation to exploit trust, not merely for the use of tricking users.
I’m sure you’ve heard about what they call the grandparent scam, where folks call up the grandparent, “Your grandchild has been abducted and we’re going to charge you ransom,” and in the background, they got somebody yelling and trying to do it, a voice that sounds like the grandchild. It’s like taking that and putting it on roids. We’re kicking it up several levels at the state of the game.
Todd Coshow:
Why is compliance falling behind in this arena? Are compliance programs keeping up with this or no?
Adam Goslin:
Generally speaking, no. A lot of the compliance programs, it depends on the compliance suite of controls that you’ve got. In some cases, the organization is using an extremely prescriptive standard for protecting certain things within the environment. When I’ve got static rules, meanwhile the real-world rules are changing quickly, that means that they’re probably going to get stranded.
The other problem, so let’s take a compliance standard that is directional in nature. I’ll just use SOC as an example. With SOC, I have to meet these certain criteria and then I develop controls and testing steps that support the accomplishing of that particular element of criteria.
The one thing where they run into problems is people are creatures of habit. I’ve already crawled through the glass. I’ve already created the controls to meet the criteria, the testing steps to meet the controls. Now we’ve got this. It’s all set. Even though the underlying tenet for the compliance program is intended to be dynamic, the reality of most organizations is that they go in, establish the rules, and it’s rinse and repeat every single freaking year.
So what is the tactical difference between the certification or standard that’s prescriptive in nature versus criteria-driven at the end of the day for the vast majority of companies? I know there’s people listening to this. They’re chuckling because they know damn well that when we made our controls and testing steps for a SOC engagement eight years ago, they’ve never gone back to tweak those things, alter them, change them, improve them, keep up with the times, any of that stuff. It’s funny that that’s the way that it works.
The reality is that attackers these days can mirror people’s identities. They can scour and scrape the internet for the less savvy users and be able to glean out their security questions that may be used to protect their digital identities. They can bypass password-based systems. There’s a lot of things that are putting the compliance arena a little bit on its heels.
I think there’s some responsibility to be had both by the folks that are putting together the certifications and standards, but equally so, the organizations that are subject to them. It’s your responsibility to take this stuff seriously and perform an annual continuous analysis of your control suite for managing your standards and certification engagement.
Todd Coshow:
Are assessments giving a false sense of security because they’re not necessarily as up to date as they could be? Where are organizations most exposed?
Adam Goslin:
I wouldn’t go so broad as to say, in general, assessments are giving a false sense of security. It literally depends on the people involved. I’ve seen organizations that are doing checkbox compliance. I’ve seen some of the shitty assessment firms that go with a checkbox mentality for the audit or the assessment. I’ve seen others where one or the other of those groups cares, and I’ve seen ones where both care.
At the end of the day, I wouldn’t necessarily equate it as much to the assessment is provisioning some false sense of security.
That said, the closer either of the parties involved, whether it’s the person going through it or the audit assessment firm, are to checkbox compliance, just trying to get her done and move on, then yeah, the greater the amount of risk that’s ultimately going to be had by the organization that was subject to audit.
I can go through, pass an audit, and still be completely vulnerable to AI-driven fraud by checking all of the right boxes.
The exposure comes into play anywhere where you’ve got assumptions of trust. Some examples are your financial approval vetting validation process, where if it’s a payment that’s over so many dollars and we have two people sign off on it, and one of the two people that’s signing off on it is virtual and faked, now you’ve got a problem.
Even vendor communication, discussions, interactions, things along those lines that you’d have with vendors where you’re trying to use a certain level of assumption of the trust factor, that starts to play into it. Notwithstanding baseline account access, accounts that are being leveraged on the platforms, and God, there’s 18 ways from Sunday to be able to bypass, violate, skirt existing controls. There’s a good amount of variability there.
Todd Coshow:
That’s fair. Considering the mechanics of modern attacks, what does an AI-driven fraud attack actually look like?
Adam Goslin:
What most people don’t realize is that the attack in its earliest stages doesn’t even look like an attack. It looks like any other person that would be scouring the company’s website, looking up stuff on members of your team, doing deep dive digging into available materials, various internet-based accessible sites, things along those lines.
All of that groundwork has been going on. But when the actual fraud attacks start to happen, they’ve been at it for a while. That’s probably one of the biggest misnomers, is people will typically relate the attack to its actually being enacted, and the reality is that they’ve been at it for a while before you can even plausibly be aware of what’s going on.
This is all publicly accessible data and information that’s out on the web, etc. We just happen to be building tooling that’s really good at collecting it up and taking advantage of it now. It’s pretty wild.
It’s highly personalized outreach. You could have cloned voices, cloned images, realistic emails, text messaging, better yet, context-aware messaging. This could literally be AI that’s set up to provision responses and respond in real time to people. There’s a lot of possibilities for things that could be happening in terms of that attack. There’s a lot of channels that they can go and head down.
Todd Coshow:
Why are attacks so hard to detect, and what’s the biggest misconception about the attacks themselves?
Adam Goslin:
First and foremost, they’re hard to detect because this looks like legitimate traffic, legitimate calls, legitimate texts, legitimate video calls. They have context-appropriate tone, timing, recognition of relationships, behavior of the individuals.
The people that are going to be very good at performing these types of attacks have their own artistic process that they go through, and they’re starting to get very good at it.
Some of the biggest misconceptions that happen during these attacks: a lot of folks are saying, “We do our annual security awareness training, so our people will spot the red flags. We give them security reminders monthly, weekly.” It doesn’t matter.
At the end of the day, you catch the right person in the right moment with the right narrative, with the right measure of convincing involved, and it’s going to be extremely difficult for them to connect those dots. It’s going to be very challenging for them. That’s probably the biggest misconception, people just leaning on the training that they’ve been doing previously.
Todd Coshow:
That makes sense. What’s next? There has to be some sort of shift to behavior. If identity can’t be trusted, what replaces it?
Adam Goslin:
It’s really going to take a deeper level. There are things that I can fake out, a person’s image or their phone number or their email or whatever. The part that’s going to be challenging for the attacker is mimicking precisely the behavior of that user. In many cases, the attackers aren’t close enough to the people to know how these users act, who they actually are, not just who do they either claim to be or who have they put forth.
When you’re looking at behavior, the behavior-based detection capabilities, you’re looking for anomalies, strange login locations. This person always, for their login or their MFA, they always go here, here, and here. When Adam logs in, what is he usually doing? He’s usually going here, he’s going to this engagement, he’s performing this function, and now I’m seeing some different traffic patterns out of a login, ostensibly from Adam.
Deviations in normal user behavior, all of these things combine together to give you a high-level overview of, again, this is a sniff test. But we’ve had a mantra for a minute or three called trust but verify. If I’m seeing strange things in the behavior patterns, then have a validation arm that goes with it.
Todd Coshow:
Why is anomaly detection more effective?
Adam Goslin:
There’s two things here. When it comes to identity, we already talked through I can fake this, I can fake that, I can fake the other thing. The perfect fake identity is not going to be capable of mimicking user behavior over a period of time. The activities of the user are almost like a fingerprint or a signature of their normal activity.
But in analyzing the user behavior, one of the biggest downsides is there is an astronomical amount of data, inputs, logs, etc., that are going to be flying in for analysis.
Realistically, we use the same tool the attackers are using, but use it to try to thwart. That’s us leveraging AI to look at traffic patterns and these anomalies. AI would be better than a human at being able to detect this stuff at scale. That’s an area where we can use the same tooling for the good guys.
Todd Coshow:
You touched on it a little bit, but what’s the fundamental shift in compliance?
Adam Goslin:
I think where compliance needs to head is in a couple of different directions. A lot of the existing compliance frameworks are built on established controls with testing steps or prescriptive things that need to be done. They’re rigorous.
We need to get a little more dynamic with the ways that this behavioral analysis fits into a continuous realm of oversight for the target organization. In a modern compliance program, we need to make sure that we’re moving toward things like continuous monitoring and behavior-based analytics, real-time alerting, identity validation that goes beyond credentials.
We talked about trust but verify. The minute that the team is getting an alert, they’re going through doing secondary identity validation, things along those lines. In some cases, the bad guys may have already got in, but if we can catch them quickly, then we’re mitigating or trying to mitigate risk here.
Are you going to be able to stop everything all the time? Probably not. But if your detection capabilities are dialed up, you’ve got a hell of a lot better shot at mitigating the amount of damage.
Todd Coshow:
Is one-time authentication and MFA still viable here, or does it not even matter at this point?
Adam Goslin:
I wouldn’t go so far as to say it doesn’t matter. We’re going to need to keep our eyeball on this one.
I wouldn’t abjectly say, “That’s completely useless.” No, that’s not true. It’s still thwarting a crap ton of stupid shit. But the reality is that we need to be in this mode of continuously reassessing our trust model.
We talked about looking for anomalies. Where are they logging in from? Where did they do their MFA confirmation through? The source IP addresses and the data, just integrating that into the process certainly is one way that we can still leverage it.
But on the one-time authentication and MFA side, I think it’s just an arena where we need to keep our eyeball on it. We need to be reassessing it.
Todd Coshow:
That makes sense. Parting shots and thoughts for the folks this week?
Adam Goslin:
I’ll go back to what I said earlier, because this is seriously important. There’s a lot of organizations that for a long time have gone with a set it and put it on autopilot notion with their compliance. There’s a certain measure of comfort in doing that. There is a certain measure of efficiency in doing that.
But in my opinion, organizations have gotten far too complacent with not reassessing, really kicking the tires of the controls that they have for their organization.
Organizations need desperately, especially right now, I guarantee you there’s companies out there that haven’t changed their controls in a decade. What good are you really doing when the whole landscape is changing under your feet, but you haven’t?
You’ve got to take that seriously. We need to have the behavioral analysis integrated into the solutions because the folks that aren’t making those adjustments, aren’t making those changes, aren’t helping to protect their organization.
Somebody is going to trust the wrong person, and the attackers are going to exploit that trust at scale. It’s not just about verifying identity. It’s about having the capability to detect that deception in real time. That’s going to be an absolute key for organizations moving forward.
Todd Coshow:
And that right there, that’s the good stuff. That’s all the time we have for this episode of Compliance Unfiltered. I’m Todd Coshow.
Adam Goslin:
And I’m Adam Goslin.
Todd Coshow:
Hope we helped to get you fired up to make your compliance suck less.
