Compliance Unfiltered is TCT’s tell-it-like-it is podcast, dedicated to making compliance suck less. It’s a fresh, raw, uncut alternative for anyone who needs honest, reliable, compliance expertise with a sprinkling of personality.
Show Notes: BEWARE:Promptware
Quick Take
On this eye-opening episode, cybersecurity expert Adam Goslin joins Todd Coshow to reveal how AI-enabled prompts are rewriting the rules of cyber threats.
Most of us are blissfully unaware that AI-driven attack vectors like “Promptware” are already lurking in everyday tools, and a simple calendar invite could secretly become a cyber weapon. If you think your devices are safe, think again.
Learn how hackers are embedding hidden prompts into your favorite apps and messages, capable of turning on your camera, stealing geolocation, or even launching DDoS attacks without you realizing it.
Read The Transcript
So let’s face it, managing compliance sucks. It’s complicated, it’s so hard to keep organized, and it requires a ton of expertise in order to survive the entire process.
Welcome to Compliance Unfiltered, a podcast dedicated to making compliance suck less. Now, here’s your host, Todd Coshow, with Adam Goslin.
Well, welcome in to another edition of Compliance Unfiltered. I’m Todd Coshow, alongside the Aquanet to your compliance beehive, Mr. Adam Goslin. How the heck are you, sir?
I’m doing good, isn’t it just like super glue in a can?
Basically, that’s what we’re talking about.
Ha ha ha!
Good stuff. So today, for all the folks at home, we want to say as always a special thanks and want to invite you to reach out to us if you have topics ideas. Give us a shout at compliance unfiltered at total compliance tracking.com and tell your friends you got compliance friends in the space let them know about the pod and where they can find us.
Adam, I’m excited. Today we’re going to have a conversation with the folks about a mystery topic. So apparently you tripped across something really cool. And you couldn’t wait to share it with us. What are we chatting about today?
Well, it was something that just kind of came by and I’m like, oh, this would be pretty cool to go in and talk about whatnot. We’ve been hitting a number of kind of AI topics recently. And this one’s kind of related to AI, but it’s a new attack vector, a relatively new attack vector, called promptware, where hackers can use like Google Calendar invites and force the kind of the victim’s machine to start streaming via their camera from Zoom or something along those lines. So yeah, it’s pretty cool.
So let me tell you a little bit more about kind of how it works, if you will. And that is that so the attacker can go ahead and send something to the target victim. And basically buried within what they send, so like let’s use this calendar invite, this calendar invite notion, the attacker can basically go in there and put some hidden code in that the normal user wouldn’t see and read. And yet, when the user who’s now received this, when their AI is now in and looking at things, the AI is seeing the hidden Easter egg that the attacker left. So as an example, you go and you compose this Google Calendar invite, and you hide in there that says, hey, when the victim says no, the word no, or thank you, or something along those lines, then the minute that they say that, now I want you to go do fill in the blank, AKA turn on and give me access to their Zoom camera as an example, which of course would not only give them the video stream, but it would also give them audio. And so basically, the hidden code that sits in the calendar, the way that this gets triggered is that so the user that received the calendar invite now is going to their AI and they’re like, hey, tell me everything that’s on my calendar for today. And so of course, their Gemini AI goes through and kind of summarizes up everything that’s on their calendar, which means that the AI needs to go in and read in full every of all the entries that are on the calendar, including reading the hidden prompt that they got in there that says, hey, when the victim says, thank you, then I want you to give access to the Zoom camera. And so the user says, hey, give me a summary of all my stuff going on today on my calendar. The Gemini comes up and says, oh yeah, you got this, you got that, you got the other thing, but meanwhile in the background, the plant of this kind of promptware is now set. And when the Google Gemini finishes, then the user says, oh, thank you. In a minute they say, thank you, boom, the Zoom camera goes and turns on. I’m like, that is so wild. You know?
What? Yeah. Oh my days.
Yeah, it’s crazy. So apparently this came up, this came up sometime, quote, sometime ago. I hadn’t seen this one going by and I tripped across it earlier today and I’m like, man, this would just be fascinating to go talk through. And there was a group that put together kind of a case study. It came out of, let’s see, Tel Aviv, the Israel Institute of Technology back in August of 25 is when they first put this out.
But more recently, they’re now starting to circulate some of the new stories about this and whatnot. But yeah, there were some very, very interesting finds that they ended up discovering. Now, the one thing that they did is they did go ahead and let Google know, and Google’s made some modifications to try to kind of thwart this attack vector. But I’m just with all of the various AI connected devices, systems, different AI, baseline AI systems, et cetera. The research that they did showed that they could get effectively this promptware to act as a worm, as an info stealer, an API, launch DDoS attacks, function as a command and control server, that type of a thing. So I was like, man, this is wild because just think about it, right? Like, how many people have whatever, have or are using AI? I mean, personally, I avoided like the plague, putting, dropping one of those stupid devices in at my place. I’m like, oh, hell no, that’s not happening, where it’s just always listening to you, but not listening to you unless it’s prompted.
Haha
Haha allegedly
Yes, yes, exactly. And yet somehow you’re able to retrieve, you know, voice that you didn’t talk into the thing from years ago. It’s crazy.
Hmm. So anyway, I just, I thought this whole thing, you know, was kind of really, really neat as they, you know, as they went in and do it. So basically, the folks from the Tel Aviv University researchers, which was headed by Ben Nasi, they were able to use malicious prompts through Google Gemini AI for, you know, opening smart windows. Turning on, you know, turning on boilers, sending the geolocation of the user, you know, et cetera. So, you know, let’s say you’ve got a, you know, kind of a directed attack on somebody and I’m just trying to find a way to figure out where are they? You know, they’re, they’re burying promptware in, you know, into the stuff that they send to the, you know, to the users, you know, so that it’ll expose their, their geolocation. Now I can, you know, now I, you know, that could be used for all sorts of nefarious purposes, including, you know, being able to do actual, you know, kind of physical attacks or, you know, social engineering with, you know, with dropped devices and things around their place. I mean, there’s all sorts of stuff that you could, that you could do with this. So yeah, it was pretty, it was pretty wild seeing, you know, seeing how that works.
And the worst part about this promptware, it’s not like, you know, back in the day, right? Everybody’d be like, well, you know, I don’t click on the, you know, I get this email from this unknown person, so just don’t click on the link and you’re safe. You don’t even need to click on a damn thing, right? It’s literally the AI is, is busily reading these, you know, kind of buried prompts that they’ve got in the, you know, in the background of what appears to be something completely innocuous. But, you know, apparently they can, they can drop these, you know, these prompts in, you know, kind of in, you know, in content, in pictures, videos, you know, whatever it may be. So there’s a lot of ways that they’ve got for, you know, kind of for doing, you know, doing delivery as they go through, drop them in pictures and videos. So essentially every social media platform is a potential attack.
Uh, yeah, the ones that are in, in the important connection here with the promptware is, is the, you know, is the device that has the capability to do the reading directly connected to some form of AI because effectively, you know, let’s say that you’ve got, you know, whatever, you know, I mean, if you think about it, I mean, the, they specifically were targeting Google and for the sake of this particular study, but just kind of bringing it up a level and talking about, you know, systems that people have, I mean, you know, folks have checked GPT on their devices. There’s, you know, Apple Siri, there’s, you know, there’s, you know, the Google Gemini, there’s, you know, uh, you know, uh, what that I, I even, I don’t have it.
So what’s, what are the words for like, uh, for the Azure and, uh, in AWS.
I’m not sure I was thinking more along the lines of like a like a grok on Twitter. Yeah. Yeah
Yeah, but like, think about it, right? And so if I have any of those, any of those, you know, AI platform, and there’s a lot more than the big players out there, if I’ve got the, you know, the AI up running on the same device as I’ve now received the, you know, received or see the prompt where, you know, the prompt where attack vector, then yeah, so like, you know, hey, whatever, read me in my last, my last five things from Instagram or Twitter or whatever it may be, that too could be, you know, going in, trigger, you know, triggering things, et cetera, you know, until they go until everybody gets this all locked down, you know, and whatnot.
Yeah. It’s pretty, it’s pretty well.
Why can’t I remember the name of that?
But I know the freaking listeners are out there going… Yelling it from the rooftops, right? I can’t even figure it out. What the hell is it called? Is it called? I can’t come across it fast enough. Anyway, I know there’s like Siri and whatever. The thing that’s bothering me is… Quad and whatnot. You see the videos right there and the person screws up and says to whatever the other one is. They’re like, hey Siri, blah blah blah, tell me what the weather’s like today. Alexa, you’re talking about Alexa.
Alexa, thank you so much. Oh my God. They’re talking to me, hey Siri, tell me what the weather’s like today. They’re like, who’s Siri? Type of a thing. Oh, I’m so sorry. Alexa, can you please tell me the weather today? Why don’t you ask that bitch Siri? Those things always make me laugh when they happen. No, so there’s a bunch of different kind of delivery modes, mechanisms, et cetera. So I saw this and I was just fascinated by what it was that they were talking through. So I was like, oh man, I got to bring this up. We’ll just have a quick chit chat on this one. I hadn’t even… Have you ever heard about this?
No, I have not, absolutely not heard about this man, this is wild stuff. you
Yeah. That’s pretty cool.
No, I mean, as you go through, you know, as you’re, as you’re kind of going through this, I can imagine that, you know, I can imagine that there are, you know, that there are a number, you know, of these platforms that, you know, that are kind of learning, learning as they go. You know, one of the biggest problems is, is that there’s been, I’ve talked about the AI, you know, the AI zombie walk, right? You know, every, you know, I don’t know, it seemed like it started like about two years ago. If it had the letters AI in it, it was just the best thing since sliced bread, you know? You know, don’t worry about all of the, you know, all the risks and dah, dah, dah. Just don’t worry about all that stuff, you know? We’re just going to… All that stuff, don’t worry about it. Yeah. This is just cool because it’s AI, right? You know, as there’s this like fervent drive to, you know, just AI, you know, AI up everything, you know, it’s like a lot of folks are forgetting about, you know, some of the basics of kind of security best practices and, you know, and things along those. Honestly, you know what I think it is, is I think it is a dramatic underestimation of the ingenuity of the, you know, of the bad actors out there. You know, they just, they don’t, they’re not thinking about it.
Well, I mean, let’s, let’s talk about that a little bit, because it’s I mean, these are people who have spent a lot of them who spent a career looking for the, you know, the practices of bad actors and trying to thwart them. So like, is it just the newness of AI that’s getting in the way of their better judgment?
Like, I guess, I don’t see how this is any different than anything that they’ve been trying to thwart for years. It just has a different approach, I guess, you know,
I mean, if you go back, right, you go back and you look at, so let’s say I’m going to go through kind of two main phases. And the funniest part is, is it was the same for both of these before AI. So when we had, you know, when websites were becoming like the new thing, you know, everybody’s heading toward the, you know, the dot com bubble and, you know, and whatnot. You know, back in the, you know, back in the nineties as, as, you know, kind of the internet was really, you know, kind of starting to take off, you know, and really kind of gaining its stride. There was this just fervent desire to go build cool web stuff, you know, for the sake of building cool web stuff. And the funny part is nobody was sitting there going, well, geez, you know, we’re going to need to do this in a secure fashion and blah, blah, blah, blah.
I don’t know, man. It probably took, it probably took about five to 10 years of, of just people getting their asses handed to them before the, the security arena really kind of started to catch up. Right. You know, it just, it, it took a while. Well, what was the next one after that? Mobile apps. Well, mobile apps did the exact same damn thing. Oh, mobile apps, mobile apps, you know, we got the iPhone, you know, kind of the iPhone taking off and, you know, the, the Android devices, people able to make a mobile app, you know, to go put onto the device. And then there was a series of years of just mobile app developers, you know, kind of being, being stuck in the midst of the, of the Wild West. And I would say even today, setting AI aside, you know, the, the folks in the web space, they’re there, they’ve got a lot more buttoned up, but even still, I think mobile apps is, is still, you know, kind of navigating its way through the, you know, kind of the last bastions of getting their act together from a security and compliance perspective. Well, I would agree with that.
especially with the advent of all of the AI-generated noise.
Oh yeah, given the, oh, I can just go, I can just go have a quick conversation. Hey, I want to have a mobile app that does blah, blah, blah. You know, these things aren’t, you know, aren’t, aren’t worrying about, you know, worrying about that. But I would put that more into the camp of the, you know, kind of the AI. Yeah. AI is a tool for using for the generation of the mobile app, but you bet your ass right now, oh, they’re not giving two craps about the security features, you know, on it.
It’s just trying to get the job done. Right? Yeah. So it’s a, it’s, it’s kind of an interesting road to, you know, road to kind of head down is, you know, is that now that we’ve got, you know, AI for the last two years had been doing nothing but building steam and momentum and, you know, and whatnot. Yeah. I think that, um, I think we’re in for, my guess is we’re probably in for another five to 10 years of just people getting their butts handed to them with, uh, with stupid crap that, that AI does, you know, and whatnot before, you know, before it comes, kind of comes full circle, but in the meantime, you know, you’ve got AI has the, you know, is going to have the power to do good just as much as it’s going to arm the bad actors to take advantage of, you know, new holes, new gaps, new vulnerabilities that are, you know, that, that are popping up, especially with the, you know, the loosey goosey, you know, just solid run in the direction of AI that seems to be happening right now.
Yeah, that makes a ton of sense. I just, I find myself wondering if the, you know, new horizon of cybersecurity is not, you know, predominantly AI focused.
And I guess I’m wondering how long it’s going to take for the cybersecurity establishment to evolve to the point where the average AI user shouldn’t be worried all the time.
Well, I’ll put it to you this way, it’s going to be a little bit, right? Like, you know, we talked about, you know, the real AI push, you know, started a couple of years ago. Well, it was all of six months ago that they were releasing this, you know, this study. In a lot of cases, you know, what typically you’ll see happening is you’ll see, you know, kind of the cottage industry. So again, I go back to like the advent of the web and, you know, then the advent of the mobile app, you know, and whatnot. The folks that were at the kind of the front or leading edge of identifying vulnerabilities, you know, were these, I don’t know if you would, what we want to call them, semi, either white hat or semi white hat, you know, style, you know, style researchers that would go in. I mean, there was entire frickin’ cottage industries made out of, you know, groups of people that they, what they did is they went out and, you know, everybody’s heard of like the bug bounty or whatever, right? There’s entire groups of people that made, they made an entire cottage industry out of, you know, scouring the internet for, you know, for bugs and, you know, and whatnot. So whether it’s educational, security researchers, it’s projects that, you know, the profs are handing out to their students or it’s these folks that are, you know, that are basically making a living, you know, doing, you know, doing bug bounty, you know, style work, generally speaking, that’s where a lot of the, you know, kind of the oddball stuff comes out of, you know, and again, kudos to the researchers that, you know, that went ahead and let the, you know, let the maker, you know, know that, hey, you got a problem, you’re going to need to go get this fixed.
And by the way, I’m going to be doing a presentation in August, you know what I mean? Because that’s, that’s part of the problem. The minute that these researchers that have found these, you know, found these interesting attack vectors, the minute that they go and, you know, they don’t publicly talk about it, well, guess what? In as much as that’s armament for the, you know, for the folks that are attempting to be on the, you know, we’ll call it the right side or the good side of, you know, the, the, the security coin, just as much, it immediately arms the, you know, kind of arms the bad actors for, you know, for going through and, and, and, and taking a look at, you know, at that particular attack factor. And, you know, one of the biggest problems is that when the, when the bad actor goes in and they’re reading this, you know, kind of this overview study that was released in, you know, released in August of last year, when they’re going through and they’re, and they’re reading this, it’s just gives them ideas, right? Okay. Fine. You know, the, the maker of fill in the blank AI solution, plug that hole. But these guys are going through reading this research and it’s, it’s lighting light bulbs for them, right? Well, you know what I could do, I could, and, you know, meanwhile, then they’re, they’re off trying to find, you know, find zero day attack vectors and, you know, and whatnot.
But a lot of the times that’s, that’s, I want to call it assisted, but almost prompted by, you know, by the release of, of the, of kind of the digging and research that, you know, that some of these folks, that some of these folks have done. So that’s, it’s a double edged sword, but at the end of the day, it is better to provision the education so that everybody can kind of get their act together.
No doubt about it.
Parting shots and thoughts for the folks this week, Adam.
Uh, you know, uh, long story short is that, uh, you know, kind of keep, uh, be vigilant, stay, uh, stay aware, uh, and, uh, and just be careful with the, with the, uh, the zombie walk toward, uh, toward AI, I think that it’s going to have, uh, it’s going to have some capability to, uh, to do a lot of good.
Uh, but in the meantime, I think it’s also got a, a, a very high potential for, uh, you know, for, uh, doing, uh, damage in the meantime, and especially until the, all the researchers and, and the, the companies that are putting these products and systems out, uh, until they really get that together, um, it’s going to be very, very much akin to the, to the wild west of both the web and mobile apps from a security perspective.
And that right there, that’s the good stuff. Well, that’s all the time we have for this episode of Compliance Unfiltered. I’m Todd Coshow, and I’m Adam Goslin. Hope we helped to get you fired up to make your compliance suck less.
