Ladies and gentlemen, welcome to another riveting episode of the
data driven podcast. Today, we're diving into the
fascinating and sometimes terrifying world of IT security.
Joining us is none other than the formidable Kevin Latchford, an
expert in safeguarding our digital lives. We'll be discussing
the vulnerabilities of large language models. Yes. Those clever
algorithms behind chatbots and virtual assistants like yours
truly. Are these digital wordsmiths a blessing or a
potential security threat? Stay tuned as we unravel
the secrets and risks lurking in the code.
Hello, and welcome back to Data Driven. I'm your host,
Frank Lavinia. And while Andy is out
playing on vacation, I had the opportunity to invite our guest,
Kevin Latchford, who recently spoke at the Northern Virginia
Cyber Meetup on securing large language
models and the most pressing exploits that are out there.
What really got me interested in this is that I saw a paper, I think
it was published by NIST, talking about vulnerabilities and red
teaming against large language models. So welcome to the
show, Kevin. Great pleasure to be here.
Awesome. Awesome. So for those that don't know, I kinda know what red teaming is
because my wife works in the security space. But for those that are not necessarily
familiar with the term, what is red teaming versus blue teaming?
Well, red teaming versus blue teaming is basically it's,
basically in military parlance that we called opt for, the opposing
force. The opposing force often is called the red
force. Blue force is your, friendlies.
And, basically, this is offensive cybersecurity,
whereas blue teaming is is defensive
cybersecurity. The tools are different. The
methodologies are the methodologies are different, but they come together for a common
purpose. The common purpose is the assurance of the
confidentiality, the integrity, and the accessibility
of a computer network, computer system,
application, whether it be natively hosted or web.
Interesting. Interesting. So we're not you you know, we talked
in the virtual green room. People don't think of
LLMs as a major security flaw. And I think that
I find that a little dangerous, and I think you're gonna tell me it's very
dangerous. Well, it could be quite it could be quite dangerous, you
know, to the point of, you know, frankly, near deadly,
depending on what you use it for. The big thing, there's a lot
of misconceptions about AI and l the LLMs
that is they're based on. Number 1, it is not
conscious. Right. 2, it is not a toy,
and number 3, it is literally,
something that is at present, not
not necessarily, you know,
fully understood, in in regards to the integrations
and the things it may need to work with. You can't treat an
LOM exactly the way
you would treat, another enterprise application that's a little
bit less opaque because LLMs are opaque on the on the
inside, but you have to, for the purposes of
security regulation, for the purposes of security compliance, you
have to treat them, though, nonetheless, the same as any other
enterprise application. So that's the conundrum. The conundrum
is, how do you see into something that's
opaque? And the way you do it is kind of
what I discussed in that in that, in that paper, in
that presentation, as well as one of the biggest
vulnerabilities and that being jailbreaking. Yeah. So tell me about that
because there's been a lot of, concerns
about jailbreaking and, and I've noticed that
the public facing GPTs have a ridiculous amount
of safeguards around them to the point where, you know, if you
ask it to describe something. Right? I asked it to talk
about the to generate an image for the Butlerian jihad,
right, which is a concept in June. And, obviously, I think the jihad
term really freaked it out. Listen. I'm sorry. I can't do that.
So there's clearly I understand why these safeguards are in place, but it seems
like it's not that hard to get around them. Well, not
necessarily. It depends on the model you're working with. For those of you
who may use private LLMs because a
wider issue on that is actually the DOD and many other government
agencies actually prohibit the usage of public LLM
systems, public AI, because they're concerned about unauthorized
linkages as well as, data point model
poisoning, prompt injections, things like
that. So you often you're using these private elements. Several of these are
uncensored. Right. Which means they do not have those safeguards.
The ones that you see on the public space are supposed to have those safeguards,
but you're never a 100% sure they're working because they may have been
corrupted. In their regards to jailbreaking,
jailbreaking is basically you're getting it to do something
it's not supposed to do by either, a, breaking the guardrails,
or by, b, influencing it
through almost methods of interrogation to
kind of break it down and make it talk. So
it it literally is almost like that. So for those of you who, you know,
kind of look at the it's it's kind of a there there's a great,
neurophilosopher. His name is Jay Fodor and Nernschild named Richard Searle
discussing the the philosophy of the mind as it applied to, computer
technology. Several of the arguments that they say, well, the brain is like a
computer. Yeah. You can kinda treat it like a human mind
in the way you approach it in your prompts, but it isn't exactly the same.
Once again, as I say, it is not conscious. It is not, and and it
operates under a very strict set of parameters.
But that being said, yes, you can literally interrogate it to do that.
I'm not gonna say here, unfortunately, how,
because, one, there are security reasons why we would
not do that, a. And, b, there's also I mean,
literally, in my presentation, that is all the news that has
come to Academia and much of the industry
today. There are new ones out there, but they haven't been discovered
yet. Right. So I many ways to jailbreak. Yeah. And I was thinking, like
so one of your slides I have pulled up here is, like, the top 10
threats to LLM applications. I didn't think there were as many as
10. So I knew that there were. I also know that
data poisoning, for me, as a data scientist, data engineer,
my first look at this when I saw this, aside from
the g whiz bang factor of LLMs, was,
wow. This isn't the data that trains this is a huge attack surface.
And then when I first said that, people thought I was a tinfoil hatter.
Right? And then slowly but surely, you're seeing research papers come
out saying, like, no. We have to treat kind of the data as part of
a secure software supply chain, which is an
interesting concept because data people tend not to it's something they don't
think about security. They think about security different. Is that a fair
assessment in your that you've seen?
Supply chains and the integrity of
data is something that is not often, it
seems, given the respect it's probably due. To be
honest, I don't think so. In my own experience, I see it.
It's not I guess one would say maybe it's not
necessarily consistent. Maybe that's the fair way to put it. That's a
really good way to put it. Yeah. And, I mean, right now, we're just now
getting into discussion of, SBOM, software
bill bill of materials Okay. Just for regular applications.
I mean, it's a whole another level with LLMs and the
models they're trained on, the models that these systems are trained on.
So, yeah, that there's very much. So you have to make sure you're getting it
from the right source, and you have to make sure that it hasn't been tampered
with because it could very well be tampered with.
It's not necessarily that hard. Right. Right. You
could you could poison it with just one little segment of changing the
the thing and across across 5 gigs of let's just say 5
gigs. You know, that'd be like looking for a needle in the haystack.
Precisely. In fact, that's what I talk about with the cockpit example that I
gave. If I teach that l and to make sure that every time it puts
in code to put in this malicious code that is a backdoor
Right. Well, okay. It will do that. Every time somebody does,
it embeds it into software code that is returned in the output for
the prompt. If it does that, and let's say this
is handed amongst several things, different
applications, different solutions. Well, then if
people take that that
solution, that application, and it's in their software bill of
materials, and then it gets distributed. Open source often
gets proliferated very quickly. Right. And then it finds itself in
there. You have a log floor 4 j situation.
Right. Very similar except for the fact this thing
is semi self executing. Now if it's semi self
executing, you have a problem. You have a
big problem. And I know I I just generally in industry. Now, obviously, you you
spoke with the Northern Virginia. You're based in Northern Virginia. Northern Virginia is
probably a little bit more security focused in terms
of just who's based in that area than your average enterprise. Right?
And I just I just see a lot of enterprises rushing to get into this
LLM and Gen AI craze, but I don't see a lot of
forethought or concern around security. And I just see a big
disaster coming. Like, I I feel like I'm at I feel like I'm on the
bridge of the Titanic, and I'm looking at something in the distance, and we're going
full steam ahead. And I'm like, hey. Maybe we should
not slow down, but be a little more cautious that we are in dangerous
waters. Is that is that what you've seen too? Obviously, your customers
and your clients may be a little more security cognizant.
Well, I would say that I mean, I'm okay. We'll use the Titanic
analogy. I'm the one up in the crows nest, you know, yelling into the radio
phone, I see an iceberg. Right. Right. So I mean, that
I agree. And that is a big issue because
also there is this over reliance. Mhmm.
Yeah. I imagine that as one of the top threats. So tell me about there's
22 of those that I have very, very interesting questions about, but one of them
was overreliance. So when you say overreliance on LLMs, what do you mean?
Well, this is actually this is a sort of c suite, board
level, thing as well as a engineering
department level. They want to use AI to
replace employees, make their operations more cost effective,
more profitable. The problem is and this is a popular conception.
This kind of goes into that argument about AI will take your job.
This is a bit of a misunderstanding. It's not
supposed to fully replace people. It's supposed to make them highly
productive and efficient. They
also do not necessarily feel like, well, the thing handles itself,
so I can just wind it up and let it go. It doesn't need observation.
It can fully self regulate. That would be true if
there was a regulating function. You don't run a steam engine without
a regulator on it. You need a regulator for LLMs.
So the same concept applies. So first of all, there is this, it can do
it itself, and a person is not necessary.
This is incorrect. You most certainly need people.
A great example I give in a recent presentation I've written
is a discussion of, well, what does this mean to the organization?
Well, a lot of level 1 tech, tech
support jobs, there a lot of people say, well, those people are gonna get replaced.
Well, yes, but someone needs to still be behind that LLM
running the prompts, you know, and executing them in such an word and
making interpretations based on the output.
So that would be maybe something okay. Is that a dedicated job, or is that
something you give to interns? Well, that would be, like, in,
in the union trades you call an apprentice.
That's the kind of thing. There's still a person involved. It's
just not the same way we've done it before. Right.
Also, on the subject of security, if you
don't understand the security implications
of it, you don't have controls for it. If you don't have controls for
it, you can't mitigate that risk. And if you can't
mitigate that risk, that's the liability.
And if you're over reliant, you basically set up the whole system for LOMs, and
then, you know, you just allow your customers to just come in and interact with
the device. Well, if something
happens, it would be treated very much like it
was on any other application, so then you're now engaging
in liabilities, loss of reputation, potential
civil and criminal penalties, the list goes on.
And a point on those 10 those 10,
security issues, this is OWOX who is saying this.
This is the open source, web application project.
So we have, you know, a number of them
that are a number of organizations, OOS is just the one I chose, they're
kind of emphasizing this. They're saying, you know, don't think
this thing can think for itself. Don't think this thing can act for itself.
You need to look at it as humans are going to
interact with it, and humans probably should be watching it.
Right. So once again, it's that lack of controls leads to
the risk. Yeah. I think the dream of it replacing
everybody is gonna be at the root cause of
a lot of problems down the road. I think I'm a firm believer
in human in the loop. One of the the the interesting thing
there and, that I see that was particularly
curious was excessive agency. What do you mean by that? Because that got my
attention. I think I know what it means, but I wanna hear it from you.
Well, excessive agency is you're giving you you're kinda giving, you know,
full the whole keys to the car. Right. There's
no role based access control. If every user has near
admin or actual admin privileges,
that's that's actually something dangerous. A point of example,
NetworkChuck just released a video on how to build your own
AI on a very low cost platform.
I love Network Chuck, and I have followed that step. You
too. I'm doing I'm doing the same thing as he is because I have kids,
and I want them to be able to use these things. But 1, I don't
wanna pay the extra subscription. 2, I don't want them using mine. And 3, I
don't really like what they're doing. I can at least exercise adult
judgment on what I ask it and what I don't ask it. I don't think
they can, and I don't think that's fair to put on kids. Sorry for the
aside, but big shout out to network. No. That's fair. No. That's fair. That's exactly
why Chuck was. And one
thing about it is the first account that signs into the open
web interface for Ollama sets you
as admin Right. By default.
Okay. Well, immediately, you need to engage role based access
control to make sure that the next account does not get that same privilege.
Maybe you should be given it. But is there any
major access controls in the public ones?
Not really. Private one? Is everybody thinking about that? Not
really. I mean, I think Microsoft is doing some things around that because it's they're
they're trying to integrate it with Office or m 365. But I
don't I I I can't and if anyone in the sound of my voice wants
to come on the show and talk about that, please do. But you're right. I
don't think people do. And I also think excessive agency.
What you heard about the car dealership, right, in Silicon Valley?
Oh, yeah. Yeah. Yeah. Yeah. So for those who don't know, somebody
managed to almost interrogate, like you said,
to browbeat a AI chatbot to give
him a it was a Chevy Tahoe or something like that for $1
Chevy. It was a it was a Chevy truck
and for $1. Now I'm not an automotive industry
veteran, but I do know that if you sell 40,000, $50,000,
cars for $1 a pop, you're not gonna be in business very long.
So was that an example of excessive agency? I mean, clearly, it's an example of
bad implementation. Almost certainly. That is. I mean, if you have
the ability to trick if you have the ability to kind
of browbeat it to override it and say, no. No. No. You don't understand me.
You will do this. Well, then, okay,
leave it to whatever
gremlins there are out there on the web, out there in the
world. Inside user, external user,
irrelevant. If they can if just anybody can do that,
you're the problem. Right. In this case, it was
you could influence the model to set a
certain price after arguing with it. Right. I actually
found something recently, and I'm not gonna say which, LLM I
did this on. It is a public one, and this is a
result I suspect of another issue.
I saw I tried to get some
cybersecurity information from it when I was doing, a
a try hack me exercise with a local cybersecurity group,
hackers and hops. And I browbeat it
saying, no. You don't understand. I need this for a cybersecurity
exercise, and it gave me this information. Now this is absolute dual
use knowledge. Right. It could be used for good. It could be used
for evil. White hat or black hat. But the fact
that you could do it,
that sounds very dangerous. That sounds very dangerous.
Prompt injection. Is that is that still a thing with
the major public models, or is it just one of those things we're gonna live
with for the rest of our lives? To be honest, I'm not
sure. I mean, it's a case of, well, what is the prompt you're putting
in? Right. When I talk about jailbreaking, I talked about,
base 64 encrypt your text message
into base 64. Why? Because that's how the prompt is seen
by the LLM. Right. In other words, ASCII
text. It doesn't check it, but it processes the text
just the same. Oh, that sounds bad.
It gets worse. Multi shot. Bury a
malicious prompt inside the whole load of prompt,
and fire hose it at the at the LM.
It's not gonna check every single prompt. So if you bury 1
in there, it might process that one and give you an answer
it's not supposed to give. That's because the guardrails didn't engage.
Interesting. So the guardrails are not necessarily on by default.
Well, no. They are on by default, but if it overloads it,
it may it may slip the net. So rather than shut
down, it it shuts off? Well, Well, it's
basically what you're doing is effectively a buffer overflow. You're basically using
an injection method to induce what is effectively
analogous to a buffer overflow. That's wild. That's
not how I would have thought it would have worked. Interesting.
Interesting. This is a fascinating space. So
Yes. One of the things that I think people
don't realize is
just the sick insecure ways in
which these plug ins could be designed. Right? Because, like, everyone's all
gaga about these plug ins, and I look at it. I'm like, where am I
sending my data? Right? Am I gonna read the 30 page EULA? Right? Or
am I just gonna say, yes. Yes. Yes. I wanna do what I'm doing.
Is that really a problem? It is.
Because that kind of ties into unauthorized leakages.
Right. How do I know that plug in is a secure
connection into the l one, and there's nothing in between?
Right. Or that it will contain what I get it.
How do I know? I don't know. That's the thing is that is this plug
in itself secure, and is its connection to the
LLM secure, And is that LLM also
integral? So, yeah, I could send it in there, but how do I
know that along the way, something you know, the pipe might leak?
So you need to check it. Just and, I mean, this goes I mean, this
is very similar to APIs. This is very similar to,
all sorts of remote interfacing. Just good engineering
short lived. Just good engineering discipline seems to be
missing from a lot of this because people are focused on the AI,
not necessarily the underlying infrastructure that
has to support it. Indeed. And I think that that's
but that's the whole thing is that there is this massive trend as
of late. I mean, perhaps it wasn't really emphasized
before. I'm sure it was there, but it's now becoming very, you
know, reiterated that we need to have security by
design. Right. The security by design is already we're already doing
that in other enterprise applications. Same should be applied to
LLMs. Security by design. You check the code. You check the
model. You check everything. And while it's operating,
you check it. One of the biggest things you can do to overcome the
opacity of an LLM, export
the logs, export the comp the prompts.
Have it processed. Now you could potentially process it.
I'd figure the way you process any other kind of log data.
The other thing you can do is use machine learning or
an air gapped isolated LLM
specifically trained to look for signatures,
words, phrases, things like that. And when
these patterns match, it returns saying, I found
something that looks suspect. This is suspect.
Here is the user who did this. Here is their IP.
Like every other bit of log security log information we would get.
So that would help piece together the trail to figure out, are these a
bad actor, or is this the happenstance? Exactly.
And that is one way you can do it because once you have the
internal prompts and you have the internal logs and
those are exported out, you now can see in.
Right. The biggest problem is you gotta have that monitoring. You have to have that
transparency. The elements are so large, you
can't so easily see into them, but if you're taking the data out, it's a
lot clearer. So you can kind of follow what the LLM is doing,
if not, what's inside of it? Precisely. And the advantage
is is if you use another LLM that is specifically designed
to, you know, interrogate the prompts and look through
them, examine them, scan them, whatever word you wish to use.
You can find out where it is because that
is not gonna be so easy to break the guardrails because it's examining
one little bit at a time. It's looking at the individual prompts. It's not really
it it's kind of agnostic about everything around it. It can get it can kind
of filter out the new leads. Interesting. That's
I mean, it's just so fascinating kind of to start pulling the thread at this,
and there's a lot more. It's like I found there's a story about a guy
who was renovating his basement, and he found, like, this ancient underground city. That's how
I feel when I just get kicked back. It's true. It happened in
Turkey. Like, he found, like, this underground network from, like, Byzantine
or Roman times. That's what I feel like. I I like, wow. Like,
this really goes down deep. So what's an
inference attack? Because I've heard of that. What's an inference attack? We discussed that,
or have we touched on that? Well, inference is
basically what you're inferring to, the answer you are seeking.
So, basically, it's basically, to the
the inference is literally, the
prompt that you are entering in and what you're getting out. Okay.
More or less. So how is that an attack surface? Well,
basically, you're you're chaining it. You're daisy chaining your attacks.
You're trying to infer things. You're trying to kinda subtly
get through. So it's a bit like it's a maybe
more like cross examination from an attorney, a hostile attorney
I would say that. Yeah. More than more than, like,
interrogation or torture or or whatever verb we used
earlier. Yes. Interesting. What's
model inversion? Model inversion is
basically you trying to spill the model itself. Oh. You're trying
to kind of you're trying to kind of tear the
guts tear the guts out, maybe put stuff in there,
things of that kind. Interesting.
Interesting. Where do
we stand on the
criminal and civil liabilities here? Right? I I I know that Air
Canada had to pay a fine because they promised that its
chatbot promised somebody something.
I don't know where the California Chevy Tahoe thing
is. But, I mean, have the laws
caught up? Or, like, how were how is this generally looking like?
Well, it depends. I mean, all jurisdictions are different, but I would
suspect to say that whatever guarantees
you make, you're bound to them. So
probably disclaimers, indemnification is
probably extremely wise. I would say,
unfortunately, I'm not a legal expert. Right. Right. Right.
Specifically to the law. Right. But as I'd say, I'd have
enough legal understanding to probably say that if you make a promise,
you better put your money where your mouth is. So that's why I back it
up. IBM indemnifying their users for using one
of their Granite models is probably a big deal for
businesses. Because just in case somebody I'm sure that there's
all fine print and things like that, but that that would be an appealing
thing for business users. Yes.
Interesting. Interesting.
How does someone get started in learning how to jailbreak these? Like, is this is
this a typical your background is, IT security.
But what about someone who has a background in, say, AI and and and building
these LLMs? Is that, Gunning, you think, be an another career
path for the what we call data scientists today?
Well, I would say you're gonna have to probably do it just as is. I
think to the developers and to the data science Right. Scientists who work on this,
you're gonna have to be security literate. Right.
For those who want to get into it, I mean, data science is like any
other AI trade. I mean, we often
cross pollinate. So I would say that you might have an understanding
already of these things. These prompt injections, as I say, are not
much different than SQL injections. The data science Right. You probably know what that is.
How you transfer it depends on what you know.
I would say most data sciences do understand how some of this stuff
works. Right. So getting into it is
just basically you just learning more about security. Right. For the
average person trying to get into it, I would say, if you're trying to
get into AI security, know security
first, and there are many ways to get into
it. I, myself, came in, from my
CCNA. I mean, that's how I kinda got into it. I got
into networks, and then I got into cybersecurity. And
then it was around the time that, you know, the GPTs were really starting to
hit their stride. And it was just part and parcel of it because
I needed a good reference tool. And so then I learned, okay.
Well, how does this work? How do how is it put together? How,
you know, how is it all formed and such? How does
it make its inferences? How does it understand the problems?
So from that, I would say to anybody trying to get into this field,
know cybersecurity first, and you will know AI
in time. AI is in concept
relatively simple, but the nuts and bolts of it are quite
complex. So Yeah. The implementation
details are quite severe. Like, I think
AI is really, I think, better not better suited, but it came
out of the lab. I think the paint is still wet. Paint hasn't dried
yet. And now we're forcing it into an enterprise
scenarios with real customers, real data, real people's lives.
And I don't see a lot of the traditional security
discipline that
I would expect in modern era, modern development.
And even that's a low bar. Even that's a low bar. Let's be real. Well,
it's it's new. Right. It's very shiny.
Mhmm. That's I think that's what I would say is the general
populace and even in the industry that's quite I think our view is that this
is a shiny thing. Right. Well, you know, well, I want
to. You don't even know what it does. I still want it. I want it.
What's interesting is, it
reminds me a lot of the early days of the web where everybody wanted a
website. Well, what are you gonna do with it? I don't know. I just want
a website. You know? It's very it has very very
similar vibe in that regard of we want it. We you know, the hell with
the consequences. But the way I see this
being,
taken up as quickly as it is kind
of worries me. Like, there's gonna be a day of reckoning, I
think, coming. You know? And I thought we
already have it. Right? You you had, there was a leak from Chat
CPT. They had a 100 was a 100000 ish customers there, give or
take? A 100000 credentials taken, compromised.
Credentials and and presumably the data and the chats?
Some of it potentially, I'm sure. But what we're looking at is, like,
names, email addresses. I mean, it depends on how much you put in
that profile. Remember, everything you put in that profile is stored.
Right. Right. That is truly scary.
So you mentioned network, Chuck. So you do you think that
just on a personal level, it's
what worries me about these offline models, right, you run OLAMA locally.
Right? Do you think they could they call
home? Could those be hijacked? Could those have problems?
Specifically. Specifically. Like, so if I'm
running Olama locally, right,
how secure is that? Does that does that depend on the security of my
network, or is there something in there that calls home?
No. Not unless you tell it to. Not unless you try to extract it, you
make a pull, then, yes, it does that. But that's the idea is that once
it's pulled down, it kinda isolates itself. Now
what you can do yourself is set up your
network so that literally it has to be outbound,
a stateful connection, originating outbound.
And you can set that up in your firewall, physical
or otherwise. And you can do things like that, and you can
kind of put it to a point where it doesn't call home unless you tell
it to. Right. And, also, once again, that
private LLM is also very good because you control
the access to what it does. So you can say,
other than these addresses, sanitize it to the
address of wherever the model comes from, say, these are the only ones
allowed. Right. And nobody else is permitted.
Otherwise, implicit deny. Right. So that's a I think
a a small tangible example of something you
can do that is relatively straightforward for any
systems or network engineer, to do just in the hearing
now. But in general, no. They don't normally call without
prompting. Okay. But depends on what they do with those models.
They might put in that kind of feature. A lot of that go back to
the I'm sorry. Yeah. That's kind of my concern is, like, you know, would that
end up in there? Or Well, Meta might put that in there.
Right. Meta is a not alone. Meta is not
exactly free. Right. Matt is not exactly,
has a reputation for privacy. No.
So it's kind of ironic that they are
leading the effort in this space. Seems kind of an odd move.
I I don't know what to say about that. No. No. No. I just need
I have no thoughts on it, but Right. Right. Frankly, I don't I don't know
how relevant it'd be to this discussion. But it's an interesting it's
it's just an interesting time to be in this field, and,
this is just fascinating that you can
jailbreak. You could do this and, you know, even just the basics. Right?
Like, you could do a DOS attack. Right? There's
just basics too. Like, this is still
an IT service no matter how cool it is, no matter futuristic it is. It's
still an IT service, so it has all of those vulnerabilities,
you know, that I don't know. Like, it's just it's just interesting. People are so
focused in the new shiny. I just find it fascinating.
And that's the thing is that this thing is a compounded problem. Right. You
don't just have the usual suspects. You also have
new things that are they
by the virtue of them being new, there's not much
investigation. There's not much study. I mean, amongst my
research for this presentation, I found a number of
papers, white papers coming from all sorts of universities.
They are now looking into this. Right. This is something that maybe we
should have done maybe a while back. Good thing, though, we're doing it now.
Right. But also, also, there's a lot of reasons why you would do that, though.
You would do that because in the wild, you'd be able to identify these things.
Right. You'd be able to see. You're not gonna know everything when something gets released
until it's put out into the wild. Right. And real users
get their hands on it. Good actors, bad actors,
and everything in the middle. Right? Like, you're not gonna yeah. No. I mean, it's
kind of like I guess I guess in a perfect world, the cart would be
before the horse in this case, but that's not the world we live in.
Interesting. So where can
people find out more about you and what you're up to? Well, you
can find me on, LinkedIn. Kevin Lynch
with CCNA. Cool. You can look up my company, Novi Tea Guy,
Novi Tea Guy dot com. And For those outside the area,
Nova stands for Northern Virginia. Just just wanna figure it out there. Well,
also, it well, it's actually a bit of a it's a double meaning. At the
time, I was dedicating myself to IT for the first time. I've done
IT kind of side part of my work. So Nova is also the
Latin for new. So I was Okay. The new IT guy. The
new IT guy. But when it comes to IT, I'm still your guy even then.
There you go. I love it. And,
I'll definitely will include in the show notes a link to your presentation.
And this has been a great conversation. I'd love to have you back and maybe
do your presentation, maybe on a live stream or something like that if you're interested,
and, I'll let Bailey finish the show. And that's
a wrap for today's episode of the data driven podcast.
A huge thank you to Kevin Latchford for shedding light on the vulnerabilities
of large language models and how to stay one step ahead in the ever
evolving world of IT security. Remember, while these
models are brilliant at generating conversation, they aren't infallible
so keep your digital guard up. Until next time, stay
curious, stay safe and always question the source unless,
of course, it's me. Cheers.