This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

 Hey everyone. I'm Drex and this is the two minute drill where I cover some of the hottest security stories twice a week. All part of the 2 29 project. Cyber and Risk community here at this week. Health, it's great to see everyone today. Here's something you might wanna know about. You've probably heard people say AI hallucinates.

What they mean is sometimes large language models like Chat, GPT or others. Just make stuff up. They give you an answer that sounds super confident, but it's not true. So why does that happen? Well, think about these models and how they're built. They don't, they don't know facts the way that you and I know facts.

They've been trained to predict the next word in a sentence based on patterns from billions of words on the internet. So it's like the world's most advanced auto complete. And if the training data didn't have the right information, or if the prompt leads into a corner, it will usually still generate something and that something might be totally fabricated, a fabricated article or a reference that doesn't really exist or a really confident sounding explanation that's just plain wrong.

Instead of hallucinating, I heard somebody the other day refer to it as bluffing, like a card player, bluffs at a poker table. The models were built to sound confident, so they make their predictions of an answer and they do what they do. They sound really confident, even when they're sometimes wrong.

OpenAI in a September 5th paper breaks this down. This whole hallucination bluff problem into a few different buckets. One is training gaps. If the model hasn't seen enough good examples of a topic, it guesses. Another is alignment issues. Again, the models are trained to be helpful and give answers instead of saying, I don't know.

So instead of silence, they spin a story. And third, there's the pressure to respond. The design favors fluent, confident texts, and it seems like humans like answers more than they. I'm not sure. The punchline in all of this is that the models don't have built-in truth meters. They're not fact checkers, they're pattern generators, and that's why open AI and others are working on some really important stuff like grounding the models and real data and improving how they say, I don't know, and giving users better tools to be able to verify what comes out from a healthcare perspective.

This makes a huge difference. I mean, imagine an AI helping to write an InfoSec policy or triage alerts, or even guide patient care if it hallucinates. While it's doing that and nobody checks bad info can get baked into critical decisions, and that's why the safest way to use these tools right now is with a healthy dose dose of skepticism and some human oversight.

Remember, AI can be incredibly powerful, but it's not infallible. The best defense against hallucinations is to stay sharp and double check and keep humans in the loop. I use generative AI all the time and what I personally found useful. Is to not use it like Google, but to give it context before I ask a question.

And an important part of this context for me is to give it a couple of really explicit instructions. One is don't make anything up. If you don't know the answer to something I'm asking, say so or explain why. There might be more than one answer to the question that I've asked. And the second thing is when you answer, show me the article or the internet reference with a functioning URL so I can go double check.

The other thing I'll say about this is don't feel bad about doing that. I was telling somebody this story the other day and they said, yeah, but don't you feel like you're micromanaging her? And I said, yeah, well, I mean, first of all, this is a technology. It's not a person, so it needs some additional supervision.

And second. The models are built to allow you to impose these context rules. I've built projects in chat GPT and that they have these additional criteria, this additional context built in that I demand that it not make things up and that it gives me a link so I can double check the answers. And I do all my work through those restricted models, through those projects, there's so much we could talk about.

But that's it for now. I'm out of time. I hope that helps. Thanks for listening. I'll put a link to the open AI paper in the comments if you really wanna dig in. And if you like this kind of content, sign up and I'll keep you posted on the latest news and webinars and podcast and insider info, all the good stuff from the 2 29 project.

Go to this week, health.com/subscribe and sign up for all the latest insights, including of course, our security and risk updates. And share this podcast with your friends and your team. And thanks for being here. I'm off to Boise tonight for a city tour dinner, and then I'm going to Orange County, California for a 2 29 project summit, uh, through the end of the week through the weekend.

So if you see me out there, say hi. That's it for today's two minute drill. Stay a little paranoid. I will see you around campus.