This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

TownHall: “Data Rich but Insight Poor” with Reid Stephan and Spencer Dorn

Bill Russell: [00:00:00] Today on TownHall

Dr. Spencer Dorn: We're estimated to produce 30% of the world's data. Hospitals like yours and mine are estimated to produce 50 petabytes of data each year. Like that's equivalent to streaming a two hour movie 25 million times.

Bill Russell: My name is Bill Russell. I'm a former CIO for a 16 hospital system and creator of This Week Health.

Where we are dedicated to transforming healthcare, one connection at a time. Our TownHall show is designed to bring insights from practitioners and leaders. on the front lines of healthcare. .

Alright, let's jump right into today's episode.

Reid Stephan: Welcome to another this week, health TownHall of Conversation. I'm Reid Stephan, CIO at St. Luke's Health System in Boise, Idaho. And today I'm joined by Dr. Spencer Dorn, who is a gastroenterologist and the vice chair and professor of medicine.

And the lead informatics physician at the University of North Carolina. That's a lot. Spencer, thank you so much for making the time for our conversation.

Yeah, [00:01:00] awesome to be here. Thanks so much, Reid.

Okay, we're gonna try something, an icebreaker game. I'm going to read you five recent, within the last month, AI themed headlines.

And then based on your expertise and your own personal crystal ball, you're gonna let me know if you think the headline today or in the future is gonna be fact or fiction. Okay. I don't get the hedge. I gotta pick one or the other. That's right. One or the other. Even though you could argue like down the middle, but I'm gonna force you to take a stand.

Okay? Okay. Headline number one, AI will improve patient satisfaction scores more than any

Dr. Spencer Dorn: current initiative.

Thats a hard one. There's so many

other initiatives, so I'm gonna say I think AI will improve satisfaction. We're already seeing it with scribes and their doc, you people like that. Their doctors are looking up from their keyboards. But that's at the high bar to clear, so I'm gonna say fiction.

Reid Stephan: Yeah. And these titles, I think, are intentionally provocative. And so that may be kind of hard. But I'm with you, headline number two. Chat GPT diagnoses are more [00:02:00] accurate than physician diagnoses.

Dr. Spencer Dorn: I'm gonna call there are some studies suggesting that chat, GPT outperforms physicians in some experimental settings, but I think in the wild I'd still call that fiction today.

Reid Stephan: number three, hospitals will need a Chief AI officer by the year of 2026.

Dr. Spencer Dorn: I think I would

call that false or fiction.

Reid Stephan: Yeah, I would, AI

Dr. Spencer Dorn: is different technology, but most hospitals, health systems have already have pretty robust governance for their digital technologies. And I think often those suffice.

Reid Stephan: Agreed. Okay. The last two, and these are related in terms of enterprise DNA Microsoft, CEO, Satya, the current CEO admits that AI

Dr. Spencer Dorn: is generating basically no value. Well, it's hard to argue with him. He sees

the, I think it's hard to show the value, and I think a lot of the value has not yet been proven, but I think I'd call that fiction. I think there's more than just hype. Yeah. There's a lot of hype. It's overblown, but I think there's [00:03:00] some substance, so I'd call that fiction.

Reid Stephan: For sure. Okay. Former Microsoft CEO Bill Gates says that AI will replace doctors in the next 10 years. Yeah, I think that's a big fiction Paul. It is just fascinating 'cause here are these two gentlemen that have this shared, you know, corporate DNA and clearly, especially with Bill. I think he's trying to be provocative.

I don't even know if he believes that, but it just speaks to the landscape that we're in. Those kind of headlines and positions, like make it challenging for folks like you and I to kind of navigate and help steer the conversations in our respective organizations.

Dr. Spencer Dorn: Definitely.

Reid Stephan: So and so on that theme, you wrote about this kind of spectrum of AI belief in healthcare.

On one end you have the doomers, like those who feel like we've kind of entered end of days, robots are gonna take over. On the other end, you have the zealots. The unbridled enthusiast who like can't get there fast enough and it's going to improve everything in your walk of life, what have you found that kind of drives the [00:04:00] mindset in each camp and what can we learn from that?

Dr. Spencer Dorn: Yeah, it's really interesting, Reid. Because I think in many ways it just reflects human nature. As human beings we tend to align ourselves with certain core beliefs or certain groups of individuals and unfortunately, we naturally like to separate ourselves. What some people would call, we like to other people, you know, they're not like me, they're others.

Yes. That's just kind of human nature. So we like to align in different camps and, you know, one popular camp are the techno utopians and they see how technology has changed various aspects of our lives, often for the better. And they want to bring those opportunities and benefits to health and healthcare.

So, they're supercharged up about this and they just see a really bright future for AI and healthcare. And then conversely, we have the cynics who see healthcare is this unmovable force. And they're deeply disappointed by the effects of digitization. They feel like it's actually created paradoxically more work [00:05:00] or it's maybe disconnected us more.

And often The people in this camp are nostalgic for this earlier, more romantic era of medicine. Whether it was truly more romantic or not is debatable. And they appreciate that healthcare's deepest problems are really not technical. They're more social and political and economic.

So we see, these two camps and I think they're valid beliefs held by both camps. But in general, I think that extremist views are typically wrong and I tend to gravitate towards the middle ground. Recognizing there's great opportunity and this stuff is hard and technology will not necessarily fix all of our problems.

Reid Stephan: Yeah, well said. I'm with you. I try and like find that medium part of the bell curve and somewhere in there is a version of the truth that's more pragmatic. And you know, as the CIO and your role as a physician leader, like we have to be psychologist, sociologist, you have to figure out how do you thread that needle of inviting the skepticism that's healthy and useful, [00:06:00] but doing without like freezing innovation or analysis paralysis, like that's the art of what we do.

It's kind of fun actually. I enjoy it, but it's not easy.

Dr. Spencer Dorn: That's true. Well, great leaders like you are thinking about the human aspects, right? Usually it's not that, I mean, obviously you need to know what's happening technically because that's what your main responsibility is. But it's really about people.

I mean, at the end of the day, healthcare is about people. It's the people that we're caring for. It's the people that are doing the work and our teams. And people are complicated. And large organizations of people are even more complicated. Very little of it is about the technology itself and much more of it is about how the technology fits into this broader system.

And, we made the mistake with EHRs of thinking that, oh, we're just gonna adopt these things and everything will be fixed and everything will be great. And overall I think EHRs have gotten a bad rap. I think EHRs have brought a lot more good than harm. But they haven't necessarily lived up to these, you know, grandiose promises that were made and projected back in the early two [00:07:00] thousands.

And I think it's largely because, you can't paint this pollyannaish picture right? Like we can't oversell technology because it's really, again, at the end of the day, not about the technology, it's about how we implement it and the people using it hopefully for good.

Reid Stephan: Yeah, I completely agree.

And it's such a human nature conditioned to want to oversimplify, to deconstruct it like really quickly and find like that solution and it's just. It's not that your idea, your comment about people being complex, organizations being complex. If I take my family of six out to dinner, it's like a challenge to figure out where do we want to eat?

Can we find a place we all like Now think about a 17,000 employed organization trying to align on things like it's going to be hard and we shouldn't be surprised or discouraged when it is hard.

Dr. Spencer Dorn: to add to that, I think, yeah. I think we also need to recognize that no one has a monopoly on all the good ideas, right?

Yeah. There's no one walking around in your [00:08:00] organization who just is always correct, knows everything. I think speaking to people on the ground is usually a good thing and you want different views, right? You want to hear competing interests and different perspectives. You want to hear from

the doomers, and you want to hear from the, you know, techno utopians because each carries some information and perspective that's worth listening to. But I think ultimately our job is to figure out, you know, how do we embrace nuance and how do we communicate clearly a vision that's grounded in reality not just hype or not just, you know, unreasonable doubt.

Reid Stephan: In one of your recent posts you talked about, and this is not a new concept, but you emphasize that technology adoption is often a people and process challenge, not a technology challenge. And I think we all instinctively know this yet, and speaking for myself, we continue to maybe underemphasize or miss that point.

So with that in mind, what are health systems getting [00:09:00] wrong about implementing AI?

Dr. Spencer Dorn: It's still so early, so it's hard to say what they're doing wrong. I think health systems are doing a lot of things right. I think for one health systems are taking this seriously and recognizing that AI is.

Different than prior versions of software and mm-hmm working hard to set up the right governance and monitoring framework so that whatever tools they use responsibly. And for, you know, for the benefit of patients in the workforce I think. One thing, maybe some are getting wrong , relates to what we discussed a minute ago of this kind of over promising overpromising that technology will fix the problems in medicine and.

You know, really, again, failing to learn from our experience with electronic health records which were built as this kind of panacea for all that ails healthcare. But, the belief that technology will necessarily improve care really is no longer credible. And so I, I guess, I would

Suggest that maybe [00:10:00] some organizations are overselling the technology. There's great opportunity, but I think we need to stay grounded in reality and remain hopeful without overpromising. I often think of, there's an artist and she's a musician, has this quote I often use, and her name's Laurie Anderson, if you've heard of her.

And she has this quote that, if you think technology will solve your problems, you don't understand technology and you don't understand your problems, and I just love that. 'cause I think it just speaks to the essence of what we're tackling in healthcare. Yes, technology can help. In some ways technology can help dramatically.

But our problems run much deeper. I mean, you know, you look at things like clinician burnout. Yeah. Yes, AI scribes are great for unburdening some physicians from all the documentation challenges, but that alone is unlikely to transform doctors who are unhappy with the practice of medicine or feel like they're, underwater to feeling like, oh, this restored my confidence in why I went into medicine.

And there are many, you know, [00:11:00] healthcare access and many of our communities, people wait weeks to months to see the right physician and AI can lead to some increased efficiency and some improvements in access, but it's not suddenly going to solve access problems. And, any dimension of healthcare where there are challenges I think AI and digital technologies can help, but they're unlikely to like fully solve the problem.

So I think that's what we need to focus on is this, let's be optimistic because we should be, there's great opportunity here. And at the same time, let's not oversell this as a magical solution for these , deeply rooted problems.

Reid Stephan: You know, I've used that technology understanding quote in the past, but just have said it.

Now I have attribution, so Laurie Anderson. So thank you for that. I'll make sure I give her credit going forward, but it's so true and as you were talking, I was thinking I have never regretted spending time to really optimize a process before applying tech to it. But I have regretted more times than I care [00:12:00] to admit where I've like short circuited that step and applied technology and I've just amplified a bad process.

And it gets back to , the core of that quote is like, just slow down and we have a, certainly in our shop, we feel an urgency, especially if a clinician comes to us with a need, we want to help, we wanna make their life better. And sometimes that well-intentioned desire can cause us to skip steps that actually make the problem more challenging and less likely to be solved in the future.

So a good reminder, totally. Okay with your lens as a physician and a system thinker. Where do you see AI making the most meaningful impact in the next one to three years?

Dr. Spencer Dorn: that's a great question. I guess the easy answer is it depends where, you know, I think it's already making an impact and, the initial view of AI in healthcare is there's just a lot of drudgery, right? There are a lot of parts to our jobs that are difficult, not necessarily rewarding, but just something [00:13:00] we have to do. So the initial emphasis I think, has rightly been on automating some of these activities and offloading some of this drudgery, like clinical documentation.

And it makes sense to start there because these are lower risk areas and people are really eager to give a lot of these activities up. So I think, you know, right now we've clearly seen AI scribes as the leading clinical facing use case. I'm particularly excited about AI to democratize medical knowledge

and either internal knowledge within your system servicing protocols and pathways and things that people may not be aware of. But even more so accessing the medical literature, right? Which depending on the estimates, could some people suggest it doubles every 73 days, that's one study.

I think that's wildly too aggressive, but the medical literature probably doubles every five years or so. And it's almost impossible to keep up with. So I think that's a tremendous opportunity to harness the power of AI to help surface the right [00:14:00] information at the right time. And combine that also with, you know, all the predictive algorithms we have

ultimately, I think AI is about making better decisions. So offloading some of the drudgery noble goal certainly welcome that. But ultimately I think the question is how can we use these tools to make better decisions, to make better predictions that we can apply judgment to, to access information. Whether it be through the medical literature or through, you know, complex patient medical records that are too hard to summarize in a reasonable timeframe. I think that's ultimately where I hope we're going is about improving medical decision making and organizational decision making as well.

Reid Stephan: Yeah you sparked a thought for me. And your last comment, like, we know we're so like data rich, but maybe insight poor. And so I think this can help there. What about pre charting? Like, in my mind that seems like that could be a great AI use case, but I'm probably overly enthusiastic about it. What are some of the cautionary perspectives that we should bring [00:15:00] to that opportunity?

Dr. Spencer Dorn: if last year was the year of the scribe, this year is apparently the year of the agent according to Y Combinator. But I think in healthcare , these will be the years of summarization. Because as you know, healthcare, there's just too much data to process.

We're estimated to produce 30% of the world's data. Hospitals like yours and mine are estimated to produce 50 petabytes of data each year. I have my notes here, like that's equivalent to streaming a two hour movie 25 million times. Like that's just like no one can. Most of that is obviously digital exhaust that no one's going to look at.

Yeah. But there is some value there. There's a study from Penn showing the average medical record is longer than half the length of Hamlet, which by the way is Shakespeare's longest play. So like there's just too much data and information and often look back to a quote this guy John Nas book said, and EO Wilson, the Harvard biologist later kind of rephrased it.

But we're drowning in information but starved for knowledge, right? Yeah. We're just overloaded. So I think that [00:16:00] summarization, whether it be pre-charting. Because specialists like me spend a lot of time before seeing patients digging through their records, which are often, I know, stored in PDFs that are not easy to search and sometimes they're upside down and they're poorly labeled.

So I think whether it's a specialist preparing pre charting or it's an inpatient physician or nurse who's taking over care of a patient I think summarization really clinical summarization is a massive opportunity that I will go on the record and say it's in my mind, a bigger opportunity than ambient documentation.

Of course these technologies will all converge, so they're a bit artificial, these distinctions. But yeah, I'm with you, Reid. I think this is a super ripe area for applying AI. In terms of your question, what the cautionary tells, I think the challenge with this is accuracy. One of the things, you know, these medical records are so long, you can't just

stuff them all into the context window and expect to get high quality output. So there are different strategies that can be taken to, you know, break these down into different [00:17:00] chunks and then re-sum. So I think that's one challenge. But the longer the documents, the harder it is to accurately summarize And while everyone's worried about hallucinations, right?

Saying a patient has diabetes, when there's diabetes nowhere to be seen that I'm not so worried about because the better company is doing this, make all their summaries traceable. What I'm more worried about is what if a summary doesn't surface important information you need to know and would've found if you did it on your own.

What if you're an anesthesiologist preparing for a case and the summary doesn't show that the patient had pulmonary hypertension or an echocardiogram last year? That could be a big deal if you start relying on these tools. So I'm more worried about errors of omission, I guess, than commission.

But I think these tools are rapidly improving and advancing and, you know, we're lucky there are several solutions in the market that are quite good that I've had a privilege of looking at. And I think this is gonna be a big potential benefit from AI in the near future, not years from now, like coming soon.

Reid Stephan: agreed. And your comment about convergence. You know, I [00:18:00] think your ambient listening vendor, whoever you have, like you should be pushing for this platform because they should be able to do not just the AI node, but the coding on the back end, the pre charting on the front end, maybe things beyond that.

So that's when you get really that flywheel benefit.

Dr. Spencer Dorn: Right. And if you look at like the published experience so far on ambient notes is pretty underwhelming. Like to over generalize. Yeah. You know, I don't know what it's like at your health system. We've been thrilled with it at our system, but if you look at the data that's been published, including out of, you know, Permanente Medical Groups, the largest experience 10,000 physicians over a 13 month period.

Only one in three clinicians actually used the scribe more than a hundred times. So only about a third of clinicians use these tools and these tools only save on average like, you know, like less than 30 to 40 seconds per note. And they're not really cutting down on after hours documentation time.

That's not to say they're worthless. 'cause the clinicians who use them love them, right? Yeah. They're really excited to use [00:19:00] them. Yeah, they seem to reduce cognitive load. They seem to reduce burnout, so certainly benefits, but the benefits are pretty circumscribed and I believe it's because note writing, well, first of all, conversations only inform about a third of the clinical note less than the inpatient world, less for certain specialties.

So the ambient scribe solutions only informing a part of the note, and the note is only a part of the workflow. So, the better scribe products,, are moving into other areas like, diagnoses and summarization and coding and all these other types of activities that I think by bundling it all together, it'll be a better user experience, but also, it'll move the needle a bit more in terms of efficiency and effectiveness.

Reid Stephan: We're getting short on time. I'm just gonna like plant a seed for maybe a future conversation that you and I have that you were describing potential errors. I think it's an interesting concept to think through the human psyche and the risk tolerance we have for a human induced error versus a machine induced error [00:20:00] and, you know, predictability impact how we might investigate root cause.

So that's an area as well that I'm just thinking about and how do we lead our team through those kind of conversations and how do we then share that information with the incredible human beings that we care for. It's just an interesting topic that we can unpack today, but maybe you can post about it and teach us all your thoughts on it.

Yeah,

Dr. Spencer Dorn: no I would love to discuss that another time. I mean, in brief we have to admit we're imperfect, right? Yes. Like, base case is not perfect, just like there's great parallel to autonomous vehicles, right? Like autonomous vehicles seem to be safer than human drivers, but Yeah. , If they crash, it's the front page Yeah.

Of the news. And we lack. The appropriate accountability frameworks for these tools and the regulatory frameworks. Yeah. Because you as CIO, you don't wanna end up on the cover of the Boise News and observe, I dunno what they, what it's called, what the Boise Newspaper is. But

Reid Stephan: yeah,

Dr. Spencer Dorn: that's probably really high on your priorities as the CIO, so, or my

Reid Stephan: [00:21:00] CEO to end

Dr. Spencer Dorn: up on there.

Reid Stephan: Like

Dr. Spencer Dorn: that's also Yes. Right. Wouldn't be good for your CEO either

so. Yeah. But at the same time, we have to admit we're imperfect and, , humans alone , we have bad days, we have bad moments. We're not all, , half the doctors are below average by definition. Right. So, we need to approach this with humility.

Reid Stephan: Yes. Well said. Okay. So to be continued on that topic that'll be great. Last question. Yeah. Kind of a wrap up. One a recommendation you have for our listeners, book, article, a person, someone that's really helped shape your thinking about AI in healthcare.

Dr. Spencer Dorn: I read a book before, like, you know, the generative AI boom.

I think it came outta May, 2018 or 2019, but it really just kind of, floored me in terms of its impact and just really opened my mind. I knew a bit, but this kind of reframed my whole perspective. It's by Melanie Mitchell who has been working in the area for decades. She wrote this book called AI, A Guide for Thinking [00:22:00] Humans.

And the take home message of the book is that we humans tend to overestimate. The capabilities of AI and we underestimate the complexity of our own intelligence. Like we're very quick to call a machine super intelligent and we're kind of quick to also say we're not intelligent. But what she argues in this book fundamentally is that we're a lot more complicated than we seem, and these tools, although they appear like magic, they aren't necessarily Yeah. So, so I'd recommend that as one of the most I read a ton. I speak to a lot of people. I'm very lucky that I get to go down these rabbit holes, but that's one that I would certainly recommend with as a cautionary, mildly skeptic, but I think grounded in reality

take on this whole crazy world we're living in. Perfect.

Reid Stephan: And we are imperfect, irrational beings for sure. And so that flavors all of this listeners, I encourage you to follow Dr. Spencer D-o-r-n, on LinkedIn and get engaged in the conversation. It elevates our community. Spencer, thank you.

its been a lot of [00:23:00] fun. I appreciate you and best of luck with your future AI endeavors.

You too, Reed. Thanks so much. This was fun.

Bill Russell: Thanks for listening to this week's TownHall. A big thanks to our hosts and content creators. We really couldn't do it without them. We hope that you're going to share this podcast with a peer or a friend. It's a great chance to discuss and even establish a mentoring relationship along the way.

One way you can support the show is to subscribe and leave us a rating. That would be really appreciated.

Thanks for listening. That's all for now..