This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Solution Showcase: The Freeing Power of Testing Automation with Phillip Furukawa and Chris Paravate

[00:00:00] This episode is brought to you by SureTest. Transform and revolutionize your health system's application testing process with SureTest ManageTest Automation Solution, SureManage. Eliminate 80 percent or more of your manual testing activities, reducing errors, and reclaiming thousands of hours for your staff to focus on more rewarding, strategic, Projects.

Continuous testing is critical and getting it right saves lives. Discover the future of healthcare IT automation with SureTest. Visit ThisWeekHealth. com slash SureTest today and elevate your healthcare system's performance.

I am Bill Russell, creator of this Week Health, where our mission is to transform healthcare, one connection at a time. Welcome to today's solutions showcase where we spotlight innovations, making real impact in health systems. Let's take a look at what's working today.

Bill Russell: All right. Today we have a solution showcase, and I am excited to be joined by Chris [00:01:00] Perta with Northeast Georgia Health System, the CDIO, keep throwing consonants in there. And, uh, Philip Awa, who is the CRO at SHO test. Gentlemen, welcome to the show.

chris: Thank you.

Phillip Furukawa: Yeah. Thank you, bill. Good morning.

Bill Russell: We're going to have a, a discussion today about a, a problem set. I think a lot of organizations are, are struggling with. Uh, but before we get there, Chris, I, I'd love for those who aren't familiar with Northeast Georgia Health System, give us a, a, a quick picture of the organization.

Size, scope, uh, your IT environment.

chris: So just over a thousand beds, five hospitals, about a hundred ambulatory locations. We serve all of, uh, northeastern region of Georgia, everywhere from Athens to Alpharetta and all the way up to North Carolina. Uh, we are a, uh, digital, most wired, low 10 health system. So, uh, pretty progressive on adoption of technology.

And, uh, we're an Epic shop, uh, uh, [00:02:00] like, like many others. Uh, Epic, workday, uh, sure. Test has been a big part of, of that solution set.

Bill Russell: So, uh, CIO since, uh, 2014,

chris: Yeah.

Bill Russell: Epic Go Live was followed that pretty closely after that, right?

chris: 2017 implemented Epic. Uh, across, uh, the whole system. Uh, so we're Epic everywhere from acute care to long-term care, to rehab, and everything in between. I, we are, uh, through and through one EMR.

Bill Russell: Well, and that, that is one of your core tenets, right? As A-C-I-O-I mean, we, we've interviewed before and you're like, look, if. If it can be done on that platform, we're going to do it on that platform.

chris: We're paying for it, right? So if you're going to pay for it, you might as well use it, uh, and maximize that use. Uh, but it's been, it's been, you know, uh, architecture, I've had the pleasure of being able to build that architecture over time. And to be really thoughtful around that. And it makes [00:03:00] supportability, uh, far more tenable and then allows us to really focus on, on driving innovation, driving change, uh, partnering with ops.

Bill Russell: Well, because of the scale of your Epic implementation, the, the Epic upgrade cycle, that's, that's like a real thing. And uh, it required a fair amount of testing every time that that came through. Uh, give, give us, walk us through what a typical. upgrade cycle required from your team, uh, maybe before SureTest, and then we'll, we'll get into, uh, what it, what, uh, problem you solved and, and what it looks like now.

chris: So like most of us who started doing integrated test. We started with manila folders and we passed testing scripts around, uh, I would say for our upgrades that wasn't too far behind, right? Uh, the challenge with an upgrade is you are reading all these, you know, elaborate release notes. Uh, and, and while Epic has provided some, some tools and capabilities in there, um, it's really hard then to inject those into your test scripts.

And then repeat [00:04:00] those. Uh, when Epic made, uh, the decision to go to quarterly updates, uh, we stayed on a semi-annual. Uh, but we're revisiting that actually. And I'll tell you the reason why we're revisiting is Epic's development cycle is, is accelerating particularly in the AI space and so often. I'm looking at something that's now available, but I can't apply it because I'm waiting on an upgrade.

So I've got, uh, for example, CDI, uh, queries and, uh, you know, CDI reminders. Um, I can't implement that capability until, uh, may when we do a double upgrade. So we're going to revisit that. We're going to look at it and, and, and probably we're going to end up moving to quarterly.

Bill Russell: With manual testing. I can't imagine without quarterly.

chris: can't do it. And that's why we went to, you know, originally we went to a semi-annual, you know, that's just Epic. But then there's all the other projects and [00:05:00] initiatives that, you know, that we're trying to thread. Um, but with testing automation, we actually think we can go to quarterly and we can increase the release cycle, uh, and actually benefit training by not by, by being more incremental in capabilities and doing more micro learning than having to do, you know, more elaborate training, more comprehensive training.

So these things all kind of fit together. SureTest has been a, a capability that we invested in. Uh, and I'll tell you, my team, you know, they, they were very skeptical. Uh, they were like, uh, I'm going to, you know, build these scripts and how's that going to happen?

Bill Russell: This, we've been doing this for years, Chris.

chris: Well, and, and we test, I mean, we test extraordinary.

So the last, um, six Epic upgrades we've had. Not a single severity one or severity two ticket associated with those. Um, we [00:06:00] implement clean, but we test, and I'll tell you, even Philips teams are like, holy mackerel. The NJHS team wants to test things and, and really kind of challenge us, right? Because this is kinda one of those areas where it's like, okay, if I test too much, there's diminishing returns.

How do I, how do I right fit this and, and what level of risk do I wanna take in those upgrades? Uh, you know, so for us, uh, you know, that testing automation is not just for the upgrades, it's for testing, uh, things that may have actually even been introduced into the environment, uh, through a change management process that, that, um.

Know, it hasn't, hadn't been as thoughtful through, so that's, that's been our enabler for us. It's also freed up resources, right? So if I'm spending less time writing test scripts and, and performing test scripts, I can spend more time applying enhancements and changes. Uh, and those things, you know, [00:07:00] sometimes are small, but they're, they impact a large number of people.

And, uh, so that's a, you know, the, all these things are, you know, competing. And, and so testing automation for us, uh, is a really big deal, uh, because it, frankly, it frees up, frees up bodies to do the work.

Bill Russell: I'm, I'm going to come back to that in a minute. I wanna pull Phillip into the conversation. Uh, when, when an organization like Uh, comes to SureTest, how, what does it look like? What does the implementation look like? What does it look like to, take what is an existing, manual process that I, I mean, I joke that you know, hey, we've been doing this for years, but. have been doing this for years. This is a known process. People, um, you know, good, bad or indifferent, they've been doing this, testing this way. What does it look like to bring them on board? And then what are some of the things you find as you sort of move through that process?

Phillip Furukawa: Sure. And so, you know, as we start having the [00:08:00] discovery conversations around, you know, how do you test today? What's in your, you know, your test library today? A, um, our team works in a collaborative fashion to determine, you know, where there may be redundancy, uh, where they need to maybe test a little bit more thoroughly.

And to Chris's point. Northeast Georgia has a, you know, highly complex environment and their team was doing a phenomenal job of testing. beauty in automation is it's going to perform exactly as we, you know, we develop it to perform and sometimes we kinda joke, computers are dumb in this sense. They're going to do exactly what it's, we tell 'em to do nothing outside of that. and, um, when you look at, you know, a way that our solution is architected, it's, it's different and intentionally. So from your traditional RPA and, and taking a, you know, if then, and that approach. We truly emulate what the end users are doing and the actions that they're taking. So we mirror that level of quality, but we're in, we're increasing the level of frequency in which your scripts are being run and the rigor that's associated with them.

So picking up those changes that are being, you know, potentially introduced into or promoted into a [00:09:00] production environment. Uh, or, or, you know, that's being eliminated by up to 90%. We're finding those things much sooner. We're delivering that back to the team so that they can fix it before it hits production and has an impact on your end users.

Bill Russell: Do, do you have ongoing testing or is this just, you know, once a quarter or Once, once every six months that you're doing this kind of testing?

Phillip Furukawa: So yeah, when you go to a family practice, right, and you go see your, your, um, your family doctor, uh, that's a one day script.

That's easy. We run that, you know, every day if it's a seven day inpatient surgery script, right? We can't run that every day, but we're going to run it maybe once every eight days. So we run those scripts on a schedule that is most appropriate for that script. But we do turn the entire library over at a minimum once a month.

So you are getting the benefit of regression testing.

Bill Russell: What does it look like when you're the first time you come in? do you guys come in, do at the elbow with the testers and sort of pick up their scripts? Or do you guys bring in a a series of scripts that they pick and choose from?

I mean, 'cause each environment's a little different.

Phillip Furukawa: It is, you know, we've got two different [00:10:00] approaches. We can take a tailored approach, which is we'll come in and, and piggyback off an Epic upgrade. So we'll leverage that integrated testing cycle that you have, uh, so that it's not a net new ask or commitment from your team. They're going to be performing that testing anyhow. We just simply piggyback that event and record those sessions via, you know, a Zoom or teams call. We have different, uh, breakout rooms for those different days of activities. Our team directs, uh, in this case the Northeast Georgia team into those different days of activity, really maximizing their time and commitment, uh, to that, uh, capture process. So when you look at, uh, the, you know, overall timeline associated with Northeast Georgia's, uh, traditional manual testing. For an integrated testing event, it was three weeks. You have, um, you know, you have, you prep the test environment, update the scripts, and then, you know, actually executing those scripts. We are able to capture all of the, their scripts in three days, three and a half days. we're, we're really thoughtful and try to maximize the time that their team would spend with us. And, uh, we do that across all of our customers now. The other approach that we [00:11:00] have is our optimized approach. We've captured around 4,000 workflows, across 28 customers today. And so what we've been able to do with that is, is we can now look at across all of these organizations, where is minimal viable coverage? You know, what does that look like across these organizations? We can pull the scripts from our de-identified library and we can, you know, put together what we feel is the appropriate level of coverage for an organization to get started rather quickly. deliver that automation, um, and then tweak it along the way. So we understand that organizations are going to be nuanced and how they test their workflows are going to be slightly different. We can make those adjustments on the fly. And the way that we do that is our team has been, uh, we've been really intentional about who we hire. So we have Epic certified analysts who have previous, previously worked for healthcare organizations. Or work directly for Epic, um, that understand the changes that are being introduced into the environment. Understand why those scripts have failed and they can update the automation, uh, in real time and keep that environment evergreen, uh, so that it's prepared for any testing [00:12:00] event.

Bill Russell: You know, it, it's, uh, Chris, the fact that you're talking about quarterly updates speaks to, uh, some of the things that SureTest talks about in their, uh, in their literature. But I, I, I, I sort of wanna stress test some of these things. SureTest talks about eliminating up to 80% of the manual testing. Did did that hold true at, uh, Northeast Georgia?

chris: Yeah, you know, I just pulled the, the data on this. I'm glad you asked me. So first of all, um, we've seen a 98% reduction in, in testing time allocated humans, touching, doing testing, 98%, which has far exceeded my expectations. We survey our teams though about the testing and how they feel about testing and what's their confidence level.

We got an 86%, um, confidence rate in the testing. Um, and so what's happening? So if I look over the last four quarter, the last four upgrades, we [00:13:00] are seeing consistently a reduction. in The number of upgraded related, uh, incidents through our monthly regression cycle. So, uh, in the last three we went from 161 regression incidents to 155 and now down to 45.

And we believe it's because we're doing this constant, uh, regression testing and we're, we're able to really make sure. Um, that things are built correctly and, uh, we're seeing issues being addressed more proactively in our, in our overall production environment. So I would tell you it has exceeded, we have almost 500 scripts, uh, in place is what the, what the latest number is.

Uh, we started closer, about 250 Uh, so we, that has grown over time. Uh, but we're getting huge value out of it, you know, and we, we, we [00:14:00] estimate in our shop and we have, uh, 367 FTEs in it in total. There's about 48 of those are in, in training and informatics, so you kind of gives you some context., So about three 20, , is what that leaves us with.

Uh, we're getting probably between seven and 10 FTEs of savings out of that. Um, and, and you know, frankly, that's not seven or 10 FTEs I've eliminated. Those are seven to 10 FTEs that I've, I've moved into enhancements or projects or other activities.

Bill Russell: Yeah, I mean, like you talked about earlier, I mean, the pace at which things are coming out of Verona right now are pretty fast and furious, and, and those people were doing sort of double duty, right?

chris: Yeah.

Bill Russell: they

chris: What happens is, you know, you end up, by the time you, you literally kick off the next upgrade two weeks after you finish. I mean, so you have now provisioned in your, in your staffing a team that's totally [00:15:00] focused on upgrades. Uh, and that's all they do. So being able to strip out the testing components allows us to focus on enhancements, uh, and, and really making sure that we've maximized that which was.

Probably the biggest challenge of being an Epic customer, even when I was at at Children's Healthcare of Atlanta, is that Epic's always cranking out these things, but you don't have the capacity to actually implement that incremental functionality, right? So now I'm, I'm reallocating that time, and that's where people enjoy, right?

I mean. I'm not allowed to do testing 'cause I like to go off and do whatever I want and just kind of click on stuff and break it. Um, so I've, I've been banished from, from testing, uh, but

Bill Russell: give me the keys to the data center. I wasn't allowed in the data center.

chris: oh, I, I, they tried that with me. That's, that's, that's a knock on block. But, uh, the point is, is nobody likes to do testing. Right. It's, it's, it's, it's so laborious. It's so monotonous to be able to bring our team. A solution [00:16:00] that really they didn't quite buy into, uh, but are really seeing that value now is, is pretty awesome.

And you know, if I, if I said, uh, we're not going to do that anymore. They, uh, it'd be a mutiny, right? I mean, we, this is just part of our, our portfolio now.

Bill Russell: Philip I, the SureTest is sort of the F-150, uh, truck of, of, uh, healthcare it in that it's, it's not sexy, but it's really functional. I mean, when you talk about 7, 8, 9, 10 FTEs, you guys, you guys throw out some numbers. UNC health saves 17,000 hours annually. And I mean, this is on your literature and, um, I think I read that white paper. Uh, and there's other health systems that are averaging 25,000 hours a year, uh, in, in savings and manual testing. I mean, that's, that's, that's uh, real work that can be done on things that are going to impact. The [00:17:00] clinician experience, uh, the, the things that matter, right?

Phillip Furukawa: Most organizations, their, their team is operating at capacity or over capacity, and so how do you give them time back to, you know, to Chris's point, empower them to practice at the top of their license.

That really improves their, their experience as an employee. Um, they get to focus on things that are really going to move the needle for the organization. You know, the, the optimizations, the configurations that. are going to drive that utilization, help drive some of that gold stars, you know, initiative that you're probably looking at.

And there's, you know, this is well known across to all the organizations too. There's no shortage of projects that are in the backlog. And so what can you get to, you know, that you, you previously were not able to get to and accomplish with anywhere from 17 to 30,000 hours back annually. And so there's, um, there's a lot of value there in that time savings of being able to reallocate it.

Bill Russell: Chris, I'm going to give you the last word here for for a CIO listening to this who's accepted. Manual testing. Um, you know, it is just the way things are. It's, I don't really want to tackle [00:18:00] that because, you know, we have other things to do and they see it maybe not as a, a top priority. Um, what, I mean, what would your coaching or what, what would you, uh, tell them?

chris: I just sat down with our CEO last week and went through our Epic executive packet, and he asked me all the reasons why we had to. Everything implemented. Uh, freeing up those resources to get after that is, is the number one reason. Uh, and, you know, some organizations will say, well, we're risk tolerant and, you know, we'll do a bad upgrade, have a few issues with an upgrade, and, uh, you'll find out how intolerant healthcare is too.

To problems, right? I mean, we need these upgrades. Clean, implemented, live business as usual, doors, open, lights on. This is a tool that will help you do that, free up those resources, and get you focused on the things that your CEO cares about, which is [00:19:00] turning up capabilities that are going to either drive down cost, or improve efficiency.

Bill Russell: Hello Chris. I wanna thank you for your time. This is, uh, is, uh, exciting stuff. I mean, we're entering a time where, um, the velocity, uh, not only from Verona, but. Just across the board, the amount of things we're being asked to do is, uh, feels to be only increasing and to be able to free up resources to, handle those kinds of things that are higher level things.

It seems like, uh, a, a real priority for many, healthcare organizations today. So I appreciate, uh, the work you're doing, Chris, appreciate the work you're doing Phillip as well. Thanks for your time.

Phillip Furukawa: Thanks Chris. Appreciate you guys.

thanks for watching this solution Showcase on Keynote with me, bill Russell. We believe every healthcare leader needs a community they can lean on and learn from. Discover more solutions and join our community at this week. health.com/subscribe. Share this with someone who could benefit from these insights.

Thanks for [00:20:00] listening. That's all for now.