Our latest in our cloud disaster series is the 2014 StorageCraft outage
that resulted in loss customer data.
A server decommissioning mistake during a cloud migration resulted in metadata
loss and StorageCraft scrambling to help customers re-seed their backups.
This may remind you a bit of what happened to carbonate, but this
is very different because here the CEO took full responsibility.
And the company I think went above and beyond to help customers
re-seed as quickly as possible.
Prasanna and I analyze what went wrong, the lessons learned for
backup providers and customers alike.
And what steps you should take if you ever find yourself.
With your backups lost in space.
We'll discuss the importance of migration processes, backup resiliency
measures and why trust is so critical in the backup and recovery business.
If you're not familiar with me, I'm w Curtis Preston and there's a reason
I'm so passionate about this subject.
During my first job as a backup admin, my company lost a huge
database and I couldn't restore it.
Since that nightmare, I've dedicated my career to making sure that
will never again happen to me or anyone who will listen to me.
We turn unappreciated backup admins into cyber recovery heroes.
This is the backup wrap-up.
W. Curtis Preston: Welcome to the show.
Hi, I'm w Curtis Preston, AKA, Mr.
Backup, and I have with me my career reverse trauma specialist
Prasanna Malaiyandi, how's it going?
Prasanna?
Prasanna Malaiyandi: I am good, Curtis, how you been?
Are you doing a little better today?
W. Curtis Preston: So for the first time in a long time, I
find myself at the keyboard.
Um, you know, in a production environment administering a
very big net backup environment.
I haven't been the guy on the keyboard in a while, and it, it was.
Fun.
Um, and doing, you know, doing performance testing and all, you know, normally
it's been somebody else at the keyboard and I'm like advising, but this was
me in a data center all by myself.
Um, you
Prasanna Malaiyandi: you back 30 years.
W. Curtis Preston: was interesting.
Uh, it's, it's not quite 30 years, but, but yeah, it's been, uh, it's
been a while since I've been the guy and, you know, and standing in what
we're going to call a data center.
It's really just a big storage closet that happens to have a, a server rack.
We're doing a, a backup of this, uh, customer's environment and, uh,
let's just say it's, it's not small.
It was, it was 400 terabytes of data sitting on a couple of little NAS boxes.
Um, so it's been, it has been interesting.
I will just say that, um.
Uh, standing in a very noisy server room talking to somebody on the phone, I could
barely hear them on my, you know, on
Prasanna Malaiyandi: just be lucky.
You at least have cell phones now.
W. Curtis Preston: I, it's nice that I have cell phones, cell phones and, and
chat and screen connect so that like.
You know, they can see what I'm seeing.
And, um, you know, all of that.
And, um, yeah, it's been interesting.
But, um, speaking of interesting backup scenarios, we're continuing our series on
cloud disasters and we have another one.
This is very much in the vein of what happened to Carbonite.
It's very similar and yet different.
So,
Prasanna Malaiyandi: Same.
Same, but
W. Curtis Preston: wanna do a quick summary of what happened to Carbonite?
Prasanna Malaiyandi: So with Carbonite, what ended up happening was they had
their own storage that they were using.
Right.
Uh, and people would upload their backups to
it, and they had a double disc failure that took
down their storage array, and they lost all their data.
And they basically told customers, yeah, don't worry about it.
We're just gonna re-upload and do another backup and get all your data back up.
And during that time, there were something like, I think it was 35 customers, 34
customers I wanna say was the number, um, who ended up having another fault
in their production data, right.
On their laptop, didn't.
Crash or whatever else happened, and they were not able to restore their
data because there were no backups left.
W. Curtis Preston: Yeah, the, I I, I think the part of their story that
bothers me the most is that, you know, the company's reaction was our
stupid storage array vendor, right?
They blamed it on their storage array vendor and they sued their vendor.
Um, and so while this story.
Is gonna start out sounding kind of similar.
Uh, it, it's very similar in terms of the impact to the customers, but the response
from the company is completely different.
And so I feel completely different about the company, even though
the exact same thing happened on the, on the customer side.
So first off, we're talking about, storage craft, which.
Is a company that had been around for a while and then had been acquired in
2021 by Arcserve, which is a company that has been around a long while.
Arcserve goes all the way back to my early days.
Arcserve was actually the first backup product I ever tested as
a, to, to potentially buy it.
And running it on NetWare and that boy that brings me back.
But, uh, so Archer has been around a long time and, um, you know, it's
been sold and, and, and I don't know what, you know, spun back out
Prasanna Malaiyandi: equity or
whatever.
Yeah.
W. Curtis Preston: Yeah.
A private equity firm.
They got bought by ca.
They got spun back out with a private equity firm and then, uh, they
acquired or they merged, Arcserve merged with StorageCraft back in 2021
and then this story unfortunately happens not long after that.
I think it was about five months after the acquisition what ended up happening.
Prasanna.
Prasanna Malaiyandi: Yeah.
So like you had alluded to earlier, just like Carbonite, uh,
they lost customer data, right?
Um, but the way it happened was slightly different.
So as part of this, they were moving from their own facilities
and migrating to Google Cloud.
Um, and they accidentally decommissioned a server before
it was already up in the cloud.
And it wasn't just any server.
It was a server that contained metadata about the backups.
W. Curtis Preston: Yeah,
Prasanna Malaiyandi: Yeah,
W. Curtis Preston: metadata is important,
Prasanna Malaiyandi: Yeah, so they basically, um, had the same situation as
what we had talked about with Carbonite, where the customers lost their data.
Now, I think unlike Carbonite, I think that, like you said, they did
a much better job of accepting blame.
And trying to get their customers or their end users back up and
running as quickly as possible.
Doing things like receding and really going the extra distance to get them
up and running as quickly as possible, including things like, Hey, if you
have more than three terabytes, we're gonna ship you a drive so you can
upload to the drive and then send it to help speed up the process.
W. Curtis Preston: Yeah, so for those not familiar with this process,
when you are a cloud backup vendor, the, you know, the hardest part
is that initial backup, right?
You know, it's one
Prasanna Malaiyandi: So
fast.
W. Curtis Preston: Yeah.
Yeah.
Damn.
Physics.
Uh, yeah.
So when you, you know, if, if you're back, you can back up.
I mean, you know, I remember, uh, you know, when I worked
at Druva, we had really, you know, multi petabyte customers.
You can back up that amount of data over the internet, but that
first backup is gonna be something.
And that's why we used the snowball edge.
which was a, an appliance that, that you would ship to them and, you know,
it already had the technology on there.
And then they would back up to that and then that would ship and it was
just mu, you know, basically a very fancy sneaker net was the way to
get the very first copy up there.
And you needed an, you needed one of those snowball edges for I think about
every 75 terabytes, something like that.
And, um, then.
And so that's what they're saying here.
And that by the way, that is referred to as the seed for those that don't
live in that world, that initial backup is referred to the seed.
So when you read the story, you're right, they did accept
responsibility and they said
the first thing that we need to get going is that seed.
And so, you know, you were talking about the, one of the way, just
like I said, the, the way, uh, my former employer would do it was they
would ship, uh, a snowball edge.
They're shipping whatever they need to ship to these customers
in order to make them whole.
One additional wrinkle, I think of this was that this was
specifically for their MSP customers.
Um, and I don't know what percentage of their business, uh, is via MSPs, but
that does, that does make an additional wrinkle in that not only are they
impacting their customer, which is the Ms.
P, they're impacting the end customer, which is the MSPs customer.
Um.
But again, I, you know, I want to, uh, you know, give a shout out,
um, to the CEO, Brandon Lacey.
I like that he, he doesn't try to pass the blame.
He doesn't try to, you know, I, I don't know.
I.
Maybe it's because he didn't have a choice, right?
You know, when you know the other vendor, they're saying, Hey, it was our, it
was our storage array, uh, that failed.
And that's why, you know, that's why our backups died in this case.
They didn't really add anything to blame it on other than, dang
it, why is my fingers here?
They didn't really have anything other to blame it on other than human failure.
Because it was a human that prematurely decommissioned a server before
that server had been migrated.
Prasanna Malaiyandi: Yeah.
And that's just sort of a breakdown of processes, right?
It's not a technology issue.
Right?
And I'm sure now they have a process in place where they're like, Hey, make
sure that the data validate that the server is actually up in wherever it
needs to be before you decommission.
Because.
W. Curtis Preston: Yeah, that is, that is probably the
biggest disappointment, right?
Is you would think you're doing this like major migration.
I remember there was Datto, they did an exfiltration out of, um, Amazon and they
built their own private data center that was a giant, and they said they believed
at the time, and I believe this was Datto, they believed at the time it was
the single largest exfiltration project ever to leave en mass, uh, from AWS.
And it apparently went flawlessly.
You, you've got to, you know, when you're doing a migration, it's like,
you've gotta have that check, right?
You've gotta have that check.
We're gonna kill the thing.
In fact, I wouldn't even kill the thing,
Prasanna Malaiyandi: You would just sort of, yeah.
W. Curtis Preston: I would, I would, uh, turn it off,
Prasanna Malaiyandi: Yep.
W. Curtis Preston: right?
I, I would, in fact, I remember, um.
I remember the very first migration that I had anything to do with, and
I'm gonna, for those of you for the fellow gray hairs in the crowd, we were
migrating off, uh, an at and T three B two 4,000, which was the at t's attempt
at a multi-processing server, right?
So the three B two 1000 was the.
First computer ever built specifically for Unix.
Okay.
Prasanna Malaiyandi: I love how you just have like these model numbers.
Just like right off the tongue, it just rolls right off.
You're like, oh yeah, everyone should know about the 3D two.
It's like R 2D two.
you might as well call it that.
W. Curtis Preston: Yeah.
What's that?
Prasanna Malaiyandi: You might as well call it R 2D two.
W. Curtis Preston: Yeah, so the three B two 1000 was the first, you know, 'cause
I think it was the PDP 11 was the one that, uh, that Unix initially ran on.
But the three B two was the first computer specifically designed to run.
Unix and it ran, uh, system five and, um, you know, directly from
Bell Labs, right at and t and the three B two 4,000 was, you know, they
attempted a multi-processing machine and it, it just, it wasn't very good.
And so we were trying to migrate off of that, and I remember that.
Basically what we would do is we would just kill things as we
believed we had migrated it off.
We would, we would turn off a database.
We would turn off a server.
Prasanna Malaiyandi: see what broke.
W. Curtis Preston: That's the, and well, yeah, and then see if anything broke.
Right.
You don't, I, I don't know.
I, I guess that's the one surprise here, is that they, they decommission
the server to the point of actually, you know, deleting it somehow.
Prasanna Malaiyandi: Or, or, I, I know they talk about decommissioning.
I do wonder if it's sort of, um, because it's metadata, I wonder if it's like
a distributed system and maybe they killed the node, uh, without, and so
since it didn't have access to the metadata, it's sort of like maybe
they weren't expecting that to happen.
Right?
So even though they say
W. Curtis Preston: Yeah, I, yeah.
Prasanna Malaiyandi: I don't know the details.
So.
W. Curtis Preston: Yeah, we don't know the details.
We're Monday morning quarterbacking, et cetera.
Um, but let's, but let's just talk about it.
When you are doing a migration, you don't kill the thing.
You don't, you don't kill the old one.
You turn off the old thing, right?
You turn it off and then you wait a while before you, uh, actually
redeploy it, delete it, whatever.
And then I would.
Then I, you know, I, I find myself wondering, well, I, I guess there wasn't,
there wasn't a backup of that thing.
There wasn't a backup of the backup, which is not that uncommon
Prasanna Malaiyandi: Yep.
Backup
W. Curtis Preston: uh, backup.
Backup.
I.
Services.
Right?
Um, and as a result, when that system was deleted, it, there was no backup
of that backup, which I think many people might be surprised by that.
What do you think?
Prasanna Malaiyandi: No.
Uh, I think people will be surprised because they think that backup
systems are highly resilient.
And of course someone's gonna do a backup of a backup, but you have to also realize
that there's additional costs associated with doing a backup of a backup.
And.
Usually backup systems, people are trying to cut costs, right?
Because this is like the tertiary copy.
And so having high availability or a replica copy of your backup system, unless
it's critical, which a lot of systems are, and people do replicate their backup
systems, but not everyone could afford to do that, and especially given that this
was for their MSP customers who are also trying to make money and add value, right?
They're looking at probably a.
Reasonable offering, right,
uh, without all the bells and whistles.
W. Curtis Preston: Yeah.
And in a world where services are often purchased on price.
Not necessarily alone, but where price is the primary motivator For many
customers, it's, this is one of those things where no one ever bought Salesforce
because they had a really good backup.
No one ever bought Oracle because it had a really good backup.
They bought Oracle because it had really good features and did
the thing they needed it to do.
Nobody's nobody, there's, there's nowhere, no ad in the world.
Said, buy Oracle, because our backup is really good.
Right?
And so the same thing is true here.
Buy our backup service because we also have a tertiary copy of
your data just in case we screw up the copy that you gave us.
Nobody's gonna even know that.
Um, and they're, and they're cer, they certainly don't want to pay for it.
All they see is that this one is more expensive than that one.
Prasanna Malaiyandi: Yeah.
The one thing I am a little surprised about though is given this is a backup
system, and it looks like they did have their metadata and data split apart just
based on what we read in the articles, I'm surprised they didn't have a self
describing format for the data itself.
I know?
there are other systems which tend to do that as sort of a fail safe, so I'm not
necessarily going to create a copy of the data, but if I, my, something happens,
my metadata or there's some corruption or whatever else, I should at least be
able to build back all the metadata based on the data that I still have access to.
I.
W. Curtis Preston: Yeah, that's a good point.
And, and that is a, another really good feature to have in a backup system
that if the, basically the backup catalog, the image database, the, you
know, the database of all the backups, if that gets messed up, it would be
really nice if the, if the, uh, you know, if you could rebuild that.
And I remember back when I was doing RFPs and the, one of the questions
we asked was if the backup catalog, um, you know, gets corrupt, can,
can we rebuild it given the backups?
And I do remember at least one major vendor.
I won't call 'em out by name 'cause it might not still be the
answer, but their answer was.
Um, you know what, they, literally, the phrase was, our backup format
is so proprietary that even we can't read it if we don't know what's on it.
And they, and they saw that as like a, like a feature.
Uh, by the way, it turned out, interestingly enough, it turned out
that, uh, it wasn't that proprietary and.
Other vendors have figured out how to read that backup date.
Yeah.
As long as it's not, um, encrypted.
Right.
That that's a whole other, whole other thing.
Prasanna Malaiyandi: So I don't think we've ever talked about
this on the podcast before.
And so is this something, I know we've talked previously about, Hey, here are
some questions you should be asking your backup vendor when trying to figure
out like what their solution does.
Do you think this is worthwhile to throw into that list of questions?
W. Curtis Preston: Of, do you have a backup of the backup?
Prasanna Malaiyandi: Either a backup of the backup or if I lose
part of my system, like my catalog, can I still recover my data?
W. Curtis Preston: I, I do think that's a perfectly valid question to ask.
Right.
And I think the other question is perfectly valid as well of
if, what protections do you, I would point at this story, what
protections do you have against this happening in your environment?
Right.
Um,
Prasanna Malaiyandi: because it may not be malicious.
It's just accidental things happen.
It's all software.
People are writing software, right?
W. Curtis Preston: Yeah, the, you know, so this, remind this, this story to
me is a, is a combination of the KPMG story and the Carbonite story, right?
Because like the KPMG story, and by the way, we have another episode on that.
Please feel free to check that out.
The KPMG story is one of failure of process as well, where.
You'd like to think that at a, at a company that size and when you're
doing something of this magnitude, I, you know, I, when I think back
on that story, I don't think they.
Thought this was of the, the, we're, I'm just trying to fix one
person's, one person's account.
They didn't think that what they were touching was a tool in this
case, uh, retention policies.
They were touching a tool that could wipe out everybody's data.
They, they
Prasanna Malaiyandi: Hundred 40,000 people.
W. Curtis Preston: 140,000 people's data with no backup.
Uh, that's a great, that's a great story.
Check that one out.
But I, the, the big thing with processes, you've, you've got to have
a very, uh, you know, the bigger the project, the, the bigger the process.
And I do remember, I.
When the, the biggest one I remember being involved in it was, it, was, it, it wasn't
a migration, it was a physical move.
We moved, a bunch of servers from one data center to another data center,
We had built a nice new big data center and we're gonna move the servers from the
old data center to the new data center.
And we had a massive process and we had tested it from one end to the other,
and we outsourced whatever we could.
So like we hired.
A professional moving company to move the servers we hired, uh, and paid extra
to have HP come in and, and, you know, unplug the servers and go ominous dominus,
you know, you may touch this server.
Right.
Um, and then, and then it got handed to the movers and stuff
and we had this big process.
Um, but there was one major failing and that was from the old
server to the new server room.
The new server room had, um, Twistlock.
You know, plugs under, underneath the floor.
And the old server room had the old standard, uh, north
American, uh, two-prong plugs.
And, uh, that was the, that was one day where I truly saved the day because
I had, uh, oh, and the electricians, that would be the ones to help us.
Were busy at the CEO's house, putting up his Christmas lights.
Um, and so, and so they didn't think we'd need electricians during our big move.
And I knew how to fix this problem, right?
I, I'm like, look, we can do this.
I just need to go to a gray bar, uh, electrical store.
And there's one down the road and it was closing in.
20 minutes, uh, by the time we realized what we needed to do, and I, I got in my
car with, with my boss and, and he looked at me and he says, I will pay the ticket.
Just get us there.
And we drove.
I drove and it was raining.
It was, I mean, there was water everywhere.
I'm driving to this gray bar and I've got it.
It's, it's, it's closing in 20 minutes and it's 15 minutes away.
Right.
I don't have time to play.
And I remember hydroplaning going around a big semi, and I remember
like, you know, swerving around a corner and all this stuff.
And I remember that when we got there, there's that scene in, uh, planes, trains,
and Automobiles where, where Steve, Steve Martin's fingers are stuck in the dash.
Uh, that was the way, uh, Lou was when we, when we got there.
But we got, we walked up to the door and it was five minutes prior to the,
you know, prior to the, the closing.
They were, the door was locked.
And, uh, and, and they, they pointed, they pointed at her watch.
They're like, they're pointing at their watch as a time.
And I pointed up at the clock on their wall and I was like,
no, it's not one o'clock yet.
Open your damn door.
And then we, you know, uh, and then I think, I think I like, I was like, please,
please, you know, please let me in.
And we got it.
And we bought all the stuff.
And I, I non.
Traded non whatever electrician, but I was trained as one in the Navy, um, which at
that time was only like five years prior.
Um, and then we wired adapters from.
A to B and save the day anyway.
Yeah, you gotta test your process and you gotta be ready
for failures in your process.
But I, I think that, you know, so the two big this, this one I feel very different.
Like I said in the beginning, I feel very different about
this one versus the carbonite
Prasanna Malaiyandi: but why
W. Curtis Preston: because they didn't try to, they didn't try to pass the buck.
Prasanna Malaiyandi: is that really the only difference in your mind is,
well, I think also in the Carbonite case, the only reason we heard about
it is when they try to sue the vendor.
W. Curtis Preston: Yeah.
Um, this.
Yeah, that's a good question.
If, would we have ever heard about the story if they, I, I don't know.
I, I, I'd like to think that it would still have made news, but all we
heard about was the fact that they store, they sued their storage vendor
in this case.
The article, right.
Um.
It starts out with this quote from the CEO, right?
In terms of magnitude is the only thing I'm thinking about waking
up and going to bed every night.
It's orders of magnitude more important than anything else that I have going
on in terms of operating this business.
Um, says the CEO right, uh, of this company trying to bounce back.
Now we haven't, I tried to find, I didn't find, this was almost two years ago now.
I didn't see.
An update on what happened.
They were saying, we're trying to get people up and running.
I didn't see any stories like we did with the Carbonite about some
customers that actually lost data because they had to restore something
while they were trying to be seated.
Uh, we don't have any of those
Prasanna Malaiyandi: Now, I know you're a big Reddit fan, and I know you
probably were browsing threads about this.
Did you see any, like, what was the sentiment in those threads
from end users, customers that, if
you can
W. Curtis Preston: that's a, actually, that's a good one.
Let me, uh, let me pull up that thread.
'cause I didn't, I didn't look too much at the thread.
This is funny.
Seems like a backup company that loses backups is no longer a backup
company, but a data disposal service.
I see a pivot opportunity.
They should certify these backups as destroyed and issued certificates.
I'd say the Reddit response is typical of every response with a cloud outage.
There are people that are just, you know, saying, ah, the cloud sucks, and,
you know, and the cloud is just, it's just another computer, it's just another
service, and you have to verify it just like you verify every other system.
Right.
Um, and there, there were some comments in the thread saying that.
You know, one of the reasons that Arc Serve and StorageCraft, uh, merged was
that they had been going down for a while in terms of quality of service.
Again, these are pe other people's comments, um, and that they, that they
weren't that surprised at, at this event.
But, um, they're still around.
They're still running.
Um, just like Carbonite is still around and still running.
I, I, I, you know, I wish them the best.
Uh, this was a major migration fail.
Prasanna Malaiyandi: Yeah, and it's also one of those things
people have, like once you lose that trust, it get, it's difficult,
especially in the space where you are that last line of defense.
W. Curtis Preston: Yeah.
I mean, people are still on LastPass.
Prasanna Malaiyandi: Yeah, let's not go there.
W. Curtis Preston: Yeah.
What's that?
Prasanna Malaiyandi: not go there.
W. Curtis Preston: Yeah, yeah.
Anyway.
All right.
Well, um, not, not a ton, not some huge lessons here, but the big one
I think here is, you know, I, I like the Maya Culpa that the company did.
I.
Uh, and I like that they knew that their priority was getting the,
the next backup, but unfortunately that's gonna take a long time.
Um, and the, the biggest lesson here is if you're doing a migration of any
kind, don't go shooting the other servers until you know that it's fully migrated.
You know?
Prasanna Malaiyandi: what, what?
Is there any lessons learned for the end users?
Like there's a, like what could they have done?
I don't think there's anything that they could have done in this situation.
W. Curtis Preston: yeah, the only, the only thing I, in terms of.
Trying to identify, I mean, what's a vendor gonna say?
This is the difficulty with the cloud.
We've discussed this in previous other things, is it's hard to actually verify
if you say, do you have processes in place for if somebody, if somebody over
there accidentally, you know, blows up a server, do you have processes in
place to be able to bring that server back or re of course they're gonna say,
yeah, that's what they're gonna say.
Um, the, the, the one thing you could think about, again, this is
expensive, but people talk about using a service that puts stuff in more
than one place or using more than one service, but people don't wanna pay
for one backup service, let alone two.
Could they do that?
Should they do that?
Sure.
Will they do that?
Nobody does that,
Prasanna Malaiyandi: Yeah.
No one's gonna spend the money in their budget on that.
W. Curtis Preston: Yeah.
Um.
I mean, it's hard enough to get people to do backups in the first place.
It's hard enough to get them to, to properly secure those backups.
Um, they're not gonna pay for a completely separate, just in case they're, you know,
uh, I will say having something like I.
StorageCraft or carbonite, die and lose all your data is not
the same as losing all your data.
Right.
It's, it's, it's all the backups of all your data.
And there will be some, could be some ramifications to your business if you
need to restore, but statistically speaking, you're not probably
going to need to restore today.
Um, just in terms of statistically Right.
Um, it's not the same.
So, you know,
Prasanna Malaiyandi: So, okay, so if I was an end user, I get a
notification from StorageCraft that says, Hey, by the way, we had an issue.
We lost all your backups.
They're gonna say, okay, I'm gonna try to help reseed.
Right.
Get all your data up there before that happens.
Because like you said, it could take weeks, it could take months.
Right.
What is the first thing that the end user should do at that point?
Right?
Once they get that first notification to make sure that they don't end up
losing all their data in case something happens to their production copy.
W. Curtis Preston: Well, I, I'll say this, you got any big projects,
uh, in process, maybe you should pause those projects, right?
Any big changes, any big maintenance windows.
Maybe you should put a pause on those maintenance windows.
Uh, the other is you should go back to old school backups for the
time being, for anything critical.
Uh, you know, you should make a local backup of any kind that, a backup of any
kind is better than a backup of no kind.
Uh, that's a really good question, right.
Um, I would put a pause on projects and I would see what I can do to get
some local backups of stuff, right?
There are plenty of open source tools out there.
That can, you know, surely somewhere you've got some disc and then you've
got some super critical servers that you really don't wanna lose.
Get some local backups of those.
Um, you know.
Prasanna Malaiyandi: or I know you had mentioned in the past, right?
You really like the iDrive service, cloud service,
W. Curtis Preston: Right.
Prasanna Malaiyandi: right.
For just doing
W. Curtis Preston: Well, the.
Prasanna Malaiyandi: backup
W. Curtis Preston: Yeah, the, the only problem with that is, you
know, again, it's physics, right?
So I'm just trying to, I'm just trying to think of something
that you can do quickly.
Um, you know
Prasanna Malaiyandi: Copy it to a different machine.
Go buy a couple external hard
W. Curtis Preston: Yeah.
Specifically for those super high value, super highly critical
servers, you know, uh, things like backup servers for other customers.
Uh uh.
Yeah.
Okay.
Well, all right, well, thanks for helping me walk through that.
That was a really good question there at the end.
Prasanna, thanks for helping me walk through this.
Prasanna Malaiyandi: No, no worries.
That's why uh, I enjoy doing these podcasts 'cause it helps all of us, right?
Just figure out what should we be
doing in case this ever happens.
W. Curtis Preston: That's what we're trying to do is to pass on these
lessons to you, uh, the listener so that you can learn from other
people's struggles, unfortunately.
But, uh, you're why we do this.
We want to turn you into a cyber recovery hero.
That's a wrap
The backup wrap up is written, recorded and produced by me w Curtis Preston.
If you need backup or Dr.
Consulting content generation or expert witness work,
check out backup central.com.
You can also find links from my O'Reilly Books on the same website.
Remember, this is an independent podcast and any opinions that you
hear are those of the speaker.
And not necessarily an employer.
Thanks for listening.