W. Curtis Preston (2): Very few people like their backup system.
So they're often thinking about making a change.
When does it make sense to do that?
And what's the best way to do it.
Today, we answer these questions and we make sure you understand the
risks of changing your backup system.
Along with the rewards.
Hi, I'm W.
Curtis Preston AKA Mr.
ATR2500x-USB Microphone & Logitech BRIO: Backup.
And I've been where you are.
I've been stuck with backup software.
I hated and wanted so badly to change.
So I know how it feels.
On each episode of this show, we dive deep on one topic,
helpful to you and your backups.
And this week it's about change.
We turn unappreciated backup admins into cyber recovery heroes.
This is the backup wrap up.
W. Curtis Preston: Welcome to the backup wrap up.
I'm your host, w Curtis Preston, and I have with me my personal
interface into the SpongeBob world.
Prasanna Malaiyandi,
Prasanna Malaiyandi: am good Curtis.
Yeah, it's SpongeBob amazing show.
So growing up I didn't have access to SpongeBob 'cause I'm a little
older than SpongeBob, but I remember I was probably in college and I
went to a good friend's house and his six-year-old nephew was there.
And that's the first time I saw SpongeBob Squarepants.
And ever since then I'm addicted, so I do watch it.
Even to this day, I may or may not have SpongeBob Squarepants socks that I wear.
Also.
W. Curtis Preston: And I have like zero connection to the SpongeBob world.
Again, you know, if you're too old for it, I'm way too old for it.
It was just a few days ago that I found out that the phrase, if you
moments at Atar, uh, is from SpongeBob.
I didn't know
Prasanna Malaiyandi: Great show.
I think you should start watching it.
W. Curtis Preston: Really?
Yeah, that's what I'm gonna do.
In the midst of all of
Prasanna Malaiyandi: a great background show to
W. Curtis Preston: Oh, is it a background show?
Prasanna Malaiyandi: Yeah.
And they have the first six seasons on Amazon Prime for free, so
W. Curtis Preston: How can I resist it?
You know, I've never actually watched an entire episode.
I, maybe I'll watch an episode or two
Well, it is time for us to get into the backup news or the, you
know, the news of, of our world.
Uh, do we wanna start?
I, I should start with yours.
Uh, our news from the AWS Storage Day back, uh, looks like about
Prasanna Malaiyandi: So this recently came out, um, it is AWS trying
to help with ransomware attacks.
Um, for folks who don't know, AWS offers a service called AWS backup,
which allows you to protect and manage some of your AWS resources, like EBS
volumes, EC2 instances, and others.
And so what they've recently done.
Previous to this new release, what they had was the ability to sort of create
a vault, is what they called it, where you kind of squirrel away your backups.
It would be stored there, it would be protected, so they
had like retention settings.
It could be supported using the immutable feature, object lock
capability of AWS and basically prevent backups from being deleted.
But this was all within the customer's account, and it was either AWS
managed keys or customer managed keys, and so that was great, but someone
could potentially still get into the system, start deleting backups.
It's not completely foolproof.
And I know Curtis, we've talked in past episodes, just actually just
recently, about air gapping and what air gaps mean and virtual air gaps.
And so now what Air AWS has offered is, let me make sure I got the words right.
A logically air gapped vault.
That's a mouthful.
Yeah.
W. Curtis Preston: fine with that term.
I'm glad that they used that term, right.
The, put the logically, um,
Prasanna Malaiyandi: what it is, is it is a more locked down vault
where unlike before, uh, where the customer was managing the keys or
using AWS managed keys, these are keys that are actually owned by AWS.
You can't delete the keys, which means if you can't delete the keys, then you can't.
Prevent access to the data.
Um, and once again, they also have the ability to set retention
that says, okay, make sure that the minimum anything between like
this retention and this retention don't allow anything to be deleted.
And once you set these policies, you can't actually go back and change it.
So it's a good thing because it prevents people from like malicious
actors, from going and changing the settings and sort of changing your
retention policy down to one day and boom, all your backups are gone.
Which
W. Curtis Preston: Yeah.
Prasanna Malaiyandi: thing.
W. Curtis Preston: It looks like, um, it looks like it's
designed to be used cross account.
So one of the things we've talked about in the past is to create an account
that holds the backups for everything.
And that's what it looks like this is designed for, because also
it talks about offering direct cross account, restore, um, which
again, I'm not exactly sure what,
Prasanna Malaiyandi: I today, order do your restores, you kind of have to
copy it back into the account first before you can start accessing it.
And so they've, I think, integrated with AWS resource account access
manager to allow instant access to those copies having to first copy it across.
Which is cool because I think that's another piece that a lot of people
W. Curtis Preston: Right, right.
Yeah.
So I, I think this is, you know, again, it's another step in the right direction.
I like the idea of having the, I do wonder, and I would like to know,
does having it ecr encrypted with AWS owned You know what I'm asking?
Yeah.
Does that mean that AWS can my
Prasanna Malaiyandi: I am sure a lot of enterprise customers would be concerned
about, um, is how do you prevent AWS or someone who has access to the AWS
infrastructure that that data is secure?
Um, in my mind, I think customers today could almost hand roll
this solution on their own.
By using a third party key management service with the current AWS backup,
where they're kind of using customer managed keys, but they're using
a third party service for that.
Uh, and so AWS in AWS fashion is like, Hey, see this as a customer pain point.
Let's build something in natively to simplify things for customers.
So kudos to AWS.
W. Curtis Preston: Yeah.
Kudos to AWS.
Uh, although I do want to ask that.
Prasanna Malaiyandi: Oh, by the way, this is in preview,
Hasn't been, yeah, so it hasn't GAD yet, but should probably be
coming out soon-ish is my guess.
W. Curtis Preston: So the next is another.
Completely different solution, but aimed at the same problem.
A lot of companies are worrying about this, the, the concern of, of
backups being attacked or whatever.
And one of the things that we have consistently said is that one of the
things that you want to do is to make sure that you have a different, you know, a
separate authentication and authorization system for your backup and Dr.
Data, which.
Which would mean that if you're an on-prem system and you're backing
up to the cloud, that you would, uh, that if active directory, for
example, was hacked, it wouldn't be able to attack your backups.
And that is, by the way, no one does this from what I can see.
Right.
One of the most often requested features that I remember back, uh,
when, when I worked at DVA was, you know, active directory integration.
It's like.
But it's not a good idea.
Right.
Um, so the, um, anyway, so this is taking, it's not the opposite
approach, but it kind of is.
So this is from Cloud, cloud casa, um, which I guess is like
a house for your cloud, I guess.
Is what they're going for there.
So it's from Cata Logic, the, the, the company, and it's
specifically designed for Kubernetes.
Uh, it's a Kubernetes backup, uh, system.
And what they're now allowing is for you to do self-hosted, uh, versions of
this soft of this software, and they're specifically billing it in that, let's
say you're running in the cloud and you want to get an air gap backup of that.
The only way to do that in their mind, or one way to do that in their mind
is to put the, the backup on-prem.
So it kind of follows the logic that we have used.
I myself, I would prefer that the, that air gap ba, you know, that air
gap, as I make quotes in the air, I would prefer that they are on some
sort of right protected storage.
Some
Prasanna Malaiyandi: and case Cloud Casa, they do say they
support an S3 compatible backend.
So if that is the case, then I'm sure that there are mechanisms
to provide immutability.
W. Curtis Preston: Yeah, I, yeah, I, I would like to see that.
But the idea here is that you can run their software now on-Prem, separating
it from your cloud environment.
I guess it's very common to run Kubernetes in the cloud.
So then you run this on-prem and so that you have a backup in, in a different, um,
again, that different authe authentication
Prasanna Malaiyandi: I think the other.
Thing that this could be useful for is for those customers who are
running Kubernetes internally, right?
It gives them a mechanism to have a completely siloed environment where,
'cause I think a lot of more modern, uh, Kubernetes data protection
platforms sort of have some sort of SaaS connectivity, if you will.
And for those people who want complete control and complete isolation, this
gives them that mechanism where if they're running something locally in
their on-premises infrastructure, they have the ability to protect those in a
simplified way without having to require centralized management and other things.
I.
W. Curtis Preston: Yeah, so it looks like, I guess if you have an on-prem deployment
of Kubernetes, you can use the Cloud casa service that already runs in the cloud.
But if you have, uh, Kubernetes in the cloud and you'd like
to protect it on-prem, you can now run the, the self-hosted
So
Prasanna Malaiyandi: are good.
W. Curtis Preston: um, options are good.
Well, there you go.
That is the news of the day.
I hope you enjoyed the news.
Today.
I thought we would focus on.
Basically, how do we make sense of everything we just covered?
We covered a whole bunch of different ways to do backup, right?
We covered like the traditional full and incremental backup system, which
may go to disc, may go to tape.
Most likely we go to disk, it may go to cloud.
We talked about CDP replication near CDP.
You know, cloud-based system.
I mean, we, we covered a whole bunch.
Like,
Prasanna Malaiyandi: You just throw a dart.
W. Curtis Preston: is just, just throw a tart.
So now you are thinking about picking a backup product.
How do you do that?
Right?
How do you pick a backup product?
Prasanna Malaiyandi: just starting with the basics, I think it is figuring
out what are your requirements, what are you trying to solve for?
Because if you don't know what those are, it's gonna be hard to pick a backup
product that meets those expectations.
W. Curtis Preston: If you don't know where you're going, you'll
probably end up somewhere else.
Um, if you've ever heard that before.
Um.
I, I, I completely agree with you I think that's a message that we put out
pretty consistently here, is that the more you can set expectations and document
expectations, the more success you will have at meeting those expectations.
If you don't set expectations and document expectations, then.
No matter how good you are, you can't compare them to anything.
Right?
So the same is true of requirements.
If your requirement, I mean, when I go back, I go back to the early days
Prasanna Malaiyandi: back in the day.
W. Curtis Preston: commercial backup product.
When I think back in the day I remember going to a bunch of backup vendors
and I went with what at that time were really basic requirements, but
they very quickly knocked out many, if not most of the backup products
that were on the market at the time.
And so and so what I remember is, and, and that's a great way to do it.
If you have solid requirements that you can dictate let's say a requirement of
you've got to be able to back up Mac oss that's gonna NOC out a bunch of vendors.
Well, why is that a requirement for you?
Well, because we run on Mac oss us.
Prasanna Malaiyandi: Well, and I think that gets to a key point.
It's make sure your requirements are justified and valid because sometimes
people are like, oh, I want a Ferrari.
Right?
But they only have a budget of like a Toyota.
So you have to make sure you understand what are the requirements that you really
need to hit versus what's your wishlist.
W. Curtis Preston: Yeah.
Wishlist versus requirements, right?
So you, you can have, you know, when we talk about requirements, I put
them in three categories, right?
There is.
Basically showstoppers.
You either do or you do not.
Back up Mac oss or whatever it is, whatever that is for you.
Oracle AWS, whatev, whatever that is, you know, for you that, that list.
Then there's g We'd really like it if you could do this.
Right.
Backups by osmosis.
That would be really nice.
Right.
Um, and those aren't going to.
Rule out products, they're gonna help you make a choice when you're confront,
when you get down to that, like three, that list of three products, all
three products meet the showstopper requirements, but one of them has.
75% of your nice hat, nice to haves.
Right?
Um, and then, and then there's like a third category of these sound cool, right?
But honestly, you know, we're, it's not gonna, it's not gonna
swing us one way or the other.
I, I, I can't think of a good example of, of what that might be, but I remember
having that, having that category right.
You know, so it's sort of like required important.
And I guess the third is
Prasanna Malaiyandi: Yeah.
And even for these three categories, Curtis, I think it's important to also
consider, it's not just what you need today, but you also wanna think about
sort of the next three years, right?
What those requirements, the must haves, the nice to haves the moonshots will be
W. Curtis Preston: Yeah, that's a good point.
Prasanna Malaiyandi: keeping around
W. Curtis Preston: Exactly.
It's a good point because you should think about the requirements, not just for
today, like you said, but for the future.
So if you are aware of an upcoming project where you're going to migrate
your entire environment to Azure.
Maybe Azure backup might be important to you, right?
It's not today, but it will be in a year or so.
Now would be a time to think about it.
And, and by the way, that's when you can, when you're having that conversation
with the backup vendor and you say, Hey, we really need this feature.
I.
a year, and they go, okay, well we're coming out with it in two to three months.
Right?
And again, get that contractually and all that kind of stuff.
Right.
But, the big thing with the needs is, you know, you, you talked about making
sure that they're tied to something.
I just wanna make sure that we dictate the difference between
a requirement and a design.
Right?
So I, I, I've used this example before.
Uh, I live in San Diego.
We have, um, Coronado on the other side of the, of the San Diego Bay.
It's not an island.
It's technically just a really big peninsula, but so many
people call it Coronado Island.
The, there are two ways to get there.
One is to go, you drive all the way down to Imperial Beach and
then you drive all the way back.
Right?
If you're downtown San Diego, that's a, like a 30 mile round trip.
Um, the other way is to go over the Coronado Bridge.
That's a mile and a half.
So what, what has that got to do with the requirements?
So, so.
At some point somebody said, we've got to get a bunch of cars from here
over to there in a lot quicker way than driving 30 miles round trip.
That's the requirement, right?
Then somebody else said, well, we, you know, we need a bridge.
No, you, you need a way to get cars from here to there.
That's your requirement.
The bridge is the design.
Another, another design would be tunnel.
Another design would be ferry,
Prasanna Malaiyandi: like the how.
W. Curtis Preston: Um, yeah.
That's the how a lot of people, they very quickly get excited about the how.
They forget to look at the why and where this happens a lot, you know, nerds like
you and me, we see this presentation you've likened a CDP to TiVo, right?
I remember the first time I saw TiVo the first time you got to pause live tv.
was like the most amazing thing, right?
You, you remember that, right?
The first time you get to pause live, I'm gonna go to the bathroom
for a minute, pause, live tv, right?
Or somebody wants to, you know, my wife wants to say something to me
and I can just pause, answer her question and I can go back to the show.
That was amazing.
Right?
Well, the same thing happens when you, you know, when somebody shows
you CDP for the first time or, uh, you know what, whatever, pick
your, you know, backup by osmosis.
The first time you see that, you get excited about that.
point is, do you need that?
Right?
You have a requirement to have an RPO and an RTO of x and CDP could meet it because
CD can meet all the way down to zero.
But what is, what is X for you and what are the other ways that you can meet
Prasanna Malaiyandi: Yeah.
And remember that X may be different for different workloads
in your environment as well.
So those are other things.
You have to think about it
W. Curtis Preston: Right, right.
Exactly.
Um, we're gonna cover that here in just a minute.
That, that, that's, we're gonna take a turn
Prasanna Malaiyandi: and, and, but wait, before you get there, since you
threw out some acronyms, do you want to define RTO and RPO for our listeners?
W. Curtis Preston: sure.
and then we'll throw on two more acronyms.
So, recovery time objective, recovery point objective, recovery time
objective, or RTO is the amount of time that a restore will take.
Right.
Um, well, that it will take to bring the system back to
operational readiness, right?
So within that.
Let's say we've got a four hour RTO within that four hour RTO.
One hour of that might be dedicated to the actual restore.
Three hours might be dedicated to getting the, the hardware back up
and running and connecting it back to the rest of the environment, right?
Uh, but that's what recovery time, objective, recovery point objective
is how much data we can lose.
As measured by time.
So one hour, we can lose one hour's worth of data.
And then we talk about RTA and RPA recovery time, actual recovery point,
actual, so this is the RTO and RPO.
That's your objective.
That's the thing you're gonna try to meet.
Um, and you maybe you have to meet RTA and RPA are what the system
that you actually have, what, what it's actually capable of doing.
Prasanna Malaiyandi: Because sometimes they don't match.
W. Curtis Preston: Especially if you didn't go through this
process on the front end.
Right.
aNother thing that's very closely related to what I said before is sometimes
you don't need to make a change.
Sometimes you see something really cool, you're like, oh, that looks
like a really cool new backup product.
Sometimes the best choice is to do nothing, right?
The question is, are you meeting the requirements with what you currently
have and slash or could you meet your requirements with what you currently
have and perhaps some re slight redesign or replacement of a, of a piece of it,
changing the, the backup target that you're using or something like that.
Or do you need, need to replace it?
Is there no way right to,
Prasanna Malaiyandi: But, but we all like new toys, Curtis.
Come on.
W. Curtis Preston: Yeah.
We all like new
Prasanna Malaiyandi: and I think the other thing people sometimes don't think about
going to that point you just mentioned is you could replace the system, but
sometimes you have to also take into consideration retraining, retooling, uh.
Thinking about your existing infrastructure, keeping that up
and running, migrating all your workloads over to use the new system.
There's a big cost with migrating a backup system that you need
to take into consideration.
If you do decide, okay, I wanna swap out my existing system.
W. Curtis Preston: that is absolutely a reason not to migrate.
If you don't have to migrate, right.
Um, realize that this is your last line of defense and every time you make a change,
you introduce instability and risk, uh, et cetera, et cetera, et cetera, right?
Prasanna Malaiyandi: Yeah.
W. Curtis Preston: It's just like everything else in it.
Prasanna Malaiyandi: The other the other thing with backup systems as well is.
If I look at it, budgets, typically, most of your money is probably
gonna be spent on the production side of the house or other things,
not so much on the backup systems.
And so you need to be wise with your dollars.
So deciding, Hey, I'm gonna swap out my existing infrastructure for something
completely different and it's not gonna make a huge difference for me.
That's kind of a waste of money that you could have been
using to do something else.
W. Curtis Preston: Exactly.
So once we have a handle on the requirements, it's
time to start talking about.
The, the various different ways that we can do this.
And I think that nowadays one of the things that should bubble up to the
top from a requirements perspective and, and should really take the lead
in this part of the discussion is when we start looking at the capabilities
of the different products is the cyber.
Aspect, right?
Because it's no good to have like the fastest backup and recovery system in
the world and the one that has the most beautiful ui, et cetera, et cetera, and
maybe costs the the least if that system can be hacked in a ransomware attack.
Prasanna Malaiyandi: I agree.
But I think no, but, but, but I think that there is various
degrees of cyber resiliency.
And you could also look to figure out, like it may not be directly supported
by the backup vendor itself, but there might be say, storage systems
that go alongside with the backup system that offer that capability.
Or how you, going back to what you were saying, Curtis, how you design the system
might lend itself to cyber resiliency.
So I do agree with your requirement that your system needs to have
cyber resiliency, but how it goes about may not be one specific
vendor providing that solution.
W. Curtis Preston: You, you may be looking at an overall solution to
solve the, to solve the problem, right?
And, and I think that that would be, you could say, well, with this vendor,
I need to buy this box and I need to buy that box, and I need to buy
this oss and I also need to buy some stuff from Amazon with this vendor.
I get all of that in one place.
Right.
Prasanna Malaiyandi: Mm-Hmm.
W. Curtis Preston: Um, so that's part of your discussion.
What, what was that?
Prasanna Malaiyandi: simplicity.
W. Curtis Preston: yeah, simplicity.
Simplicity is, is the, it is the, or.
Complexity I will say is the enemy of security.
I, I think, right?
So yes, while I agree with what you're saying, the sometimes you can get the
things that you need from multiple pieces.
I would also argue if you had two systems that are equivalent in every other
way, the one with fewer pieces wins.
Prasanna Malaiyandi: Oh agree.
For sure.
W. Curtis Preston: Yeah.
Um, and so you look at all of the different aspects, things like RTO and
RPO, you really, really should look at ease of use, ease of configuration, ease
of basically daily management, what kind of daily management that you have to have.
And you should also be looking at.
The, again, these security aspects.
I think when we look at requirements, I, I think perhaps we should just do a
whole separate episode of what are the things that I think are table stakes
at this point for a backup system.
Now, you're right.
Some of these things may be provided by a second vendor, for example.
I think that at this point, true actual complete immutability.
Is table stakes.
The, the problem is that there's a whole lot of companies that use the word
immutability when it, it, it's, it's not
Prasanna Malaiyandi: Like air gap.
W. Curtis Preston: Yeah.
Like the term air gap.
Yeah.
Um, and so you, you have to ask specific questions to find out, you know, because.
You have to ask questions.
For example, if I change, you have to ask them.
I, I'm a little devious.
I dunno if you know this, but you have to,
Prasanna Malaiyandi: I never expected that.
W. Curtis Preston: you have to ask the questions in such a way that it
makes it sound like you're asking for something good, but you're
actually asking for something bad.
Let me give you a perfect example.
Your product offers immutability.
What if I change my mind?
Is there a way for me to, as the administrator, undo the immutability
of a particular set of data?
I.
And, and give a reason as to give an example.
Uh, let's say we decided we're no longer backing up an
entire section of the company.
We sold off a company or whatever, and we, you know, we, we'd like
to reclaim that storage, but we turned on the immutability flag.
Um, is there a way to undo that?
The best answer?
Yeah.
The best answer is no, sorry.
The worst answer is, oh yeah.
You just, you just push the thing.
You just push the thing.
It's not really immutability, it's just, uh, you just, you just
push the little, the super secret button that's over here, right?
Um, that only we know is there.
Somewhere in the middle is, yes you can, but we make you jump through 10 hoops.
Um, and we do all sorts of manual, human based, face based verification.
Although as I, as I say that, I'm like, I immediately go deep fake.
Um, and I, I
Prasanna Malaiyandi: that.
Can't be trusted anymore.
W. Curtis Preston: yeah.
Goodness gracious.
What a world we live in.
Um,
Prasanna Malaiyandi: soon.
This podcast will be a deep fake.
W. Curtis Preston: how do we know this isn't a deep fake, I'm just saying.
Prasanna Malaiyandi: Ah, good point.
W. Curtis Preston: I mean, all you have to do is watch a
couple of those videos where.
You, you know, it's a fake video, but it really doesn't look like a fake video.
It's a little freaky.
Um,
Prasanna Malaiyandi: Oh.
W. Curtis Preston: yeah.
So how do you, those are, those are your answers.
And I think that that is to, that's question number one in this system.
If I, so for example, if you use AWS and Object lock, the answer is no.
Prasanna Malaiyandi: Unless,
W. Curtis Preston: Right.
Unless, unless
Prasanna Malaiyandi: unless you are using No.
So there are two modes of object lock, if I recall.
One is governance, one is compliance.
I can't remember which one is more strict, but one does allow the admin
to change it, the other one does not unless you delete your account.
W. Curtis Preston: So you should be using whichever one is the, the one
that doesn't allow you to delete it.
Right.
Prasanna Malaiyandi: Yeah,
W. Curtis Preston: And, uh, although there, there is,
Prasanna Malaiyandi: least I believe that's the case.
W. Curtis Preston: yeah, there is still that question.
And I, and I really want to get an answer to this que a
definitive answer to this question.
What happens when I delete my account?
Now I've been told that it doesn't go away right away.
And that, um, there's a way to recover that account if it's,
uh, if it has optic clock on.
But I haven't, I don't see that in writing and I haven't, and I
certainly haven't, uh, tried it.
That's what I need to do.
I
Prasanna Malaiyandi: Try it, Curtis.
W. Curtis Preston: Just create an account, create object clock, delete
some data, or put some data in there and then delete the account, and then,
Prasanna Malaiyandi: See what happens.
Yeah.
W. Curtis Preston: so what else?
Uh, we, we look at the different, um, you know, the different areas of
functionality for the different areas of your environment, and then to allude
to something that you had said earlier.
I, I think it's important to talk about the fact that, let's talk about
basically best of breed versus all in one.
So is it better to have one backup system that does everything, or three backup
systems that do three things really well?
That's a question.
Prasanna Malaiyandi: So assuming that everything is equal, it's better to have
the one system that can do everything.
However, that's usually not the case.
W. Curtis Preston: Yeah.
Prasanna Malaiyandi: And there might be certain workloads that are more optimized
and significantly better than that.
all-in-one solution in some areas,
W. Curtis Preston: Yeah,
Prasanna Malaiyandi: which case, for those specific cases.
And so for those specific workloads, you might decide, okay, everything else
I'm gonna protect with this all-in-one.
And for this one particular specialized use case, I'm
gonna use this special product.
W. Curtis Preston: and this is kind of what I wanted to talk
about is that the answer should be.
It's okay to do, to, you know, to, to have that specialized workload
protected by the specialized product that specializes in that workload, right?
If you're unable to meet your requirements with the more general product, that's
really the only time to deviate.
And this is, this is just like when, when I talked in the beginning.
The only time to move off of a backup product is if you can no
longer meet your requirements with the current backup product.
The only time to deviate from the central design and the central, you
know, one sort of product to rule them all is when you can't, um,
you can't meet your requirements for one part of your organization.
With the, you know, the, the one big backup product that you're
planning to do everything with, and then suddenly somebody brings in.
I don't have a modern equivalent of this, but I'm thinking about back when.
You know, back in the day when I was in the data center, we had like
nine different types of Unix and we had, we had the top three databases.
We had Oracle and Informix and Sybase, and, and pretty much any popular flavor
of Unix we had in the data center.
And then we had an AS400.
There was no backup product in the world.
That backed up Unix and AS400.
Right?
So we needed a separate backup product for the AS400 because there was just,
there was just literally no other way.
That's an example of what I'm talking about.
Um, can you think of a, of a modern equivalent to that?
Prasanna Malaiyandi: At one of my previous employers, one of the things that I ran
into was when you had very, very, very large databases running on tier one
storage, backing them up was too painful,
W. Curtis Preston: mm-Hmm.
Prasanna Malaiyandi: and I'm talking about hundreds of terabytes.
W. Curtis Preston: Right.
Prasanna Malaiyandi: And so there was a solution developed called Protect
point to allow direct backups of those.
From the storage rate to a data domain system
W. Curtis Preston: Mm-Hmm.
Prasanna Malaiyandi: and intended only for that particular use case, right?
But it allowed them to actually get a backup done rather than all of the
pain that they were going through, or the fact that they weren't able
to backup at all to start with.
W. Curtis Preston: Yeah, I, that, that's, I think that's a good example.
I was thinking when I was trying to come up with a modern equivalent, um, I.
I was thinking of like, maybe you're not running VMware or HyperV or KVM, you are
running some other hypervisor, right?
I, I know people that work at some of these other hypervisor companies
and they're, they're doing just fine.
They're doing, they're having an exciting time right now with the
Broadcom acquisition of VMware.
Right.
It's a, it's, it's fun days for these guys, right?
So you're obviously not going to be able to use your.
Mainstream backup product to back up the non-mainstream virtualization
product, you're going to have to have something else for that product, right?
Um, and that, and that is okay.
Uh, if you have no other choice, um.
Prasanna Malaiyandi: Yeah.
But yeah, and that's just life, right?
You need to back up that data.
It's important to you.
You'll figure out a way the perfect.
The other example, Curtis, as we were chatting, I was just thinking about this.
Most SaaS applications, right?
Most of those are the common vendors don't always support it.
And so you might have one that requires a specific backup product to back it up.
W. Curtis Preston: exactly, because we're gonna back up our SaaS apps, right?
Prasanna Malaiyandi: No, SAS doesn't need to be backed up.
W. Curtis Preston: Don't make,
Prasanna Malaiyandi: I am not serious.
Yeah.
W. Curtis Preston: When I look at all of the different ways
that we do backups, I think that.
Uh, the cyber requirements need to be, I think they need
to be front and center, right?
Because it doesn't matter how good the system is, if it disappears in
a ransomware attack, it's no good.
And I think that the ways that we can do backup today, that offer s restores
that are so much quicker that, um, it.
May cause me to waffle a bit on this.
Don't do it unless you have an absolute requirement bit.
And here, and here's what I mean by that, when you think about cyber recoveries,
when you think about recovering from a ransomware, uh, attack, well, it
might not be an absolute requirement.
For a normal restore to be of a certain speed.
Right.
The recover, the RTO, the, the previous part.
I, I guess I'm not, I'm not changing my answer.
I'm just saying I'm, I guess I'm saying, I'm going back to the beginning and
saying, when you're determining your RTO and RPO, just think about the cyber
aspects When you have a cyber attack, you are going to spend your entire
RTO and probably past your RTO, just figuring out what needs to be restored.
Prasanna Malaiyandi: Yeah.
W. Curtis Preston: So then think about the likely recovery that
you're going to need to do.
And think about how that potentially affects the RTO capabilities
of a backup product or a DR product that you're buying.
Because basically they're gonna finally figure it out.
It's taken 'em two weeks and they're gonna be, okay.
Prasanna just now just restore these 17 boxes, right?
And they're not gonna go, oh, well, okay, well I guess I
can start my 24 hour RTO now.
No.
You got no minutes at this point.
So I guess what I'm
Prasanna Malaiyandi: go run as quickly as you can.
W. Curtis Preston: yeah, yeah, exactly.
Um, I'm just saying that perhaps in the modern climate, the, the restore aspects,
perhaps you should give a little bit more weight to products that can do restores.
Really, really, really, really quickly.
This, I get, this is what I'm saying.
I'm, I'm kind of waffling on the requirement, but I'm kind of not,
because you're not gonna have the RTO set to two weeks, right?
You're gonna have the RTO set to something probably in hours, and
when you get a cyber attack.
It's gonna, whatever, whatever you set it to, it's gonna eat it up.
And then you're just gonna need, need to be able to restore as quickly as possible.
Um, and so just think, just figure that into, when you're talking to,
uh, vendors, when you're thinking about their backup products.
Um,
Prasanna Malaiyandi: I like that.
W. Curtis Preston: yeah, it's all cagey and weird and, uh, it's just,
I'm trying to, I'm just trying to.
Get you to think about that.
If you're thinking about changing your backup product, just think about how,
think about what a likely cyber recovery will actually look like, um, and how
you'll, um, it, I don't know if I've told this story on the podcast, but
I'm pretty sure I've told it to you.
I go back to.
One of the first major restorers that I participated in, and I was
at a bank and we had a, we had a NOC right network operations center.
And when we were doing a large restore, there was someone who was at a console
that was a remote console to system we were restoring and they were sort of,
they were the operation center, right?
And so they were talking to people on phones and you know, we had, we
had, uh, um, I'm trying to remember.
I don't, I don't think we had.
Flip phones.
I think we just had, we just had phones, right?
Regular hardwired phones and beepers.
Right.
Prasanna Malaiyandi: Yep.
W. Curtis Preston: And I remember that, um, we had the guy in the data center.
Uh, I only saw the other half of this story.
I didn't, I didn't see this, the, the, the funny part
Prasanna Malaiyandi: Because you were out in the data center
W. Curtis Preston: 'cause I was the one in the data center doing the thing.
Right.
So.
In the NOC.
There, there was the, the, you know, the guy sitting at the, um, at the,
you know, the console and standing behind him were two managers.
His boss's boss and his boss's boss's boss standing, you know, and it just
so happened that their first name, they're both, first, both of their
first names were Tom and, One of my cohorts was talking to this guy on
the speaker, on speaker phone, not knowing he was on speaker phone.
Prasanna Malaiyandi: Oh
W. Curtis Preston: And so we were in the middle of this big recovery.
And, it's just what I thought about, you know, when you come
time to do the restore, you've got, you've got the attention of the,
you know, the powers that be right.
So he said, you know, so yeah, where are you?
Oh, I'm in, I'm in, you know, this data center and you know, where are you?
Oh, I'm in the NOC, you know, sitting here at the console.
And he goes, let me guess.
You got Tom and Tom over your left and right shoulder.
And apparently Tom and Tom just took like one step back.
Because yes, when the feces hits the rotary oscillator,
that's exactly what happened.
You've got all that attention, a very unwarranted, unwelcome, uh,
well not warrant, unwarranted.
It's warranted, uh, unwelcome.
I.
Prasanna Malaiyandi: All eyes on you.
W. Curtis Preston: Yeah.
So, uh, just my final thought, just make sure that the first time you're firing the
backup system in anger is not the first time you're firing your backup system.
Right.
Make sure that you've got a solid handle.
Once they say, go, and I know I've got 15 systems to restore, I
know how long that's gonna take.
So that you can, you know, and hopefully that, that you can reduce
that time down as, as much as you can.
Prasanna Malaiyandi: Yeah,
W. Curtis Preston: Anyway.
Um, well this has been fun.
Anytime I can reminiscent over a, uh, a scary event from my
past, it's a beautiful thing.
Well,
Prasanna Malaiyandi: which you have quite a lot of, I must say.
W. Curtis Preston: I do, I do that.
I'm a storyteller.
What can I say?
So thanks for hanging out, Prasanna.
Prasanna Malaiyandi: as always.
Thanks Curtis.
W. Curtis Preston: All right.
And thanks to our listeners.
We'd be nothing without you.
That is a wrap.