Listeners of this podcast have heard why snapshots aren't backup.

You've also learned why replication isn't backup either.

But what if I make a snapshot on one array and replicate it to another array?

Is that a backup?

A lot of people would say yes.

We'll also need things like reporting and cataloging, of course, but I would

argue that snap and replicate also known as near CDP is one of the most

efficient ways we have of protecting data.

Hi, I'm Debbie Curtis, press and AKA Mr.

Backup, and each episode of this podcast, dives deep into a backup related topic.

We turn unappreciated backup admins into cyber recovery heroes.

This is the backup wrap up.

Welcome to the show.

Thanks for listening today.

I am your host, w Curtis Preston, and I have with me the guy that

made me wait for him today.

Prasanna Malaiyandi

I am good, Curtis.

I am so sorry for making you wait.

and, and why did I wait again?

So you could finish the last few minutes of a series that you have already

Well, it's, and it's not even the entire series, it is just

the episode that I was on.

Now, granted, I'm near the end of the show, so it is sort of getting

to the cliffhanger phases, but I was enjoying my lunch while watching

the last bits of a show, and

Yeah.

then Curtis, and I was like, Curtis, I need five minutes.

And then I was like, no, I need seven minutes because

there's still six minutes left.

Well.

It is time for the news of the week.

the first news of the week falls into one of my favorite news categories.

Do you know what category that is?

Things that you should be backing up that people don't

realize until the data's gone.

Yeah, I was just gonna say, I told you so.

So what do do you wanna do?

You wanna jump right in on our first news

yeah.

So this is actually something that I ran into on Reddit, which

I for some reason get cis admin.

Subreddit, uh, articles in my feed and it was like, Hey, did anyone notice that

they have data missing from Google Drive?

And I was like, oh man, what is this?

And then it slowly started picking up.

I think the register carried it.

Bleeping computer and other folks as well, but.

Basically what happened is people realized all of a sudden that some

of their data was gone and that they didn't have any of their files and

other changes since May of this year.

Since May of this year, and this is being recorded in the end of November,

so that is a long amount of time

and, and Google has officially responded.

And what they're saying as I was looking at these, these instructions and,

and it basically said like, um, don't mess around with your drive right now.

Right.

So they were, they're saying don't disconnect.

Don't do any structural changes to your drive.

And to me what that says is their pro, that there was a suggestion that maybe

somebody did a rollback of some snapshot.

Is that

Yeah, and I think we should be clear, this isn't just general

Google Drive, so if you just access it through the web portal, right?

All of that stuff still works fine.

That has all your data.

This only specifically affected customers who were using Google

Drive for desktop to connect.

So that I, I gotta

Oh, is that no longer

I.

I'm seeing comments from the, that was what we thought a few days ago, but if

you follow some comments, and again, it we're, we're, we're coming this from,

you know, from the outside and we're not personally being impacted, but

there are people who are saying that they've never used the desktop version

and they're experiencing the problem.

Those people may be wrong, it's just a couple of people.

Um, but they're saying that they're experiencing the problem, but.

So I, whoa.

Which is in, yeah, that's news because I was worried about

that, so I went and looked.

And Curtis, I know we do, we use OBS as a backup for us recording this podcast

and we upload that to Google Drive.

And I did go and check to make sure, because we just uploaded one a couple

of weeks ago, and that still exists.

So I don't know if whatever we had shared it with, it looks like

not everyone is affected, but.

right.

It looks like there are some random set of folks who somehow,

some reason are affected by having some of their data gone.

Yeah, so it looks like, uh, I don't remember exactly where I read it,

but the idea is that someone rolled, basically rolled the entire drive or, or

a section of this entire drive back to, uh, essentially the end of April and.

So, and that's consistent with what they're saying of like, don't do,

don't do any work in this right now and don't do any structural changes.

'cause what I think they're going to try to do is to basically undo that

action, which would then put the, all of those same customers back to what

happened before all of this happened.

And if you're making any changes in there right now, those changes

will be undone by that change.

What's a little disconcerting is that there isn't, there's no, you know, we,

we talk a lot about how companies respond and Google isn't doing those things.

It's.

very, very little information out there about this.

Yeah, there's just this one page that it basically said, Hey, don't do anything.

There isn't, I haven't seen any updated stories I checked before

we recorded this, and I'm a little, a little concerned about that

I'm also surprised that.

They gave service engineering or whoever else the capability to even roll back

production to a snapshot that far back without checks and balances in place.

Now, I don't know the what happened, right?

This is all just assumption, but it's a little scary that someone

had that sort of capability.

It's a lot scary.

Uh, imagine if you're a company that is using this, you know, you've

got all sorts of stuff stored in there and you just rolled it back.

Uh, yeah, I, I, I hope that there's an update on this.

I hope we know more than we know right now, but, uh, if you're, if you're

a Google user, it's time to just double check what's going on in your

Yeah.

And this is one of the other reasons why you should back up

your data, even if you use Google

Yes, yes.

This is, this is why I put it in the, I told you So category, you know, the,

the Google, you know, the, the cloud is, is a magical place, but it's not magic.

So, um, let's take a look at this other story.

And it has to do with a ransomware attack on a hospital chain in

Nashville, Tennessee, and they've got.

30 hospitals and 200 care sites around the country.

Oklahoma, Texas, New Jersey, New Mexico, Idaho, and Kansas.

And they were forced to divert patients from a, a number of ERs and one of

the other things was that people weren't able to book appointments

at, um, you know, their usual doctor because the patient portal was down.

I just wanna say.

It wasn't that long ago.

Do you remember when the ransomware groups, they specifically

didn't target healthcare?

Um, because it tend, you know, people can

die, but that is

clearly gone.

And remember there was the case in Germany, I think, where a patient died

because an ER was closed and they had to re reroute them to a different one.

And that's, I think when it came out where ransomware actors were like,

yeah, maybe we will avoid hospitals.

Yeah, but clearly not here.

Right.

They targeted this group and, the only update that I've seen is that.

They, they are starting to restore some services.

The, the company said that they did notify law enforcement.

It did say that they, um, that they contracted a cybersecurity firm.

These are all good things.

These are the things that we like to hear.

These are what people should be doing.

Um, but we don't yet know if they were able to restore services

or if they paid the ransom.

Uh, we don't, you know, we don't know yet.

And I think the other big thing is these are your medical records, right?

And another unintended consequence is some of these particular

facilities provide specialized care for certain types of, uh, ailments.

And if they're.

They're providing that specialized care and then they're down.

It's not like they can just divert that care to some other place.

So yeah, this is a real mess.

Um, you know, I just, the, the thing I think we can take away from this is what

it's again, what, what I've already said.

I like that they contacted the law enforcement.

I like that they contracted with the cybersecurity professional.

The key there is that you want to start having those conversations.

Now you want to identify a cybersecurity firm that you can

contract with, that you can work with.

One of the ways to do this is to con, is to talk to a cybersecurity.

Uh, like if you, if you have cybersecurity insurance, to talk to them, uh, and see

if they can put you in touch with somebody now so that you can prepare, uh, you know.

Rather than, um, going to Google and saying cybersecurity first,

Yeah.

the middle of your, uh, ransomware

attack.

So that's, uh, that's hopefully what we, yeah.

A little bit too late.

All right, well, that is the news of the week,

All right.

This week on the backup wrap up, we have another backup to basics

topic, and I wanted to talk this week about near CDP and which is

near continuous data protection.

And I, I think in order to do that we have to sort of back up a little bit and

talk about the things that we've talked about that have led up to this point.

Uh, these are modern backup and recovery methods.

Basically things that have been birthed in the last 20 years,

basically in the 21st century.

Before we talk about near CDP, I think we need to talk about the

things that have led up to this point.

And, uh, we're gonna talk about replication, snapshots, and what

we call continuous data protection.

So let's talk about replication first.

Do you wanna take that on?

Yeah, so replication is basically you're taking data in one system and

replicating it to the other system.

So the second system is an exact copy of the first system.

And in the case of synchronous, there's no data loss, right?

So your RPO is zero.

And yeah, so it is in sync.

It's basically a mirror.

And that also means you don't have multiple versions on that secondary side.

So if you have a logical corruption or you have a user error on the primary,

it's just gonna replicate it blindly to the other side, and that's what you get.

Exactly.

It makes your stupidity just more effective is what the way I, the way I

like to say it, and it doesn't matter whether you're, I mean, I suppose

maybe if you had an asynchronous replication, you could, may, maybe

there's a big enough buffer that maybe you could stop a disaster if

you did something really stupid.

But you'd really have to be on the ball, I would think,

uh, to, to do that in general.

The, the, the replication.

Replication will be great for Dr when your site blows up, but

it will be really worthless if you're the one that blew it up,

Yep.

If, if you had dropped a table or, or you got a ransomware attack,

which I think we can all agree is.

Uh, a big deal, right, right now,

Oh yeah.

Uh, and by the way, each of these that we're summarizing have their

own episodes back before the episode.

So, so if you don't, if you're not familiar with these topics, these,

this is just a review of them.

They, they each have their own episodes.

Yeah.

So I think next in the list of topics, I think we talked about CDP next.

So do you wanna talk about continuous.

Yeah.

So basically continuous data of protection is replication with a back button, right?

It, it, it, it works very similar to replication except that the way it

stores the data on the other end, it does it in such a way that you can bring.

The, you know, the destination back from the bad thing, right?

The great thing about replication is that it's incremental, right?

That it's block level and it, and, and it, you know, it can keep up with, you

know, relatively speaking, real time of what's going on in your production site.

The bad thing about replication is the exact same thing, right?

So, so CDP gives you the ability to go back in time.

If you did something stupid, like drop a table, get a ransomware attack, have

some sort of logical corruption, it gives you the ability to go back in time.

It has a couple of different ways that it does that.

Um.

amazing.

Curtis, why isn't everyone using

Yeah, it wa if, if we were having this conversation, say 20 years

ago, everybody was gonna do CDP.

Uh, the only problem is it's, it is expensive af right?

Uh, not just the cost of the software itself, it's also the cost of all of the

IO and all of the storage required to restore essentially every single change.

From, you know, during the entire recovery continuum that you are, uh,

trying to be able to support and, uh, the, and, and so there are very few

actual, I think, true CDP products.

There are some that are very specific, like Zerto, I think,

uh, would be a CDP product.

The, there are some, uh, the EMC recover point.

I know that a couple of the other products that I tracked have now been

acquired by other companies and they're just a product on their portfolio.

The, um, but the, basically the problem is it's just too dang expensive, especially

if we're gonna use it for everything.

Right.

Um, and then we have snapshots.

Now you used to work at a company that.

Did a snapshot or two.

And by snapshots we mean storage snapshots, not the ones up in

AWS, which are entirely different.

Yeah.

So storage snapshots let you take a virtual copy of a particular volume

file system, et cetera, and keep it there so you can quickly go back to it.

If you need to restore really rapidly, it's all stored locally, which is great.

And some companies, actually, I would probably say most companies these

days allow users to browse snapshots.

So they don't need to call up the IT help desk and be like, Hey,

can you restore this file for me that I accidentally deleted?

It's already there in the system.

They can manually browse it, they can pull the data out themselves,

self service, it's awesome.

Saves the backup team a bunch of time having to do restores.

The downside of snapshots though, is it's on the local system.

And when we talk about backups and the purpose of backups, you wanna

make sure you have a copy that's independent from that primary copy.

When you have a snapshot, if something happens to that system, if someone deletes

that volume, then that snapshot is gone and you've lost your quote unquote backup.

So a snapshot is not a backup.

And I will caveat that with what Curtis said earlier.

Snapshots have changed their names based on what the vendor decides to implement.

So an EBS snapshot isn't really the same as what I've just been talking about.

It is completely different.

They actually make a copy into AWS S3 that is independent from the production,

and therefore it doesn't follow what we've been calling snapshots,

even though AWS calls it a snapshot.

Yeah, exactly, and, and I think they're not the only cloud vendor to do that.

I also know, for example, our previous employer calls their backups.

They call them snapshots, which I didn't like it when I worked there and

I don't like, I still don't like it.

But yeah, so when we're talking about snapshots here, we're talking about

storage snapshots, like what you would see in a NetApp or, uh, and there

are different kinds of snapshots.

There's copy on, right?

There's redirect on, right.

And again, there is a whole separate.

Episode just on that topic.

So my memory is that I coined the term near CDP back in the day.

They just, they just called it snapshots and replication.

And as you may recall, CDP was all the rage.

And I remember thinking that.

CDP was very expensive.

And because of that, very few people are going to use it.

They might use it for their severely, like tier one applications, but they're not

gonna use it for regular every day data.

And what was more common back in that time was that most people would use.

NetApps for that type of data.

I mean, at that time, NetApp was kind of, you know, ruling the roost

of the, of the NA world, right?

Network attached storage, and they were very big on snapshots and

then replicated snapshots, and they

and replicate.

I.

Um, you know, you could do multiple tiers of that.

They were happy and you could, you could

Yep.

Replicate the data all around the world.

exactly.

Exactly.

And I liked the term near continuous.

And I remember, um, one of the folks that I interfaced with was, um, uh,

storage Zilla, which is, uh, mark Toomey.

Uh, he lives over there in Cork.

And, uh, that was my, that was my attempt to do a cor for anyone who listens there.

And I remember he just, he just really hated my term.

Like, he's like, continuous is a binary term, you know, like, like immutable.

It's a binary term.

It's either continuous or it's not.

You can't be near continuous.

Like, like it's like saying you're near pregnant, right?

Pregnant is a binary term.

And I'm like, yes, but we do use the word like nearly dead.

Right there.

There aren't times when we do put the word near next to a binary term, and I

just felt that this was a world that was much closer to continuous than it

was to what we thought of as backup.

Backup at that time, and honestly, even to today.

I think, I don't know.

This is one of those, like, I don't know for a fact, but I'm

pretty darn sure that most people still just back up every night.

Yep.

Right.

It's like if your RPO is 24 hours or less, you are probably doing some

form of, I'm just gonna use quote unquote replication, which is all the

stuff we just talked about, right?

Which includes Async sync, CDP, near CDP, which by the way, I also don't like

the word near CDP, but that's just me.

Well, you just have to get over it 'cause you're on, you're

on the podcast now, buddy.

Yeah, but, and then everything beyond 24 hours is probably backup.

And I know as technologies change and everyone was like, Hey, database backups.

I wanna do it more frequently than every 24 hours.

Let me do log backups and all the rest of that stuff.

That's when things sort of backup, sort of started reducing the RPO

Right?

and started moving down into that near CDP space.

And, and again, if you're not familiar with the terms Rt, o and RPO, you

really should be recovery time objective, recovery point, objective.

It, it literally drives all backup design, right?

Recovery time objective is how, how long have, have, you know, us and

the, the, the, what do you call 'em?

The, um, sorry, the stakeholders.

What, what have we and the stakeholders agreed that it is an acceptable time

for the recovery to take, right?

We, we have to be able to bring the data back in four hours, right?

And then our PO is how much time, how much data we've agreed

that we are allowed to lose.

By a measurement of time, not, you know, we could lose 10 gigabytes of data.

It's, we could lose one hour or four hours or 24 hours worth of data.

That's what RPO and those two things drive backup design

Yeah, and I would say that it's also useful beyond backup design.

I think anytime you're talking about.

Data protection, disaster recovery, backup, all of these things

always take into consideration.

RTO and RPO.

Yeah.

Uh, no one cares about backup window anymore.

It used to be that was that, that drove a lot of backup, uh, design.

But, uh, you know, luckily we, we've, I think we've tackled the backup

window problem, so you would probably call what we're about to talk about

snapshots and replication instead

Snap and replicate.

And actually when we went back to the replication issue or replication

episode, I would actually call async replication snap and replicate.

But that's because of how I entered the storage space and

the technology with NetApp.

So that's what I, when I think of async replication, I

think of snap and replicate.

Interesting.

Um, obviously it doesn't have to be snap and replicate.

Ay replication could just have a buffer.

Right, right.

And a lag is just a snapshot.

Right.

So every six hours I do that.

That's my lag.

Gotcha, gotcha.

Yeah, I, I would say that.

Snap and replicate would be a way to do a sync replication for sure.

And the, I, I think the more common way people would probably

just use this term, uh, snap and replicate, and I'm fine with that.

Uh, this is one where I, where I, I'm not going to battle for the term.

I do like the term because I think it's a lot closer to continuous.

What's that?

Is it

not, it's not trademarked.

Feel free to use it.

There are some systems where you don't make the snapshot on the primary system.

You replicate the data, and then you make the snapshot over there.

My problem with that is that when you're making the snapshot, you often have to.

Interface with the thing that's writing the data to the snapshot, right?

So you want to put Oracle in backup mode.

Take a VSS snapshot, take a VMware snapshot, whatever it is, do the,

do the thing that you need to do to get the data to be consistent.

Then we take a snapshot, then we replicate that snapshot.

I don't like replicating

You don't, you don't, you know how people deal with that.

I'm laughing at it because I've actually worked with groups and

products that actually do that.

Right,

so, uh, one way you can solve what you're asking Curtis, is when you

take your snapshot, you are queing the application and you issue the

snapshot command to the target.

But, but the problem with that is that we need to make sure that the bits are

By QCing.

Yep.

I find, I find that very.

I find that messy.

I don't like it.

I'm

I It's not clean.

Yeah, it's not

Yeah, it's not as clean.

I, I like clean.

So when we're talking about, you know, near CDP or snapshots of replication,

the really nice thing about it is that you can take essentially as many

snapshots as you'd like to take within the limits of your storage system.

I, I don't know what, do you know what ONTAP is up to these days?

I am guessing probably a thousand.

Yeah, that's a lot of snapshots,

Yeah.

You could take a snapshot every minute for the first hour.

You could take a snapshot every hour after that, et cetera, et cetera, et you can

take the snapshots as much as you want.

And then basically what you're doing is you're replicating the changes that are

contained within that snapshot, right?

Um, and

it's much more efficient because the storage array itself is

keeping track computing those differences and sending 'em out.

So it's much, much faster at doing that than reading the data out, figuring

out the differences and sending it.

Yeah.

The challenge, I think, is that it is a storage level solution,

which means that you need to do the interfacing up to the application.

Sometimes the storage vendor can help you with that.

Sometimes you're on your own.

I've been in both scenarios.

But at today though, Curtis, I wanna say most backup vendors

integrate with most storage vendors.

Or, and it may not be a hundred percent, but if you're picking like the major

common ones, I'm guessing that most backup vendors have API integration

with the storage vendors APIs in order to be able to trigger that snapshot.

Yes, you can.

So the question is, do they both interface with the application and with

the storage snapshot at the same time?

All I'm saying is you need to look into that, right?

If you're taking a snapshot, you need to do your best to make sure

that that snapshot is, is application consistent, um, ver versus the

alternative, which is crash consistent.

Right.

And, and by the way, let, let me just, yeah, let me just use that, lemme just

talk about that term for a minute.

So if you are not.

Making a snapshot with the, in, in partnership with an application,

you're creating what's called a crash consistent snapshot.

It's called that because it is as consistent as a crash.

You, you're essentially like, it's like you flip the power switch off

on an, on an operational storage array and you get what you get.

Yes.

Nothing is moving, but.

Stuff was moving.

So your mileage will vary.

Well, nothing that was committed to DI or committed by the storage array.

Any rights that were committed by a storage array has been preserved.

Anything that was in flight may not have been committed.

And as an application, you might have to do some recovery steps once

a storage array comes back because you don't know what the state is

Yeah,

because some of those in-flight rights might have been

committed, some may not have.

And there are those who say, look, you know it works 99% of the time.

You just take more snapshots.

And if this snapshot doesn't work, maybe the previous snapshot will be,

I'm just, I just, I try to avoid crash consistent snapshots whenever I can.

Right.

I would.

I, I agree.

For the most part, but there are cases where you could use a crash

consistent snapshot at a more frequent basis and do like an application

consistent snapshot, say once a day.

So even though, so you can potentially

have that as your backup of your

Yeah.

Yes.

As a backup of your backup.

Right.

And that

why?

Why would you do that?

I'm guessing the answer to that question would be, I.

Perhaps if doing an application consistent snapshot has an impact on

the performance of the application.

Uh, I know in the case of Oracle, for example, when you put it in backup

mode, it changes how it stores the redo logs, which could, which can

have a minor impact on performance.

And so perhaps you only do that once a day when nobody's using the

database and then you do the crash.

Consistent snapshots more often than that, I, I don't have a problem with that.

Yeah.

Yeah.

But just relying on, yeah, and then you just take that snapshot

and then replicate it off.

Right, and the, the beautiful thing I think of snapshots and replication

or near CDP, is that what you have?

I'm glad that you find that term so amusing, what you have at the um,

I think that's why I can say that I coined this term 'cause nobody

else seems to want to use it, so I must have coined it and I love it.

Um, so the, um, and it's in at least two books, two that I wrote.

I don't know if it's anywhere, I don't know if it's in anywhere else, but,

uh, I don't care what you'd call it.

We're just talking about snapshots and replication.

Just don't,

the 15 years that I worked,

don't call near C-D-P-C-D-P because NetApp definitely tried that one.

Right?

It is not continuous.

The, the reason I was laughing is yeah, the 15 years that I worked

in the storage industry, I'd never come across near CDP ever in the

way that you're talking about it.

Yeah.

Yeah, yeah.

And I'm fine.

I'm fine with that.

So again, I'm still taking credit for coining it, even if nobody uses it but me.

It's not like the 3, 2, 1 rule or anything like that.

Um,

One other thing I wanted to mention about Snap and replicate that I don't

think you covered yet is there are some vendors when you're doing Snap

and replicate, you may not always have to have the same snapshot retention on

your source array and your target array.

You might, for instance, decide I'm only gonna keep 30 days worth of

snapshots on my production system.

And on my secondary system, I'm gonna keep 90 days worth of backup, uh,

worth of snapshots so I can go back.

Some systems allow you to set different retentions for snapshots on both sides.

Some may not.

So you should also, once again, look at your vendor, see what's possible.

But I know for some folks, instead of having to go beyond that 30 days and

say, okay, now I have to go to my backup infrastructure and pull data off of

it, they might be able to say, okay.

If it's not in production because it's beyond the 30 days, let me go

check my secondary storage system.

Okay?

I have 90 days worth of snapshot.

Can I restore the data from there?

Right.

Yeah.

I, I, I love that idea, right?

'cause it, one of the, one of the nice things about this idea is that you

could have, maybe have a more expensive primary storage array and you can have

a less expensive storage array that's based on Sada, for example, as you're.

As your a backup system.

And another thing, by the way, that you can do with a near CDP setup is that

you can use that secondary site to give, I'll coin a new term near C DP plus.

So, so near CDP plus is snap, replicate then back up, right?

Use that snapshot that's on that.

On that target and then back that up with some other method that isn't 'cause one

of the downsides that some people pick on, uh, snapshot and replication is that

your entire, basically storage and backup infrastructure are all within one vendor.

And the, the worry is about this idea of a rolling bug that somehow

takes out all of ONTAP one day.

And it takes some, it takes everybody's primary, uh, and their

secondaries along with it, so.

The other issue also with that just snap and replicate, is if you say, have a

backup proxy, so you're backing up your NASS system, you're using a proxy, which

is basically a backup client to mount that snapshot and copy the data off.

One of the challenges you have is when you mount it.

To the storage array.

That backup client looks no different than any other production client,

and so when it ends up reading the data, it could cause performance impact

because it has to read the entire file system on the source to figure out

what's different and move the data off.

This, of course, isn't integrating with the native snapshot storage APIs that the

storage vendor provides, but is actually just reading it like a normal file system.

When you do snap and replicate, you can actually mount the

snapshot on the target system.

And do your backup off of that, and therefore you're not affecting your

production application because you're not impacting the IO on that system.

Or you could use our friend Steven's favorite thing, the NDMP,

Yep.

You could use NDMP

the network data management protocol.

Which was, which was another solution.

This is like to, this is technically off topic at this point, but there was this

other way to back up, uh, NAS systems.

Well, it's still around.

Is that you can back up essentially to tape.

DMP is generally meant to go to tape, uh, or to virtual tape.

And, uh, it was meant to solve the issue that you mentioned because

it would recognize it as a backup process and then deprioritize it.

Uh, nice.

It,

I.

Yep.

There's another use case I wanna talk about with SNAP and Replicate, and it's

not necessarily backup related, but there are many companies who have a distributed

environment and they need performance.

And so what they sometimes do is they will snap and replicate to multiple

systems as kind of a fan, as kind of a fan out, and then they would have

clients read from those target systems because they're consistent at some

point, and use that as, uh, read.

Optimization rather than all these systems trying to hit a single production system.

And these secondary systems could be in the same building.

It could be spread across the world.

So you're now sort of doing read load balancing and you're leveraging

the snap and replicate technology in order to move a copy of the

data to close to the clients.

Yeah, that, uh, by the way, that's, we, we, I don't think we really mentioned

this before, but that's one of the best things here, is that that secondary

target, and maybe even a tertiary target could be very far away because

you're doing asynchronous replication, so you shouldn't be impacting the

performance of the, of the primary array.

Uh, at least not much anyway.

Um, but that, that's, we can put that generally speaking as far

as we want to from the primary.

Yep.

So I'd say the final thing that we would say about snapshots and

replication is that that which we've already sort of alluded to, and that

is that your backup vendor may support this as just another way to backup.

Production data.

Right.

Most of the popular NAS vendors, especially nas, uh, are gonna

have something like this.

And then, uh, the more popular they are as a NAS product, the greater

the possibility that they will integrate with a, a backup app.

Right.

So, um, this is just another way to backup up, especially your on-prem storage,

although some of these vendors are now starting to offer actually for quite some

time now, are offering cloud versions of these typically on-prem products.

Um, so anything, can you think of anything else that we should talk about?

Persona,

I think that covers it all quite a

it's just a, yeah, it's, it's, it's a great way, I think to have a very tight

RPOA ver a really tight RTO, right?

The RTO is really small.

'cause basically you just start using the snapshot that, that you,

that, that there's no restore.

You can start using like the replicated snapshot immediately while you're

restoring the primary snapshot.

Right?

That, you know, that's sort of the beautiful thing of, that.

You might get a.

Reduced performance.

Um, but so, so the RTO, it can, you can meet a really tight RTO, you

could do snapshots very frequently.

So you can also meet a, uh, a really tight RPO, um,

I did have one thing to add since you were just talking about it.

So one thing we didn't talk about, which I think is.

Super awesome about snapshots is we mentioned previously that snapshots

are read only, which is great if you wanna pull some piece of data out

of it or something else like that.

But if you have applications where you need to actually do some recovery process,

you can actually take a snapshot, which is read only, and most storage vendors allow

you to clone it into a read write volume that you can then mount and connect to

your and do your recovery process again.

Again, without occupying the full amount of space, because it's all

based on the snapshot, spins up a copy, allows you to do the recovery process.

It's read, write.

You could do all your testing, your restore verification, which

we always talk about on the podcast is go restore your backups.

And once you're done with that and you validate, you can quickly toss

it away, and then you're good to go.

So that's another benefit of sort of snap and replicate, is you can

do all this verification on your secondary system without once

again impacting your production.

Right.

There are a lot of advantages to the snap and replicate style of,

you know, I'm calling it backup.

Right.

And, uh, this is one of them is, is that, you know, the, basically that the.

The, the replicated copy stays in native format, and that leads, that

leads to all sorts of possibilities.

One of which I think probably the best of which is, is all of this you, you

can do automated recovery testing.

Right.

Automated cloning and then, uh, tested recovery.

And that way you're, you're validating the actual snapshot that

you would like to use for recovery.

So yeah, I, it's, it's a really great way, it's a really great way that I think

maybe not enough people take advantage of.

So hopefully, um.

You know, you've learned a thing or two.

And, uh, with that, I wanna say thank you for, uh, joining

us and of course, persona.

This was one where you really shined, I think.

'cause you, you know, your,

This is, this is what I lift and breathed.

Yeah.

Yeah, yeah, exactly.

Exactly.

So, uh, uh, great, great having you on again today.

And I promise I won't harp on the near CDP term as much.

It's gonna take off.

Uh, we'll see.

We'll see.

Maybe I'll, maybe I'll do it in Spanish and then, uh, it'll be, it'll be better.

Uh, and, uh, so thanks to the listeners.

Thanks for, thanks for listening because, uh, that's really the

only reason that we do this.

That's a wrap.