You found the backup wrap up your go-to podcast for all things
backup recovery and cyber recovery.
In this episode, we tackle a confusing phrase, snapshot backup.
I know what you're thinking, Curtis.
Isn't that an oxymoron?
Well, that depends on what you mean by snapshot.
We're gonna break down the difference between what I'm calling
traditional storage snapshots and what vendors like AWS call snapshots.
Which are actually more like backups.
We'll talk about copy on write?
Redirect on write.
VMware snapshots that are just weird.
How to turn your snapshots into actual backups that follow the 3, 2, 1 rule.
Uh, pretty much everything that you need to know about making.
A backup snapshot.
If you don't know who I am, I'm w Curtis Preston, AKA, Mr.
Backup, and I've been dealing with this topic for over 30 years, ever since.
I had to tell my boss that there were no backups of the production
database that we had just lost.
I don't want that to happen to me.
I don't want that to happen to you.
That's why I do this.
And so, enjoy this episode of the backup wrap up.
Welcome to the show.
Hi, I am w Curtis Preston, AKA, Mr. Backup, and I have with me my QOS advisor
Prasanna Malaiyandi how's it going?
Prasanna?
I am good Curtis, and I'm glad that we don't have a random five second lag
every time we talk, so That's good.
Yeah.
So, uh, we figured out, we, we were, we started to try to record this
and there was this massive lag.
Um, and, um, I realized that basically like I counted 'em real quick.
There are four people currently watching Netflix.
Uh, I was able to stop one of them, but I don't control the other ones.
And so you came up with the idea of, you know, what about QOS?
And so I, I have a firewall.
Um, and, uh, so I, I enabled SmartQ, I put this website on it and I
said, give me, gimme all the stuff.
Gimme everything I can.
And uh, so now it's, you know, beautiful.
So I guess that's a plug for firewall as well, not a sponsor.
Yep.
Or for any router software that actually supports proper QOS, not like most
Yeah.
consumer level ones, which don't do a great job.
Yeah, so this is, I I've been very happy with the, with
the firewall since I got it.
Uh, every once in a while I use something, you know, fancy on it when this is nice.
So, um, so, um, we haven't recorded in a while.
You, you went away.
I went away.
I went on vacation for the longest vacation I've been
on in seven years I think.
So I was gone out of the country for three weeks, which was nice.
I was off in India for a wedding, visiting family.
Lots of travel.
So much good food.
Curtis, um, and
Yeah.
I might be addicted to caffeine.
So
In India as you've had, right, uh, they serve in the south, they
serve filter coffee, which is sort of a small cup of coffee, which
is very, very concentrated though.
And, uh, I was drinking three or four a day.
Now this is normally someone who drinks maybe one cup of
coffee a day, other than when
Yeah.
office where it's maybe two.
So.
Yeah.
Well anyway, well welcome back to the same time zone.
Very excited.
Thank you.
but It is hard with the 12 and a half hour time zone difference.
You know, it was like
Yeah,
we get a brief window to chit chat, or in the evenings we get a brief window to chit
yeah, yeah.
I basically wake up and call you and then before I go to bed, I call you.
Right.
Um, and, um, uh, yeah, so a lot of people don't know that it's,
that it's 12 and a half hours time difference, which is just odd.
yeah, I don't know why they do the half hour.
Yeah, just funny.
Maybe they, they saved the half hour to drink the filter coffee.
So, uh, all right, well, today we're gonna talk about, uh, you know, and I
thought, you know, you thought maybe we'd covered this already, or at least
recently, and, and we really haven't.
So this is a, a word that comes up a lot in, uh, in the show.
And that is this word snapshot, which.
If, if we, if we go outside of it, it, it, it means that, you
know, it's, it's a picture, right?
That that's what the word snapshot and also.
There's another word that's gonna be used in this episode, which is image, which
is really just another word for picture.
But in our world, an image is very different than a snapshot.
And as you've often heard me say, words mean things.
And um, I want to make sure that this is sort of a, I'm gonna
call it snapshots basic episode.
So if you're, if you're just wondering what you know, what are snapshots, what
are the different types of snapshots?
Both what I'm gonna call the traditional snapshot, which is the ones that you and
I, you know, grew up on, if you will, and then something that is relatively new.
Again, comparatively speaking, new, uh, that is also being called
snapshots, which is actually something completely different.
So, um, uh, and then what purpose they serve, uh, et
cetera, et cetera, et cetera.
Any, any other sort of introductory things you can think of before we get started?
no.
I also did like your analogy of a snapshot is like a picture
that you would take right
Yeah.
of like tech, right?
And,
Yeah.
thing to add is that's important is it captures a moment in time.
Yes.
Right.
A
One moment in time.
Sorry,
and
go ahead.
but, so we're not referring to a video or whatever Apple calls
their eight second or six second
Yeah,
that they send over.
Right.
That's not what we're referring to.
It's a single
Yeah.
So first let's talk about what I'm gonna call the traditional snapshot
and, um, which I, another word, it's funny now that I think about it.
This other word that I'm gonna use is also sort of like, um, it's
also a word that is used outside.
Id to also mean.
You know, looking at a thing, right?
And, uh, the word that I'm gonna use is view.
Mm-hmm.
So, uh, if anybody has any experience in databases.
Right.
There is this concept called a view, which is a different view into the database,
which gives you, it looks different than, um, than the, the, the entire database.
Right.
It is a, it is a view, a a particular point of view, if
you will, into the database.
I don't want to go too much into that for two reasons.
One is not really relevant.
Two, I will probably mess up the description, but, um.
Because it's one of those areas where it's right at the edge of my, of my experience.
I am not a DBA, um, but I've often had to pretend to be one
in order to back up databases.
But this goes to your concept or your, the, the point that you were
making is that a snapshot, uh, is a view of the storage at a particular
point in time, so it is a view of your typically file system, but also
it could just be a volume, right?
A virtual volume at a particular point in time and what your, when
you look at that snapshot, the actual blocks that you are looking at, or
you're copying or you're accessing.
What you are looking at.
Is going to like where the blocks come from is going to be based on how the
snapshot is actually being delivered.
Sometimes you are going to be looking at the blocks from
the original storage device.
Sometimes you're going to be looking at blocks from some
sort of cash snapshot area.
Mm-hmm.
Um, and so this view that you're looking at is this sort of virtual
view of a particular point in time and it just keeps track of everything.
And it says, okay, at this point in time they go to ask for this block.
It knows what was where.
So it goes, at this point in time, block A was.
You know, at this status, well block A is still that status over on the original
storage device, so we're gonna go get it.
Or if you ask for block B, you go, block B was this status, and we can
see that that status has changed.
And so we're gonna get that block from the snapshot area.
Um,
And
go ahead.
does a different implementation.
Yeah.
think most of.
As of what I know right now is they're all kind of, all the blocks
are shared, and like you said, as things change, new copies are created
that contain the updated version.
But there's all this metadata and mapping, all this stuff is what
the storage vendors, that's really like what they focus on, right?
Is how do they make it fast?
How do they make sure that you have that plausible point in time view to look at.
Yeah.
And, uh, we're gonna talk about, uh, like a couple of different
ways that this actually happens.
Uh, but the, but when you think about the way I described how that works,
the important thing to understand is I said that if you grab blocks, one
part of the, you know, one set of blocks where you're going to get those
blocks from is from the original.
Thing, right?
The, the device, the volume, et cetera.
And the other way is you're gonna get blocks from the snapshot area, but
the, but the thing that you should infer from that is that in order for
the snapshot to function, you need the original device to continue to be
Mm-hmm.
functional and et cetera, et cetera, et cetera.
Yeah.
Um,
thing to
yeah.
that's important for snapshots Curtis
Mm-hmm.
is that snapshots are read only, right?
Which is a critical fundamental, uh, the, the property of a snapshot, right?
Just like a
Yes.
you take a picture, you're not modifying that picture,
Right.
Good point, good point.
Um, and because otherwise it wouldn't be a snapshot of that point in time.
Now you can take a snapshot and you can make it read, right?
But at that point, it's really no longer a snapshot, right?
Um, it is a completely different sort of function.
Yeah, and typically for those ones do have that snapshot, and then
normally when you make it read, write, you're creating a new entity.
So the snapshot still exists, but then the all the writes go
Correct,
entity.
correct.
Correct.
Um, so there are.
Two different ways that snapshots, again, traditional snapshots are typically done.
And there's what I'm gonna call, um, sort of the the traditional Traditional, okay.
And that is this concept.
And this is pre NetApp, basically right before NetApp started doing snapshots.
'cause NetApp really kind of reinvented snapshots and, and how they work, right?
Before NetApp, you had what was called the copy on write Snapshot and where
it gets, or CCOW copy on write Snapshot and where it gets that, that name is
that you, you have a block, right?
When, when you make the snapshot at the very beginning of making the snapshot,
all blocks are in the original device.
Because nothing has changed.
The moment you go to do something, you go to update, you know, a block.
The question is what happens now?
So in a copy on write situation, the um.
We're going to, before we change that block, we're going to copy that block out
to the snapshot area so that later when we go to, um, access that block, we say, oh,
we need the block from this point in time.
That block, at that point in time, is only available over
in the snapshot area, right.
What's important to understand is that when you go to update a, a block
with a copy on write, um, snapshot, it has to do a right of that block.
It has to move the data, then copy, you know, write that.
Old block in the snapshot area, and then it's gonna write the
new version of that block.
So there are three IO operations for every right operation
in a copy on write snapshot.
And then the more snapshots you have.
And the more, and the longer you keep them, the more, uh, of your blocks
that you're going to have to copy every time you, uh, write new data.
Is that, how did I do with that explanation?
yeah, you did.
Well, um, the other thing also is at some point you end up with a
Yeah.
As well, just because of how writes are being done.
And remember all the time, these aren't just small blocks, right?
So even if you go modify, say a hundred bytes in a block, you still
have to write that entire block back,
Yeah.
sort of the underlying building block for the file system.
And
Right.
this could lead to a lot of wasted reads and writes and IO that you're consuming.
Are you, did you just say that the block is the basic building block?
It is funny, the words that we use, right?
Um, yeah.
And so, um, what's it, I, I think one of the most important things to understand
about a copy on write Snapshot set up is that over time, if you have a lot of
snapshots and you keep those snapshots for a significant amount of time.
The performance of your primary array because it's having to do all of this
copying, you know, on, you know, on write.
The performance of that, both the read performance and the, and
especially the write performance can be significantly degraded over time.
Yeah, and I'm just thinking, don't know any systems to my
Mm-hmm.
that still use copy on write.
I am sure there are some, but I, but I would agree with you that
most newer storage vendors realize sort of the evil of copy on write.
You know, the, the, you know, it's, it's, I. It's, it's, it's sort of like
the famous Maya Angelou quote, right?
Like, we did what we did when we knew what we knew, but now we know
different and we do different.
Right?
Uh, that's a, that's a massive, uh, I'm sure misquote of, but I think I
got, I think I got the concept there.
And I remember being at a large, very large oil and gas company
and we were, um, you know.
Helping them do a, it was an, it was a RFP, right?
Um, for, for just a massive, uh, storage change.
Right.
And, um, they knew how many snapshots they wanted to create, and they knew
how long they wanted to create them.
They were already a, a NetApp customer.
The, the problem with being a NetApp customer was they knew every.
Bad thing about NetApp, right?
They, they, they could see right through the stuff that they said that was bs.
Right.
You know, every vendor has got some amount of bs, but they were a customer,
so they knew everything that NetApp, you know, they knew every blemish about
NetApp, but they, and they were bringing in other vendors to see if perhaps they
could get what they get from NetApp.
Uh, while changing the vendor.
And one of the things that they want is they wanted 90 days of user
browsable snapshots, and they had all this data to show how much money
that that was saving them because of the number of user generated
restorers that were happening, right?
Because one of the great things about having snapshots is that if you give
them access, your users can just go to the right area and they can see their
directory from a different point in time.
Right?
And so they, they had all this data that showed that they had all these user.
Generated restores and um, uh, and so they said we want 90 days
of user browsable snapshots.
I won't name the vendor, but let's just say it's a fundamental law of physics.
Um.
That vendor said, uh, if you do that, you know, you're, you're, you're, 'cause
they were, you know, they were doing copy on write snapshots at the time, and
they basically said, you're going to have a significant performance degradation.
And they, and they asked them like, how bad, and they just sort of, you know.
Spitball the number, and they said it was 50%, like a 50% performance
degradation, but they're like, we actually don't know because no one does
that, no one does that with our storage.
Right.
Um, and, um, uh, yeah, so it's, it's a really big deal.
Right.
So that is,
Yeah.
I. Copy on.
Right?
Which is the old school way of doing snapshots.
And if you're currently having any problems with your snapshots,
maybe you investigate and see if they're copy on write?
That's gonna be your core problem, right?
Um, and, and it's gonna fall under the category of
doctorate hurts when I do this.
Well, don't do that.
You need to do fewer snapshots or whatever.
I just realized there is one vendor that still does copy on.
write.
And do you know what that is?
Who?
Microsoft with VSS Snapshots?
VSS.
The volume shadow services.
Yeah.
I was just thinking about that.
I'm like, there has to be someone, and I was like, oh, yep.
Yeah.
And, and by the way, that brings up a, a really good, uh, thing that maybe
we should have covered earlier on.
And why do you, why do you make snapshots?
Right?
There's two reasons.
And VSS is the second reason.
The first reason is that it acts as sort of, kind of like a backup
Mm-hmm.
that it gives you this view, this place that you can go get and get the file
the way it looked three weeks ago, three months ago, six months ago, whatever.
However long you keep your snapshot.
Go ahead.
can we call it little B backup?
Because, because here, here's my
I what you're saying?
Yeah.
gets, because it doesn't follow the traditional
Yeah.
backup of 3, 2, 1 rule and everything
Yeah, that's why I'm saying it's kind of like a backup, right?
Yeah.
Um, it, it's not really a backup until you copy that snapshot
to some other location, right?
So, and it also needs to be managed and all that kind of stuff,
and you need to know what's in it and all that kind of thing.
Anyway, but to, to follow the 3, 2, 1 rule, you need to at least
copy it to another location.
Hopefully that location is offsite and now we got three
copies and two locations we got.
Okay.
So, um, actually, if all you're doing is.
Snapshot of replication, you're actually not following 3, 2, 1 either because
it's not, you don't follow the two, you don't have, you know, 'cause the two is
meant to be two different risk profiles.
And if you, if you go from one filer to another filer and you're, yeah.
Okay.
Um.
The second reason that you make a snapshot and VSS falls under this
typically, and that is what you're doing, is you're creating a stable
point in time that is now read only that you can then use to back up.
Mm-hmm.
You can point your backup.
And most backup vendors that are, that support windows, they integrate with VSS.
And what VSS do?
You, you, you talk to VSS and, , basically, you know, quick
summary there, there's, there's this concept of a, of a VSS writer.
You talk to the VSS and you say, Hey, I'm here to do a backup.
What kind of, um, you know, what kind of stuff do you have?
And they're like, we have SQL Server.
You're like, okay, do a snapshot for SQL Server, and then you, and then
that gives you a, a stable point in time to back up this, this device that
otherwise is moving around, right?
So it, it gives you a read.
Only copy that isn't changing while you're backing it up.
And then if you're just there to do a backup, it actually immediately
deletes the snapshot as soon as you're done with the backup.
Right.
So that's the second reason that we use a, a snapshot, which is
just to create a stable point in time from which we're gonna run.
I'll call it a big B backup to use your, to use your, your terminology.
All right.
So that's copy on write?
Uh, do you wanna describe, redirect on write.
Yeah.
So this is basically what NetApp does
Yeah.
Although they would say no 'cause they say there's a slightly
different, but that's okay.
You know?
Yeah.
NetApp is probably most famous for this with its write anywhere file layout
Yeah.
right?
Which basically says when you're going to actually do your writes, you don't
need to overwrite the existing block.
Right.
gonna pick any other random block, and then you're just
gonna keep track of where it is.
So all new writes go to all new blocks, so you don't have to ever
worry about going and updating all the metadata and copying data out, and so
you're efficient in your write code.
Pap and managing snapshots becomes a lot easier.
And so the where the term redirect on write, is instead of overriding the
block in its current location, you're just gonna redirect that, write?
Put it in another location, and then your snapshot manager just needs to keep
track of where the old location was.
So the big difference between a redirect on write, so there's a,
there's a plus and minus here.
The plus is much better performance,
Yep.
right?
Which is why.
Pretty much most vendors have gone with that as the newer.
Um, the downside is snapshot data and production data all
in the same volume, right?
There's no snapshot area.
It's, it's everything.
So if you keep snapshots too long, you can actually fill up your, your
volume, um, and have a problem.
Yeah.
And they, that's why they do try to do things like setting up a
snapshot reserve space so you can automatically start deleting it
once it gets to certain too high.
But yeah, there are cases where that could also still be problematic.
So,
Yeah.
And, and this is why, because you can do snapshots essentially.
With impunity, you can do as many snapshots as you want, as often
as you want, without, um, and, and keeping them as long as you
want without any zero performance.
Issues as long as you keep enough storage around for your snapshots, right?
Uh, 'cause then we start getting low on blocks and we start, you know,
then we, that, that, that, that, that's a totally different problem.
So as long as you have enough space for your, uh, for everything,
then you shouldn't have any performance from redirect on write?
Snapshots.
Yep.
And I think with it now, I think the latest NetApp snapshot limit
is I think a thousand snapshots.
It used to be 2 55 for a while, but
Yeah, the,
a
the famous 2 55 and yeah.
Yep.
It's funny how that number just sort of, well, 2 56 really is the number,
but 2 55 is, we can't do 2 56.
We can do two.
Well, it's 2 55 because active is always zero.
Oh, there you go.
There you go.
Yeah.
Um, so that's redirect on write.
And that, you know, and we, we really talked about it very quickly, but
basically it means that you can have as many snapshots as you want, as long as you
want without any performance degradation.
So that's generally what you find in most common, uh, snapshot providers right now.
other thing I wanna also mention, copy on write?
Versus redirect on write.
Mm-hmm.
discs, it made a huge benefit For Flash.
It makes an even bigger impact with Flash there are issues around
right amplifications when you're
Mm-hmm.
um, as well as how many wear cycles you can have on your flash.
And so redirect on write is actually much better, both from a performance
and longevity perspective for Flash
Right.
was on disc.
So
So the prob the problem there is that you're worried that you're having
too many writes to one cell, and each individual cell in, um, flash has.
A limited number of times that it can be overwritten, right?
And so the worry that you have is if you have too many writes to the same block,
you actually end, uh, to the same cell.
You actually end up, that cell can become, uh, no longer usable
but basically each cell has a finite number of times that you
can overwrite that cell and you hit that and that cell is done.
And now you know, it's like a bad block on a, or a bad sector on a disc, and
you gotta move on to the next sector.
and, and one other thing to mention too is I know so far when we talk about
writes, people are probably thinking, oh, you're just writing a block of data.
But remember with that block, you have a bunch of metadata
associated with it, right?
Indirect blocks that point to other blocks that contain metadata and everything else.
That all needs to be updated by a file system.
So even just doing like a hundred byte right,
Mm-hmm.
require, say.
20 other blocks to be updated as
Right, right.
Exactly.
Yeah.
So those, so those are the two sort of what I'd call traditional snapshots.
Right.
Um, and now, and the second one is now what I would call
sort of the standard snapshot.
yeah.
is a third.
Really weird way that his snapshots are done, which I'm
just gonna cover really quick.
And that is VMware.
So they do a, um, I, I believe it's a copy on write, but what's real
actually, actually they do a, I'm gonna call it a weird on write?
So what they do when you write a new block.
They don't write the block to the, and and by the way, this is my understanding
of the way VMware's, um, snapshots work.
If I am now wrong, please tell me, but this is, I've checked repeatedly and this.
I believe is still the case.
So what would happen is when you take a snapshot in VMware, usually for backup
purposes, um, when you go to write a new block in VMware, it doesn't write
the new block in the production area.
It writes the new block over in the snapshot area, leaves the
original version of the block on the disc, and then it stays that way.
Until you delete the snapshot, in which case it updates all the blocks to the new,
Yep.
thing.
And, and, which is why, and, and I, I administered VMware for, for
a while at least not really in a production environment, but, but
you know, in a lab for a while before I realized this was the case.
And so it means, what that really means in practice is you really can't use.
At all.
You can't use VMware snapshots the way you can use NetApp snapshots
or even copy on write snapshots because the performance is way worse.
Right.
Uh, at some point you go to delete snapshots and you think deleting
snapshots would be a nice thing, except when you delete the snapshot, there's
this flurry of IO activity to update the primary volume with any of those blocks
that are now the, the current block.
It's the messiest thing I've ever seen.
So two things to add.
One is for VMware, it's okay if you keep those snapshots around.
For a short duration.
Yeah.
for a short
Yeah.
it's fine, but don't think about using it for like that 90 day user restore use
Mm-hmm.
that you were mentioning earlier,
it for a day or so.
Right.
And then delete it.
Right.
Make that a normal part of your thing.
Exactly.
And then the second point is with all of these snapshots, VSS VMware, most
of them support two modes of operation.
One is sort of a software based snapshot approach where they're
dealing with all of this.
So like Curtis mentioned with VMware, right?
They're keeping track of everything.
The second thing, second way that they also support it, is they
integrate with the storage vendors.
So when you go say VMware, go take a snapshot.
VMware will do some things and then it'll call into the underlying storage
provider and say, okay, now go do a hardware snapshot and lock in that data.
In which case then you don't have all of these issues that Curtis just
Exactly.
That's a really good point.
Thanks for, uh, bringing that up.
Right.
So VSS and VMware and maybe other sort of software level snapshots, they
do have this concept of talking to a storage array and then putting that
storage array, making, making, letting the storage array do the hard work.
Right.
Which is, which is.
I, I think certainly the preferred way to do snapshots is to do it in, in hardware.
Yeah.
. So the final way of snapshots, and this is the snapshots that most of you
know, because most of you are very, you are your head's in the clouds, right?
And that is, and, and I'm, I blame Jeff Bezos, um, because it
was Amazon, I believe that first started using the term snapshot.
To actually refer to something completely different.
If you go into a, you know, if you go to a number, it used to just be
like EBS, the Elastic Block storage.
Right.
Um, that, or you take, uh, a snapshot of a EC2 E image.
Yeah.
RDS Right.
You know.
If you take a snapshot, as I made quotes in the air, it doesn't do
anything like what we just said.
It creates what I would call an image copy, right?
Of that thing that you just took a snapshot of and it creates a
basically bite for bite copy of that.
And typically, I think it's, I'll just speak in terms of a, you know, AWS,
it creates that bite for bite copy in.
S3, right?
It stores it in a special reserve area in S3, and so when you create a snapshot
in pretty much anything in Amazon, what you're actually creating is a backup.
You're creating an image copy of that device.
You're putting it in S3.
Um, you can even, um, you can often specify where an S3, right, so you
can specify that you want that backup to go to another region, for example.
Um, and you can also copy these snapshots around, um, and, um, yeah,
But you can't access it directly via S3,
correct, correct.
Like, yeah, that's why I was saying it's like a special, like reserve area.
NS three.
Yeah.
it's
That's for, that's for a lot of safe safety reasons, right?
Yeah.
It's called a snapshot, but it honors a lot of our backup
with a big B requirements.
Yeah, because it's an actual copy, you can specify that it be sent to a, to
another region, to another account.
Right?
And that's what I wanted.
I want it in another account.
I want it in another region.
I don't want you to just do an EBS snapshot, for example, to the
same availability zone, uh, that.
Isn't, doesn't really follow the basic 3, 2, 1 rule, which is
something we talk about a lot.
Yeah.
But I do, I do think it's important to understand that a snapshot in,
in A AWS is so much, most, so much closer to what I would call a backup.
In fact, depending on how you do it, it definitely is a backup
than what we typically talk about when we say snapshots.
Right.
And, and it's interesting because there are vendors that have made
a, a deal of, you know, they're like snapshots are not backup.
And they've been saying that for so long that they continue to say that.
When what we're really talking about is AWS when it really is
a snap, it really is a backup.
it's interesting you bring this point up because think other vendors, other
traditional storage vendors are also now starting to go the same route
where they're saying our snapshots can now be offloaded into object storage.
Right.
Yes.
I, yeah, I have seen some of the, some vendors do that.
Yeah.
Right, which is very similar to what AWS has been doing, but I think
they still call them snapshots.
Right?
So
Yeah.
of this, you have to, I think it's important as administrator, an IT
person to really ask the question.
When someone says snapshot, what do they mean?
Yeah.
Right.
I'm gonna, and we had, we used the phrase snapshot backup in this,
um, in the title of this episode.
And this is a snapshot backup, right?
So it, it's some would see snapshot backup as like a, like a oxymoron,
like, like military intelligence, you know, um, that, you know,
two words that can't go together.
But if by snapshot you mean what they do in AWS and I. Similar, uh,
you know, other cloud vendors, then I'm fine with calling that a backup.
Just make sure, so the question is, how do you make a snapshot backup?
Well, there's two ways.
One is if you're talking the traditional type of, you know, storage snapshots,
you need to copy that snapshot to another, uh, storage array.
Hopefully one that's offsite.
Um, hopefully one, you know, my dream would be.
Is to another vendor or you, then you, you, you change the
form of the snapshot, right?
You know, you're doing, you're doing, you know, filer to filer and then you're,
you're backing that up to some other, uh.
Object store
system, right?
Yeah.
might do NDMP, who, you know somewhere, uh, Steven, uh,
his, his ears are perking up.
Um, the, um, uh, anyway, but if it's, but if it's, if it's a cloud vendor,
typically, and again, I can really only speak with, with authority and
AWS, but from what I've seen, the other cloud vendors are very similar.
When you take a snapshot in AWS, you're actually taking a backup.
And when you're doing that.
You can use AWS backup to sort of control all of this, which is AWS backup is
really just a control plane for all the other stuff that's going on, and
you can make sure that you know that you're following the 3, 2, 1 rules.
You have a different account, different availability zone, different region even.
Um, yes, it makes it cost more, but it actually makes it a real backup.
So, questions for you.
Yeah.
So you mentioned that sort of doing snapshots, replicating it from
one filer to another filer doesn't quite meet the rules of the two.
Right?
In 3, 2, 1, backup.
Uh, in the 3, 2, 1 rule.
What if so many of these vendors now allow you to run virtualized
instances in public clouds?
Mm-hmm.
So I could run a NetApp instance in AWS
Yeah.
My answer is no.
I.
because it's still the same software version.
Yeah.
Okay.
Yeah.
Now AWS
it's better.
Better, you know, good, better, best.
It's better than the other thing because at least it
isolates you from the hardware.
But it's still, the os still the same os.
And would you also consider the same like AWS offers file systems, right?
So they have like support for NetApp by FSX.
Would you consider replicate if it's, I'm not even sure
Yeah, I don't, I don't know.
I don't know if I can speak to that.
Right.
I don't know if I can speak to that.
Right.
And then, and then
do you have to be for the two is my question.
I, I, again, you're, you're just trying to, it, it's sort of a
good, better, best thing, right?
The, the, the more different you can be, the better you are.
It's difficult in this cloud world, right.
Um, the, um, and some would say, well, isn't a w aren't AWS snapshots the same?
And it's like.
They're actually a little bit different because it is a full image
copy, um, and it's stored in S3.
If something were to happen to your EBS volume because of a bug in EBS,
you could still use that snapshot, which is stored in a completely
different system, which wouldn't have the same bug and store that in.
You could restore that into EBS.
that your volume wasn't corrupted by some EBS bug
Well, but you're, but S3 isn't EBS.
no.
But when
Oh, you mean, you mean before?
Well, if it was created, if it was created, if it was corrupted
by the EBSB, then you know Yeah.
You're, that's gonna happen no matter what you do.
So, so the I, I like where you're going, Curtis.
I think there's also sort of the practicality aspect that we have to
also look out for, because ideally, like you said, right vendor, you
have multiple vendors, right?
You're using different technologies, but then you have to balance that with
pr, sort of the practicality, right?
Are you gonna be able to understand all these various technologies,
build out the skillsets, integrate it, pay for it, right?
We all know how small backup budgets are versus production budgets.
Yeah.
Yeah, yeah,
just.
And, and I've always tried, you know, throughout my career to try to be.
To say, look, if you're asking me my opinion, you know I'm
giving, I'm giving you the bar.
Right, right.
And you know, and I understand that people have to, they have to, they have
to live in the real world where, where backup is not the most important thing.
Ah, it hurts me to say that.
Um, right.
Um,
but at the same time, maybe there are certain use cases where
you can focus on it because of the high value of the data or
Yeah.
like that, where it is important to have
Yeah,
type solution.
also want to add, you know, we talk a lot about the 3, 2, 1 rule, and I
say that because the 3, 2, 1, like if it doesn't follow the 3, 2, 1 rule
and it's just not a backup, right?
I'll say the number one.
Number two, I think I'd be remiss to say that also I. You know,
we're starting to talk about things like the 3, 3, 2, 1, 1 0.
You know, you do want at least one of your backups, your copies
to be on immutable storage.
Right.
To be on truly immutable storage.
One that even you can't delete even if you want to.
Right.
Um, because only then is that backup, uh, protected against a ransomware attack.
Yeah.
Or a, or a, or a direct bad actor.
You know, a, um, what's the, what do, what do we call it?
Um.
threat.
Thank you.
You know?
No, there's a. Uh, uh, a rogue admin, right?
Uh, you know, a rogue admin, either a rogue admin, right?
'cause there have been rogue admin stories, you know, uh, but also someone
who is able to gain access to your administrative account and then go
and delete all your backups you need.
Only if even you can't delete it, even if you want to, is a backup, truly immutable.
Um, and so you can do that with snapshots.
You just need to copy them into something else.
Yeah.
All right.
This was fun.
Yay.
Welcome back, Curtis.
Yeah.
And our, our lag wasn't too bad on our thing.
Hopefully our QOS did its job and hopefully our recording, uh, works.
And with that, that is a wrap.