This week, we're talking about one of the most fundamental technologies
Speaker:underpinning today's data protection systems and that's replication.
Speaker:What is it?
Speaker:How is it different than other methods?
Speaker:And if it's so great, why don't we just use it for everything?
Speaker:And of course what's the difference between synchronous and asynchronous.
Speaker:And why does that matter?
Speaker:Hi, I'm W.
Speaker:Curtis Preston AKA Mr.
Speaker:Backup.
Speaker:And I've dedicated my 30 year career to helping people like you keep your data
Speaker:safe from all that would do it harm.
Speaker:This is the backup wrap-up.
Speaker:Welcome to the show with me, as always, as a guy who seems to never
Speaker:be happy until I have problems.
Speaker:Prasanna Malayandi.
Speaker:How's
Speaker:I'm good.
Speaker:I'm just trying to think what problems do you have right now?
Speaker:Talking about, I'm trying to edit the podcast and you're giving
Speaker:me problems about my network.
Speaker:That's,
Speaker:what I'm talking about.
Speaker:yeah, it's.
Speaker:What I do, you need someone to ask the hard questions,
Speaker:Somebody's got to do it.
Speaker:The network's going all right.
Speaker:Those that have followed, I do have this new tool.
Speaker:I have, um, the, uh, Firewalla.
Speaker:version 125 of Curtis's home network, architecture.
Speaker:Yeah.
Speaker:but I now have this Firewalla that gives me a lot more
Speaker:information than I had before.
Speaker:And unfortunately it verified, my new ISPs, thing that I use too
Speaker:much internet and I have to buy.
Speaker:the unlimited package, which is sad because it's another 50 bucks a month.
Speaker:But anyway, so let's get to the news of the week
Speaker:I'm going to let you take the first story, uh,
Speaker:Yeah, yeah, yeah, so the first one is an article that came out recently.
Speaker:it is about a backup compa I guess, would you call them backup or data
Speaker:protection company that recently, Raised around a funding, it's called Alcion.
Speaker:I actually used to work with the founder previously at an old job.
Speaker:and they actually raised a round of funding and it was actually led by
Speaker:Veeam, which is interesting because Veeam, as we all know, is also a backup
Speaker:company and they are investing in Alcion, who is primarily targeting backups
Speaker:for Microsoft 365 for SMB customers.
Speaker:today?
Speaker:That's their initial target market.
Speaker:I'd say that the biggest difference between the two companies,
Speaker:one is a software product, the other is a SaaS offering, and
Speaker:that makes them very different.
Speaker:And I guess Veeam felt that it was different enough.
Speaker:I like the quote in the article.
Speaker:It said something along the lines of that, that from Veeam, that
Speaker:this problem of ransomware is a big enough problem that we're going to
Speaker:need the whole industry to solve it.
Speaker:And so they invested in what.
Speaker:is essentially, I would call it a partial competitor, right?
Speaker:They do compete for the Microsoft 365 space, I also know the founder as
Speaker:well, and, I think they've taken an interesting angle with a very AI based
Speaker:approach, a very security based approach,
Speaker:Yeah.
Speaker:And one of the things that I'm actually looking forward to is seeing if this
Speaker:is going to force other vendors to reconsider how they approach the problem.
Speaker:And.
Speaker:Everyone in all the vendors raise a bar when it comes to ransomware
Speaker:protection and detection and other things like that, which we all
Speaker:need.
Speaker:Right?
Speaker:Anybody that's listened to this show knows that we're concerned about that problem.
Speaker:my second news story, it's from the enterprise strategy group, which for
Speaker:those of you that have known ESG for a while, they were, A few years ago,
Speaker:acquired by TechTarget, and there was a, 2023 survey of IT professionals.
Speaker:The most interesting number that I see here is that they said 65 percent
Speaker:of the almost 400 respondents said their organization were using some
Speaker:sort of cloud based, backup system.
Speaker:And the way they worded it, that could be either a cloud based
Speaker:service or they're storing a copy of their backups in the cloud.
Speaker:That, to me, really, that's a very different number, wouldn't you say, than
Speaker:a few years
Speaker:Yeah, everyone would be like, I don't trust my data in the cloud.
Speaker:How do I get my data up into the cloud and bring it back and all the rest of that?
Speaker:In fact, I think one of the interesting things was even just
Speaker:the title of the article, right?
Speaker:It's like cloud backup and disaster recovery evolved toward maturity.
Speaker:You know, that just kind of says it all, right, versus before you're
Speaker:fighting tooth and nail with customers, trying to persuade them, Hey, it's
Speaker:okay to store your data in the cloud.
Speaker:But
Speaker:Yeah, so the not so good number was saying that half of the surveyed orgs
Speaker:recovered 75 percent or less of their cloud data per incident on average.
Speaker:That sounds really bad.
Speaker:that is not, I agree with, Christoph Bertrand, which said that's just
Speaker:not acceptable for, such a mission
Speaker:I guess the question I would have is, it would be, we probably need
Speaker:to compare that to what's your recovery rate of on premises backups,
Speaker:because that's not going to be 100%.
Speaker:and then I think it's also not fair to compare it because how
Speaker:long has traditional backups, been around and had time to evolve
Speaker:versus cloud backups, right?
Speaker:It hasn't had that time to evolve and mature.
Speaker:Fully as these other products
Speaker:Yeah, I would say that part, having formerly worked at a cloud data protection
Speaker:vendor, I would think that the problem that I see with this number is that they
Speaker:do conflate people storing backups in the cloud versus people using a cloud service.
Speaker:I would be very surprised to learn that.
Speaker:People that are using a fully cloud based service, if they, if their recovery's
Speaker:failed, because the other, it's really, the cloud just happens to be a part of it.
Speaker:All of the same sort of misconfiguration problems that lead to problems
Speaker:with traditional backup are going to bleed over into a cloud.
Speaker:Just because you're using cloud
Speaker:but it's the cloud.
Speaker:It's magical.
Speaker:Come on Curtis.
Speaker:Yeah
Speaker:Yeah, it is magic.
Speaker:the one other interesting number from this survey, and there were
Speaker:some other interesting numbers, but just for brevity purposes, this other
Speaker:interesting number, they said 16 percent of the, respondents said that they.
Speaker:reviewed their, their cloud data protection strategies annually.
Speaker:that is, that's a very different number than back in the day.
Speaker:Back in the day, that number was like five years, maybe three years.
Speaker:Certainly not annually.
Speaker:Backup historically has been an incredibly sticky process.
Speaker:And, this idea that they would review it annually, I think it's a
Speaker:I think it also makes sense though, because you like with the five
Speaker:year, three year, five year, right?
Speaker:You have to worry about hardware depreciation, right?
Speaker:Upgrade cycles all the rest With cloud, right?
Speaker:There might be new technologies, cheaper ways to do things, right?
Speaker:Your workloads might change.
Speaker:what you had running on premises might be migrated to the cloud.
Speaker:now you have to reconsider it.
Speaker:How do I back it up?
Speaker:And so I think that's probably why looking at these things more
Speaker:frequently, like annually, probably starting to make a lot more sense.
Speaker:Well, that is the news of the day.
Speaker:on this episode of the Backup to basic series, we're going to be talking
Speaker:about replication, different kinds of replication, the good things, the bad
Speaker:things, and what they, you know, what they have to do with backup and, and
Speaker:I, I,
Speaker:it a backup,
Speaker:I, I, I I thought you were going to talk about like the replication
Speaker:technologies that they have in like Star Trek and, Right?
Speaker:Where it's like, create me,
Speaker:No, that's, that's not replication.
Speaker:That's...
Speaker:What is that called?
Speaker:what are those things?
Speaker:Transporters?
Speaker:Oh, oh, no.
Speaker:the making, the stuff, right?
Speaker:The food
Speaker:Do they call them replicators?
Speaker:Yeah.
Speaker:I think they call them replicators.
Speaker:We are, we are not discussing replicators like in Star Trek, although that does,
Speaker:those do sound magical, by the way.
Speaker:And by the way, like if you could do transporters, of course you
Speaker:could do replicators, right?
Speaker:Because all replicators are is storing a backup copy of the food and then
Speaker:saying this is what it is, right?
Speaker:Do you think that they have backups of those backups?
Speaker:The copies?
Speaker:Yeah, no, the copies of the food because imagine you wipe out the
Speaker:copies and then you've basically killed off humanity because they
Speaker:wouldn't have any way to make food.
Speaker:I
Speaker:Or maybe they're immutable, or maybe they're immutable copies, you know?
Speaker:Maybe they're immutable copies.
Speaker:I would, I would hope that no one can go in and change the
Speaker:food before it gets replicated.
Speaker:Um,
Speaker:On to a more serious matter.
Speaker:let's do a more serious matter.
Speaker:Well, so we, we've moved on the previous episodes.
Speaker:We talked about sort of regular kind of backups and they were, they were backups.
Speaker:That require a restore basically is, is what we were talking about.
Speaker:We've now moved on to types of backups where the restore
Speaker:is essentially already done.
Speaker:Right.
Speaker:Um, and replication.
Speaker:And so I'm, I'm, I'm putting these and, and by the way, you know, for those that
Speaker:don't know what I'm talking about, we are basically working our way through.
Speaker:This book, Modern Data Protection, which is my book, which is available,
Speaker:you know, wherever books are sold.
Speaker:And, um, you know, I might even put a, put a link in the show description there.
Speaker:Um, and the idea here is what we're talking about is methods of backup.
Speaker:And again, remember when I say backup, I I'm very broad in that term, methods
Speaker:of backup that support instant recovery.
Speaker:What do you want to talk about?
Speaker:What instant recovery is?
Speaker:Yeah, so, normally when you think about, you took a backup, you
Speaker:need to, something happens, you need to get back your data, right?
Speaker:If you had your data stored on tape, you gotta go fetch the tape, you gotta pull
Speaker:the data off of tape, copy it somewhere else, and now you actually have your data
Speaker:and you can do something with it, right?
Speaker:With Instant Recovery, It's really, your data is already available for you.
Speaker:You don't have to have any sort of processing that you have to do before you
Speaker:can actually start accessing your data.
Speaker:Yeah, I mean, it's a beautiful thing.
Speaker:I think that maybe, I dream that sometime in the future, this is
Speaker:the way all restores will be done.
Speaker:And the cloud really does make this possible.
Speaker:We could talk about that all day long, but we just need to work through
Speaker:the different ways that, um, we're going to create an always on readily
Speaker:available copy of the data, and the first that we're going to talk about is
Speaker:Before you get there though, I think one important point to cover is, I
Speaker:know you said in the future maybe everything will be like this, right?
Speaker:I think if you had a gazillion dollars, yes, Because there
Speaker:is a cost associated, right?
Speaker:Which I think is important to cover, right?
Speaker:With sort of using these various technologies that might make it more
Speaker:difficult now Like you said, maybe the cloud helps with some of these but there
Speaker:is still a cost associated with Keeping data on something that is instantly
Speaker:available Versus say using a tape or some sort of offline Media where the
Speaker:cost may not be just the cost of the storage but think about from power cooling
Speaker:Right all those other aspects as well
Speaker:Yeah, I was speaking in the future when these problems have all been solved.
Speaker:Um, I think even today that there are services, there are cloud storage
Speaker:services, that if used properly and judiciously, can provide you A ready
Speaker:to go copy of your data pennies on the dollar that costs you, you know,
Speaker:next to nothing to, to have and to possess and to have it ready to go.
Speaker:You will then pay through the nose if you use them, right?
Speaker:I'm thinking of course of like Glacier, Deep Archive, Instant Recovery.
Speaker:Right?
Speaker:There are services that are still storing the data on disk, but storing
Speaker:it in such a way, I don't know, we don't, you know, neither of us have
Speaker:ever worked at AWS or any of the cloud providers, but they're storing it in
Speaker:such a way that they've made it so cheap.
Speaker:To, to, to have that copy, to have it offsite, to have it in,
Speaker:you know, in another region.
Speaker:I just think that as that continues to happen, that we will eventually
Speaker:get to a place where all DR copies can be held like this.
Speaker:Can I have a dream, Prasanna?
Speaker:Can I have a dream?
Speaker:Can
Speaker:You can have a dream, but I think, but yes, but before, sorry, before
Speaker:we also continue on, I think we also need to define what we
Speaker:mean when we say instant, right?
Speaker:Because instant,
Speaker:when we say instant recovery, right?
Speaker:I think it's helpful to talk about that because instant is a spectrum, right?
Speaker:Like you were talking about for your Glacier instant restore feature, right?
Speaker:It's not like, Oh, I want my data.
Speaker:It's available within the Cloud.
Speaker:Like, snap my fingers, it's there.
Speaker:Right?
Speaker:It does take time, but that time may be acceptable to you, right?
Speaker:It might be, say, minutes, not days.
Speaker:yeah.
Speaker:So instant is kind of like continuous.
Speaker:It's a binary condition.
Speaker:It's sort of like immutable.
Speaker:It's a binary condition.
Speaker:It's like pregnancy, right?
Speaker:You're either pregnant or you're not.
Speaker:And so it's either instance or it's not.
Speaker:But it's yet another one of these terms where it's not truly binary.
Speaker:Right?
Speaker:It's just, we're defining instant recovery in this point is, I don't need
Speaker:to do a restore to do the recovery.
Speaker:The data's already been restored.
Speaker:It might take a few minutes to make it accessible to me.
Speaker:Yeah, sorry, and, and just for people who may not have heard our previous
Speaker:episodes, do you want to quickly cover differences between restore and
Speaker:recovery since you just talked about it?
Speaker:A restore is essentially the act of bringing back the, you know,
Speaker:the data and putting it in place.
Speaker:The recovery is the process of sort of bringing that data online and using it.
Speaker:And, and sometimes that recovery involves massaging other things.
Speaker:It involves massaging the data and it involves doing other things.
Speaker:When you're done with the recovery.
Speaker:Everything is working again.
Speaker:When you're done with the restore, everything is ready to be working again.
Speaker:Uh, so depending on how good or how complicated the restore is,
Speaker:the two may be, they, they may happen at the same time, but in a
Speaker:complicated situation, you do the restore and then you do the recovery.
Speaker:this case, uh, the recovery would be this instant of, um,
Speaker:you know, like I said, just,
Speaker:Everything's ready to go.
Speaker:Or sorry, everything.
Speaker:has, is available.
Speaker:Yeah, exactly.
Speaker:Uh, we don't have to do a restore.
Speaker:And I would say that in a true instant recovery, the size of the, of the thing
Speaker:being recovered should not be a factor.
Speaker:Whereas in a restore, it very much is,
Speaker:Oh yeah.
Speaker:right?
Speaker:Yeah.
Speaker:Yeah.
Speaker:Okay.
Speaker:So with that in mind, we're going to talk about the first method that could be used.
Speaker:And, and is used actually, it is used quite a bit to do an instant recovery.
Speaker:Uh, as we often do on the show, let me take you back in the day.
Speaker:Yeah.
Speaker:So back in the day, there was a very sharp dividing line between.
Speaker:Backup and recovery and disaster recovery.
Speaker:And for backup and recovery for what we also call operational
Speaker:recovery, you had a backup system.
Speaker:You went to tape and, and when you did a restore, you, you came back
Speaker:up, you came back off of tape.
Speaker:Anyone with a tight recovery time objective would never build,
Speaker:uh, their DR system based on tape back in the day, right?
Speaker:Even as we move to disk based backup and recovery systems, anybody with a
Speaker:tight RTO is not going to build their disaster recovery system built on
Speaker:having to do a restore first, right?
Speaker:So back in the day, if you worked at like a financial trading firm or
Speaker:something like that, and you wanted to have a disaster recovery plan because of
Speaker:course you wanted that because you had the OCC that was looking into you and
Speaker:Regulation,
Speaker:could say we're not a bank, right?
Speaker:You used replication.
Speaker:You had a backup and recovery system that did operational recoveries
Speaker:and file recoveries, and maybe even recovering an entire server.
Speaker:But if you had data that truly mattered, you used replication.
Speaker:And, and I know you worked at a company
Speaker:Yeah, that, was my first job out of, out of
Speaker:college.
Speaker:Yeah, So, let's talk about what replication is, uh, how does
Speaker:it typically manifest itself.
Speaker:Do you want to just...
Speaker:Sort of define what replication is?
Speaker:so replication is basically taking data, making a copy, and sending
Speaker:it off to some other system, right?
Speaker:At a high level.
Speaker:Now, there's various nuances and combinations that other system
Speaker:could be the exact same vendor, the same model, all the rest.
Speaker:As the original, it could be a different vendor, right?
Speaker:A different system.
Speaker:Right?
Speaker:So there are different ways you could do it.
Speaker:There are different protocols for transferring that data.
Speaker:You could use protocols that some of these vendors have built in
Speaker:for communicating between them to make things more efficient.
Speaker:Because when you have to send data across from one system to another,
Speaker:doing it over and over, because it's not like it's a one time operation, right?
Speaker:Depending,
Speaker:Yeah.
Speaker:If I could interrupt, I think that's one word missing from your definition.
Speaker:If I could insert at the beginning of your definition, the word continuously.
Speaker:Right.
Speaker:Because anybody can copy the data over.
Speaker:I think replication implies that we're continuously copying it
Speaker:over.
Speaker:The idea is that it, that it's happening all the time, right?
Speaker:We're going to talk about the difference between asynchronous and synchronous.
Speaker:So sorry for the confusion for those, but basically you have some data, you
Speaker:know, in A and you, and you have a copy of that data in B and that copy,
Speaker:that, that B copy is continuously, for lack of a better word, updated to
Speaker:be the same exact thing as what's in
Speaker:Yeah, so going to Curtis's disaster recovery scenario, right?
Speaker:Something happens on system A, right?
Speaker:System B has a version of what system A looked like at a point in time,
Speaker:and you can fail over to system B, and that's how you're instantly
Speaker:up and running, because system B
Speaker:I think one thing we should also cover, which I want to talk about
Speaker:earlier, I know you threw out the term recovery time objective.
Speaker:I think another point to talk about here is also recovery point objective, right?
Speaker:Yeah.
Speaker:Well, go ahead.
Speaker:You brought it up.
Speaker:So yes, in the case of when you're doing synchronous replication, yes, there's
Speaker:still going to be an RTO on that.
Speaker:But that RTO, you should be able to measure in seconds to minutes, depending
Speaker:on how long it takes you to bring up system B and make it available for
Speaker:your applications and other things.
Speaker:So that's kind of
Speaker:like RTO side.
Speaker:Do you wanna talk about the RPO, Curtis?
Speaker:Yeah.
Speaker:So the RPO is, is basically the amount of data that you
Speaker:agree that you can lose, right?
Speaker:The recovery point objective is we say we can lose an hour's worth of data.
Speaker:We can lose a day's worth of data.
Speaker:We can only lose five minutes worth of data.
Speaker:This is.
Speaker:You're setting your recovery point objective.
Speaker:What, to what point must you recover?
Speaker:And the fastest RPOs in the business are going to come from these types
Speaker:of products that we're going to talk about in the next couple of episodes.
Speaker:The first of which being.
Speaker:Uh, replication with a, with a, you know, a replication
Speaker:system, you could get an RPO.
Speaker:Uh, you could meet an RPO that's as close to zero as possible,
Speaker:actually seen zero,
Speaker:Yeah, I'm sure you can do zero.
Speaker:It's just that
Speaker:so
Speaker:I'm gonna
Speaker:could happen,
Speaker:So it's in a very, very unique case.
Speaker:So I don't know if you're aware of this, there is a mode on
Speaker:some systems called domino mode.
Speaker:It's out there in the industry, right?
Speaker:And basically what you're saying is, every time I accept a write
Speaker:on system, A, I'm gonna make sure it's replicated, it hit system B.
Speaker:It's acknowledged before system A acknowledges a client.
Speaker:Yeah,
Speaker:you're guaranteeing every
Speaker:basically, it's like a two phase commit for storage.
Speaker:A two phase commit is a term from database, um, world, which makes
Speaker:sure that it's written both to the log and it's written to the, you
Speaker:know, before you acknowledge that the, the transaction has happened.
Speaker:It's High
Speaker:latency, very expensive, but if you have those requirements, there is an option.
Speaker:But like you said, most
Speaker:people will...
Speaker:So the question is what matters more to you, performance or never losing any data?
Speaker:So let's, speaking of which.
Speaker:Let's talk about the difference between synchronous replication
Speaker:and asynchronous replication.
Speaker:Um, synchronous is basically what you just described that, um, well,
Speaker:it's not necessarily what you
Speaker:Yeah, not fully to domino
Speaker:we are going to, we are going to replicate every byte.
Speaker:As it's happening, right.
Speaker:And literally every single, you know, the moment it, a change happens here, the rep,
Speaker:the change is going to be replicated to the, to the destination, to the target.
Speaker:Right.
Speaker:And, um, the, the good side of synchronous replication is you're going to have.
Speaker:The smallest RPO is possible, right?
Speaker:Including, I guess I, I, I, I guess I was aware of it.
Speaker:I didn't know that that, that I didn't know the term dominant mode, but,
Speaker:but I was aware of that, that idea, the, this is going to give you your,
Speaker:your tightest, uh, RPO as possible.
Speaker:The challenge with, uh, this is a little thing called the speed of light.
Speaker:The, um, it takes, so, you know, one of the things that we.
Speaker:That we know is that we want the recovery copy to be some
Speaker:distance away from the primary
Speaker:in the same rack.
Speaker:not in the same rack, not in the same town.
Speaker:Um, there are sadly some stories from 9 11 where there were companies
Speaker:that had their hot site, which was a synchronously replicated copy of
Speaker:their data in the, uh, other tower.
Speaker:And so there were companies that ceased to exist on 9 11 because they lost both their
Speaker:primary and their secondary copy, right?
Speaker:You don't want to have the two copies in the same place where a disaster
Speaker:could wipe out the entire town, a flood, a earthquake, whatever.
Speaker:And so you want to have some distance between.
Speaker:uh, the two.
Speaker:Now after 9 11, the, I don't know which, um, which governmental organization,
Speaker:but after 9 11, the, the fed, the federal government, not the fed, which
Speaker:we say the fed or you know what I mean?
Speaker:The federal government started to dictate that A recovery that, that a, that a
Speaker:synchronous copy of the data needed to be stored more than it was like 250 miles.
Speaker:And, and everybody basically came back and said, that's not really feasible.
Speaker:It's possible, but not feasible because going 250 miles.
Speaker:At the speed of light, every single time you replicate, every single time you
Speaker:update a block on every piece of storage, um, basically really pulls down the, um,
Speaker:the performance of the primary system.
Speaker:And the other thing to also mention is when you're doing synchronous replication,
Speaker:you want to make sure that you're...
Speaker:Destination system, right?
Speaker:Your replica copy is as performant as your primary copy, right?
Speaker:Because otherwise, if it starts to lag and slow down, eventually you're going to
Speaker:start falling further and further behind.
Speaker:And usually these technologies, after a while, they'll just stop accepting writes.
Speaker:And they'll kind of fall back into like an asynchronous mode and
Speaker:have to then catch back up, right?
Speaker:Which you don't want because that affects your RPO.
Speaker:Yeah, you don't want.
Speaker:Yeah.
Speaker:So yeah, so it's got to be basically the same, if not better
Speaker:performance on the backup system, which is crazy talk to most people.
Speaker:Right?
Speaker:Um, but that, but that is if you want an RPO of zero.
Speaker:Synchronous replication is literally your only choice, right?
Speaker:Let's talk about asynchronous replication and that is basically
Speaker:not that we're, we're not, we're going to build essentially a buffer.
Speaker:Uh, of writes as the, know, and we try to not let the buffer get full, right?
Speaker:The, basically the, what it does is it disconnects the, the writes on the
Speaker:other side with the writes on this side, we may, and we'll try to keep up
Speaker:with the pace, but writes can continue to update and you might get a spur of
Speaker:writes, uh, on the, on the primary side.
Speaker:And then they fill up the buffer and then, you know, the, the buffer gets emptied.
Speaker:And that I think works as long as essentially your, you know, your bandwidth
Speaker:and your latency is, is sufficient.
Speaker:Um,
Speaker:And,
Speaker:go ahead.
Speaker:and I just want to chime in that I know you called it async and async
Speaker:threw me for a while because in my mind async is different, but that's
Speaker:just based on my background, right?
Speaker:Um, other vendors might refer to this as like semi synchronous, right?
Speaker:Where it is not fully synchronous.
Speaker:Right, but
Speaker:Well, not synchronous.
Speaker:Asynchronous means
Speaker:know, I know, I know, but this is just some vendors.
Speaker:So if you are looking at vendor terminology, right, and looking at
Speaker:their vendor technical specs, what, so here's a question for you, Curtis.
Speaker:What are some things that a user can look at for a vendor's technical
Speaker:specs to determine if they support async replication, like what
Speaker:we're talking about right now?
Speaker:Well, I mean, I don't know what other words they would use to describe
Speaker:it,
Speaker:I know NetApp uses semi synchronous.
Speaker:For what you're describing.
Speaker:whatever.
Speaker:That's just mark.
Speaker:That's just marketing BS.
Speaker:Okay.
Speaker:It's either synchronous or it's not synchronous.
Speaker:You're either replicating everything as it's happening or you're not.
Speaker:Okay.
Speaker:Like, so the question is, so the question to the vendor, it's not, is, can my target
Speaker:copy ever get behind my primary copy?
Speaker:And the answer is yes.
Speaker:And it's like, well, well, how much are we talking?
Speaker:Right?
Speaker:So the question is, because that's going
Speaker:the, so that's
Speaker:actually the key, yeah.
Speaker:Yeah, exactly.
Speaker:So if you can build up a buffer of an hour of changes, you could lose an
Speaker:hour of changes on the, on the target end if you lost your primary, Um,
Speaker:and.
Speaker:what if you said like, I want, so, okay, so then I'll take back my comment.
Speaker:In your mind though, if I said I want to have a replica copy
Speaker:updated and my RPO is 8 hours.
Speaker:That's still async.
Speaker:Well, what else would you
Speaker:Okay.
Speaker:No, just, just checking.
Speaker:Yeah.
Speaker:I mean, again, you're either synchronous or you're not
Speaker:So then, so then I take back my comments, right?
Speaker:So if I look specifically, that's why I just wanted to make sure.
Speaker:You stand corrected,
Speaker:Yes.
Speaker:So if I look at, sorry, I used to, I'm more familiar with NetApp and NetApp's
Speaker:replication technologies, right?
Speaker:But if I look at NetApp, right, they have synchronous, which we talked
Speaker:about, which matches perfectly with what we were talking about.
Speaker:They have semi sync where it allows some amount of lag.
Speaker:I think it's up to 60 seconds.
Speaker:Right.
Speaker:But it does like what you said, buffering, and then they have
Speaker:sort of their async, right?
Speaker:Where it allows you to go beyond, I think it's from five minutes to, you can set a
Speaker:replication every five minutes to days,
Speaker:And those are just two different, those are just both asynchronous with different
Speaker:settings, as far as I'm concerned.
Speaker:Um, and, and you could also, like, if you were a net app and I don't think they do
Speaker:it like this, but if you waited until you took a snapshot and then replicated only
Speaker:the bits necessary for that snapshot, that's still kind of asynchronous,
Speaker:just a technology.
Speaker:That's a means.
Speaker:yeah.
Speaker:But generally here, we're not talking about snapshots and things, because
Speaker:that's what we're going to talk about.
Speaker:In a following episode, generally what we're talking about is two,
Speaker:you know, block devices that are being continuously updated to be
Speaker:exactly the same with possibly a lag.
Speaker:If there's a lag, then it's asynchronous.
Speaker:If there's no lag, then it's
Speaker:Does it have to be, so I'm going to ask another question.
Speaker:So
Speaker:you are
Speaker:I know I am.
Speaker:So you said block devices.
Speaker:Would you consider anything that also replicates a VM at a VM block level
Speaker:Yeah.
Speaker:Yeah.
Speaker:I mean, in the end, it's still a block device underneath, right?
Speaker:It's not what we're replicating.
Speaker:It's just that, um.
Speaker:We don't, we don't have this conversation when we're talking about object,
Speaker:Or files.
Speaker:We don't, we don't, well, even, even file replication, we don't really talk
Speaker:about synchronous and asynchronous when we do file replication, I don't think.
Speaker:I think that's always asynchronous.
Speaker:At least I don't think so.
Speaker:I think this is generally a block discussion, right?
Speaker:There may be exceptions to that rule, but generally this is a block discussion.
Speaker:Even if this is a NetApp, right?
Speaker:Really what you're replicating underneath the NetApp is file.
Speaker:It's a file device, but underneath you're replicating blocks.
Speaker:Um, all that, all that discussion just from one word.
Speaker:Um, so, you can also have a hybrid of these two things.
Speaker:Can you think of an example of that?
Speaker:Is
Speaker:Yeah.
Speaker:So take your bank example, right?
Speaker:So you're a financial customer in New York.
Speaker:You want to make sure that your data is synchronously
Speaker:protected to someplace nearby.
Speaker:So maybe you go over the river to New Jersey using synchronous
Speaker:replication, and now you need to make sure that you're protected in case
Speaker:something happens in the East coast.
Speaker:And so you might use async replication and send that copy off to say, Utah.
Speaker:that when we go through the woods?
Speaker:Yes.
Speaker:See what I did there?
Speaker:Over the river and through the woods?
Speaker:Um, yeah, so you might, and that was, I remember back in the day,
Speaker:we had a synchronous copy that was literally like right next door.
Speaker:I mean literally, when I say next door, I mean Like it's the next
Speaker:rack over, um, that was, uh, like an A or asynchronous replication.
Speaker:And then actually, will you remember Symmetrics and or,
Speaker:or, or EMC and, and NetApp?
Speaker:They, you'd have like three.
Speaker:You'd have like four storage arrays, right?
Speaker:You'd have the,
Speaker:Oh, you can have
Speaker:a lot more.
Speaker:Yeah,
Speaker:you'd have the synchronous that was right next to it, the asynchronous
Speaker:that was down the street.
Speaker:And then another asynchronous that was the other.
Speaker:Yeah,
Speaker:and sometimes there are customers who could use it for like data
Speaker:distribution use cases, right?
Speaker:So I need to
Speaker:make sure that my database is available locally in different countries.
Speaker:So I'm just going to replicate it all.
Speaker:yeah.
Speaker:So synchronous, both forms of replication have a lot more applications
Speaker:than just backup and recovery.
Speaker:Um, so I'm glad you mentioned, you know, or you VMs, there is another.
Speaker:Not file and not VM thing that gets replicated, which is very, where
Speaker:we still have sort of the same discussion about synchronous versus
Speaker:asynchronous, and that is databases,
Speaker:Application level synchronous
Speaker:right?
Speaker:And, and, and I'd say that in that case, basically.
Speaker:When we, again, that can definitely have two phase commit, right?
Speaker:So in the database.
Speaker:You're replicating a transaction or, or something.
Speaker:I don't have a better word.
Speaker:I know not every database has transactions, but I
Speaker:don't have a better word for
Speaker:It's the atomic unit, whatever it is, or sorry, the unit of
Speaker:Yeah, an atomic unit of a thing, you changed a thing in the database, right?
Speaker:And you're replicating that, rather than the blocks underneath.
Speaker:And, well, you could be replicating the blocks underneath, and that's
Speaker:what we're talking about over here.
Speaker:But at this point, we're talking about database level,
Speaker:application level replication.
Speaker:And when you do that, you have the choice of either immediate
Speaker:consistency, or eventual consistency.
Speaker:Which is essentially analogous to synchronous and asynchronous.
Speaker:And, and what made me think about it was when you were talking about that,
Speaker:there are other reasons that people replicate and, um, there are databases
Speaker:that allow replicating data, you know, across the world and the best.
Speaker:Example that a lot of people are used to is the DNS database, right?
Speaker:That is an eventually consistent, replicated, um, system.
Speaker:If you've never, if you've never made changes to DNS.
Speaker:Let me tell you, it can take a minute.
Speaker:I recently, literally in the last week or two, I've been making changes to
Speaker:various websites that I own to, you know, change their, uh, DNS entries
Speaker:and CNAME entries and things like that, and they don't all immediately happen.
Speaker:Um,
Speaker:And yeah,
Speaker:but
Speaker:they will eventually get there.
Speaker:That is essentially the database equivalent to, um, to, uh,
Speaker:asynchronous replication.
Speaker:Now, let me ask you a question.
Speaker:Prasanna,
Speaker:mm hmm.
Speaker:if replication is so amazing and we can have the tightest RPOs, and, and, and
Speaker:I think we can agree that Tighter RPOs are better than looser RPOs, right?
Speaker:An RPO of zero or a minute is way better than an RPO of 36 hours, which is what
Speaker:we typically get with a backup system.
Speaker:Why don't we do replication for everything?
Speaker:And by the way, money is not the issue.
Speaker:You knew what I was going to say, uh,
Speaker:It's not, well, it is an issue besides money, right?
Speaker:Because replication is
Speaker:yeah, so the
Speaker:don't we do that?
Speaker:so the biggest problem with replication is for most of the replication technologies,
Speaker:when you replicate it, you only have that one copy on your target system, right?
Speaker:And so, If you had a user who dropped a database table and you replicated that
Speaker:command over to your replica, guess what?
Speaker:It gets dropped on your replica as well, and you have no ability to recover.
Speaker:And
Speaker:so that's one of the biggest limitations of relying on replication,
Speaker:or at
Speaker:And same would be true of any change on the, on the primary system, which
Speaker:includes things like ransomware, right?
Speaker:So if there's a, if there's a hacker goes in.
Speaker:and deletes all your tables, or just, you know, corrupts your, your data,
Speaker:deletes all your files, whatever.
Speaker:It just, if a virus does, whatever, whatever, you know, if there's any
Speaker:kind of cyber attack on the primary system, it simply makes that attack more
Speaker:efficient,
Speaker:And because the entire purpose of replication is to create a duplicate
Speaker:copy on your target system, right?
Speaker:Immediately, if you ask for an RPO of zero, you're going to get an RPO of zero,
Speaker:now, now I will say, I will put a caveat that, wait until we talk about snapshots,
Speaker:we will come back to this specific point,
Speaker:Right.
Speaker:There, There, are, there are back backup data protection based systems
Speaker:that use replication at their core.
Speaker:And don't have this problem.
Speaker:This is just, just replication, right?
Speaker:It's like, why don't we use raid for everything?
Speaker:And then we don't need backup.
Speaker:Well, same exact reasons, right?
Speaker:Exactly.
Speaker:Uh, raid does help with media failure and equipment failure.
Speaker:And, um.
Speaker:Site failure and power failure and, and, and, and, but it is completely defenseless
Speaker:against cyber attacks and stupidity.
Speaker:Or logic issues.
Speaker:Well, I'm just, you're being nice.
Speaker:See, this is why this is,
Speaker:Well, I was thinking software bugs, not user logic.
Speaker:Oh, okay.
Speaker:Well, that's still stupidity.
Speaker:It's just, you know, Different
Speaker:Yes.
Speaker:person who made the bug.
Speaker:Um, well, this has been fun.
Speaker:Uh, but yeah, this whole thing of, um, this idea that replication is where
Speaker:we're continuously updating the data.
Speaker:Now, now that you see like this idea of, of, um, you know, why we were wrestling
Speaker:over the word continuous, right?
Speaker:Because.
Speaker:Is asynchronous continuous?
Speaker:I think it's still a continuous process.
Speaker:It's just continuous with an asterisk, continuous with, with, with a buffer.
Speaker:Um, and I think that, well, in fact, I know that today is.
Speaker:Way better than it was back in the day.
Speaker:You back in the day, you had to have the same vendor on both sides.
Speaker:It, it, it sold a lot of EMC, right?
Speaker:You had to have the same vendor because the replication was based on the box.
Speaker:And now there are a number of products.
Speaker:one of which we've had on here, right?
Speaker:We had Datacore on here, which, which does replication between, you know,
Speaker:uh, disparate, different architectures.
Speaker:You can have a very expensive primary system.
Speaker:And a less expensive target system.
Speaker:You can use it as a way to upgrade and move your technology.
Speaker:You can move your older systems to be the target system, knowing that if you
Speaker:need to use it in a disaster, you will suffer a performance loss, but you
Speaker:don't have to, like you did back in the
Speaker:day.
Speaker:You don't, you know, when you primer, you don't have to have the same, you know,
Speaker:and so you get, you could cost savings.
Speaker:You don't have to do it that way, but it gives you the flexibility.
Speaker:And I think that modern day replication systems are much more
Speaker:forgiving of things like You know, outages and, and things like that.
Speaker:Right.
Speaker:Um, and, but, you know, but they still don't protect you from, you know, mistakes
Speaker:and cyber attacks, which is why we have to talk about the alternatives, which we'll
Speaker:do on later episodes, but, uh, so you were, you were unusually, not unusually,
Speaker:you were excessively argumentative in this episode, but I thank you anyway.
Speaker:that, that, I try, I basically spent all of my, almost all of my career
Speaker:working on replication technology, so.
Speaker:Yeah.
Speaker:And I, I, yeah, you worked at a company that was like, it was trying
Speaker:to say, well, there's asynchronous and there's near synchronous.
Speaker:And I'm like, those are two words for asynchronous.
Speaker:As I was editing this episode, I realized that.
Speaker:I was being a little too pedantic about continuous or not continuous.
Speaker:The thing with replication that really sets it apart from other data
Speaker:protection technologies, is , one is that the copy is an actual copy.
Speaker:It is a, it looks exactly like the original, unlike what we do in the
Speaker:backup world, where we tend to put stuff in a container or something like that.
Speaker:Um, that's one big thing.
Speaker:That's separate separates replication from other backup technologies.
Speaker:And then also the idea is that it is, it is incrementally updating that.
Speaker:Whenever we do it, whether it's continuous asynchronous or even periodic.
Speaker:I really forgot about how, like there are vendors like NetApp.
Speaker:That actually do periodic replication.
Speaker:There's nothing wrong with periodic replication.
Speaker:It's still replication.
Speaker:Um, so you've got continuous asynchronous and then periodic
Speaker:that, what, what sets replication apart from everything else is this.
Speaker:Whenever we do the next replication.
Speaker:What we're doing is we're transferring the, the, the things, whether it's
Speaker:database records or blocks, we're transferring the things that have changed
Speaker:since the last time we replicated.
Speaker:So I hope that.
Speaker:Helps Muddy the waters.
Speaker:Just a little bit.
Speaker:You're either synchronous or you're not just like in.
Speaker:In the following episode, we will be talking about a
Speaker:term near continuous, which I actually coined, but, uh, a lot
Speaker:of people think isn't a thing.
Speaker:I don't know
Speaker:if
Speaker:think, they're like, you can't have...
Speaker:Then again, they'll say that continuous is a...
Speaker:Um, is, is, is a binary, uh, condition.
Speaker:So anyway, well, um, I want to again, uh, say thank you to our listeners.
Speaker:You are why we do this.
Speaker:Remember, this is an independent podcast and the opinions that you hear are ours
Speaker:I hope this episode has been helpful.
Speaker:That's a wrap.