This week on restored all, we've got a backup practitioner that has used

multiple backup products, including the one where I happened to work.

I hope you enjoy this week's episode.

Hi, and welcome to Backup Central's Restore it All podcast.

I'm a host w Curtis Preston, a k a, Mr.

Backup, and I have with me my colossal Chore completion

celebration companion Prasanna

Woohoo,

. W. Curtis Preston: Malaiyandi

How does it feel, Curtis that.

my Lord.

it's, you know, uh, there are no words.

I, I, I will technical, you know, basically, you know, to,

so to be honest, I have finished the part of the project that.

everyone can see.

I'm still sitting on carpet in this one room, which is my office.

Um, and uh, but everything else has been completed, which means that we're

gonna have guests over for Christmas.

So that's

And, and this is basically for the listeners who

may not realize this is Curtis's.

I'm going to replace all the flooring in the downstairs by myself

and take up tile, which has been.

Glued down with super glue, basically

Yeah.

and

was, there was what?

There was carpet, there was tile, you know, large, large format ceramic

tile and also Pergo in one room.

So I took all that up and it was approximately 1500 square feet of

luxury vinyl tile that I laid down.

it was like three pallets that were

sitting in your garage, right?

Yeah.

Uh, and I'd say about, um, I don't know, about a 25% into the project I was.

what have I done?

Like, because I, I actually got hurt.

Like, I, I, you know, I'm, I, you know, I'm in my fifties, I'm down on my knees.

My, my, that, that was a mess.

Um, and then I got sick, so, but yeah, the really funny thing is, um, for the, for

the long time listeners, you know, that sometimes my granddaughter comes on here

and at the beginning of the project when she saw how much work it was, her comment

was, this is gonna take like five months.

and I was really insulted at the time, but, uh, it took actually a

little over five months due to my injury and my, and my sickness.

But, but yeah, at this point it looks beautiful out there and I, I, I don't

think the mic would pick it up, but right now, just outside my door is

our new, uh, Roomba that we bought,

To keep it.

of that.

Yeah.

it and span

Yeah, exactly.

Because the one thing I noticed with having, you know, the hard floor

versus the co carpet is that you see and feel every little bit of dust.

And so, um, that, uh, so yeah, so we, we added that, that it's, it's

now done, its first full vacuuming and now it's trying to do its second.

So it's a fun, it's a fun time.

At the Preston household

Well, at least it's all done before Christmas,

Exactly.

Um,

I'm sure your wife is quite pleased.

yeah, she really likes, she really likes decorating

the house for Christmas and we really weren't able to because of, you know,

the fact that yeah, it was a construction zone up until just like two days ago.

so now here's a question, Curtis, are you gonna quit

your day job and go do INS flooring installation as a full-time gig?

absolutely not.

There was this guy, and by the way, shout out to there, there's a website called

for anybody who's thinking about doing it.

It, it wasn't like difficult per se other than the fact that you're on

your hands and knees the whole time.

And, and one thing that was invaluable was this, there's a guy, um, who.

Has a site called, so that's how you do that.com . And he has a

service where you give him like, I think it's like a hundred bucks.

and then he guides you through the project.

You give him your floor layout and he's like, start in this room, go

to this room and here's the room where you're gonna work backwards.

And um, you know, and that was invaluable throughout the entire process.

Um, so shout out to him.

Um, yeah.

Thanks Joe.

Anyway, so let's get on to our guest who's sitting there going, what kind of

podcast have I come on At this point, I didn't sign up for home improvement stuff.

He has been in it for over 20 years and was most recently the Vice

President of Technology, architecture and Innovation at Maximus, a

government administration company.

If you're looking for an experienced bright IT manager, we think he'd

be a great addition to your team.

Welcome to the podcast, Albert.

Uy,

Uh, hi Curtis.

Good morning.

Thank you for the introduction and I'm glad to be here.

uh, have you recently completed any major?

Do it yourself projects

Yes, actually, like you're saying, I'm doing the Christmas

lights and, and kind of upgraded my laundry, so did some cabinets and sink.

So I'm a, I'm a diy like you, so d i

right.

All right.

Fellow, fellow DIY person

respect.

Um, sometimes, you know, you bite off a little more than you can chew.

Uh, you, you know, you're halfway through the project and you're like, why?

You know, why didn't I just call the guy?

The solar project, Curtis

Yeah.

We won't talk about the solar project.

The solar project was, yeah.

Speaking of taking six months.

Um, yeah, but it, but so ha Is it the, the, the laundry thing?

Is it finished?

Yeah, it's finished.

Um, do you wanna hear an interesting thing I wanna share?

So, you know, I, you know, I got a point.

Install the cabinet and install the sink with the plumbing,

and I'm like, okay, I need it.

I need to get it certified, so I need a certified plumber.

Right?

So they say they usually charge like $90 plan trip and stuff, but when they heard,

uh, I just moved into a new house, the price went from $90 to like five, $600.

So I know when they hear like, oh, you just moved into a new house.

And so I went and to uh, forget it, I can do it.

I went and just did it and I'm, I'm pretty sure I, I would,

yeah, I, I did a great job.

So my wife said, Hey, thumbs up.

why they would

Yeah.

Yeah.

When they hear a new house, the price just went up the roof.

I don't know why.

From 90, 90 to a hundred to like four, 500 dollars.

Just, just to just install the plumbing, the, you know, the, the, the, yeah.

So

Wow.

crazy.

Yeah, it is crazy.

Uh, yeah, I recently redid some, you know, some stuff in the kitchen and,

and I, and I, I did call the guy cuz plumbing is one of those things

where it, it's not, Technically very hard, per se, at least much of it.

Some of it is actually really hard, uh, but it's, but it, but it's, it's one of

those things where it's not, it's like, it's not technically difficult, but you're

on your back and you're in these tight, tight, tight spaces and all that stuff.

So it's definitely in that category where I'm like, you know what?

And, and also the one thing about plumbing is if you get it wrong,

you might not know right away.

Right.

It might be just a slow leak and then you find it out way later

and it costs you way more money.

Um, we won't talk about the bathtub incident of, uh, 2000

Well, you know another thing that reminds me though, I

know you're talking about plumbing, but I feel the same way about cars and brakes.

Right?

It's like a brake job.

It's like get someone to do that.

And I know, Curtis, you're laughing because I know you do your own breaks, but

Yeah, I, that was, that was the way I felt for a long time

until I, until I found out it, that, that,

that they actually weren't, that.

Yeah, they're not, and, and, yeah.

Yeah.

Uh, and generally with breaks, they either work or they don't, like, it's

not like you, you know what I mean?

Anyway.

Yeah.

But I, but I, I, I respect the, the not doing it as well.

And uh, and I was gonna share the trick is you have to

put the right washer and clamps.

And like you said, I went and put a pan after my work is done.

Uh, just to make sure there's no leak.

I left it for a month and there were no leak.

So,

Nice, nice, nice.

Yeah, so, well, we're not talking about plumbing today.

We're

Are you sure we could

Yeah, we could, we could do a just a DIY podcast where I

talk about what I've done and you talk about what you've seen on YouTube,

Yep, exactly.

Yep.

Um, and, um, maybe, maybe Albert and Albert and

I can, can start a d i y podcast.

Um,

I was recently watching a YouTube video of a guy trying

to clean up a 30 by 60 saltwater pool,

Wow.

clean it out.

What do you

sorry, clean it up because it was full

of algae and everything else.

He had just moved into a house and he had no pool experience, and he gave

up and he just hired a pool service.

They wanted five grand to bring it back to life, and then 1200 a month.

See, this is why I do stuff myself.

I had the same problem, five grand.

Yep.

Well this is a 30 by 60, Curtis.

Oh, but still, I mean, 30 by 60 is big,

Yeah, it is

Yeah.

okay.

One, one more.

So, one more.

So years ago when my, I have a pool, and by the way, you don't need a pool.

You need a friend with a pool.

Um, that's a life lesson to anyone.

Please take that lesson.

You don't need a boat.

You don't need a pool.

You need a.

with, with both of those, or, or one friend with one each.

Uh, my pool, my pump went down and I didn't have the money to replace it right

away, and by the time I replaced it, I had a swamp in the, you know, I had a, a,

it was disgusting and I actually, if you can believe this, and a bunch of stuff

had been dumped in the pool like, like leaves and twigs and all that kind of

stuff, I actually put on my scuba gear.

and went in my pool and went down and cleared out all the

debris in the bottom of my pool.

And by the way, I was completely blind cuz it was just green, right?

Just a hundred percent green.

And I'm down there with a, with a bag and I'm just, just

crawling around in the bottom.

I had myself super weighted so that I was just laying on the bottom and I'm

pulling out the, the twigs and the rocks and the, you know, And, and because I

knew that like any one of those could, could mess up the, the, the pump, right?

The Well, no, the, the, you know, the,

Oh, the Impalas?

W. Curtis Preston:

crawley thing, you know?

W. Curtis Preston:

Yeah.

W. Curtis Preston:

So, so I, I, I, so the, and then, and then I just dumped it.

W. Curtis Preston:

I just dumped.

W. Curtis Preston:

The pool, right?

W. Curtis Preston:

Um, and instead of trying to spend hundreds and hundreds and hundreds of

W. Curtis Preston:

dollars of chemicals trying to bring what it was back to life, uh, but that

W. Curtis Preston:

was an expensive, just the process of dumping the pool and refilling the

W. Curtis Preston:

pool, uh, was an incredibly expensive process because they charge you, you

W. Curtis Preston:

know, uh, what it is, is it, it impacts both your water bill and your sewer.

W. Curtis Preston:

um, because they're like, oh, they're, you know, cuz they, they look at how much it

W. Curtis Preston:

impacts your sewer bill for like the next quarter because they see how much water

W. Curtis Preston:

you put out.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

And so, uh, so the secret is to fill it with your neighbor's hose,

Good tip.

Oh, Curtis.

i y tip, fill your pool with your neighbor's hose.

Um,

Hmm.

I kid.

I kid.

So, so, Hey Albert.

Uh, you, you've been in it for a while, for a minute.

As the kids, say,

Hmm.

um, and, you know, and, and I'm sure you've been in and

around backup systems quite a bit.

From your perspective, how has backup gotten.

Better and worse, like over, you know, you've seen, uh, if you've been in it 20,

you've been through the tape world, you've been through the conversion to disc world.

You've been into the, you know, into the, um, obviously into the cloud world.

Uh, what, what's that?

What's that been like right.

I would say people are lucky to be, uh, managing backup today compared to

20 plus years ago because like you said, when I first started, there were only

like one cartridge or there's no robotic, and, and the tape is, can only do like

megabyte versus gigabyte and all that.

And then, and then you know, you have to start with on-premise and then

now you got cloud and all you need to do is good internet connection.

So, you know, it's almost like night and day comparison.

Right.

So talk to me about, um, so first, you know, we took, you know, you, you

were in that tape world and I, and I grew up in that tape world, right?

My, my.

The backup tape that I first, well, technically the first tape I did a backup

to was actually a nine track tape, which I don't, I don't remember what the capacity

of that, but it wasn't much because, um, because I remember that the, the density

of that tape was 120 bits per inch.

It was a single.

It was a single, um, recording stream.

Right.

Um, and then I, I have, I'd have to look up, I, yeah.

Anyway, so that was my first, and then I remember working with,

you know, eight track tapes.

I'm sorry.

Um, eight millimeter tapes, four millimeter tapes, uh,

then, um, dlt, then L T.

, right.

Um, grew through all of those.

Um, d do you remember, were things ever good with that world Because

I remember like, for a while they were bad because the drives

themselves were really unreliable.

Didn't the drives?

My memory is that the, the drives got more reliable, but then they got too fast.

There.

Was this, the shoe shining problem of the tape drive being too fast for the job?

Uh, how, how about.

Right.

Yeah.

Um, well, I was gonna say, well, the key thing is when you're doing backup and

disaster recovery is r p o and r t o.

RPO stands for recovery point objective, and RTO stands

for recovery time objective.

So, like you said, um, more, uh, two decades ago because of the speed wise,

um, there's three types of, uh, backing up full backup differential and incremental.

Um, because of the slow speed you have to do of a lot of incremental, right?

However, people don't realize when you restore, restore, that means you

have to restore a lot of incrementals.

Right.

So as the tape device goes fast and fast, you can go less incremental

and do more differential.

And today with the speed, you can also do full backup.

So what I'm trying to say, we use it when you restore, you always

want to start restoring, using full backup and then whatever differential

based on r p on R T O based on, you know, restore your differential.

And if not, you're incremental.

Uh, if I make sense.

Yeah.

You do.

Yeah.

Prasanna, you, you remember?

We just, what?

It was

I think it was like two episodes ago.

Yeah.

We just did an episode on backup levels and, uh, I

prefer the term cumulative incremental

Yep, yep, yep.

different companies

use the term differential to mean

different things.

Um, but yeah, that was a, that was a huge thing back then was the,

the, how often we're gonna do a full, how often we're gonna do a

differential or cumulative, incremental.

And then obviously we did incrementals every day if we

weren't doing one of those other

Prasanna Malaiyandi:

Wasn't there a Hanoi Tower?

Yeah, there was, the Tower of Hanoi thing was about

different levels of backup real.

That that was a huge thing.

Compare that to what we do now, which I, I would say when you look at,

um, and by the way, I, I, I'll throw out our usual disclaimer Prasanna,

and I work for different companies.

I work for Druva, and, uh, Prasanna works for Zoom and uh, this is

not a podcast of either company.

Uh, and the opinions that you hear are ours and sometimes they're all mine.

They're all Curtis's.

I don't give Prasanna a word in edge, a word to speak.

Um, and, um, uh, and by the way, be sure to subscribe to this podcast

so that you don't miss anything.

And also, Yeah, just click right.

Just click subscribe right now so you don't forget.

And be sure to rate us, just, uh, scroll down to the rating thing.

Give us, you know, five stars and, uh, give us a comment.

We love, we love comments.

Always helpful.

Always.

It helps.

It helps other people find our podcast, the more comments that we have.

And, um, But, uh, and, and also, uh, you know, in full disclosure, uh,

Albert is a former Druva customer.

Uh, he's not, uh, you know, he's not employed right now, so he is

not a current Druva customer, but, uh, he was a former Druva customer.

So we'll be talking about Druva a little bit in this episode.

A little bit more than normal, I think.

But, um, so what I was about to say was, You know, with a product like Druva, and

we're not, we're not the only ones that do it, but with a product like Druva, you

do a, what I call a forever incremental.

You don't do repeated fulls, but.

, but the backups are stored in such a way that each backup behaves as a

full, from a recovery perspective.

I know we always talk about sort of the incrementals,

but something that I always like to bring up is when you talk applications

is after the incrementals, then you gotta worry about all your archived

redo logs and whatever the nomenclature, the application uses, right?

Which just adds more and more to the time it takes to recover.

And, and that's a very good point.

So I said the first requirement you knew to know is the R P R T O.

The second I'm a big, um, follower methodology of people, first

process, and then technology.

So just to share with you, I also used to work at GameStop,

which is also a Druva customer.

Um, I was able to reduce the 72 hours of Dr.

Uh, r t o to 24 hours.

and when I mean people process technology, um, because, uh, they were like siloed the

service team by themselves and database.

So when we were doing disaster recovery, um, the server guys have to restore the

file first and then database comes in and then use their database back up.

To restore the database.

So, you know, there's two different methods of, you know, um, restore, right?

So yeah, that's why people, you know, get together with the people first and then

create the process to make the people work together and then apply the technology.

Yeah, sometimes you people tend to forget about

those first two parts, right?

They're like, oh, here's an awesome technology.

It'll solve my problems.

But then they apply it, and then it doesn't work the way they think,

because the people in the process part, they haven't worked through.

And it's like, oh, the technology's bad.

But it's like, no, the technology isn't bad.

It's just you only did one third of the problem.

You have to also worry about the people in the process.

And, and another thing I wanna share is when I was at

Maximus, um, because of the people, there were only two tier support.

People requesting the restore and then the backup team restoring.

But the thing is, you know, it queues up because the backup

team are the only restoring.

So I created a third tier, like a middle tier.

Um, we gave, um, the database team power, user rights, server

team power, user rights, site administrator, power, user rights.

So instead of going to a backup team, the site support team can, can restore

site files, database can restore database files, and server team can restore

server team server files, and then application can restore application files.

By doing that, the sla, um, we were able to meet SLA because it's not entirely,

uh, dependent on the backup team.

it's almost a little bit like self-service.

Yep, exactly.

Yeah.

You know, you, you bring up a really, I, I think a really good,

another part of technology that.

That has happened in the last, I'm gonna say 10 years, and that is the idea

of role-based administration, right?

It is probably a little more than 10 years, but, um, what I remember

again, back, back in the day, uh, if you were gonna administer

a backup system, you were root.

Right.

You, you had root, you logged into the system as root.

You pulled up the, the, the ui, uh, we called it a gooey back in the day, right?

You pulled up the gooey and you ran that gooey as root.

And if you didn't root, because root was the, was the, the user ID that

had the power to get all the files.

That's why you had to run it as route.

Right.

Um, and that, but that was a horrible thing because I remember back in the

day, Saying, you know, I, I, I would joke about it, but I would say, you know, be

friendly to the backup person because with, with, in order to do backups,

not only did I have root on the backup adminis, the backup system, I had root

on every system because I needed to.

So I had the ability to log into every.

Destroy your world, right?

And then go to the backup system, or maybe before that, go to the backup

system and destroy all the backups, right?

So be really kind to the backup person because they have the

ability to destroy your entire world and leave no evidence, right?

Um, that's, that's what we did away.

With this idea of role-based administration, this idea that you

gave, because back then, if you wanted, you know, the Oracle DBAs

to, to, to do their own thing, you had to give them root, right?

Which I think we could all agree.

They don't need root, we don't need Oracle, uh, permission.

Right.

Um, and of course I had Oracle permission back then too.

Um, so this idea that you could give certain groups of people the ability to

do things, The way you described it was it was a way to make things better for them,

but it's also a way to make things better for you in that you gave them just, you

know, we, we have this concept of least privilege that we talk a lot about, right?

You gave them just enough power to do the job they needed to do, but

without giving them the ability to mess up the rest of the world.

Does that sound about.

Exactly role based

don't give him the keys to the Kingdom

Yeah.

Yeah.

So, so let's talk about that a little bit.

So, I know Druva has role-based administration.

What other things did you do with that?

Like how, how did you divvy up responsibilities?

yeah, so role based, so we'd just like to say database

team can only restore database application, application server

server, uh, site administrator site.

. And the reason is, um, you know, we have data centers, but

we also have remote offices.

Uh, because of, um, government services.

What we have is what you call a tl, uh, authorized to operate.

So some of the files has to be in the remote office before we can

put it in the cloud or data center.

So that's when the site administrator come in and they

restore whatever locally to their

Also when when you say site administrator, you mean

administrator of that particular

office.

Yes.

And what about, um, I know, I don't know if you've, if you've

done this, but what about separating.

Other things within the backup system.

Like some, I've seen some people where they're like, we, we give this

person the ability just to run the backups, but then they don't have

any ability to say delete backups.

And then, you know, we have another person who can configure backups,

but they can't restore backups.

I've seen some very interesting stuff.

Uh, can you think of anything else from the people that run the backup system?

Anything that you did?

So, so the backup team, we used to call them, well, the team was called

e edp, enterprise Data Protection Team.

So they schedule a backup, so they do all the backup based

on the project requirement.

Mm-hmm.

So in the, on the roll base, um, the roll base are able

to restore, but usually it's the EDP team that schedules the backup.

Mm-hmm.

So on the database team, um, because of the project requirement

based on the r p rtl, you know how many times you do full versus incremental.

So they schedule it.

And then one thing we made it easy is because we went to the cloud,

we also, um, uh, created a tagging.

Um, you know, you can create a tag on your cloud instances.

So we have a, um, a tag called backup, and then they would put a value,

uh, which is a string, and then they use regex to go to the string to

say, Hey, how often do we want full?

Uh, you know, is this a database?

Is this a file?

Is this so.

Mm-hmm.

. Mm-hmm.

. Mm-hmm.

all the,

without having to require some two people talking together.

that's correct.

That's correct.

Did you have anything?

What?

What would you do if you had a new resource?

Cuz I'm assuming you used like auto discovery and things like that.

If you had a new resource that needed to be backed up but someone

forgot to put a tag, did you, did you have anything for that?

Yes.

So, um, we have a delivery team.

Um, so we automated everything with the help of Druva.

We created Lambdas on aws, so for, um, for operational readiness.

So when a project goes live in two weeks, we already checked

two weeks prior to going live.

Those tags have values, so everything's shared.

So, did I answer question Curtis?

that warms my heart.

Uh, Albert the idea that someone would wait to go live on a project,

they would wait to make sure backups work before that project goes live.

That, that warms

my heart.

And, and, and Curtis.

With GRS Health, we create what you call backup of service.

So every, every instance cloud instance being created has

the site value of the backup.

We already know, uh, if that instance will be backed up, if that

instance will be patched, and if that instance will be monitored.

So those three functions are in the tags, and we have a operational

readiness team that goes in and makes sure those tags are values.

And then we also have, uh, uh, account owner, account owners.

We have technical point of contact, uh, for those project and for those instances.

So everything's labeled, everything has to be labeled.

So if not, uh, um, which we partner with the security team,

we have what you call compliance.

So before they go live, we make sure the compliance is a hundred

percent before they go live.

If not, uh, if not, we create, um, incident priorities.

Um, we create, um, a day, um, well, two weeks before it goes live, we

create a P two, and then one day before it goes live, it becomes a P one.

In other words, you need to fix it before it goes live.

now the two weeks before.

Are when someone tags it.

Are you actually doing backups at that point to make sure that the entire

process and everything works, that things can be restored because it hasn't quite

been pushed to production yet, correct?

Right, right, right.

So, so we follow, um, two SLAs, a corporate SLA and a project sla.

So before you, you know, when we deliver everything, follow the

corporate sla, and then if the project SLA is stricter than the corporate

O overrides the corporate sla.

Hmm.

So, so when we deliver, we based on those sla, so those values

are filled out based on those SLAs.

Make

that.

I love, yeah.

I, you know, when I started again, a hundred years ago, when

I started backup sla sch, I mean, we didn't have anything like that.

I

mean, at, I mean, at best we did have, we had, we, we didn't, I, I

think I was in the backup business.

Probably five years or so before I heard the terms RTO and RPO

and by the way, I agree with you, that's where it starts, right back then

again, you'll remember this back then.

We spent so much of our time talking about like backup window

and um, you know, we didn't, we hardly talked about restore window.

We talked about backup window because that was the thing we had to do every

night, whether or not we could fit our backups within the backup window.

So this idea that you basically, you agree on the SLA first.

and then you design backups and other things, uh, to meet those SLAs and that

all of this is done as a cooperation between the business and the technology

folks before the server goes into, or the application goes into production

again, that this is what Prasanna isn't just what we talk about all the

time, right?

That.

That con that, that, that cooperation between the business and the Yeah.

The stakeholders and the, and the people.

Uh, or the, or the technology folks.

There, there's another thing by the, there was another group that you talked about,

and again, this warms my heart as well.

You talked about partnering with the security folks, um, you know,

in this world of ransomware.

Hmm.

I think that partnership is more important than ever

before, um, that folks on the security side need to have some basic understanding

of what, what the way backups work.

And folks on the backup side need to have a basic understanding

of, you know, security.

Well, thi this concept that you talked about, you know,

role-based administration, that's a security concept, right?

Least, uh, least privilege is a security concept.

. And, um, so, so let me ask you this.

How have you, over, over the years, how have you dealt with one of the

problems that, that I see in it, which is the, the, the idea that nobody wants

to do the backups , so nobody wants to be in charge of the backup system.

Uh, how, how did you deal with that before and how was that any different, uh, when,

you know, when you, uh, went with Druva.

Right.

Well, going to the cloud is very helpful and since Druva is also cloud

native, it's very helpful and one advantage of working in government

services were highly regulated, right?

So, so we, you know, because we're highly reg regulated, security

kind of dictates what we need.

So actually our operational readiness is monitored by, How about compliance?

We monitor all the, all the security compliance and operational compliance.

So before going live, so let's say if your backup choice has, you need backup

agents, make sure backup agents, you know are installed, we check on that and

then make sure the tag values for the backup agent is full, you know, field.

And then same with patching, and same all that.

So compliance is almost like our key, like you said, security, uh, and it

covers both security and operational.

and

what about get, getting the person to actually

getting somebody to actually take the responsibility for the backup process?

Is that, is that less of a problem for you in that, in that,

world?

Um, yeah, because, uh, the delivery team, so whenever any

instance, um, gets, um, created, it follows the corporate sla, everything.

So all the engines install, everything's automated.

Uh, the, the security agents installed, the monitoring agents

installed, the backup agents installed.

So everything's automated.

And then what we do is we use the compliance report to go in and check, make

sure those, um, those agents have, he.

and make sure those agents also have values on their tagging.

So it's all automated and we

like.

actually Yep.

It sounds like it's part of a team rather than one person

that's just responsible for backups.

It sounds like the team is responsible for backups.

Is that okay?

That, I think that's perhaps one way to solve

Yeah, and I'm guessing that having this automated

process also helps you avoid sort of like the shadow IT issue that we heard so much

about like five, seven years ago, right?

Where a department's like, Hey, I need to spin up something, but I don't want

to go through normal it, so I'm just gonna go swipe my credit card, get an

AWS account, and start running right.

Right, right.

Yeah.

And, and, and it sounds like I, I think that automated process was

also enabled by the way Druva works, you know, again, before and after.

W you know, you could automate to a certain degree, pick

your favorite backup product.

You can automate that to a certain degree with one major caveat,

and that is capacity, right?

Um, a, a backup server or, or, you know, cluster of backup

servers can only back up so much.

It can only handle it from a, from a comp, uh, compute perspective,

from a throughput perspective.

Most importantly, from a capacity perspective, when the bits

are full, the bits are full.

And you can't just, you can automate all you want.

Your backup system's just gonna be like, uh, I don't know where

you expect me to put this stuff.

Uh, you know, but with, but with Druva, you can a hundred percent

automate it because the compute and additional capacity that you need is

just automatically added to the system.

Does that, does that seem about.

Yep, that's right.

Just following up on that sort of the automation side, I think

one thing I'm curious about Albert is.

Enabling of this automation, the APIs, that a lot of, because previously,

like Curtis was talking about, right?

You went into a gooey or a UI and you manually clicked all these

buttons and now you have sort of this automation side of things.

How important are APIs as you start to look at this new world, right?

And the enabling of integration of backups into these process.

Um, APIs are very important and don't forget, there's different flavors.

APIs, you have public APIs and private APIs, so most time you get

public APIs, but you have to see if the public APIs support role-based,

blah, blah, blah, everything, right?

So APIs important.

One thing though I wanna add is, uh, what's more important than API is culture.

Like automation.

So when we start implementing automation, everybody's like, Ooh, if

I automated, I'm gonna lose my job.

Right.

So, so, so the one thing I did was promote the culture first before, you know, you

have api, but people don't wanna use it.

It's useless.

Right.

Yeah.

So, and how I promoted Yeah.

And how I promoted it.

I, um, I create, you know, sometimes it's nice to mix fun with work.

So I started with automation.

Hmm.

So, and I create a process in automation games.

Hey, automation games is not just for developers.

So I kind of come up with a, uh, idea board.

So if, if, if a project manager have an idea, uh, and then other people will see

the idea and say, Hey, I wanna join you.

So the developer will have a developer joining him, have a QA joining him, or.

and then together as a team, they create this automation.

They, and then they submit it on the automation games and

then, you know, and all that.

So, so kind of created automation games to promote the culture,

and then people get the point, Hey, this is actually helping me.

Uh, I can do, I, I can spend my time doing more important

stuff than just doing backup.

Right?

So,

Right.

Um, yeah, I like that a lot.

Be because, you know, you, you really, um, I flash back to a.

I remember, um, you know, a hundred years ago, uh, when I worked, and

here comes the name again, when I worked with Stewart, um, . It's a name

that comes up a lot on our podcast.

Uh, there was a, there was a guy at, at that company.

That his thing was, he was the NetWare administrator, remember NetWare.

Yes, no.

And, and he had zero interest in modernizing the

infrastructure because it meant that he wasn't gonna have anything else to do.

, right?

Uh, that, that problem is.

I think that, I think you, you really hit the nail on the head that this,

this problem with au the, the problem with automation is people think,

well, if I fully automate the world, then I'm going to be out of a job.

And, and I would say that what you do is you automate the stuff that

you, that's just the mundane, the stuff that has to happen every day.

One perfect example is backup.

So that you can then do the interesting stuff, which is like, we can talk

about hunting for the bad guys, right?

We can talk about watching the environment.

Something that that can be assisted by technology.

Right?

And I know, I know Druva has that as well, where it has the posture and

observability features and it can.

Look for bad things going on.

But I think that that's also a very human thing to do is to, number one,

watch for the, for the bad actors.

And number two, um, continually, you know, you talked about that idea

board continually figuring out how we can make our security better, right?

Uh, because in in this world, there is, there, there has been no time where I

think cybersecurity is more important.

Yes.

Uh, actually I'm glad, uh, Curtis, you touch, uh, what people don't

understand is there's um, um, there's three data situation.

You call it data at res.

Data in motion and data in use, right?

So like you said, mostly when you're doing data backup, you wanted data at rest.

So one thing we're looking at is actually we're looking at

Druva is data resiliency, right?

So that's when security and cybersecurity comes in.

So you don't just bank it out when it's a data arrest.

You also have to protect it during InMotion and uh, at use, right?

So,

Right.

so that's in terms of cyber.

Uh, one thing I wanna mention too, on automation, um, One trick, um, because my

pre my predecessors tried automation game it, you know, coupled them and it failed.

Uh, one thing I also learned is you need to make it open.

You know, the guy, the person before me said, Hey, okay, automation

game, everybody has to use Ansible.

But the thing is not everybody knows Ansible.

So when I create automation, I open it.

You can use dos batch file, you can use Lambda, you can use, uh, batch,

uh, you know, batch script, and.

and that's how it's successful.

So, and, and then what, what we needed to do is find a tool that will support

all open scripting or, or programming.

So,

Nice.

Uh, yeah, I like that.

Yeah, because not, not everybody knows.

I mean, I remember back um, everybody was jumping on Pearl.

and all I knew was, all I knew was Bhe.

Right.

Um, I was an old Bhe guy and I, I learned Pearl and I, I wrote, I remember

writing, uh, just to learn Pearl.

I remember writing a monitoring program that, that created a webpage.

of how net backup was running at the time.

Right.

Um, this was, uh, I actually remember the first time that, that, that, that, uh,

system, I dunno what to call it, program, uh, the first time I went live was

actually at Amazon back in, back in 1998.

Uh, it was a long time ago.

Um, I, you, you mentioned earlier about how you made Dr.

Faster.

Um, when you talk about that 24 Dr.

Pro, 24 hour DR process, uh, talk to me about how, how

does that 24 hours break down?

Um,

what, what does it consist of?

yeah, because, uh, everybody was operating in, uh, silo.

So the, the, the server team, uh, used a product called, um, at that time was

using a product called CommVault, and then the database team was using SQL

backup, so the old database between SQL backup and then the server team

will use Combo to back up the sql.

So when you do diver dr, you do reverse, right?

So the server team comes in and restore the combo file, and then

after that, the database team has to use that file and, and, and, and

restore it into a SQL database, right?

So that's like a, just the process alone is 72 hours.

But since they, we created the process.

So instead of using SQL backup, it's just one file.

So when the server team restore it, it's already the.

You don't need the database team to come and restore this file into

a SQL database, if that makes.

you, you made the actual restore process itself.

You, you took out steps in the restore process itself.

Right, right.

And then it cut costs to this.

And I was talking about at that time there were no cloud yet.

So when we are doing VR exercise, um, we had to fly in everybody.

The file backup team, the database backup team, you're

talking about a good 15 people.

So, uh, 15 people, um, not doing their day-to-day job, just

doing the DR for three days, so,

Right, right.

Yeah.

We, we did that back in the day.

We would, we didn't have to fly anybody in cuz we were all in the same place.

But, but we did it over the weekend cuz we were a bank.

Um, so it's not like we're gonna take the bank down for a few days.

We did it over the weekend.

. Um, and, um, we would, uh, one of the, one of the rules that we

had back then was that the person responsible for the backups, which

was me, was not to run the Dr.

right.

Um, because the idea is, it's supposed to be properly documented enough that the,

the other person, you know, that another person that's capable should be able

to follow the documentation and do it.

I will tell you, in the three years that I was there, not once did we

successfully do the , the restore from beginning to end without having

to come in and, and, and help.

I mean, it did work.

It's just.

You know, the, the process was so complicated and so convoluted that,

um, you can only document it to a certain point and then, right.

And, and so I, I, I do think that's, you know, you talked about making

it so much simpler with a, with a single step restore process.

That seems like that would be the, the biggest boon to

getting things restored in.

it's interesting that sometimes people forget about that.

Instead of looking at how do we document this process?

It should be, how do we simplify this Right.

and start thinking about that.

Yeah.

How can we make this process so simple that documentation is like,

pull up the UI and log in, or, or

Well, well here, well here's a, here's a question for you.

Right.

Whenever you go buy a product today, how often or do you go and like open

the instruction manual, read through the instruction manual before using it?

Right, Curtis?

Or you just got a Roomba, right?

Did you actually read the instruction manual or were you

like, oh, I'm just gonna use it?

Well, that was the quick start card that I did need because

there, there were some steps that I had to, but it was just a two-sided of a card.

Right.

Um, but you, you, you did make me think about, again, in this process we bought

a new, um, we bought a new dishwasher.

And there was this, I, I was putting in the dishwasher and I had put it in

essentially without, without really, I mean, I had looked at the manual a little

bit and, and, and I had it installed.

And then we wanted to, um, I wanted to level it and it was level, but I

wanted to raise the level of it so that it fit better in the, in the,

in the uh, counter and.

I was doing it.

Like, I was like, well how hard could it be?

You just turned the things right.

Um, right.

And my wife in the middle of this and I was getting frustrated cuz it wasn't

working the way I would expect it to work.

And my wife was like, have you looked at the manual?

And I got mad.

I was like, dang, I don't need the manual to turn some screws.

And it turned out I looked at the manual and I was doing it all

wrong because they had changed.

They had changed, uh, Yeah, so the manuals can be helpful, but

yeah, the process should be simple enough that you can do that.

Uh, so you know, Albert, I wanna thank you a lot for coming on.

Is there anything that we, that maybe you wanted to talk about that

we, that we haven't covered yet?

If I could go back to what you said about how backup has progress.

So if you remember 20 years ago, there's no cloud.

So, and then most of the time you change your DR provider every two years.

Right?

You know, popular ones are like Sunguard and ibm.

20 plus years ago, if you go with a provider, uh, you say,

how many servers do you need?

So 20 years ago you send this team going in and then they installed

the os, they installed the app.

and then 10 years later, uh, I don't know if you heard, they're

searching as bare metal restoration.

So you, so I did that too.

So what you do is you put a bare metal orchestrator on that location.

Uh, you, you, you, you say how many servers and then, you know, based on the

nick, um, then you have to spare bare metal restoration to restore it, right?

And then now today you got the cloud.

So it's gets simpler, simpler, uh, and faster.

Yeah, because with, with the cloud doing Dr.

I, I've always thought that Dr.

Cuz you know, some people are like, oh, he says that cuz he works for Druva.

I, I've only worked for Druva for five years, I've said this for many years.

And that is that.

The Dr.

The, the DR is the killer app for the cloud.

Because what you want is you want, uh, you know, a thousand servers right now.

and you don't wanna pay for them until they arrive.

Right.

Until you need them.

And, but with, with a, with a product like Druva, you can do the

restore part in advance, right?

And then it just automatically brings the servers in when it, when it needs.

And, you know, you talk about a simple DR process, it's just literally a one button

and then it automates all of that stuff.

Bringing all those servers online, all of the network settings, all of that stuff.

. Um, so yeah, I, I, I think if, if, if you haven't experienced DR in the

Cloud, uh, I'd say give it a try.

If you're pr, if the product you have currently offers it, give that a shot.

If it doesn't, uh, you know, they'd be happy, happy to talk to you over at

Druva, uh, to see how easy Dr can be.

Uh, and, and I'd also say cloud, it makes it so much less expensive

that it can bring Dr down to.

The masses, if you will, right?

It used to be it was only the, the companies who had lots of money that

could afford the sun guards, right?

This, this is literally companies of any size should be able to

automate their DR environment.

And, and Curtis, I would like to add has to be a

hundred percent cloud native.

So just to share with, that's reason why we went with Druva.

So when I was at Maximus, um, they had a project to go from on-premise to cloud.

. So the backup team was reporting to the engineering team.

At that time, I was managing the operations team.

They had two years to do it.

after 18 months, they just did one remote location and, and then they gave

me the team and say, Hey, finish it.

And I got six months left from the 24 months.

And you know, you're talking about several remote sites and big data centers.

So they said, here's the solution, implement six months.

I'm like, okay, you tried it for for 18 months and you couldn't do

it and you're giving me six months.

Fortunately I was technical and I look at the solution, I'm

like, this is not cloud native.

This is hybrid.

And that's the reason why it's taking forever.

Because when you talk about hybrid, that means including pro premise and on

premise equates to you need provisioning.

So, you know, I spent two months using their solutions that

there's no way this is gonna work.

I only have four months left and I have like so many sites and

because I had Druva experience in my previous job, which is with Gamestop.

I went in to the executive team and said, Hey, if you want me to get

this done in four months, uh, uh, we need to go with my new solution.

And they gave me a thumbs up and I was able to do it in four months.

And how I did it in four months is because I know how Druva works I've spent the

first month with, uh, what I call A P O V, I don't call it poc, proof of value.

Yep.

And I had the Druva team help me.

We created playbooks.

So all I needed to do is I know the solution works, I just create the.

A playbook for database, a playbook for application, a playbook for

server, a playbook for sites, right?

Spend a whole month creating the playbook and then project

management team help PMO help me come in and spend the three months.

And actually I told 'em I don't want to get in done in three months.

I wanna get it done in two months.

Because if you give them three months, they're gonna use three months.

Right?

right,

So, so I only have four months left.

One month for p.

Three months employment, but I told him two months and we were

done in two and a half months.

So two weeks before the deadline, because of expensive, on-premise, uh, license

renewal, um, we just, we had time to check and make sure everything works.

So we got it completed in four months.

Using the solution.

So just share.

And I did apply the people process tools.

I know the technology works.

Now I have to figure out the people, the database team, the server team,

the role base, like you said, in creative process, how we can make all

of them work together and how everybody follow the same playbook and all sort.

So

Nice.

I like it.

I like it.

Right.

Just to share

Thanks so much for, thanks so, so much

for coming on the podcast

uh, thank you for inviting me.

Uh, pleasure meeting.

and Prasanna.

Thanks again and thanks for helping me celebrate my little.

Woo-hoo.

Nice to meet you, Albert.

Likewise.

Thank you.

Happy holidays.

yeah, absolutely.

Happy holidays and thanks to the listeners.

Remember to subscribe so that you can restore it all.