Hi, and welcome to Backup Central's.

Restore it all podcast.

I'm your host, W.

Curtis Preston, AKA Mr.

Backup.

And I have with me my continuing advisor on my consumer backup project

Prasanna Malaiyandi, how's it going?

Prasanna.

I'm good.

Curtis.

You know what?

We have the expert, the backup anorak Daniel Rosehill coming

next week on the podcast, so

yeah.

can definitely pick his brains.

Yeah.

You know, it's been interesting cuz, and it was funny, it was on the

podcast where I really, I really, I had a moment where I was like,

I'm not really backing up I photos.

Right?

Like, I'm, I'm backing up, you know, I, I use iCloud, but as, as we have

discussed, iCloud is not a backup.

iCloud is a sync.

Right.

And if something catastrophic, if you, if I ever got hacked, um, and somebody

got a hold of my, my iCloud password or my iPhone and then just decided to

massively delete everything, I, if I caught it soon enough, I would be okay.

Cuz I do have like a deleted items thing.

Right.

And as we discussed, I have, well, I don't think we'd discuss on the pod,

but as part of this project I found out I have 11,000 photos in, in iCloud.

So,

isn't as much as a lot of other people, you know.

I'm sure there are

know, I am not,

you.

yeah.

As, as I, as you and I were talking earlier, I'm not, you

know, on one end, I'm not Cecil b Dilla and I'm not, you know,

photographing and filming everything.

On the other hand, I'm not Prasanna because you use your phone camera,

like you use your, uh, Tesla,

Yeah, pretty much.

Yeah.

Almost never

Okay.

So you've had your Tesla, how long now

four years,

and how many miles do you have on it?

I think like 11,600, 700, something like that.

And it works great because my maintenance cost has been zero.

My electricity cost is really minimal versus gas,

Cars are incredibly reliable when you don't use them.

Um, anyway, yeah,

powered cars, even if you don't use them,

you still gotta change the oil.

You still gotta do everything else, you know?

So,

I'm not, we're not doing a, we're not

doing an e an ecar thing.

But anyway, but yeah, we've, we've been having some fun with, uh, with this

project of figuring out the various ways.

Right.

Uh,

Prasanna Malaiyandi:

and, and I think, yeah.

Prasanna Malaiyandi:

I was just gonna say, I think you should mention to the listeners what you're

Prasanna Malaiyandi:

current, what you are currently trying to do for backing up your iCloud photo

my current, uh, uh, uh, I don't know what,

I don't know what, what, what, I don't know these different methods.

My current method that I am trying is Google Photos, and it turns

out Google photos, it, it's the only one that I've found so far.

Uh, well, the only one that I've.

Well, there's maybe one other, which is iDrive, but Google Photos, cuz

the problem is that on an iPhone you can turn on optimized storage.

And so I have like, I don't know, somewhere between sixty

and a hundred gigabytes.

We're not quite sure of photos up in, uh, iCloud and, but I only have four

and a half gigabytes on my phone because it's, I'm using the optimized storage.

But apparently Google Cloud photo, Google Photos pulls down a high res

whatever high, the original version from.

iCloud and then backs

Prasanna Malaiyandi:

that, that's our theory.

Prasanna Malaiyandi:

That's our theory.

That's the theory.

Well, it's, I, that's what it says in documentation.

We shall see what we shall see.

Um, and then we will report on the results here and then,

and, and I'll blog about it.

I'll backup Central, uh, because IO iCloud is not a backup.

The number of articles that I read that told me to use iCloud to

back up my iPhone pissed me off.

Right?

Like, it, it was like 95% of the articles that I found on how to back up, uh,

my photos basically said, OI cloud.

I'm like, ah.

Because.

Because for most consumers, right, they're probably not going to do what

you're about to do, and they don't care.

And so turn on iCloud.

At least you have something else other than whatever's on your phone,

Yeah.

So we're gonna, we're gonna have an answer for the three people in the world,

all of which are probably already on this recording, the three people in the

world that actually care about having an actual backup of their, of their photos.

Anyway, all right.

Well, we're gonna bring back a, a longtime friend and a

returned guest to our podcast.

Uh, he is one of the few people in this industry that, um, make me feel young.

Uh, we welcome.

We welcome.

And he's also the, uh, the technologist, extraordinary and

plenty of potentially at Vast Data.

Welcome to the podcast, Howard Marks.

How's it going, Howard?

I'm really happy to be here cuz you know you guys went on your

little podcast and you said something about using flash for backup being stupid

Yeah.

Yeah.

I might have

But, but, but, but wait, I wanna clarify,

Howard, that was from Curtis.

I was the one who was like, yes.

Oh, he's

What about Howard?

Oh,

W. Curtis Preston:

throwing me right under the

throw you under the,

W. Curtis Preston:

So, we'll, we'll, we'll

you don't have to explain that to me.

I've known Curtis 35 years.

that's that.

We will, we will, uh, we will, we'll get to that topic.

I will give you a chance to defend your, your, your Honor.

Um, why don't we start with an update.

It's been a while since we've had you on the pod.

Why don't we start with an update on, uh, how much more vast, vast

data is, uh, since we had you on.

Well, you know, from the financial side, um, we announced at

the beginning of this year that we've hit a hundred million a year in annual

recurring revenue cuz we've organized ourselves as a software company even

though the experience customers have is look, appliances on my data center

call vast when something goes wrong.

Um, we arrange for customers to buy the hardware so that we are a software

company, makes life easier for us.

Um, the other big thing is that our friends at HPE

just made announcement of a product of theirs called GreenLake Files

that will be powered by our software.

So before today, if you wanted a scale out, expandable, low cost all flesh file

and object system, we would facilitate your buying hardware from the OEMs that we

deal with, and we'd sell you the software and you'd have a system that was running.

Now you can buy that from HPE as part of GreenLake, and that includes management

through the GreenLake Cloud front end.

So you can manage the GreenLake for files along with GreenLake for Block

and the Compute and all the other servers that are part of GreenLake.

So they've taken our software, married it to their control plane

and run it on their hardware.

And is for, sorry, for those who may

not be familiar with GreenLake.

GreenLake is more of a, I don't know, a managed or a hosted environment done by

It, it, it, it's an as a servicey.

So there, there are both consumption and CapEx models the way I understand it.

But you know, you don't log into the block array that and create a LUN.

You go to the cloud website and you create a LUN and their

control plane does that for you.

And so it's got more controls and you do don't have to keep the detail.

Yeah.

So then you're, you're probably paying

for what you provision then,

Uh, you can do that or you either way.

And, you know, that's, that's, you know, kind of an H p E finance question.

But my understanding is they do it either way.

And I'm sure for vast data, right.

That's a huge win, being part of that offering.

Well, it's, you know, first of all, it just gets hundreds and hundreds

of boots on the ground out selling our software and our whole concept of you

can do all flash for as cheaper, cheaper than other guys can do spinning disk.

And why do you want spinning discs as opposed to flash?

Other than that, they're cheaper.

You know, I can't think of another advantage.

Um, and so we, you know, for most workloads have narrowed

that down or reversed it, and so, All flash cheaper than disk.

What a great idea.

Um, so we've got, you know, a, all those HPE sales guys going out there selling it

as an HPE product, you know, it's not like Qumulo or Scality, where HPE was reselling

those products to run on HPE servers.

Mm-hmm.

Um, and you know, who you called for support was.

Well, is this a server problem or is this a software problem?

It's HPE GreenLake for files.

HPE takes the support calls.

It's a full HPE product.

Um, it's our software underneath it.

W. Curtis Preston:

Yeah, it's interesting.

W. Curtis Preston:

So you know, you talked about the, you, you know, you said you, you.

W. Curtis Preston:

You, you have a r r.

W. Curtis Preston:

So basically your customers are paying an annual fee to you based

W. Curtis Preston:

on the size of their storage

we, so, so we make, we make our money on a, what

we call a Gemini subscription.

That is, you know, in capacity units, we sell it at a hundred terabytes.

HP can sell it in different ways, um, and it's per year.

And we guarantee that we'll write that agreement for any piece

of hardware for 10 years at the

right, right.

right.

I remember

Because, you know, it's not spinning disks.

They don't start failing a lot more often in year five and six.

And so if you want to keep it for 10 years, keep it for 10 years.

If you decide you want to replace some of your hardware in your five

or six, because the new denser or faster hardware is more attractive to

you, uh, but you bought seven years of support, we'll transfer it on the,

you know, terabyte per terabyte basis.

Right.

Gotcha.

Um, yeah, that's a pretty good deal for you.

And by the way, I'll, I'll, um, I, I was gonna, uh, compare it to something

else, but, but it, it made me remind me of our, uh, disclaimer Prasanna.

And I work for different companies.

I work for myself, he works for Zoom.

And, uh, these are our opinions, not theirs.

And, uh, be sure to rate us at, uh, your favorite podcast or give

us all the stars and comments.

It helps other people find us.

If you think we're amazing, then maybe other people will do so as well.

Uh, reach out to me, uh, @wcpreston on Twitter, or w Curtis Preston

at gmail, and, you know, to be part of the conversation.

And we'll see.

Um, you know, we'll get you on.

So your arrangement with HP reminds me of our arrangement with Dell.

Basically it's the whole boots on the ground thing.

Uh, you get to put your product in front of a whole, you know, giant number of

other people and it's great for you.

It's good for them.

Their customers get the benefit of your, uh, technology with, with the company

that they already, you know, know and

And you know, and, and we all know that there are loyal H P E

customers who you know now it's a lot more likely they'll buy this product cuz

it's got that stamp of approval on it,

all of which works for us.

Yeah.

Yeah.

Well, congratulations on hitting a hundred million.

Um, wish you the best of luck on your way to the, you know,

doubling that and triple in that.

Um, last time you were on, we talked about, we, we alluded to, I think,

a little bit about how you do dedupe or dedupe-like stuff that's a little

different than the rest of the world.

And, and, and that it's better, you know, these are the, you know, the,

the, you're saying it's better.

So I, I want to give you a chance to talk about that, and then

we, we guarantee it's, we guarantee it's better because I'm

a vendor and without a guarantee you shouldn't believe anything I say.

Okay.

All right.

That sounds good.

So how so how, so first off, how is it

better, and then why?

It's better cause it reduces data further.

Um, And the why is how it works.

So, you know, at the beginning it's really pretty simple.

We do variable chunk deduplication with a variation of the rock soft method.

So if there are insertions, we re re re-sync relatively quickly and the

deduplication gets more effective.

Mm-hmm.

Um, we do z standard compression on the data.

We then throw some data specific encryption algorithms at the data for

things like, oh look, it's numeric data.

Well that means it's only gonna vary within this range.

We'll store deltas.

And so whichever of those compression methods reduces this block of data most.

Mm-hmm.

we run.

Um, because we're doing so the, the data path is writes go to storage class memory.

Mm-hmm.

then get act and then all of this data reduction happens

as we migrate from that writebuffer to the capacity flash layer.

And since it's after the act, as long as we're draining that buffer fast

enough, l how long in time it takes to move any piece is irrelevant.

And so we have time to go, ah, let's try five different compression algorithms.

Use whichever one works best.

Interesting.

Yeah.

do you do,

go ahead.

Go ahead.

do you?

And that's actually very interesting how you can, like you said, by

storing it in memory, right.

You're not impacting client latencies at all.

Right.

For them it's like, Hey, right.

Went through.

And then you have this time to do the parallel, uh, computation.

Yeah, just, just accept it.

It's storage class memory, so it's an S s D, so it's persistent and

there's no batteries and protection and you know, a panic when power goes

Yeah.

Now, when you are running these algorithms, like I know AI and ML

is all the hot topic everywhere you look these days, right?

Are you guys doing anything around that in terms of trying to smartly detect

which compression algorithms based on

We we're, we're not doing that in the data path right now.

You know, frankly, the running the five doesn't use that much

compute that it's worth it.

Um, we're using AI in our cloud platform, so if you have multiple

clusters, there's a cloud site you can go to and see one dashboard.

Um, and we're using it for the capacity projections.

So it's like, oh look, here's how much capacity you're gonna need

six months from now while you're filling out your budget request.

Let me tell you what you're gonna, there's AI behind that so that it

smooths things like, oh look, every three months they do a cleanup.

And so let me factor that the AI is good enough to factor that kind of thing

in, but not in the data path.

But let's get back to the data path.

Yep.

yeah, your, um, your comment when you, you know, there

was a comment you were like, as long as we're clearing the buffer quick enough.

Um, and, and I would agree with you, um, you know, how, how do you ensure that

that happens, I guess is, is one question.

Well, first of all, it becomes a parallelism issue.

So we have a large number of compute nodes, all of which are

draining this buffer in parallel.

And so when the buffer hits a high water mark, more threads

to D stage, it gets spawned and allocated across the parallel system.

Now, if there's a huge influx of writes, and you know, we're talking.

Tens of gigabytes per second for hours on the smallest system.

Mm-hmm.

Um, then we'll start introducing latency into the

writes and apply back pressure.

Okay.

Okay, that makes

But, but you know, that's, you know, literally,

you know, it, it don't

ha it, you know, the mechanism is there just in case,

right.

happen.

And when you, and because you have more than more

capacity, I guess, more throughput at the capacity level than at the storage

media level, is that why you can just increase the number of parallel

threads and you don't have to worry about the backend being a bottleneck?

so in, in, so our, our building block, we call a D Box

or a data box, and it's got some.

S scm SSDs.

We started with Opta.

We now mostly use K oxia FL six, and then a larger number of capacity SSDs.

And they, you know, it's whatever the cheapest we can get or the cheapest that

our OEMs use is, um, the P C I E lanes.

Feeding the small number of S C m SSDs is generally the bottleneck.

Okay.

And so we can paralyze reading data out of s C m, the

writing to cut to the capacity.

We have a lot more capacity ssd, so there's plenty of bandwidth to write

Gotcha.

So what, why'd you stop using OC Octane?

Um, well first we decided just to get a second

source because it's a good idea.

Um, and then I, Intel

turned out to be a really good

of, then, then Intel decided to get out of the business.

Um,

and, you know, we have supply agreements with Intel.

They still have a warehouse full of wafers.

Um, but it, you know, the, the performance advantage wasn't worth the complexity.

So we've chunked on these variable sized 32 K average blocks and we de-dupe them.

But in addition to running a strong hash.

To validate identical, we run a series of weaker hashes against the same

data blocks, and these weaker hashes are designed to generate the same

hash value for inputs across a narrow range of cryptographic distance.

So if two blocks have a sm, so cryptographic distance is the

number of bits you have to flip to turn block A into block B.

If block A is within X bits of block B, this hash will

generate the same hash value

Okay.

from a data reduction point of view.

If two blocks generate the same hash value and are a small cryptographic

distance part, they have long common strings between them.

And will therefore re compress with the same compression dictionary.

So the first block that generates one of these similarity hashes, we just

compress and store when the second through MTH block generates the same hash.

We recall the first one and we used the dictionary from the first

one to compress the second one

So you get better compression

we can store it compressed without the overhead of

storing the dictionary a second time.

yeah.

Yeah.

Yeah.

and it becomes essentially the difference,

Yeah.

So instead of storing a bigger block, it's like just very, very small Deltas

because they are cryptographically

right?

that,

that's

similar.

The mathematicians would say it's a limited cryptographic distance.

That's unique.

I've never heard of someone doing that.

Have you, Curtis?

just this guy that we had on the podcast a little

Yeah.

that looks a lot like Howard.

It, it, you know, nobody else is doing it now.

Um,

Is it, are you patenting it or,

there are patents around it.

I don't, I haven't looked to see exactly

gotcha.

Gotcha.

to.

Yeah.

That

Um,

cause reading patent applications makes my brain hurt.

I see, I thought that you would, let's

say you got two chunks, right?

And you run the really weak, but much faster, I'm assuming, uh, hashing

algorithm, and that you would say these two blocks definitely aren't the same.

And so let's not do anything else other than com.

They're not, they're nowhere.

They're, they're, they're cryptographic distance.

I think you said so far apart, there's no point in running the

stronger ddu, uh, thing on it.

Um, that's

where I

it, it turns out, it turns out even with a weak hash, the

number of identical hashes that are not identical data is so small that

the cost of testing is ignorable.

Yeah.

And especially if it's in flash, like probably

the,

is

compare is so rare.

It doesn't matter that it's expensive.

Yeah.

And the fact that you're not doing this in line, right.

So it's all been de

Or

it, it's not trench.

It's in line, but it's not.

client.

act,

Yeah, yeah.

Exactly.

it's, it's it's post act, so it doesn't have any impact on latency.

But you know, the, the S CM is a one-way writebuffer.

We write new data into it, it gets demoted to the capacity flash layer

and there's so much bandwidth in the capacity flash layer that reads from

there actually faster than from the scm.

So there's no reason ever to promote it back.

Right,

Um, but the other thing is we keep all the metadata in that s scm.

So as you expand the system, you add another enclosure that's got more s cm

and more capacity, the DUP hash table and the similarity hash tables all grow

with it.

So it's one data reduction realm regardless of how big a cluster is.

We don't have to store that DUP table in memory.

Yep.

And so you know the whole, well, flash would be great for backup,

except I can't afford it as well.

If you've got three or four conventional PBBAs,

Mm-hmm.

you know, first of all, the vendors of PBBAs charged

you a lot for that disc storage.

You know, they, that's a high margin product.

Yeah.

as soon as you have two of them, you have two deduplication realms.

Right,

And we might talk about data duping 10 to one.

That doesn't mean all your data dupes 10 to one

right.

50% of your data at least is unique.

Yep,

Some of your data ddus a hundred or a thousand to one, and most

of the benefits you get is from that data that ddus a hundred or a thousand to one.

Well, when you got two boxes, it's not a hundred or a thousand, it's 50 to 500.

yep.

And every time you add a new box, you're, you lose some of that benefit as well.

Right.

So,

you can dup all our episodes down to

like four or five comments.

Right?

Like back up, back up, all the things.

Well that

3, 2, 1.

Just 3, 2, 1.

that requires the next version of, uh, AI deduplication

that can take out the idle banter.

Exactly, exactly.

Our episodes will be like five minutes long.

So just to summarize or just to close on that, so

we talked about the how you guys do it.

So because of all these technologies that you're leveraging or mechanisms,

right, that's how you're able to offer that guarantee, right?

That's better than anyone else.

Yeah, we, you know, we use Z Standard.

It's a slightly newer compression algorithm than anybody else

does, cuz we started a little bit later than everybody else.

So we got to pick the latest one.

Um, and then we have those, you know, the additional, well, oh, they're numbers,

let's just store the differences, tricks.

And then we do deduplication on variable block, which is

as well as anybody does it.

And then we throw in similarity as, oh, here's another unique

trick nobody else does.

And so the combination is, we are confident that as long as you're send,

you know, we guarantee as long as you're sending us unencrypted data,

that will reduce it better than the other guy, whoever the other guy is.

And if we don't, we'll provide the capacity so that you

didn't pay any more money.

Cuz

Yeah.

Yeah.

No, that's a great guarantee for end users and customers, especially

with budgets these days, right?

It's like, Hey, I bought this system.

It doesn't quite meet my expectations.

I can't go back to my boss and ask for more money.

So

Well, you know, the other side of that is, you

know, just, it's really a very simple scale out architecture.

So you don't buy today what you think you're gonna need in three years.

You buy today what you think you're gonna need in a year, and then you

can buy more when you need it later.

Or if, as some of our customers have found out much to therin of

our sales guys, their data reduces better than they expected and they

don't need anymore in the next year.

Well then you're just ahead of the game.

So, and I know maybe you could talk in gener generalities,

but sort of like if I was a customer who had one of the competition, PBBAs, right.

And I now use Vast, right.

I buy a vast system, sort of like, is there, like what is the savings that

I normally see in terms of storage?

Like if I had like a hundred terabyte P B B A, actual

if, if you, you know, a hundred terabytes is small for us.

okay.

Or say

a

So if you had, if you had a petabyte P B B A, um, then you

know, you're probably storing four or five petabytes of logical data on

it.

Um, and you bought a, you know, a petabyte of usable from us and you'd

probably store 25 or 30% more on it.

Okay.

But that petabyte, P B B A is as big as you can buy that P B B

A, there isn't a two petabyte P B B A.

Yep.

And the real difference is at restore time

Hmm.

because PBBAs are scaled.

For backup speed, not restore speed.

They don't even have restore speed on the spec sheet anymore.

And backups are not sequential operations nearly as much as

you think they used to be.

And you

know, when

Curtis, when when Curtis changed block, block tracking, incremental forever,

Right?

of those things make the backup and the restore much more random.

yep,

And so if you're backing up to a disc based p v a,

your restore speed is like a fourth or a fifth, you're backup speed.

yep.

If you're backing up to a vast, your restore speed

is five times your backup speed.

Cause we're, cuz we are designed to serve.

Re primary storage applications where reads happen much more frequently

than writes cuz the reads come from all the capacity SSDs, the

writes have to go to the s scm.

Um, and so what, where that really starts to get important is when, when we start

talking about ransomware attack, cuz 10 years ago Curtis and I used to teach

seminars and we'd go, yeah, 90, 95% of your restorers are, you know, the file.

Somebody screwed up.

And you know, if it's on A P B B A it'll be restored in a couple of minutes.

And if it was on

tape, you'd go find the tape and then a couple of minutes.

And so, but you don't know you've been ransomware attacked till

thousands or hundreds of thousands of files have been encrypted.

Yep.

And so now you have to like use something like instant recovery to

check back, you know, is this backup good?

You gotta do three or four quick looks without restoring, which

is a great feature, but you know, requires a relatively high speed

backend to work relatively well.

And then you're gonna find, okay, this is my last non good point.

And then you have to restore and you are gonna have to restore a lot

of data and restore speed starts to become really important then.

Mm-hmm.

And then the kicker is, and the lawyers in the insurance company

won't let you use the, the system that was infected for another couple of

weeks cuz it's evidence or we have to get somebody in to clean it and certify

that it's cleaned well, if you know you can run a VMware NFS data store

on VAs, you can just restore to VASc.

Now it's a bad idea to run your primary and your backup on the same

system for more than a day or two,

Right,

but, Compared to not running your primary and just, you

know, if your choice is backup only or primary and backup, and if this one

system dies, I'm really in trouble.

Not that part of choice for me.

I want my users back up and running.

As soon as the lawyers let me get tacked to the old system, or my

VAR gives me a new system, or I have someplace else to storage, VMO

to, I'm getting that stuff off there right away.

But that might mean I'm up a week earlier and a week earlier is a lot of time.

my objection to flash for backup has

been for two primary reasons.

One is, is expensive af right second.

Do I really need it?

Right?

Like, because there's, there are a lot of things that we can buy in life, right?

Uh, like I, I need to move fertilizer.

I can totally borrow Prasannas, uh, Tesla and it will do it, right?

But, but is that what I should be using for that?

Do I need, do I need a Tesla to move fertilizer or will

my Prius do

need Prasannas.

Tesla to move fertilizer if you ever want prasanna to speak to you again.

no, that's, that's true.

By the way, the Prius has been used to move fertilizer just for the record.

Um, but so, so that's the thing.

It's like, there, there are a lot of things, like, this goes back

to the c d P, the c d P argument that I made back in the day.

It was the same thing, the same two arguments.

One was c D P was too damn expensive, right?

And then the other was, does anybody actually need.

The, the, the, the functionality that C D P provided.

And the answer is yes.

0.1% of the population needed what C D P provided.

And that's why you don't really see c D P as a choice very, very often these days.

Right there, there, there's a one or two companies that do it now, um,

and all the other products have died.

So those are my two arguments.

It's, I, I already know what your argument to the second one is gonna

be because you just gave it, I think.

Um, so

why

about for cost?

Well,

talk about costs?

so, for cost, it depends what flash systems you're talking about.

Mm-hmm.

Um, I will give you that most all flash systems are

designed to be fast as possible for a small amount of data, because that's

what you need to run the Oracle databases that make companies work.

And so if you, you know, it's a block, you know, it's block storage to be low

latency to support O L T P and therefore expensive because that's, you know, if

that system goes down, you count by the second how much money you're losing.

And so you have always bought expensive storage for that.

Um,

sort of the, sort of the true normal sort of, if I, if

I can use this word, pure flash array.

Yes.

the, that type is designed for that, right?

Um, that's technically pure with a small p, but it works the other way as well.

Um,

talking either way, you know?

Yeah.

You know, I could name half a dozen other products, but

And they're just too expensive.

of it, you know, it's, we're gonna design a system based on

having a, a pyramidal tiered system.

this is the one at the top.

Yeah.

And if you assume you're gonna build a tier system, then you

want the one at the top to be as fast as possible, and you kind of

don't care how much it costs because

you'll just put stuff that doesn't deserve it on the next tier.

Yep.

Philosophically our idea was we're gonna make something that

delivers performance for everything but the very, very top there.

Mm-hmm.

And goes down in cost to where Well, if you use enough

of it, you don't need those tiers.

You don't need the complexity.

Right.

So, you know, part of our story is as you consolidate workloads, you have

workloads that need performance, and you have workloads that need capacity.

When you add capacity, performance comes with because in,

you know, spindles, how many SSDs?

Yeah.

A hundred SDS is so much performance.

200 SDS is twice that much performance.

So if you take the applications that need capacity, And you put them on the

same system as the applications that need performance but don't need capacity.

The performance that the capacity creates is used by the applications

that need the performance and the cost of the performance is brought down

because you've used that much capacity and you get in a vir virtuous cycle.

I think I followed that.

yeah, it, it, it's basically

by

if you, if

that's

common.

if you, you're, paying 10 x for 10% and one x for

90%, then you're paying a hundred.

If you have one tier that costs a hundred,

Mm-hmm.

why have two tiers?

Yeah.

And when you use capacity, that capacity comes with performance.

Mm-hmm.

And that means that performance is available

to other applications that didn't need the capacity.

So you don't need to have separate systems, you just have

So, so if I could, if I could try to put this

in, in, in just different words, but it'll say the same thing.

If I've got a hundred QLC disks, right.

Um, and, and these are how big

15 or 30 terabytes.

the each, each one, right?

Each one

So if I've got a hundred, I've got one and a half

tear, one and a half petabytes.

Did I

do that

right of raw?

Yeah.

Okay.

All right.

So I've got one and a half petabytes of raw capacity.

And what you're saying is we can just take a slice off the top, if you will.

You know, we used to call short stroking the discs.

The, obviously you don't need to short stroke a, a flash, but you're

basically saying, we're just gonna take a slice off the top, uh, of these

150 discs and we're gonna get this massive performance slice, uh, for,

for the 10% that need that performance.

And then the rest will just put wherever we need to put it.

Is that, Does that sound about

right?

I'm, saying all those SSDs create one pool of

performance and one pool of capacity,

Right.

and a workload can draw from either one as much as it needs.

Gotcha.

Is it separate capacity and performance pools that you

then assign to Applic?

Okay.

It's just one

pool that includes both

that, and now, you know, and you can use q o s, you can say,

okay, this workload gets a hundred thousand iops, or, or 50 gigabytes

per second, and this other one gets

different.

yeah,

yeah,

And

you'll just use

And so you

you need to

performance, right?

But you know, if you've got, um, you know, your backups and you've got the developers

who wanna do run, live copies of the database, well run it all on one system.

It's, it's an all flash system.

It's fast enough to run the database.

It's almost as if you're saying, You've built a

system that works for all workloads except that 1% or whatever, that's

like that very, very, very high end.

And you're saying you have one common architecture that allows it

to deal with, regardless of if your workload is capacity focused and not

very performance, it doesn't need a lot of performance or it's high

performance and maybe a little capacity.

It's all a

and and it doesn't matter whether your definition of

performance is bandwidth or iops.

You know, it's like all but that very lowest.

You know, we, you know, we're an all flash system lightly loaded.

We deliver one millisecond latency.

yeah,

You know, some systems can deliver half that

and some rare applications care.

But you know, between that and the 10, Tencent, a gigabyte, well,

there are 20 terabyte hard drives and super micro servers and you

know, they don't do any iops, but you can write to 'em pretty fast.

You know,

Yeah.

in between we can cover.

So we're dancing around.

You're saying why you could be cheaper, but let me,

let me just put a, lemme just put it right, you know, sort of, I'm

assuming that you get into competitive bids with PBBAs on a regular basis.

Yes, sir.

Okay.

How do you do there?

They're easy.

Those are very high profit margin products for

the

so you're, so you're, saying you can come in

less expensive than the effective price of the typical P B B A, even

though you're using all this flash.

Yes, sir.

Okay.

Because that, that's the short answer.

I like the long answer.

That's, I like the long answer.

You and I

live in long, right.

Um, yeah, but in the end, it doesn't matter if it's still more expensive.

yeah, the, you know, the long answer is, you know, we use the

cheapest flash we can get because we designed the system to treat flash well

and understand how to minimize wear.

We ha our erasure codes have 3% overhead at I at large scale, so we're not wasting.

Space on raid, we reduce data better than anybody else does.

So you know, we're getting as much capacity in there.

Um, and then when you start saying, okay, it's 30 terabyte SSDs, so you get a lot

of capacity and a little bit of space and a little bit of power, and the power

and Rackspace start to add up as costs.

Um, especially when you start looking at the fact that the leading PBBAs are

still using eight terabyte hard drives because that rehydration tax of turning,

Hmm.

making everything random, well the bigger the hard

drive gets, the worse it is cause.

One hard drive is a hundred iops.

Yep.

Doesn't matter whether it's a one terabyte hard

drive or a 20 terabyte hard drive.

And so they're just reaching the, the world.

The land of diminishing returns on IO density.

They can't go any lower.

And now the sheet metal and the power supplies and the SaaS

expanders are becoming a larger and larger percentage of their cogs.

And they mark 'em up a lot cuz there's a lot of IP in there in terms of

software and they have to make a margin.

Um, and so we just don't have most of those problems.

Yep.

you're, you're, also marking up due to your ddo, right?

I mean, you

Yeah.

Or you know, some of it, you know, some small portion of the difference is, you

know, compared to the guys, those the best P VBAs, we still do a little bit better.

but but when we go into a customer who says, no, no, no price, this as

if you de-dupe the same, we're still coming in with a lower selling price.

Yeah, I, I'm not surprised about that.

The other thing, Howard, I wanted to bring up, I know you

mentioned sort of dis drives and the a hundred iops limit, right?

That each of them typically have.

The other thing that I've also seen is as the drives get larger and larger, anytime

you have to do a raid, rebuild, right?

And you're talking like a 20 terabyte drive and it just takes longer and longer,

and now there's a potential for failure,

Yeah.

Well,

becomes a lot worse.

you know, I do a lot of, you know, resilience calculations

and people just don't realize how big a factor rebuild time is

in the probability of data loss.

Uh, we had one customer share with us.

They ran, you know, the leading.

Scale out system before us and the average for their rebuilds was 53 days.

Oh

wow.

That's two months.

yeah, that's two months during which time your data is exposed

and you know, could be slightly exposed if you are already running.

N plus three could be really exposed if you're running n plus

one, like some vendors recommend.

So it all depends.

right.

Okay.

So, so I think, I think, you know, you, you've definitely

covered the cost argument.

Um, the, and, and it's, I think if we just back up, you've

already covered the why, right?

The why.

would, why

is today's Restore different?

Ransomware.

Yeah.

stores.

The stores bigger.

And the restore location is less well known

What do you mean by that?

you may not be able to restore back to the infected

system cuz it's still evidence,

okay.

Understood.

Okay.

right?

You need someplace to restore to.

And you know, having it where the primary in the backup are duped to each other

probably gives you that in a pinch.

Yeah.

Have you

seen.

emphasize in a pinch, cuz

Yeah,

you know, we've, we've all been trained, no, no bad idea.

Don't mix the strip, don't cross the streams.

Um, but, but that's the, if you have a backup on the primary, you

don't, you don't have a backup.

But if you have to choose between backup and primary, I'd rather have primary.

Have you seen customers actually do this

in the field with fast systems?

Oh, we have several customers doing really

large scale backup to Vasst.

Um, we had one customer who was kind of shocked cuz they were

doing encryption in net backup.

And so they expected us to not reduce data at all, uh, but they were doing encrypted

net backup backups of Oracle dumps of the same database over and over again,

encrypted with the same encryption key.

And we started seeing about 20% reduction just because even when you encrypted, if

you're backing up the same data, it looks

the same encrypted as it

right.

There's like sort of two questions in my head here.

One is, and, and they're, they're very much related.

One is the, the whole backup container problem, right?

Meaning that you get the net backup container and the.

Arc serve container and the backup exec container, you know, and they

all stored backup data differently.

And you have that issue.

And then you, but you, there was something that you alluded

to that I found interesting.

You said commonality between the backup and the primary, but the

backup is in some weirdo format.

So are you able to get backup or commonality between the

backup and the primary?

Now in that case, it's

more likely we'll see commonality between multiple primaries.

You know, it's more like you

restored 17 windows VMs

Does the way that you're doing ddo make the

format problem any less problematic?

Right.

O Only in that we reduce them all as opposed to if you were relying

on the data movers to do reduction.

So, you know, kind of the most common case is the storage guys like backup,

you know, com Vault or net backup or Veritas or Veeam or whatever they use.

And the Oracle DBAs don't trust them and insist on doing, doing, dumps.

Right.

And so, you know, if you're doing both to, you know, they

just give the Oracle DBAs, okay dump to this NFS mount point on the vast.

Well then we'll reduce all of those dumps as well as anybody could reduce

all of those dumps and your data mover.

You'll do data reduction at multiple stages to manage the network traffic.

And then we'll do the final dup at the end cuz we're finer grained.

And the sim similarity works really well for things that are duped course

grain, cuz the edges all look similar.

And so when

we, when we run, you know, we have a probe you can get as a VM that

scans your data and reports back, this is how much it would reduce.

And this is how much of that comes from each of these techniques.

And so when we run, when we do that with data from a data mover,

d duper, those are usually, you know, 128 K or big blocks because

they have limited memory available.

And so we see more similarity cuz we're finding those pieces Finer

just to, just to make sure I understood correctly.

So the, one of the question that I didn't really ask was, you know,

when you buy a, you know, pick your favorite P V B A, they tend to support.

These five backup products, and if you buy a different backup

product, well, they're like, well, we don't understand that format yet.

And so then they have to go and do some development work to figure

out how to crack that container.

Do you not have that problem or have you done that

We, We, have not optimized for any of these backup applications,

And yet you

still get better duped than the other guys.

Our, our general case data reduction against all of these reduced

data types still gets better reduction.

You know, we are not, you know, scanning for the timestamps in Oracle rack

dumps and, you know, that level stuff.

Right.

Not to

agnostic, right?

to say we won't in the future, but our, you know, our data

reduction was written for primary storage.

It just so happens that being an

N F S or an S3 target for a backup data mover is a simple case of primary storage,

Yeah.

it just works.

Right.

Right.

Hmm.

What do you think Prasanna,

in the future?

So I, I had no complaints to start with.

Uh, the one question

I lost this argument?

I think I might

I think, I think you lost this one.

Uh, Howard, the one last question I had was, I know some of these backup vendors

support the ability to do source side due duplication by integrating with

the purpose-built backup appliances.

Does VAs support that?

Are you guys planning to support that?

I know you're looking, you just

previously said, right, that you're

don't, we don't, um, I've never been really comfortable with the

use of client side CPU for that cuz client side CPU is valuable for other things.

Um, I think, you know, doing a pass at some level in the data mover.

Mm-hmm.

It's like, okay, we'll we'll de-dupe at the media server at some

large grain so that we're not transferring 50 copies of windows over the network.

Um, is perfectly reasonable thing to do cuz it's a network

bandwidth management technique.

Um, things like Didi Boost are, you know, let's offload this from the,

the P B B A to the client and we'd just rather do the work ourselves.

Um, and in our architecture, since you can just add more servers at the front end

and you just have to buy the servers, we don't even charge for that software.

If you need more compute.

To do more

you just scale out.

to more dup, you just add a few more servers as opposed to stealing 5%

of the cycles of all of your VMware hosts, which means you now have to not just

buy servers, but you have to buy another VMware host, another VMware license.

All the other stuff you put on a VMware host starts to add up.

Yeah.

Gotcha.

Well since, well, since you stepped into my neighborhood

now Howard, I will have to say that source side, DUP done correctly,

speeds up the backup and reduces the C P U utilization on the client.

But I, I want, I can't speak to the, to the implementations

you were talking about.

Um, I can only speak to the one that I am obviously very familiar with.

Um, cuz there, you know, there that, that is the off discussed

thing of like, well there is a,

It's

know, there's a

pen.

it's also a different case because of the assumed

bandwidth at all the stages.

right.

You know, I'm, I'm kind of assuming that there's

a lot of bandwidth for short

distances in the data center

Well, all right.

I, I concede this battle, Howard, I lay down my sword.

Um,

Okay.

I

you know, I mean, you, what's that

You expect

to what?

in the mail,

Um, yeah, I'll send you, I'll send you something.

Um, alright, well, uh, Howard's been great.

Uh, glad to hear the update and glad to, you know, I, I remember we did,

now that I heard you describe it, I, I think we did cover it in the last one,

but I think you went deeper this time and that, that's good to hear this idea

probably.

you can, that you have the, that you have the, the bandwidth

to, to, to, to how many different ways did you say you try each block

for

there's five compression algorithms and, and then there's a strong

hash and a number of similarity hashes.

I can't remember offhand what they are,

Gotcha.

I got, I thought I heard you say 15 total ways.

I thought I

heard you say

that.

it on that order

Gotcha.

So the fact that you can take each chunk and try 15 different ways to

compress it and pick the one that works the best is pretty damn cool.

Um, and, um, you

know, it's just on

of it's just cuz we can parallelize it so well

Yeah.

that too, right?

Uh, the

fact that, you can, you know, that's a.

one takes.

We're doing a lot.

Yeah,

it's that, that beauty of the scale out architecture, right?

That you can just pass that out, um, like that.

All right, well, thanks for coming back, especially to, it's really funny

how we had you, you're like, you're listening and you're like, Hey, you said

mean things about the way I do things.

I will, I accept your challenge.

And I'm like, all right, come on back.

Uh, happy to do that.

And we'll do that with other people.

By the way, if you're out there listening and you're like, the thing that Curtis or

Prasanna said is wrong, we will be happy to have you on and have us prove to you

why you're wrong or, or in this case,

In this case, uh, right.

Yeah, we concede.

Well, I mean, I mean, I think my concerns are certainly valid and

there are certainly vendors out there that are like, yes, we can

certainly sell you this appliance for the purposes of backup, because

recovery speed is really important.

I'm like, but it costs five times the cost of this thing over here.

I don't like how much better could it possibly be Anyway,

so that's, that's where those arguments tend to come from, so.

Alright, well thanks.

Thanks Howard for

and given, given the marketplace, they're

not completely unreasonable.

Uh, you know, we, we just do things sufficiently different that,

you know, if you think restore speed's important than we do,

right, right.

Yep.

Yeah.

Yeah.

Ransomware.

Ransomware.

All right.

Once again, ransomware, you know, I don't know.

What do you, what do you call it?

Uh, trump's all, um, although I don't enjoy that word as much

as I used to for some reason.

Um, anyways, thanks for, thanks for your questions for,

Uh, I try.

I try.

And Howard, great to have you back on the podcast.

Hopefully you'll come again.

Always pleasure.

As long as I keep winning, I'll keep coming back.

and we thank you to our listeners.

Uh, you know, we're nothing without you.

Remember to subscribe so that you can restore it all.