This week's cloud disaster story is a shocking one from 2009.

And one that really chapped my hide at the time.

Today, we dive into the turbulent history of Carbonite, a once prominent cloud

backup vendor from posting fake reviews on Amazon to a major data loss incident

and a multimillion dollar lawsuit.

Carbonite's story is a cautionary tale for both customers and

vendors in the backup industry.

Join me and my co-host as we uncovered the details of Carbonite's missteps, including

their use of woefully, inadequate storage arrays, lack of data, redundancy,

and failure to take responsibility.

For a huge customer data loss.

We'll also discuss the lessons we can take away from Carbonite's mistakes.

One big discussion is guidance on what customers should look for when they're

evaluating a cloud backup provider.

You'll learn what happens when you blindly trust vendor marketing.

And you will learn the importance of thorough due diligence when

entrusting your data to a third party.

If this is your first time listening.

Hi, I'm w Curtis Preston, AKA Mr.

Backup, this podcast is my attempt to make sure that what happened to me 30

years ago will never happen to you.

My employer lost their purchasing database and I couldn't restore it.

I dedicated my career to making sure that that wouldn't ever again happen to me.

And now I'm passing it on to you.

We take unappreciated backup admins and turn them into cyber recovery heroes.

This is the backup wrap up.

W. Curtis Preston: Welcome to the show.

Hi, I am w Curtis Preston, and I have with me a fellow possessor

of useless knowledge Prasanna Malaiyandi, how's it going?

Prasanna.

I am good, Curtis.

Yeah, I was, uh, I, yes, I do tend to have a lot of what I

would consider random knowledge.

Not always useless.

I know sometimes it's helped

W. Curtis Preston: Okay.

You do have useless knowledge.

You do also have useful knowledge.

We, you know, I've kidded you about it in time.

I'll be doing something and I'll be like, you know, working on that timing chain on

my Prius, a car that you've never honed.

And you're like, well, did you do the thing with the thing with the thing?

And I'm like, how do you know this?

And you're like, well, YouTube, you know

I watch a lot of YouTube XI

W. Curtis Preston: But recently you and this person to which I am married, have

been kidding me a lot about my recent,

I don't know if it's,

W. Curtis Preston: useless knowledge.

I don't think it's recent though.

I think you've had this in your head for a while, and I've only

discovered it recently that you have this random knowledge in your head.

W. Curtis Preston: so, so I.

Well, it's relatively recent, by the way.

It, it's within the last year, uh, that I asked myself, what are the different

kind of palm trees in California?

And I found out that, first off, I found out that there's over 3000

palm tree species in the world, and that there's something like 50 to

90 palm trees depending on who you ask that grow, you know, here in

California.

Yeah.

No, no, no, no,

no.

50 to 90 that grow, period.

There's

only one species of palm tree that is native to California.

look at this.

You're already correcting me, the expert,

the.

W. Curtis Preston: yeah.

And that, but, but with all adding all of that up, if you can correctly

identify roughly five species of palm trees, at least in Southern California

where I live, you can cover 99% of the palm trees that you're going to see.

Right.

And I point this out to people.

I, I, I kind of, I bring it up.

That's the

weird part is I bring it up, I'm like, Hey, uh, look, it's, you know, and, and I

did this with this, this lady that I was talking to, and I was like, oh man, I got

a lot of king palms in this neighborhood.

And she's, and she said, you know, the different kind of palms

Curtis is like, my life mission is

W. Curtis Preston: like, yes, yes I do.

Would you like to know?

And so I start sharing with, with, with this person, uh,

the different kind of palms.

And then she said, can you tell me the kind of palms that are in my yard?

And I said, yes, I can.

And for the record, she had a combination of queen palms and king palms.

Queen palms are from South America.

King palms are from Australia, and uh, only the Mexican fan

palm is native to California.

And for those of you that have looked at California, been to California,

the Mexican fan palms are the ones that grow like 150 feet tall.

Yeah.

W. Curtis Preston: Right with the tiny little sprig of green on the top, uh,

they're all, they're also 20 feet tall, but they, they're the ones that grow

to, to the really, really tall ones.

Um, so yeah.

But you got, you and my wife have been making fun of me and

my, my possession of this random

knowledge.

so are you sure that this woman was not

paid to ask you this question?

W. Curtis Preston: No, she was not big.

I, I am pretty confident that she expressed genuine interest

in the species of palm trees that were in her neighborhood.

Now?

Now, now are you sure that she expressed genuine interest or now that you're

replaying the story in your head, you think she was expressing genuine interest?

W. Curtis Preston: I am convinced Sir, and I stand by my story.

Okay, so now that you've finished

palm trees, what comes next?

W. Curtis Preston: I, I don't know.

It, it, it, it, it has to come upon me.

It has to be, I, I need a muse, you know?

I need to be inspired.

You, you know what inspired, I think last time when you

did this, we used to walk all the time.

W. Curtis Preston: We did.

That is what happened.

We used to walk together separately, walk together

separately, right, uh, on the phone.

And I was realizing in my neighborhood the incredible diversity of palm trees.

Now that I have this knowledge, I need to redo that walk and then see, um, see how

many of them are actually, uh, you know.

The Just five.

Just of the five.

Um, yeah.

So anyway, uh,

speaking

so, so

W. Curtis Preston: what's that?

So listeners, if you are walking and you encounter

a palm tree and you're on Twitter slash x, post it there and tag Curtis

and he will respond.

Yep.

W. Curtis Preston: Yeah, I will respond if it's one of the five that.

Um, yeah, it was kind of funny.

Um, what is funny is when I, like if I, if I'm trying to teach somebody

the difference, for example, between a queen palm and a king palm and,

um, they don't get it at first.

And I'm like,

it's so obvious.

Like, look, and at those two trees, the difference between a king and a queen

is so obvious and it takes 'em a minute.

And then, and then they have the aha moment.

And

I'm, and I feel.

It's like, it's like the moment when somebody brings up the 3, 2, 1 rule out

of, out of nowhere.

And I say, yeah.

So speaking of the 3, 2, 1 rule,

so, So here's my question, Prasanna,

we.

Have been talking a lot about cloud outages and cloud disasters and, um,

so far they've all been like a cloud provider that is supposed to do the

thing and the, and in every one of the cases, if the customer had just

had a backup, they would've been fine.

So.

Right.

Right.

in, but the only thing I wanna, I

was wondering though, is OVH

W. Curtis Preston: A proper backup.

Yes.

W. Curtis Preston: You know, we, we rely on these backups

a lot and so today's story.

You know, the story that went live yesterday really gets me upset because

in that story, the vendor lied.

I.

To their customers.

They said that, you know, it's proven in court that, that the

vendor lied to their customers.

They told them that the server was, was in a fit, at least one customer.

We know that they specifically said, I want a server in another data center.

And they said, you got it.

And then it turns out that they didn't

have it.

Because contractual contracts and

operations don't always align

two separate people, two separate org

W. Curtis Preston: Yeah.

Yeah,

W. Curtis Preston: One of the things that we tell people to do is to,

is I'm a fan of Cloud-based backup.

I never would've guessed.

W. Curtis Preston: I think it's the killer app for the cloud.

I really do.

Right.

Which is why this story bothers me so much, and that is that we

have a story of a, of a backup vendor that, you know, lost.

Customer data

and it's, it's so frustrating from the story.

Right?

So

well, well it, I would say that it's not even just lost

customer data, but also the way that they've been operating isn't necessarily

what you would look for in a company that you're trusting your data to.

W. Curtis Preston: Yeah, there are multiple incidents of, um,

I'll call it everything from, you could call it lack of candor.

You could, you know, depending on how you want to characterize

it, some of it is outright.

I don't know if I'm gonna say lie to the degree that OVH did, because that

was, you know, proven in court that they purposefully misrepresented.

Um, in this case we do see them misrepresenting things, but, you know,

do we know for a fact, has it been proven in the court that it was a lie?

No.

But the activities that we see happening.

Weren't good.

Um, and so what are we talking about?

Prasanna?

are we teasing?

What are we teasing?

So the company,

W. Curtis Preston: a big

name,

A big name, right?

This is a name who, a, you know, say four or five years ago you couldn't go

to an airport without seeing this name.

It was everywhere,

right?

What are we talking about?

We are talking about the company called Carbonite.

W. Curtis Preston: Yeah.

Right.

It was a large company that recently was acquired or merged

and is now part of OpenText, uh, which is a Canadian based company.

W. Curtis Preston: Which is gonna figure into the story.

yes.

So maybe we should roll back things and just go way back

in time and talk about Yeah.

Mm-Hmm.

W. Curtis Preston: let's go way back in time.

So first, um, you know, I'm gonna bring up a story.

that It's not on topic, but it's relevant.

Okay.

A couple of months before the key story that we're gonna talk about happened.

Carbonite got busted by essentially a, a part-time, like internet sleuth, right?

It is just some blogger guy.

Um, and what they got busted, uh, doing was posting fake reviews on Amazon.

Right.

And the, the, this, this sleuth uncovered it, you know, he busted.

I, I remember, you know, I wrote a blog post about it back when this happened.

This was in, by the way, this was 2009.

And, um, this, this person found that, um.

Uh, that basically they had created, uh, a combination of like fake

accounts and, and in some cases just really poorly disguised accounts.

The, and all of the metadata pointed that, um, that these reviews that have been po,

these really positive reviews that have been on, on Amazon were in fact written by

either, um, Carbonite employees or by, um.

Uh, people related to Carbonite

employees, right?

And this guy, this guy uncovered it and, uh, he blogged about it.

And then New York Times covered it, uh, a blog called Pogue' s Post.

And by the way, this, this is one of the stories that's, it's a little

difficult for us to cover because some of the things that we're referring to.

ar Arnold all, they're not left like the original blog that started,

the investigation seems to be gone,

right?

Um, the, I don't know if the entire blog is gone or just the blog post.

Luckily the New York Times article is still around that refers to this.

But yeah, so they were posting, um, and, and they, what I remember was

that they tried to pass it off as nothing and then they, you know,

they just sort of got busted doing this and that, that just, and that.

Story broke just a couple of months before the big story that we're gonna talk about.

Any, any thoughts about that?

Like I just think back to like 2009.

W. Curtis Preston: Yeah.

in the days where like people weren't

doing it all that much, right?

Posting fake reviews, and you didn't have the smart

algorithms to try to detect this.

You didn't have all the sort of review farms or paid reviews and all those

things happening like you do today.

And so maybe they just thought, yeah, maybe we could just get away with this,

and like that was probably, they did.

They probably didn't think that there would be a risk of get being caught.

Right.

W. Curtis Preston: Yeah.

One of the people that was posting a, a a review was the vice

president of marketing at Carbonite.

Right.

Um, so, so that's sort of the, the, the

timing under which, this happened.

What's

by the, by the way, we're not saying it's bad to post reviews.

It's, there's nothing wrong with that, right?

We're just saying it's bad to post reviews, pretending to be other

people trying to give an unbiased or, or give an unbiased opinion.

W. Curtis Preston: Yeah, it is actually, um, you know, you can get in

trouble, like in the us you can get in trouble with the SEC if you do that.

Later,

W. Curtis Preston: Yeah, yeah.

Like the SECC will resurface in the story.

Okay, so,

so here's what happened.

2009, a couple of months after that story basically.

Um, it's, it's interesting, so I'll tell it not in, this is sort of like Star Wars.

I'm gonna tell it in the order that it happened, not in the order

that we found out about it, right?

Because how we found out about it is.

Beyond my comprehension.

So what happened was that Carbonite had outsourced the backup part

of their backup software to an IT vendor that then purchased.

Disc drives from a company called Promise and $3 million worth,

by the way, $3 million worth of hardware from this vendor.

They then had a, uh, was it a dual disc failure?

Is that that what is that, what

we, Yeah.

They had a dual disc failure in their RAID arrays, and as a result of that, they

lost the backup data of 7,500 customers that doesn't mean that 7,500 customers

lost the data that they cared about.

They lost the most recent backup.

And then what they were able to do is they, they, they

immediately restarted the backup.

And they're saying that what we now know is that 54 customers out of that 7,500.

The data that they were backing up was damaged in some way.

Their PC crashed or they

deleted it or something.

And then they went to go restore data and they were unable to do so because

the data that they wanted to restore was in that, that data that was lost,

Yeah.

W. Curtis Preston: um, which is.

Formerly working at a backup vendor, that number seemed high to me.

The, the number of customers that would restore 54 out of 7,500.

Right?

That's

0.1%.

Yes, but right.

That's just customers who encountered another error while before they

could, because who knows how long it took 'em to back up.

Right.

It might have taken them days depending on how much data they had, 'cause they

had to basically back up everything again.

W. Curtis Preston: Yeah, I, I guess I'm just saying that, you know, knowing

what I knew about, about our former employer, people just don't restore stuff.

They just don't do a lot of restores.

I would be

surprised,

Prasanna Malaiyandi: unless they needed to.

though.

Right.

And in

these cases, yeah.

yeah.

So, so, and that's their hope, right?

It's that they just hope that nothing bad will happen while everything

is getting backed up again.

And of course, for those 54 people.

W. Curtis Preston: Yeah,

Did

W. Curtis Preston: it's sort of like how you hope that another disc drive won't

fail when you're only using raid five.

Um, you hope that that won't happen while you're rebuilding.

So having worked for storage companies in the past, I am

W. Curtis Preston: Yeah.

but not surprised that they were not able

to handle a double disc failure.

Um, but.

It is a little crazy that they built a system like that and where it's

like, yeah, this is good enough.

Um

W. Curtis Preston: Let's get, let's get to that.

Let's get to that in a minute.

I wanna ask you a really big question, Prasanna.

huh.

W. Curtis Preston: How did we find out about this?

Oh, so it was published in some articles right after

the fact, but the only reason that this was even picked up by the news was because

Carbonite decided to sue the vendors.

W. Curtis Preston: Yeah,

breach of contract and breach of warranty and

W. Curtis Preston: the IT vendor and the storage vendor.

Yep.

In order to recoup some of the costs.

And that's where all this information came out as they were seeking damages.

W. Curtis Preston: Yeah.

And I remember being, you know, like backup guy at the time, and

I mean, this is what, 15 years ago

and.

I remember like to the day, it's like 15 years ago and I re my fir.

What's interesting was timing wise, 15 years ago I had my

first Mac that had Thunderbolt.

Hmm.

W. Curtis Preston: Okay.

Why does, why does that figure into the story?

Here's why.

At the time I was running a company and I needed, I needed a decent sized

storage array for my, um, by the way, the the computer I'm talking

about is literally right over there.

It's literally sitting over there.

It's a, it's a I, um,

what, uh,

Hi Mac.

W. Curtis Preston: The, yeah, the I, the new imacs,

right?

And it had Thunderbolt and we were doing video editing on it, and we

needed a nice sized array to buy it.

And the only arrays that you could buy at the time that were

thunderbolt were promise arrays.

And when you assert at the time, the promise arrays, they, they

were, I mean, they were great.

I, you know, I had it.

I actually still have the promise array somewhere, and.

But the thing was that they were also known for being the cheapest

arrays that you could buy at the

time.

So when this story broke, I.

I, I I sort of realized a bunch of things.

I'm like, wait, this big cloud backup company is using the cheapest storage

arrays that, and they're just using like regular old storage arrays.

Not like good ones.

They're not, you know, they're like, they're supposed to be the cloud.

They're supposed to be, they're supposed to be using advanced technology and,

and, you know, so, so there was that.

The other thought I had was.

Wait, they just have like one comp, they just have one

copy of each customer's data.

And it's not even using RAID dp, it's not using mirroring, it's not using RAID dp.

So, so all it takes is one double disc failure

And and you're done.

And, and then the other thing was I, I said to myself.

Why, if this was me first off, I wouldn't have, I, I don't think

I would've made that decision.

I think

I would've done something very differently.

Second, if I had made that decision, it's the, I'm gonna publicly

announce to the world that I made that decision and like, what, what

was going through anybody's mind?

It

seems like a big PR

snafu.

W. Curtis Preston: big, PR snafu, a bad business decision to buy

the arrays in the first place.

I'm not saying that promise arrays are bad.

I will say that having that much data without having RAID DP bad,

right, that much data without having mirroring or something like

that, that where a double disc failure just takes your data out.

Double disc failures happen all the time.

You, you, you used

to work at a company that did this?

That's why.

So there is a quote, and I don't know if we, yeah, and I will cover it now.

Uh, where basically, uh, the CEO David Friend

W. Curtis Preston: at the time.

he's no longer the CEO.

He was the Yeah.

Yeah.

So he had explained sort of, okay, this is why they had crashed because we were using

Promise Technology and we're using RAID, and we suffered as single disc failure.

So he said that we switched to a popular Dell server that uses RAID six.

That allows a loss of three of the 15 drives simultaneously

before you lose any data.

The configuration is, in theory, 36 million times more reliable

than a single disk drive.

The chances of three out of 15 drives failing at the same time are almost dill.

Now, having worked at multiple storage companies, I want to call,

uh, BS on that, nearly nil, right?

W. Curtis Preston: On the three drives, failing at the same time.

Yeah, having worked at companies, especially given

the size of discs these days, what I've experienced is that during the

times when you're actually doing a raid rebuild, that's actually when you're

putting additional stress on all the

other discs.

As you're doing a whole bunch of reads, recomputing parody, in order to write

out and recover the new disc, that's actually the time in which those

other discs, if there's any weakness in them, that's when they fail.

W. Curtis Preston: I feel a story coming on Prasanna.

What's the story?

W. Curtis Preston: The story is my oracle, the, so this is the

first time, so I, I, I, I left.

I was no longer like an it.

I.

Regular IT practitioner.

I was now a consultant and I was at this major oil and gas company and

I was just supposed to be the admin.

I wasn't supposed to be the backup guy, but I went around and I've seen

that the backups were just broken.

And, you know, and I, and, and one of the servers that really bothered

me was, it was, it was a Solaris.

I remember it was running Solaris two three, which at the time was a

seriously buggy version of Solaris.

And, um, I was trying to get the first ever backup of this Oracle database,

which was a three 300 gigabyte,

three oh

gigabyte Oracle database.

Massive, huge, right?

And, um.

It was what was called a DSS decision support system.

It was all the rage back in the day and it was the kind of thing that took

imports nightly from the mainframe.

And, um, it had never been backed up, ever.

And I had to argue with the DBA that um, he didn't want me to do.

He didn't want, he didn't want me turning on archive log mode because AR archive

log mode corrupted Oracle databases.

That was his

stance.

Yeah.

W. Curtis Preston: I'm like, is that the stance of Oracle or is that just

some random thing that's in your head?

And it was the latter.

Um, and then I, um, and he, uh, he said, well, if it doesn't corrupt it, it

cau, it causes performance, uh, lower, you know, it lowers the performance.

And, um, I was like, well look, we've gotta get this

thing backed up at least once.

And so either I shut down the database and back it up one night,

or, which they didn't want to do 'cause that's when they did the night,

the nightly uploads, or I need to put it in archive log mode

at least for a couple of days.

So I can do a hot backup of it.

And um, and then I went to go, I, I went and that's when I looked at

it and I saw that not only was it Solaris two three, but it was Solaris

two three, completely unpatched,

right.

It, it, it had been like a couple of years

one wanted touch

W. Curtis Preston: Patches and, and so I, I knew because I had already

encountered it at this client, that if I put the backup software that I was using.

The features of Solaris that it used during a backup would cause it to crash.

If I kicked it off, I knew I had to put on the latest jumbo patch of

Solaris in order to do the backup.

And so, uh, I did.

And uh, I got it rebooted.

And this was like a Friday afternoon and I got my first ever backup

of this 300 gigabyte database.

And then over the weekend, five drives failed.

Right.

Should never have rebooted the server, Curtis.

W. Curtis Preston: yeah.

Yeah.

So, um, so I came in Monday morning and, and basically the, the drives failed.

I got, and by the way, it was the only an additional wrinkle was this was a place

who didn't buy Sun Service from Sun.

They bought like pieces

and

parts

and.

put it together themselves, and then,

W. Curtis Preston: Yeah,

yeah.

But this server was from Sun and so I was able to call sun, get sun in

over the weekend, replace the drives, you know, rebuild the file system,

restore the database, and it was up and running come Monday morning,

and I was cock of the walk, baby.

Right?

And then this, this, um, this, uh, this DBA comes up to me

and he sees me and he's like.

I bet you think you're hot shit, don't you?

And I'm like, yeah, yeah, I kind of do.

You know, I'm feeling pretty good about myself right

now.

And he said, well, I'll tell you what, you don't get any credit.

I was like, why is that?

He goes, well, he goes, I'm not exactly sure what killed the server or those disk

drives, but I think it was your fault.

He is like, I have a couple theories.

One is you did this backup and it exercised the disc drives more

than they were used to being used.

And so that's what caused the thing, uh, to fail.

Or it was, uh, the fact that Oracle was in, uh, arch archive

log mode and that's maybe what caused it or the fact that you put in that

big jumbo patch, um, and one of, one of those things killed the disc drives.

And so since you killed the disc drives you get no credit for, for saving us.

I was like, okay, whatever dude.

So

yes, multiple disc drives in this case, five disc

drives can fail all at once.

This is why we back up.

This is why we do mirroring.

This is why we do all kinds of, I, I guess it just really disappointed

me at the time, and so it, it was really surprising that a company who

this is what, this is what they do.

They, they were just using the same RAID arrays, you know,

that I'm putting on the back,

on the back of my iMac.

I was very disappointed.

Um, I, I, I spoke a lot there,

but it just bugged me when it happened.

What

yeah.

No, I, I agree with you.

It's, and the challenge is their marketing messages and everything else at the time.

Right.

Portrayed it as, uh, high quality.

I.

Backup solution for customers.

So they don't need to worry about backup and everything else that

that comes along with it, right?

They're like, yeah, we will deal with everything for you.

And when customers put their trust and their faith in that, uh, the

company didn't quite meet the bar.

W. Curtis Preston: Yeah.

Yeah, and, and I guess it's just.

Even to this day in the subsequent articles that came out after the original

article, when the CEO David friend was going to the people say, Hey, you know,

I just wanna correct the, this, you

It's not 7,500.

Yeah.

W. Curtis Preston: it wasn't 7,500 customers that lost data.

It was 54 customers lost data because only 54 customers couldn't restore.

Right?

But still in those further comments, he was still like blaming, promise.

Like he even, you know, um, and by the way, I do wanna say I was

able to find the, the, docket, uh, the actual filing and , the

lawsuit was settled outta court,

right?

So there's no, there's no public record of what the actual thing is,

but I highly doubt that, um, you know, they're a storage array vendor

disk drives fail and sometimes monitoring stuff fails and.

You know, I, I, I just, I could not, for the life of me fathom the idea of, of

suing your, your storage array vendor.

It's a little bit like when Musey sued

Google.

because they deleted their own data.

Right.

Um, yeah.

I, I just don't understand that.

So there was an article from Backblaze, uh, which for the record

is a competitor to Carbonite, but I still agree with their point.

And they were saying that they felt that the coverage of this.

Incident missed.

One of the crucial points of this story, and that is that a backup vendor

outsourced the backup and, and they made a, you know, that they, that

they made a point of saying that, that it's like Google Outsourcing search.

Right.

And, and I, and I have to agree with 'em, right?

They, they weren't saying they were reselling backup.

They were, they were saying they were a backup vendor, and you would

think that they would deal with that.

Right?

Um, and, uh, at the time, the, the response to that was that

the CEO said that they now, now, write their own software.

Right.

So, I don't know.

The, um, yeah, in, in that blog it says Building robust online

backup technology is difficult.

There's certainly lots of complexities involved to ensure data is

backed up, redundant and secure.

It is the role of the online backup service provider to have

the technical expertise and laser focused to work through these items.

Pushing it off to an outside company seems a bit risky.

Yeah.

Well, and the fact that we started with this right as the 3, 2, 1 rule.

They were not doing the 3, 2, 1 rule at all, you know?

W. Curtis Preston: yeah, yeah, I mean, you know, there are, there

are probably a bunch of vendors out there that, for example, are relying

on, you know, backup vendors that are storing only one copy on S3,

right?

Yeah.

Now as three stores, three copies.

But you still are, you still are relying, but in that case, you're relying on what

arguably is like the most tested storage array service in the world, right?

Yeah, exactly.

W. Curtis Preston: Yeah.

Uh, but as opposed to promise, which was a little known

product that, that I'm willing to bet.

The reason they bought 'em is they were the least expensive.

That's

why I bought 'em.

Yeah.

W. Curtis Preston: That and the fact that they did Thunderbolt at the time.

It was really hard to get Thunderbolt based storage array

I think at the time it was probably OWC and Promise.

W. Curtis Preston: Yeah.

Yeah.

And OWC was probably more expensive, so I

didn't buy it.

Um, so what happened after that story Prasanna, which I didn't even know

about until we were researching this, this story until you brought it up.

So apparently I came across while looking

for lawsuits against Carbonite.

Uh, it looks like they just settled one, um, in January, uh, 31st of 2024.

So a couple of months ago from this recording, uh, for 27 and a

half million dollars apparently.

So Carbonite was acquired in 2019 by a Canadian, Canadian

company called OpenText,

W. Curtis Preston: So five years ago

they were acquired.

Yep.

But before they were acquired, Carbonite apparently had made some statements that

misled investors and they were releasing a new backup solution, uh, called

Server VM edition that they promised was super strong and extremely competitive.

That launched in 2018 October, and the company said the product would

allow businesses to recover virtual machines data from a single location.

And then they got sued by investors because they said the product, it never

worked and that it never once successfully backed up a customer's data and that

the executives basically knew it did not work and still made those statements.

And so this was just settled for 27 and a half million

dollars to resolve the claims.

W. Curtis Preston: Yeah, it's interesting.

So they accused the company of violating the Securities Exchange Act of 1934.

After this news came out and the stock price fell more than 24% the next day,

uh, the judge initially dismissed the case, but the court of appeals reversed

'em in 2021 and, um, and certified the, the class as a class action.

And then it looks like they finally settled it,

um, and agreed to pay $27 million.

That's a lot of money.

Yeah.

W. Curtis Preston: I wonder how many promise arrays that'll buy.

Sorry, that was, that was, that was just mean.

Um, I, yeah, I don't, I, you know what I, what I used to, what I used to say

about Carbonite was, you know, just Google, like if you're, when you're

looking, when you're looking at vendors, just Google their name and lawsuit.

'cause at the time, Carbonite was the only one that that would come up with.

With results.

What's interesting here, like, it's sort of like with with OVH,

why in the world would anyone use.

The, a vendor, why would they go public?

Why would somebody invest?

And it's because these stories, they just go under the, under the

radar.

I think a, I think a, a, you know, an IAS vendor like OVH, who

showed that they don't know how to build data centers that they

they

went IPO.

W. Curtis Preston: Didn't, yeah, they didn't do fire suppression.

They didn't do, um, you know, I Why would you, why would you ever

want to do business with them?

And I have a similar feeling.

This is just my feeling, my opinion, if a company is so, so,

two incidents of clear misjudgment.

One in terms of the original design and outsourcing to another vendor, and

then the second, the decision to, to make this public by, by blaming the

vendor.

What that, what that says to me is you take no blame yourself.

If, if they had done a mea culpa, if they had said, Hey, uh, we really screwed up.

We shouldn't have trusted everything to this vendor.

Uh, maybe even trashed him.

I don't care.

Right.

Now that I think back on it, that's the part that bothered

me the most in the, in the story

is that to this day, they take no responsibility for that.

Uh, they also, they also take no, they know, um, there's no, um, wrongdoing.

They admit no wrongdoing in the SEC, uh,

the settlement.

Yep.

W. Curtis Preston: Uh, at least they're consistent.

Prasanna Malaiyandi: So, yeah, so I, I think.

Like you said, this is useful as you're, because I don't

think people actually do this.

So when you're searching for a vendor, we always talk about go read reviews,

go talk to other people, look at, uh, analyst results and or analysts

opinions and things like that.

Look at like the IDC surveys and, uh, the Magic Quadrant from Gartner.

I don't think that we've told people, go Google the Vendor plus lawsuit.

I.

You know,

W. Curtis Preston: Yeah.

go look and see what has come up.

Like is this a company that you are comfortable doing business with

and trusting that they will stand beside you when things go wrong?

If things go wrong?

W. Curtis Preston: Yeah.

And, and I think it's important to say that I have no ax to grind with

these, with these folks, right?

I, I don't work for a competitor.

I don't, they didn't personally wrong me.

Uh, they've never discussed me publicly that I know of.

It chaps my hide.

It just the idea that that.

You accepted no responsibility for that.

And then again, apparently even up to recently, it, it appears

that you were doing stuff, um, that seems a little shady,

Yeah.

Now granted that was 2018 when they made those statements, but.

W. Curtis Preston: That was not that long ago, man.

That was just before Covid.

That was like, I was like, yes.

Covid iss like a day like Covid, COVID.

The entire period of covid counts as like a day.

I, yeah, that was 2019.

Yeah.

Now the one thing though that we have to state is since the

acquisition by OpenText, things could be different, right?

W. Curtis Preston: that is true.

So I think we should at least give 'em that benefit

of the doubt that things could

W. Curtis Preston: That's actually a really good point.

Thanks for bringing that up.

You know, we're, we're commenting mainly about the company that was.

Carbonite before it was acquired by OpenText.

And perhaps this acquisition of OpenText will significantly change things, uh, or

has already significantly changed things.

This is really what, what we need to, um.

When I think about like the OVH situation, what can we learn from this?

What can people do to protect themselves from this when

using a cloud backup vendor?

I would, I would number one, be asking them what kind of

storage technology they're using.

Right.

I would

ask about the availability and redundancy,

W. Curtis Preston: Yeah.

Um, you know, are you using storage arrays or are you using, are you using S3?

Are you

using, you know, what, how are you storing my data and how are you making

sure you know that it stays available?

I, I, think though, as a user, maybe what

underlying technology is less important because I could see some vendors

maybe would not discuss that I.

But at least ensuring the characteristics that you expect of S3.

So for instance, is it geographically spread out?

Right?

Is it redundant?

Is it a replicated copy, independent copy, right.

Those sort of things.

W. Curtis Preston: I understand what you're saying.

They might not want to say, but like, you know, where we used to work,

we just say, yeah, it's, it is on S3.

Right.

If you could say that,

I think that gives you a certain amount of, right.

Just like you just said, is this designed to survive, um, you know,

a fire, like what happened at OVH?

Is this designed to survive?

Um, you know, a multiple disc failure.

What happens in a mul, in, in, in a, in a catastrophic failure of,

of, you know, because let's just, let's just, let's just say this.

'cause here's the thing.

A single fire would've done the same thing.

Yep.

W. Curtis Preston: And then who are they, who are they gonna sue then

their fire extinguisher company.

Right?

What the core problem here is that they had a single copy of customer data.

yep,

W. Curtis Preston: That's it,

right?

No redundancy.

No.

No geo redundancy.

Prasanna Malaiyandi: Or if they had Acho go

W. Curtis Preston: redundancy.

A derecho.

You love talking about that

derecho, don't you?

I love talking about the DRE show.

It's an

W. Curtis Preston: I never

even heard of a derecho until we

talked about it on the show.

Exactly.

W. Curtis Preston: Yeah, it's a hurricane that starts over land.

For those of you that don't know what a derecho is, and don't ask me why.

It's the same as the word or really close to the word for right.

In Spanish.

Um, any, can you think of any other, you know, things that

a, that a customer can do to,

Because you know, the, you know what this reminds me of?

You know, there's, you know, you remember the old thing of like, on the

internet, nobody knows you're a dog.

Right.

It's like what can a customer do to ensure that they don't go to

essentially a another vendor that behaves the way Carbonite did back

in 2009?

I think another thing you should do, and it's not

a guarantee, but at least gives you some protections, is take a look in

their SLAs and in the contract, right?

Do they talk about disaster recovery redundancy?

Do they talk about SLAs?

Do they talk anything about geographic separation for storage?

W. Curtis Preston: Right.

Right?

If it's there, that's good.

If it's missing, ask the questions.

W. Curtis Preston: Yeah.

Yeah.

I think ask as many questions as

Prasanna Malaiyandi: yeah, it's not foolproof.

as we found with, like we found with OVH.

Right?

It was written in the contracts, but it still didn't mean anything.

But at least it gives you a more legal standing for if there is

a lawsuit that comes from this.

W. Curtis Preston: Yeah.

Um, and, uh, I, I hope, I hope the audience has appreciated this.

This is our best attempt to, to deduce.

I'm looking up here at my browser and there are like 17 tabs open to the story,

to the different story, all of which we're trying to cover the same article.

And, and one of the reasons for that is that the, the,

the original article is gone.

The original article in the Boston.

Uh, globe.

Is it that original article is gone and I couldn't, I also couldn't

find it in the internet archive.

I tried doing that.

So we had to look at a whole bunch of different blogs that were, uh, essentially

trying to summarize the same article.

But, um, you know, this is our best attempt at trying to,

to see what we saw from all

these various

and, and it may not be a hundred percent accurate, right?

We're just going off the information

W. Curtis Preston: Based on the articles that we have.

Yeah.

Yeah.

Yeah, All right.

Well, it's been not fun.

It's not a fun one.

I don't like it.

This is

my world and I This is, this is, this is people that lost data

it's,

W. Curtis Preston: that did what we told 'em to do, and they still lost data.

so it's not fun, but I think it is hopefully

eye-opening for folks that they need to be more inquisitive and

not just blindly trust vendors.

W. Curtis Preston: Yeah.

Never blindly trust vendors.

Yeah.

W. Curtis Preston: Yeah.

All right.

Well, uh, thank you Prasanna for your usual

analysis.

I, I enjoy these.

I actually like this series, Curtis.

I hope our listeners do as well.

W. Curtis Preston: I.

Yeah, I'm, I'm enjoying the series as well.

It's been nice to go back to these stories that I remember from 15 years ago, and

then we're, you know, in some cases, like the OVH Fire, we're seeing some follow up.

This one, and in this one we saw some follow up we didn't expect

from quite a bit later, which technically unrelated, but you know.

Anyway, um, I, you know, listeners, we appreciate you.

Uh, you know, we'd be nothing without you and, uh, be sure to subscribe

so that you don't miss an episode.

That is a wrap.