W. Curtis Preston: Spoiler alert, the company in this episode ceased to exist
due to what happened in this story.
Disasters happen, and since the cloud is just someone else's data
center, they sometimes happen in the cloud, and sometimes they
take companies along with them.
This episode is the first in a series called Cloud Disasters.
Each episode tells the real story.
Of a company who failed to back up their cloud data and suffered as a result.
And these aren't podunk cloud vendors either.
Every vendor covered in this story is a major vendor.
I'm not kidding around when I say you should back up your cloud data.
I don't care that there are those who think I'm just pedaling fud.
Newsflash, the only reason we've ever backed up anything is because
of the fear of losing it and the uncertainty and doubt we have in
the system that we're backing up.
Fans of the podcast know, the whole reason that I became a backup specialist is that
in 1993, I was unable to recover a very important Oracle database for my employer.
I didn't want that to happen to anyone else.
So I've dedicated myself to helping others protect themselves
from feeling that awful feeling.
This show is aimed at you, the unappreciated backup admin, and we want
to turn you into a cyber recovery hero.
This is the backup wrap up.
Welcome to the show.
I'm your host, w Curtis Preston, and I have with me my continued and necessary
Tesla consultant Prasanna Malaiyandi.
How's it going Prasanna
persona
Prasanna Malaiyandi: Pana.
I am doing well Curtis, and how have you been?
W. Curtis Preston: I, well, as you know, I have been.
Fiddling.
Prasanna Malaiyandi: Uh, why am I not surprised?
W. Curtis Preston: I have been, fiddling with the automations of my Tesla now.
Tesla's come in, I'm gonna say two battery flavors, right?
There's NMC and LFP.
Nickel Manganese Cobalt and Lithium Iron Phosphate.
And I have the latter.
And the latter is not supposed to be subject to the same don't
charge it to a hundred percent unless you need it right away.
Issue.
Uh,
Prasanna Malaiyandi: In fact tell
you to charge it to a hundred
W. Curtis Preston: I do tell you to charge it to a hundred
percent at least once a week.
Um, but I was charging it to a hundred percent every day.
And I was thinking that I, that even though it doesn't have necessarily
the same issues, I could still get better battery life by not charging to
a hundred percent unless I needed to.
Uh, which, uh, for me is not all the time.
And so, but I'm also very absent-minded, and so I couldn't like lower the
percentage and then remember to, to, to, to raise the percentage later.
And so, uh, I found this wonderful app called Tessie, and I've been obsessing
over it for about a week or two.
Prasanna Malaiyandi: A week.
It's been nine days.
W. Curtis Preston: Yeah.
Yeah.
I will say if there's any listeners that are a Tesla owner, I.
Uh, you should just get Tessie, uh, T-E-S-S-I-E in the app store.
I bought it for the automation.
What I also got was this immense amount of analytics and reporting
and, um, all this great, great stuff.
And also reminders.
That's another, speaking of being absent-minded, it will tell me, hey.
Idiot, you're home and your car's not plugged in.
Prasanna Malaiyandi: Yeah, because how many times has that
happened to you, Curtis, that you've come home and you're like,
W. Curtis Preston: you know, a couple, uh, enough that it was annoying.
Uh, it, I will say it, nothing is more annoying than, you know, basically
driving your car down to, you know, the electrical equivalent of fumes.
And then, um, and then getting up in the morning and going,
okay, I'm ready to drive today.
And you're like, oh, crap, I gotta go to the.
I gotta go the supercharger for 20 minutes before I could do anything else.
Um, yeah, so it's got, you know, it's got that the notifications, it
reminds me to, to rotate my tires.
And also gives me analytics about my driving and my efficiency and, yeah.
So, yeah.
So happy, happy, happy, happy, happy.
Um, but you know, who's not happy?
The people that are gonna be in this new series, they're not happy.
And this is a new series.
One of the things that you've heard us say is what Bana.
Prasanna Malaiyandi: Back up the cloud.
W. Curtis Preston: Yes, backup up the cloud.
The cloud is not magic.
There is no such thing as the cloud.
It's just somebody else's computer.
Uh uh, the cloud doesn't magically back up itself.
Despite what you may have been told the cloud there are parts of the cloud where.
People generally agree with me.
I, I think probably the best example would be something like AWS, you know, like EC2.
They're like, yes, we know EC2 needs to be backed up.
Um, and, and then we just argue a little bit over how that's going to be done.
But when we get to the extreme it, the other end of that, we get the
SaaS world, we get the Microsoft 365 lovers who say, oh, this, that,
this doesn't need to be backed up.
you know, or G
Suite and, we have gone back over the last, uh, several years and selected, um,
a number of, you know, poor victims of.
This belief and, um, I, I can think of no better, uh, story to start this
out than code spaces because it, I, I think it was kind of the first,
Prasanna Malaiyandi: it was probably the first that was sort of well
publicized as well as having very dire consequences for not backing up the cloud.
I.
W. Curtis Preston: Right.
And it, it's interesting from a timing perspective, it happened in 2014.
Which to me is the year of the beginning of the massive level of ransomware.
Now, I know ransomware actually goes way back longer.
You know, much longer be before that.
But to me, 2014 is really when I started seeing ransomware kind of everywhere.
And this was technically a ransomware attack, not in the traditional sense
that we think of today, but, but it was, so Code spaces.com was a site, and this
is the, the irony of all ri ironies.
And, and I will say that unlike Alanis Morissette, I actually
know what the word ironic means.
Um, I love the song.
It's just, there's so many things to that song that are not in any way ironic.
They just suck.
Um, code spaces.com was a site to store your code.
It was like, um.
Prasanna Malaiyandi: GitHub.
W. Curtis Preston: It was like, it was like a GitHub and they had
many, many customers and it was a safe space to store your code.
Hence the name code spaces.com.
Prasanna Malaiyandi: Yeah, so basically like you mentioned, it was a place
companies could store their code and this was way back in the day.
And so it's like, hey, if you just have your own code sitting locally
on your system, because not everyone was comfortable with the cloud, they
offered a service that allows you to store your code there and keep it safe.
And now according to a cash version of their website, because of course
you can't find anything anymore about them, uh, they had over 200 customers.
A week or 200 companies a week using their service, which isn't
like small beans, right?
And yes, it's not like the thousands of millions, but there are 200
customers who now no longer have access to their code anymore because
of what happened in code spaces.
The other interesting thing is according to their websites, and I will quote
backing up, data is one thing, but is meaningless without a recovery plan.
Not only that, a recovery plan and one that is well practiced and
proven to work time and time again.
Code Spaces has a full recovery plan that has been proven to
work and is in fact practiced.
W. Curtis Preston: That
Prasanna Malaiyandi: do you think about that, Curtis?
W. Curtis Preston: That sounds really good.
Um, you know, based on what we know happened, clearly they didn't test for
Prasanna Malaiyandi: all scenarios.
W. Curtis Preston: Right.
And they specifically didn't test for cyber attack.
Um, but.
Prasanna Malaiyandi: the time could you fault them really?
W. Curtis Preston: I don't know.
Well, they had poor backup design.
Just, you know what, what we're
gonna get to, what we're gonna find out.
They had full backup.
They had poor backup design.
They failed to follow what persona?
Prasanna Malaiyandi: The 3, 2, 1 rule.
You know, we haven't talked about this in a long, long time.
Do you wanna quickly mention it to some of our
W. Curtis Preston: Yeah.
So the 3, 2, 1 role, and by the way, many companies have said
it's gotta be more than 3, 2, 1.
Yes, I agree.
Right?
These days it has to be more than 3, 2, 1, but if it's not 3, 2, 1, there's no point
in talking about the other ones, right?
Having at least three versions on two different media, and the, the,
the idea here is on, on things that are subject to different.
Risk profiles, right?
Maybe it's disco and tape, maybe it's, uh, on-Prem and off-Prem.
Maybe it's, you know, on-prem and Cloud.
Maybe it's, um, you know, a different region, et cetera.
And then one three, the one is make sure that there's something off site.
Um, they had neither the two nor the one,
but we're gonna get to that in a minute.
Prasanna Malaiyandi: and, and for those listeners who wanna know more,
we actually had an episode with the person who coined the term 3, 2, 1, who
comes from digital photography, in fact.
And so you should go take a listen to that episode.
We'll
W. Curtis Preston: Yeah, we'll put.
Yeah, we'll put a show notes.
Peter Krogh.
Um, great guy.
So what happened?
Well, we have that a hacker gained privilege credentials that we still
don't know how that happened, right?
They're saying probably through phishing or possibly through, uh,
stored EC2 access keys in a public code
repository.
Prasanna Malaiyandi: Which happened a lot back then and still does today.
W. Curtis Preston: Yeah.
Yeah.
The, um, there was a, uh, security researcher at Tripwire that said, that
this is a problem for people how to, how to, manage authentication codes like this.
And, uh, they said they, they had seen thousands of EC2 accounts
abused after storing their EC2 keys in public code repositories.
Ouch.
Right.
Um, but we don't, so we don't know.
Someone
got access.
Yeah.
Prasanna Malaiyandi: Would it be ironic if Code Spaces was using code spaces
to store their code and they left a public repository with their EC2 key?
W. Curtis Preston: That wouldn't indeed be ironic.
Um, but yeah, so we don't know exactly how this hacker, uh, this,
you know, bad actor got access to the, to the environment, but they did.
And the first thing that they did was they started a DDoS attack.
Uh, what is a
DDoS attack persona?
Prasanna Malaiyandi: This is a DI distributed denial of service, so
you have a bunch of servers outside hammering various servers at a company.
Causing it, flooding it with a lot of traffic, which then
causes it to stop responding.
So if you're code spaces, someone did a DDoS attack on you now, you wouldn't
be able to serve and function as a service to those 200 paying customers.
W. Curtis Preston: Yeah, the really interesting thing about this is that the
DDoS attack was apparently subterfuge.
Right.
Um, you know, look over here
while I, nothing up but
Prasanna Malaiyandi: which I think is still common today, right?
There are still a lot of companies who they're trying to hide their tracks
and they're like, Hey, everyone's gonna fight, and DDoS were common.
And so they have a plan in place and everyone's scrambling there where you're
like, Hey, look at my left hand while I'm doing something with my right.
W. Curtis Preston: And then I, I think the, I don't know if interesting
is right, but the hacker left contact details for themselves in
the customer's Amazon dashboard.
Right.
Uh, this was, this was, I, I think, I think maybe we haven't mentioned it.
This was an AWS customer.
Prasanna Malaiyandi: And this is why we call it ransomware, right?
Even though it's not your traditional ransomware like we
think today, it more or less is.
W. Curtis Preston: Right.
Yeah.
So then, uh, I'll quote from there.
There was a page, by the way, there was a page.
Uh, basically code spaces.com died after this, and it became a, they,
they replaced it with just one page that says, here's what happened.
And we're, I'm gonna read quotes from that page that are no longer
available because they sold the domain.
Um, upon realizing that somebody had access to our control panel, we started
to investigate how access had been gained and what access that person
had, uh, to the data in our systems.
It became clear that so far no machine had access had been achieved due to the
intruder not having our private keys.
Um, that's what they thought, right?
Um, but things
turned, uh, yeah, turned ugly.
Uh, why don't you talk, talk about
the next one there.
Prasanna Malaiyandi: So what was the response?
So Code Spaces did a smart thing and one of their first response.
Actions was to change all of its EC2 passwords, but quickly, code
spaces discovered that the attacker had created backup logins, which any
sensible person's going to do, right?
You're never gonna say, Hey, I'm only gonna have one admin in my environment.
And so the attacker was able to create all these backup logins, and so now they
were able to just go back into the system.
And continue doing the attacks.
And once they realized that Code Spaces was trying to actually recover and take
control from the attacker, the attacker then started to go and just delete things
from the control panel because they had super user access at that point.
Right.
So they could do whatever they wanted.
W. Curtis Preston: Yeah, somewhere in here, in the version of the story
that I have, I remember there was a d, there was a, there was an attempt
at a ransom, basically give us this amount of money, or, or, or we're
gonna, you know, do bad things.
Um, but the, but you know, they, they obviously didn't want to pay the ransom.
And then, uh, then what
happened?
Prasanna Malaiyandi: And so finally code spaces got their control
panel access back, but not before.
The attacker had caused quite a bit of damage, so the attacker had gone
removed all the EBS syn snapshots, S3 buckets, all the amis, which are the
Amazon machine instances, some EBS instances and several machine instances.
And this is a quote from the same webpage that Curtis was talking about.
In summary, most of our data backups, machine configurations
and offsite backups were either partially or completely deleted.
W. Curtis Preston: What I, I, you know, I did.
I never noticed that phrase before.
And they said And offsite backups.
What?
Offsite backups.
What I mean, what be, because the hacker only had access to
the one account.
Prasanna Malaiyandi: I am guessing that they replicated
to another AWS region within the
W. Curtis Preston: Okay.
Okay.
Another zone within the
same account.
All right.
Prasanna Malaiyandi: Or or another region.
Yeah,
W. Curtis Preston: Yeah.
yeah, But still within the
same account
Prasanna Malaiyandi: Possibly within the same account.
W. Curtis Preston: Yeah.
Um, yeah, yeah, possibly you're right.
We don't know for sure, but if it wasn't the same account, then the hacker had
to gain access to multiple accounts.
Right.
Prasanna Malaiyandi: Yeah.
Yep.
W. Curtis Preston: Um, yeah.
So basically this is the equivalent of blowing up somebody's data center, right?
Um, because basically they just.
In a matter of a few keystrokes, they just deleted essentially everything, right?
Everything that mattered, or enough things that mattered that they, um,
you know, uh, took out the company.
Prasanna Malaiyandi: One of the things that I don't know if you found any
information about Curtis is did they ever reach out to law enforcement
or even AWS's security operations to be like, Hey, I have this issue.
Can you help me?
W. Curtis Preston: Yeah, we don't, I, I, I'm going to.
Assume that once you know, the feces hit the rotary oscillator,
I'm sure they called AWS.
I mean, of course they called AWS, but what we know is that
right, if you know AWS isn't magic, and if you didn't follow the architecture
and do the things that you were supposed to do, A-W-A-W-S can't
undo that.
Prasanna Malaiyandi: Yeah, well, I was just wondering before it changed, its EC2
passwords, if they had reached out to AWS, if they could have helped them in some
way or been like, Hey, here are the best practices for locking down your account,
W. Curtis Preston: Yeah, I'm, I'm gonna guess based on how they responded
that they did not do that right.
Um, I don't think their response was the best thing they
could have done at the time.
Um, so then we have, um.
So here's again, I'm reading from their co their, from their quote
here, code spaces will not be able to operate beyond this point.
The cost of resolving this issue to date and the expected cost of refunding.
Customers who have been left without the service they paid for, we'll
put code spaces in an irreversible position, both financially and
in terms of ongoing credibility.
No kidding.
Um, as such, at this point.
We have no alternative but to cease trading and concentrate on supporting
our affected customers and exporting any remaining data they have left with us.
Prasanna Malaiyandi: Ouch.
W. Curtis Preston: That is, um, that's a tough one.
So basically, you know, hacker gets in Hacker, uh, you know.
Offers are ransom.
They try to, instead of paying the ransom, they try to lock
the hacker out unsuccessfully,
Prasanna Malaiyandi: Yep.
W. Curtis Preston: and then the hacker deletes the company, the uh, right.
I mean, they deleted basically everything, you know, as much as they could get
access to in that, you know, in that
account.
Prasanna Malaiyandi: That's crazy.
W. Curtis Preston: Yeah.
Yeah.
This is one of those where it's like, I, I always used this story to
recommend backup design for the cloud.
Even though I don't think I've ever encountered someone who says,
well, I don't need to back up EC2, I don't think I've ever heard that.
Prasanna Malaiyandi: Yeah.
W. Curtis Preston: I have very commonly found people whose backup design was very
similar to this company's backup design.
And, um, so, well let me ask you this.
What do you think, um, what could they have done differently to stop this?
To prevent this from happening.
Prasanna Malaiyandi: I would say the first could have been take your
backups to a different account.
In a different region, or even in the same region, that would've
at least kept your data safe.
W. Curtis Preston: Yeah, I, I, yeah, I, I completely agree with you.
I would probably just say as long as you, as long as you're, I, I guess what.
I'm, I'm trying to, I'm factor, I'm thinking in my head, like,
from a cost perspective, does it cost extra to send to another
region?
Is that,
Prasanna Malaiyandi: Yeah, usually it does.
Yep.
W. Curtis Preston: Okay.
So
Prasanna Malaiyandi: But they already have an offsite backups,
right?
So.
W. Curtis Preston: right?
Yeah.
So.
Um, I, I think that your backup should be copied to another
region and another account.
I, I actually, and, and, and, and I'll say that, you know, my, my
opinion is somewhat peppered by having worked for a company that does this.
But there are companies that will backup your cloud data and then get it out
of the cloud into their cloud, and I think that's as secure as it can be.
Right.
And I think that it should be then stored in some type of immutable type offering.
Um, basically get it, get, get it out of the region for security against bad things
that might happen that aren't hackers.
And then get it out of the account to secure it against hackers and the.
If that costs you money, figure out a way to do that that costs
you as little as possible.
And, and, and I do think the companies that can back up, take the, the
incremental data and then maybe de-dupe it before they pull it out.
If they can do that, you can minimize the egress cost of moving it out.
Um, the, the other thing, um.
Yeah, so, so they didn't have it in a different account.
They didn't have object lock turned on.
Um,
Prasanna Malaiyandi: Object lock did not exist back
then.
By the way.
W. Curtis Preston: what's that?
Prasanna Malaiyandi: Object lock didn't exist back then, so.
W. Curtis Preston: Okay, so we won't, we won't blame them for that.
But, but, but since object lock exists now, we'll say this is what you should do.
Uh, but you know what did exist back then that they did not use multifactor
authentication?
Prasanna Malaiyandi: Oh, for their access into their admin account or into their
W. Curtis Preston: Yeah.
Yeah.
So if somebody gains access to your admin keys and they're able to log in.
If you don't have MFA you, you have zero protection against someone
either stealing or accidentally, you know, inadvertently getting access
to administrative level keys and, uh.
I mean M-F-A-M-F-A-M-F-A mfa, I mean, how many, how many times
do we have to say it right?
Um, good password management, MFA and, uh, patch management.
We, we say this all the time.
If you just did those three things, you'd stop roughly 90% of attacks.
And in this case, if they had had MFA, this, uh, bad actor
would not have been able to
gain access.
Prasanna Malaiyandi: Yeah, and and I think it's important to say
just because you use MFA and patch management and everything else.
Doesn't mean that you don't need backup, you still need backup because
that is how you are gonna recover from this, plus other issues as well.
The security side of things just sort of helps to protect you from
letting the hackers in to some extent.
Not gonna be a hundred percent foolproof, but hopefully, like
Curtis said, protects you.
And 80, 90% of the cases.
W. Curtis Preston: Yeah, to borrow from and totally abuse
a quote from Shakespeare.
There is more on heaven and earth than that is dreamt of.
In your philosophy, there are many, many ways that your
data can be attacked, deleted.
Set on fire exploded.
Sucked into a sinkhole.
There's so many different things that can happen to your data.
That's why you have backup.
And backup protects against all of them.
And, and we're saying backup and Dr and all of those things that come with it.
Um, but uh, the other thing that they also didn't do was this idea
of, um, you know, least privileged.
Do you
want to talk about that?
Prasanna Malaiyandi: Yeah, so normally you do not want, in a company, you don't
want the intern to have the same level of access as your CEO or your IT admin.
And so you wanna be able to say, Hey, whatever access a person needs
to something, that's all they should have access to and nothing else.
And so you wanna have, make sure that you are focused on that and
don't just say, Hey everyone, you guys have admin credentials so you
can do anything and everything.
Because if one person who inadvertently gets compromised,
now everything is exposed.
So you wanna scope down their access to only what they need and that's it.
W. Curtis Preston: Exactly.
Now, when, when reading the, uh, articles about this, one of
the things that other people I.
Uh, dinged this company for, was that they didn't have an established
procedure for locking down the account.
And when I thought about that, I just found myself wondering.
I.
Huh?
How exactly would that happen?
And the best that I could find, you know, how, how would you do that?
And the best that I could find is that you would have a secondary account
that has access to this account that you then have a procedure to do things
like, um, disable, I think about.
You know, what, what is, what is the cloud equivalent to blocking somebody out?
And I, and I think the, the, the quickest way would be to
disable particular I am profiles.
Um, there, there was some, I, you know, and I'm not, I'm not
an expert on this, uh, I don't think you're an expert on this.
I would say talk to your cloud company.
Talk to your cloud provider.
Ask them, Hey, I am worried that one day my cloud entire environment
might become compromised.
How can I automate, basically locking everything out?
Right?
Once we determine that a hacker is in our environment, I would really like
a button that I can press from another account that shuts everything and down.
And everything like this can be automated and yes, that is a, uh,
you know what, what, what's the term?
The, you know, yeah.
The nuclear option.
That is the nuclear option.
But once you have an a hacker in your account to me that that
would be the proper option.
Shut everything down.
Uh, except for like, I would think create a new IAM profile that you can
use after you've done this, and then nuke everything that isn't that, um.
And, and that should be automated and that, and I don't think this
is something we normally talk
about.
Prasanna Malaiyandi: I don't think it's something we talk about, but
it has to be out there somewhere.
I'm sure that AWS or Google or pick your favorite cloud provider,
they probably have a procedure.
I think the danger is you don't want it too automated because there's also the
risk that a hacker or someone else could trigger that and shut down your company.
So it's sort of one of those nuclear options.
So you don't wanna make it too easy.
But I'm sure that
W. Curtis Preston: Yeah.
Prasanna Malaiyandi: a.
Yeah.
W. Curtis Preston: use the nuclear option to.
This is one of those where this, this is the nuclear button, and so you just,
maybe you have an account that just does this and that account is like
completely separate from everything else.
Like we, we talk about having an account.
That is the backup account, right?
That is used for backups and no one ever logs into this account.
And you create it in such a way that if anyone ever does log in,
does log into the account, it sets off alerts everywhere and Right.
It, it can be like a honeypot account, but this account, um, yeah,
I'm not sure how to do, again, I'm not an expert in this, but I would,
I would create a separate account.
I would make that account as secure as humanly possible.
Again, ask your cloud provider how to do that, uh, the, the best way to do that.
But I just noticed that everybody said that almost everybody, they're like, they
should have had procedures for what to do in this situation, and, and they didn't
have them.
I think
perhaps,
Prasanna Malaiyandi: right?
W. Curtis Preston: what's that?
Prasanna Malaiyandi: At least manual procedures, right
W. Curtis Preston: At least manual procedures, right?
Because I don't think that changing the passwords on IM profiles was
the, the quickest way to do that.
Right?
I think the thing they should have focused on perhaps was kicking out,
currently logged in sessions, uh.
Yeah, I don't know.
And the problem is, it's like, how, how do you, you, you have to,
you have to build your incident response around your environment.
And one of the things that they could have done is maintain an inventory of,
um, basically privileged, super privileged accounts and look and see if there
were any new ones.
Prasanna Malaiyandi: Well, that's the thing I was going to mention is they
should at least have had monitoring.
When someone adds a super privileged user, they should have been flagged
about that immediately, right?
Because that's not a common occurrence.
W. Curtis Preston: Right.
Prasanna Malaiyandi: And so monitoring, alerting is also looks like something
that people should be doing to catch these sort of issues as well.
W. Curtis Preston: Yeah, and, and, and I'd love, by the way, I'd love
other suggestions from listeners.
I would love to hear from you.
If you go to backup wrap up.com, there's actually a button on there
that you can leave voicemails.
You can send us notes and, uh, you know, you know, if you'd like, we can actually
even play your response on the air.
I would love to hear better suggestions than we have from a security perspective
because I'm, you know, I'm Mr.
Back.
I'm, I'm not Mr.
Security.
I, I play, I play a security on tv.
Um, the, uh, but, but, but, uh, the summary statement.
From a backup perspective, if they had simply followed the 3, 2, 1 rule, if they
had made another copy of their backups in another account, in another location,
if they had used a cloud provider to do this for them so that then a copy of all
their data was stored in, in a completely different company, if they had done any
of those things, they would've at least had a copy so that once they got on the
other side of the attack, they could have.
Then recovered all the data because that's the, the true disaster here.
As that once they've been attacked and once the attacker gained access
to their account, they were able to delete all their data, both
their primary and their backups
Prasanna Malaiyandi: So I know we've been focused solely on code space as the
company, but I think there's also blame to be placed on those 200 companies who
were using Code Spaces for not also having a backup of their data and relying solely
on code spaces as their service provider.
W. Curtis Preston: You know, that is an interesting, we, you know, we
tend to focus here on the fact that it was a provider, but what this
really was, was a SaaS provider.
Prasanna Malaiyandi: Yep.
W. Curtis Preston: Right.
Prasanna Malaiyandi: Yep.
W. Curtis Preston: Yeah.
So we don't know what happened to those other companies.
And so this is, this is a double lesson, right?
If you're, if you're running in the cloud, make sure you've got
a backup of that, of that data.
If you're using a SaaS provider, make sure you have another copy of
the data that you're putting in that SaaS provider, because that would be
another way for at least the, the, um, the, the, the thing that's difficult.
Here, again, I agree with you, the thing that's.
The, the difference here is that unlike many of the SaaS providers, this company
specifically said, Hey, we got it.
We got your backups, this data, and it's tested and it's all this stuff, right?
Um, the, um, you know, I just had a thought.
Um, if we go to LinkedIn and we search for code spaces.
Find people that used to work in code spaces.
Oh, that would be, I wish, I wish we could talk to somebody
that was involved in this, but
uh, I think we can
Prasanna Malaiyandi: I'm sure there are probably NDAs.
W. Curtis Preston: What's that?
Prasanna Malaiyandi: There's probably NDAs.
W. Curtis Preston: Oh, they're probably NDA.
Yeah.
All right.
Well back up the cloud, I told you so.
Any final thoughts for you, persona?
Prasanna Malaiyandi: No, I totally agree with that, and I like this because I
know we bring up code spaces a lot, so I think that hopefully our listeners now
understand why we talk about it and what they should not be doing, and why we
harp so much on things like the 3, 2, 1 rule on MFA, because you don't want your
company to have to shut its doors because they were unable to recover their data.
W. Curtis Preston: Yeah.
And sadly, this will not be the last company that.
You know, basically ceased to exist, uh, because they didn't
properly back up their data.
Prasanna Malaiyandi: Yeah,
W. Curtis Preston: All right.
Well, uh, thanks for, uh, joining me persona, as always.
Prasanna Malaiyandi: anytime and looking forward to see your analytics on your car.
W. Curtis Preston: I will see what I can do and I will also thank our
listeners, we be nothing without you.
Thanks for listening.
And be sure to subscribe so you don't miss an episode.
That is a wrap.