ATR2500x-USB Microphone & Logitech BRIO-2: Backup is not archive and archive is
not backup frequent listeners to this show, know that we say this all the time.
This episode.
We're going to dig a little bit deeper and we're going to talk
W. Curtis Preston (2): about exactly why that is the case.
It really comes down mainly to eDiscovery.
And why backup systems make a really bad tool when you get any discovery requests.
If you live in a place where you are likely to get any discovery request,
or if you're required to keep certain information, For compliance reasons.
You need to listen to this episode.
And interestingly enough, it ends up with kind of a surprise ending,
one that came to me actually only after actually editing the episode.
If this is your first time listening.
Hi, I'm w Curtis Preston, AKA Mr.
Backup, and I've dedicated my 30 year career to backup and recovery, disaster
recovery, and anything near to that.
And this podcast is dedicated to those unappreciated backup admins.
We turn you into cyber recovery heroes.
This is the backup wrap up.
Welcome to the backup wrap up.
I'm your host w Curtis Preston, and I have with me someone who
apparently has much better ability to remember to do things than I do.
Prasanna Malaiyandi
W. Curtis Preston: how's it going?
Persona.
Prasanna: I am good.
Curtis, what did I remember that I, that you seemed to forget?
Oh, that's
right.
W. Curtis Preston: So we, uh, you know, in keeping with, you know, my
philosophy, we do two recordings here.
Uh, partly because we use a cloud, well, chiefly, I think because we use a cloud
recording service that has been 98%.
Reliable.
And then sometimes it just doesn't have the recording, which is,
which is incredibly frustrating.
So we make a backup recording and we use something called OBS,
which if it was just you and me, we probably could just use OBS.
But, uh, you know, we, we have guests and installing OPS, isn't that
helpful?
. so anyway, but you remember today.
And
so we have
our backup.
Prasanna: Yeah.
And then I know we're gonna get to the news in a second, the topics, but
I think one of the things we should also mention is, uh, not only do we
keep the recording of OBS locally, but we also upload it to a Google Drive.
W. Curtis Preston: we do.
We
Prasanna: And one thing though that people may not realize.
W. Curtis Preston: Two, one buddy.
Prasanna: Exactly.
I think one thing though people don't realize is I know you created a shared
folder for us and throw things in, and over the weekend I got into a little bit
of a hissy fit because I was like, why are you consuming my storage, Curtis?
W. Curtis Preston: Yeah, I don't understand that at all.
Where I
share a drive with
you
Prasanna: you don't share drive.
You share a shared folder, which is different than in enterprises
where you create a shared drive.
That has different association than creating a shared
W. Curtis Preston: So you mean when you put it in there, it's still in your
account, but it's sharing it with me.
Oh,
that's,
Prasanna: it's basically an organization
mechanism.
W. Curtis Preston: Okay.
All right.
Well, hey, you know you learned something, uh, speaking of Google Drive, though.
Uh, I think we have good news.
I, I think this is good news.
This is a follow up on the big Google Drive story that I, that I
believe we reported on last week.
And this was this thing where, uh, a bunch of Google users, there were like
several hundred that had clicked the, this has also happened to me button
and where basically all of the data since May had had disappeared and.
The, the good news is that Google released a, uh, basically instructions
and a blog post that talked about what to do, and it turned out that
the data was never really gone.
It was just data.
It that, that, so first off, this.
And you were right the first time you said it.
When we talked about it last time, said that this only
affected those with Google Drive.
And I said, well, there are some people saying that they have the
problem and they weren't using the desktop version of Google Drive.
And you may recall I said they may be wrong, and it looks like they were.
So, um, basically you needed to download the updated version of Google
Drive and then click Restore from backup, which is interesting because.
It means that there are files that are not quite in the drive but
are stored, still stored locally.
These are files that were not yet synced and that basically this
should solve your problem, um, because it only affected files that
that had not yet been synced to.
Which I want to say, um, how do you not know that your drive
has not been sinking since May?
Um, right.
Yeah.
I, I
don't know.
That's
just a thought.
Prasanna: that it's kind of crazy.
It's been like seven months and no one's noticed the problem.
Like no one, like if you're using Google Drive for desktop, you've never gone
to the web interface and been like, Hey, let me check if my file is there.
Try to pull it down or use the Google Mobile app.
You know, that's kind of crazy.
I.
W. Curtis Preston: Yeah, I think it's thi this really kind of drills to the heart
of why I'm not that big of a fan of Google Drive as, or any similar drive like,
uh,
Prasanna: One drive.
Yeah,
W. Curtis Preston: Drive, um, is that the sinking process
is
not.
Managed and reported on the way that a backup process typically was.
So a sync failure apparently isn't bubbled up to anyone because you would think
if they're using Google Drive desktop version, Google Drive would be flashing
messages to them saying, Hey, you know, you haven't synced for seven months.
You might wanna look into that.
Um, yeah.
Whereas a backup app is something that allows for centralized
monitoring and, you know, especially when we talk about, um, companies.
Anyway, I digress that, that that isn't what I wanted to talk about today.
Prasanna: I do wanna say though, kudos to Google though for actually providing an
update on both what the issue was, quickly resolving it and providing a patch.
W. Curtis Preston: They did figure it out.
They did fix it.
And it looks like, um, my, my favorite part of the instructions
were if you click the Restore, you're gonna either get, uh, two messages.
Either Restore was successful or you're outta space if you get the second one.
Um.
You should really go get some more space.
It just, they did like in a nice little professional way, and I
really wanted them to go, Hey dude, man, clear out your drive, you
know?
Prasanna: Uh,
W. Curtis Preston: Uh, anyway.
Anyway,
so
you want, you wanna talk about the 23 and Me
Prasanna: So I came across this article recently, so it looks like 23, and
me had a security breach recently, and it was kind of alarming What was,
what the media was talking about.
They were like, oh, 23 and me was breached.
All this information is available on something like.
I think it was like, what was it?
6.9 million I think was the number of users and it's like, wow.
And they didn't have any other information.
Right.
So it's like 23 and me for those not familiar, it's
W. Curtis Preston: Sounds very
Prasanna: It's like an ancestry website where you can upload your
DNA profile and it'll help connect with other people who may be related
to you that you may not know about.
And so with that DNA profile, it's like, wow, if hackers got into 23 and
me, they're able to pull down your DNA.
And what could they do with that?
Like all this data could be used for nefarious purposes
and other things like that.
W. Curtis Preston: They could
Prasanna a clone.
Prasanna: in 10 years.
Right.
But, and so it was very scary.
And just yesterday or uh, yeah, yesterday they just published an
update from 23 and me saying, Hey, we figured out what the issue was.
And once again, like the Google, I'm glad they were very transparent, so.
W. Curtis Preston: Yeah,
Prasanna: So what ended up happening was, uh, hackers basically use credential
stuffing to access 14,000 accounts.
Uh, what credential stuffing is, is they basically.
Figured out that there was another website that was compromised.
They downloaded those username and passwords and they use those compromised
credentials and just started atta uh, using those same logins on 23.
And me.
And like we always talk about Curtis, people should be using
different passwords on different websites and they should definitely
be using a password manager, right.
W. Curtis Preston: If, if, if everybody did what we tell them to do, which is
don't use the same password anywhere.
And use MFA, right?
This hack would never have happened if, if these 14,000 and
I, I, I, I'm not, I don't know.
It, it sounds like victim blaming it.
It is what it is.
This is your job as a person, as a company to, to, to prevent the, you
know, the stealing of your information.
And if you just do all the things that you're not supposed to do, you know,
your chances of getting hacked are much
much higher.
W. Curtis Preston (2): An interesting update to this story since we
actually recorded this news item.
And that is that.
23.
And me basically came out and said what we just said, which is, you know,
if you hadn't reused your passwords, Uh, you know, from other sites on our
site, you wouldn't have gotten hacked.
And they got lambasted for it.
Basically, they said, you know, 23 and me is blaming their users for their hack.
It wasn't a hack, right?
It wasn't a hack of 23 and me, it was, they went and got other people's
credentials from, you know, their user's credentials from other sites.
And they tried those credentials on 23 and me and they logged in.
I, you know, I I'm, I have to say.
Um, you know, I'm pretty much on 23 and me side here do not reuse your credentials.
On other sides.
Number one, number two, turn on MFA.
Right?
If, if, if you had done both or either of those things, Uh, you would
not have been subject to this hack.
I, you know, I don't know what else to say.
Prasanna: Now, so it's not as bad, right?
So it's 14,000 people.
They didn't go to the backend systems and pull in all the data.
They went through the front door, right?
And so they saw whatever that user could have seen.
Now.
The downside is it wasn't just those 14,000 people and
their information, right?
Because with 23 and me, you could say, Hey, who else am I connected to?
And you could look at their
DNA relatives is what they call it.
And you can also build a family tree, which also exposes a personal data of
people you're connected to potentially.
W. Curtis Preston: Right.
Yeah, it is.
It is an optional feature, but it's probably an optional feature
that people use quite a bit.
And you know, it shows, you know, it shows a lot about you, right?
You're, you know, you know, obviously things like when you last logged in and it
shows the people that you're related to, which may expose family relationships that
you had not intended to expose publicly.
I.
Uh, there, the scariest part was, it, it, it said, it talks about DNA segments.
It, it shows that the DNA segments that you have in common.
I, I haven't looked at that specifically, so I don't know exactly what that means,
but I, I think it's important to say that what it doesn't, what you're not
able to do as a 23 and me customer, is to download your actual DNA profile.
Right.
The thing that, that I was joking about earlier, you know, cloning,
you're not, you're not able to do that.
And so while this sounds like a huge breach, the only thing I think that
23 and Me could have done to prevent this is to force MFA on all customers.
And you know, it's something that you as a company, I think should think about,
um,
that.
You know, especially if you have sensitive data.
Right.
And I, I can't think of any data more sensitive than DNA profile.
Right.
Um, that's about, I mean, what the customers could have done
is you use a different password and, and turn on, I'm sure.
23 and Me has MFA enabled or available to you, but many people might not use it.
Apparently 14,000 people, at least 14,000 people don't use it.
So
Prasanna: I also do wonder if companies should start integrating
with things like, what is it?
Am I owned?
right.
Those and I Pond.
Yeah.
Am I Pond?
Yeah.
Right.
And it's those websites that I think companies should start
thinking about integrating with and being like, Hey, there's already
a list of breaches out there.
So has this username password, the hash of the password been used elsewhere.
So they could
W. Curtis Preston: Oh, that's actually, I really like that
idea Prasanna of basically.
Proactively going through your username, um, and password database
using the MI pond, uh, database to say here is, but you won't get, I
guess the only thing you'll be able to notify, you'll be able to notify a
person that this username, this email address has been owned in another site.
And notifying.
I was, I was thinking for a minute there that you could figure out if
the actual
password
Prasanna: I wonder if you could use
W. Curtis Preston: used.
Prasanna: But
W. Curtis Preston: Well, you could use that, but they don't put the
password in the am I pond, right?
They just say that, um,
you
know, um,
Prasanna: Maybe that's a service that they should offer.
W. Curtis Preston: who am I,
pond?
Prasanna: For companies to integrate, be like, Hey, we have, because they
must have the hashes, you know, because they're getting the data from somewhere.
W. Curtis Preston: Right.
Yeah.
There's a, there's a, I think there's, there's some money to
be made there, I think, right?
Um,
Prasanna: yeah.
The one other thing I would say is, and they didn't go into a lot of
detail, so the breach happened back in October, so it's not known how
long it took before they saw the issue and how also the attackers were.
Forcing themselves in.
For instance, if the attackers, and I'm sure attackers don't do this, right, but
if they start coming from a region that you normally don't log in from, that
should have been flagged immediately.
You know?
So their intrusion detection system should have realized, Hey, you're
logging in from the east coast of the us.
You normally log in from the west coast.
Yeah, we should probably check and see.
W. Curtis Preston: I, but I, I agree.
I, I mean, I agree, but I'm just thinking about like, I don't know any, like
the only, the only website or service that I log into like that where it
says, Hey, you're in a different spot, is Netflix, and the only reason that
that's the case is because I'm not supposed to watch outside of my home.
They're not, They're not,
thinking
Prasanna: But, but, uh, there are other websites though, like that look
and see, okay, are you logging in, like Google, for instance, right?
Hey, you're logging in from this remote place.
Are you sure that's you?
And they
W. Curtis Preston: a different place.
Prasanna: And they send you a email, right?
Or Apple does it
W. Curtis Preston: If, if,
if you've
enabled
Prasanna: No, no, no.
But it doesn't even have to be MFA, right?
At least a notification.
W. Curtis Preston: sure?
Prasanna: Well, well, I think at least getting a notification.
W. Curtis Preston: Okay.
Okay.
That, yeah, that's true.
That's something that they could do, right?
They could notify users.
Yeah.
Hey, uh, just so you know, someone's logging in from Russia
with your,
um, of course
that,
yeah.
From a new browser, because you can fake VPNs, right?
You can use VPNs to fake.
Yeah.
Prasanna: So I think there are
W. Curtis Preston: So, I, I,
Prasanna: and I
don't know what systems they have in place at 23 and me, but hopefully they are
reconsidering what systems they have to prevent things like this from happening.
W. Curtis Preston: Yeah, agreed.
I, I guess just short version here is this wasn't as bad as
we thought it was initially.
This wasn't an attack, this wasn't a backend attack.
It is officially a breach because it was, you know, their company.
This was basically a bunch of people who could have prevented the breach
by just changing, you know, using different passwords and enabling MFA.
Um, it sounds like they could have done some additional things to, to,
uh, ameliorate this, uh, as a company.
And we also, we don't yet know if they, um, um, if the, like the hack, you said
that this, this happened in October.
Did they sit on it for two months or did they just not know
for two months?
That's also
Prasanna: Now the one thing is they did bring in forensic
investigators to figure out what's going on, so I will also give 'em
W. Curtis Preston: so maybe it, maybe it took, yeah, maybe it took that long.
All right, well that's the news of the week
All right.
This week I thought we would dive, uh, a little bit deeper into archive,
if you want the basics of the difference between backup and archive.
There are a couple of episodes from a little bit, a little bit ago.
Um, it just says, what is Archive and retrieve as opposed
to what is backup and restore.
Prasanna: We will put that in the show notes, right?
In case you want to go back and listen to that.
W. Curtis Preston: yeah, we'll put a link.
Yeah.
Thanks.
We'll put a link in the show notes to that, uh, episode.
If, if you wanna go listen to that before you wanna listen to this, we're
gonna go a little bit deeper into what is Archive 'cause it is very different.
What, um, do, do you remember how I, how I, um, sort of separate the two?
Prasanna: What was it?
It was, I believe one of the points you talk about is backup is.
Restoring your, what your environment looks like to a point in time
that plausibly existed, right?
Archive is you get a whole bunch of data back and the way you
search is very different, right?
Usually backup is, I wanna go back to a point in time archive is I'm looking
for all the emails from Curtis or those types of things where you're, what you
end up with from an archive perspective.
Is never a plausible point in time, or it may never be a plausible
point in time in the life of that
W. Curtis Preston: right.
Absolutely.
Prasanna: Is that fairly accurate?
W. Curtis Preston: That, that's absolutely it, right?
So it's basically a, a, a backup is to do a restore and
an archive is to do a retrieve.
And the difference between a restore and a retrieve is that a restore
is basically restoring your system back to a particular point in time.
Even if it's just one file, you're restoring one file.
If you're just doing one file.
Technically, I suppose you could use either, but generally we're talking about
many files, and so with a restore, you're restoring the system to the way it looked.
Generally speaking, just a few minutes ago or perhaps yesterday,
basically to the most recent backup.
Um, and, and possibly to something before that, especially in the
case of a ransomware attack.
You want to pick a particular point in time because it's before the ransomware
attack happened, but the whole point is to bring it back to a point in time.
That's a point in time that you know, whereas with archive, it's about, I
need some information that matches a particular set of criteria that I.
Um, and I don't even know where that information might be.
Right.
That's, that's another big difference between a backup and an archive, is
that with a backup, we know we're restoring system A, you know, Apollo,
we're restoring it to the way it looked yesterday with an archive.
We are, you know, you gave an example, we're looking for Curtis'
emails, um, or we're looking for any documents that Curtis created.
In this span of time.
It could be emails, it could be written documents, it could be
drawings, it could be whatever, right?
A, a perfect example of this is, let's say I work for a firm where my job is
to design stuff, design widgets, and.
What goes into designing those widgets?
As I work there, I'm going to create drawings.
I'm gonna create, uh, do we still call them CAD drawings?
I don't if we even do we still, yeah, we
Prasanna: accurate.
I think it's still accurate.
W. Curtis Preston: Yeah, I think so too.
Um, I'm gonna create drawings of what I'm doing.
I'm gonna create conceptual drawings, perhaps, uh, actual physical drawings.
I'm going to have emails where I go back and forth, Prasanna,
what do you think of this?
And you're like, yes, I like it.
I'm gonna have documents where I describe the requirements
for the widget that I'm making.
Can you think of anything else that I might
Prasanna: It's like meeting recordings.
W. Curtis Preston: Meeting recordings.
Absolutely.
Meeting notes where we discuss, uh, what I'm up to now, why
would all of that matter?
Because it's, let's say a couple years down the road and I've left
the company and I've gone to a competitor, and suddenly the competitor
comes out with an identical widget.
And, uh, my former employer is now accusing me of stealing
intellectual property.
And what, so what they want is they want proof that I worked
on that widget at this company.
And so, um, they're going to, they want all these things.
They want all the emails, they want all the documents, they want all the CAD
drawings, they want all the meeting notes.
They want all of these things.
And they, they all have, and the widget, you know, the widget has a name.
Right?
That's one of the reasons we have project names.
Right.
Uh, the widget has a name and so we just want every email where I mention that
project name, I want every drawing with it's named after it, et cetera, et cetera.
Or just maybe if we, if we wanna cast a wide net, we just want
any CAD drawings that Curtis did from this time to this time.
Can you think of anything else?
Prasanna: One other key aspect is that project may not exist on any production
storage system anywhere else, right?
The only copy may exist in this archive system.
And you don't even know what system it initially existed on.
Maybe that system has been retired, right?
You don't know.
You don't care.
What you're really focused on is finding everything associated with this project,
or like you mentioned, any CAD drawing that Curtis did between this timeframe.
W. Curtis Preston: Yeah, exactly, because this may be happening.
A few months, a few years, many years later, you don't have any knowledge
of the servers or the applications.
You don't remember if you were using Google Drive or perhaps maybe there was
an outage in Google Drive and you switched over to Microsoft 365 or the vice versa.
And you, you don't remember those things, and that's why you have an archive.
And you can go in and just ask for information.
Show me, uh, drawings that, um, you know, show me the CAD drawings.
Show me the emails, show me the, um, uh, any documents that Curtis was working
on that have these phrases in them.
Um, because that is, um.
The, the, the thing that you touched on is a really important part is that
you don't know where this stuff is.
The other thing is, it's not a point in time, it is a range of time, right?
It's going from basically anything Curtis worked on in 2023 and, um,
because again, we've accused him of stealing an an electric property.
We wanna show that he created it here.
So we wanna just see everything.
Uh, and then you have a team once you do that.
Prasanna: I think the other key though with the archive is, especially depending
on the system, it doesn't matter if you, Curtis had created a, like a working
document and then deleted it, right?
It doesn't matter any of that, because as long as the archive
captured that copy, it's still there.
Regardless of what you do and the time, if you delete it or whatever else, the
archive holds that copy, so it's there,
W. Curtis Preston: Right, right.
Prasanna: So you have all this documents, right?
From 2023, everything.
Now you need some ability to go through, right?
And usually you will have your legal team or someone else go through and
sort of filter and really figure out, okay, what is relevant, what is not?
Because there might be thousands of documents that Curtis created in 2023,
and so now you need that ability to figure out, okay, which are the ones
that are relevant for this IP case versus which are the ones that are not?
Because you don't, you wanna find the needles in the haystack, right?
Not give everything over to, for discovery purposes, right?
For legal discovery purposes.
W. Curtis Preston: So we've talked primarily about e-discovery as one of the
reasons we archive, and that is probably.
The primary reason I think many companies archive, they may actually have a
regulatory requirement to archive, to basically be able to show any version,
any, um, you know, any conversation.
I know that like, for example, I.
Uh, financial trading firms have to show any conversations with customers.
So they archive every conversation with a customer, whether it's audio
or text or, you know, anything.
Like they have to archive that so that they can then search it later.
When you, uh, when the customer said that you promised them a
certain financial return and you're like, uh, never did that.
Right?
And then it's like, uh, yeah, actually you totally did that.
I think that probably is the primary reason many people archive, but
if we look at the other reason to do archive is storage management.
You wanna talk about that?
Prasanna: Yeah.
So before we talk about why you would archive to get storage management,
I think it's important, like you mentioned to talk about the cost.
So production data, right?
Sitting on tier one storage, it's very expensive, and as data ages typically
the value of that data starts to reduce.
So a project that you worked on three years ago, you're probably not gonna go
back and touch it, think about it, right?
But at the same time, you might have need to access it maybe at
some point in time, or like the example Curtis you gave, right?
It's a widget that you created no longer really needed.
You don't need to keep all that data around.
So you'll keep like the final version of the widget, but you don't need
all the working copies and working examples that you created along the way.
And so.
You wanna save all that space.
So what you could do is there are different solutions most people
would go about and say, okay, let's archive this project and move it
to cheaper lower cost storage that doesn't need that high performance.
At the same time, allowing me to search and find it because that
is critical for these use cases.
So what people do is they would want to archive old data sets, things
that they don't actively need.
And so you move it off to a archive system that allows you to store at a much lower
cost than the tier one production storage.
And typically they give you this additional functionality like
being able to do the searches that we had talked about before.
Now one of the challenges with sort of archive systems, right, and moving
the data is you have to be able to identify the data before you can move it.
And for some organizations that's very difficult to do.
And so you will see a lot of times where people are like, okay, I'm just gonna keep
things on my primary tier one storage.
But within that system they have other tiers of storage.
Like I can move it off to, uh.
Serial a, uh, a TA disc or I could move it to object storage and the storage
array itself will automatically deal, deal with all that tiering for me.
So I don't need to worry about it.
It's still all seamless, but it is important to note that that's not archive.
You don't get all the capabilities to be able to find your data.
You don't have that protection necessarily to make sure a user
doesn't go manually delete the data.
You don't have that ability to search all copies of all the data.
It's really more like a backup system that tries to emulate
archive, which we've talked about before, is not an archive system.
W. Curtis Preston: Yeah, I would describe, I mean, you know, you can, you can do,
basically one of the things that people do is they maintain the same structure.
Of what they have in the primary side.
They just move it into a less expensive place.
They move it from S3 to Glacier Deep Archive.
Right.
And you can, you can maintain the same structure and, and you're right, that's
long-term retention that isn't archive.
The idea behind archive is again, just like archive isn't backup long-term
retention isn't archive either.
And there are backup systems that just move old data, old backups out to long,
you know, to less expensive storage.
And if you actually need those backups, then it, it brings it back again.
That's more kind of an HSM style thing than it is archive.
That would be hierarchical storage management, for those of you that,
that haven't used that term before.
Um, but a true archive.
Is going to allow you to bring it back in a different way.
Right?
So we talked about earlier, so this is sort of the most basic type of
archive, is we, we do that search, we identify all of these files, all of
the CAD drawings, all of the emails, all of the documents, all of the phone
calls, all of the bi, you know what, all of the information having to do with
the, um, you know, the Booyah widget.
And then we archive that into a, uh, essentially a digital box.
That's the way I like to call it.
And we put all of it together so that when it's five years down the road and
somebody says, you know, this reminds me a lot, this is one of the things we call,
you know, institutional knowledge, right?
This reminds me a lot of that thing Curtis was working on back in 2023.
Remember that?
What was it called?
The Booya Project, right?
And then you go to the archive system and you search on Booya and poof,
there are all the emails, all the, you know, all the, um, whatever, all
of this stuff having to do with this.
I, one of the things I like to liken this to, have you ever watched a case,
uh, an episode of the show, cold Case?
You ever watch that?
Okay,
Prasanna: That's probably the one show like detective show that
I've never seen or crime show.
W. Curtis Preston: Yeah, so in cold case, every episode of Cold Case
involves this warehouse where they have these boxes, you know those
like off the kind of office type
Prasanna: boxes?
Yep.
W. Curtis Preston: Yeah, the file folder boxes.
Right.
And um, and basically it's like.
You know, Steve got murdered and there's a Steve murdered Steve box, right?
And it'll say like, it'll have like date and time and basically they put
all of the stuff, all of the evidence, all of the notes, all of the stuff,
and they put that into a box and they put it on the cold case shelf.
And then somebody's pulling that out.
And again, they get all of the stuff.
This is a digital version of that, so that when you remember and you say, um.
Uh, you know, we want, we want to go back to that project, and
I have a perfect example of this.
I used to work for a, or I did some consulting work for a satellite
company and that satellite company, once they, they used to make a lot of
satellites for China, and then some time passed, some significant amount
of time passed like several years.
And then, uh, China came back and said, you, you remember that stuff
that you made for us back in 1998?
You remember those satellites?
We want 20 more of them.
We, we don't want any improvements.
We just want, like, we want exactly the ones, those worked out really well.
We want a, you know, we want a bunch of them, you know, we want 20 more.
And they had an archive system.
They were able to pull up all those drawings and then poof.
And then just produce, uh, essentially carbon copies of what they made.
That's.
Sort of the old school archive and you, you need to be able to attach
metadata to it, a project name, other things, so that when you're searching
for it, you're able to find it.
Prasanna: Yeah.
Or another example is animation studios, right?
Typically when you make a sequel, you're going back and pulling up old
frames and old animations to reuse again, and so being able to quickly
find those right and pull them back up saves your animators so much time
rather than them trying to recreate it.
W. Curtis Preston: Yeah, absolutely.
And the other, uh, I, I know that w with, you know, we, we
have some insight into that.
We've had somebody on here, uh, who, who's, who's worked there, Jeff Rochlin.
And, you know, we know that, that they do both sort of cloud versions.
They also do like hard copy versions.
They use optical media.
Because they know they don't wanna lose it.
Right?
So they have multiple copies, so then they get back there.
So that is one purpose of, uh, archive.
But I, I'd say the more, do you agree with me that the more
common reason is that e-discovery
Prasanna: I think so.
I think so.
I think just going to storage management.
I think a lot of people see storage management or storage costs as
cheap enough not to hinder their users from moving the data and
then them going, having to go and try to figure that all out.
That I think it's not as big of a motivation motivator as it is from
a compliance and e-discovery case.
W. Curtis Preston: Being told, you know, you need to do that
from a compliance standpoint.
And so, and you touched on this a little bit earlier in that.
You were saying that if people like create something and delete something, it would
still be in the archive And, and I think that's an important distinction because
we need to talk about the way this works.
A basically, a real time archive system is the only way that's gonna happen because
if you're just running batch archives.
Let's say just in the same way we, we run batch backups at the end of the night.
If you're just running batch archives, you wouldn't get those
intermediate, uh, versions, uh, or, or files that, um, you know, emails.
Like you, you, you might send an email and then you realize, oh, I said something.
Prasanna: And go delete it.
W. Curtis Preston: in that email, then you go delete the email.
But if you have an actual email archiving system, it's watching and it's going to
archive every single email that goes out and every single email that comes in.
Prasanna: You know what this reminds me of?
W. Curtis Preston: go ahead.
Prasanna: This reminds me of our CDP discussions because this is
literally what you want is you want CDP for an archive use case.
W. Curtis Preston: Yeah, exactly.
You are essentially doing, uh, that's a continuous data protection
for those that missed that episode.
And it is, you, you definitely want, um, sort of, you know, it's, it's real time.
It can be it, it can be asynchronous, right?
But it's real time replication of every single.
Whatever it is that you're archiving, you do file system archiving.
You would need to do file system archiving.
You would need some sort of plugin to the file system to be notified of any
file changes, which I would assume would be available from most filers.
Like it would be something that you would plug into a virus detection program.
Right,
Prasanna: And it doesn't have to be every single change.
It's only when those changes have been committed, right?
So when you close
W. Curtis Preston: right, right.
Right.
Um, and so the idea is that you would, uh, you, you're, you're
storing that every single thing that goes out, both good and bad.
And, um, and so, and when you have a, this is sort of, I would call this
like a real archive system, right?
So like it's, if it's four e discovery, you need to be able
to go in there and there are.
I don't know, 30, 40 different pieces of metadata attached to any particular
item, object, whatever you wanna call it.
Right?
Obviously there, these include things like the author, the, if there's a, a document
name or a subject name to an email.
If there's, uh, if it's an email, who it was sent from, who it was sent to,
the date it was sent, uh, the content of the email itself or the document itself.
Um, what, what else, what can you think of other
Prasanna: I was thinking like if it's a document that was shared,
like who else it was shared with.
Who had access?
When they had access?
Who left comments?
Who made updates?
W. Curtis Preston: Right.
All of that stuff would be shared or stored in a realtime, you know, in a
realtime archive system so that you can say, I wanna see all the documents
that Curtis created, and I wanna see everybody that saw those documents.
Right?
So, so you can search by, you know, document owner, document creator,
uh, email owner and creator, and, um.
Let's say you have an employee who has, uh, accused a company of a hostile
work environment, and they say, uh, I got all these emails from all kinds
of people saying all kinds of things.
Okay, well, show me all the emails that were sent to this person.
Over the time that they worked here.
And then we're gonna, and then we'll do calling again, right?
We'll go in and we'll do, we'll do one big search to pull and just get
a giant pile of emails, all of the emails that were sent to this person.
And then we go in and we search for phrases and words.
Uh, that should not be in a business email.
Um.
And so that's why it's, that's why it's a two step phrase or a two step
process, not, you don't want to do 10 e-discovery polls against the, the email.
Um, so why don't we, um, why don't we do this with a backup system?
Prasanna: because backup serves a different purpose, right?
It's intended to restore data, not retrieve data.
And there are some systems that try to do this and mix the two
W. Curtis Preston: Probably the one that comes to my mind the
most would be CommVault, right?
They have a common technology engine between backup and archive.
Um, I don't, and I, and I'm not saying this at like, I.
I'm not saying that they don't do it, I just, I haven't
spoken to anyone who does both.
Theoretically it could be done with a single system and they do claim
to do it with a single system.
I just don't know anybody that does that.
Uh, I'd love to hear from anybody that is using either Commvault or
anything else that is doing both backup and archive with a single system.
Again, it's, it's, it's theoretically possible, but generally speaking,
backup systems don't store the data.
In a way that, um, you, you can't search against it like that.
The, the first thing a backup system is want to, okay.
What system are you restoring?
Uh, I don't know.
Right.
I'm just looking for emails.
Oh, so you wanna restore the email server?
What's the email server's name?
Uh, I don't know.
It was five years ago.
I don't know.
Um, and, uh, you know, part of the way we were on, we were on, uh, Google, uh, email
and then we switched over to Microsoft 365 and then we were, you know, um, a
good email system would have all of the email from all of those, regardless of
which hosting provider you were using.
Prasanna: The other thing also is that for a lot of these use cases, it's also
that full text search index, right?
To be able to find not many backup products can do the full text
search across all your different data sets and data types, right?
There are e-discovery products that allow you to search what people said in a video.
And show that as, Hey, who was Curtis talking to in this
meeting that got recorded?
And be able to pull up the phrases and look at the transcripts.
W. Curtis Preston: Yeah, that is a a really good point.
There are some limited backup systems that are able to restore a
file based on its full text, right?
You can search against the full text of a file, but again, you're
gonna find one file, right?
Um, the, and again, generally speaking, you start with the system you're
restoring and then you, and then you, uh, call if you will from there.
Whereas this, this is casting an archive system is catching a
much, casting a much wider net.
Prasanna: The one question I had for you, Curtis, is
W. Curtis Preston: Yeah.
Prasanna: typically who operates the archive system versus who
operates the backup system?
W. Curtis Preston: Well, it's going to, the answer to that
question will be de will depend on.
Whether we did this for storage management purposes or free discovery
purposes, if we're doing it for storage management purposes, uh, it
can be just about anybody, right?
That is qualified to operate it.
But if we're doing it for e-discovery purposes or compliance purposes, it's
going to be someone that is specifically trained in compliance and to make sure
that they have the right requirements to make sure that you have both the.
The initial creation of the archive and then the retention of the archive because
there may actually be laws and rules on.
How long you can re uh, actually retain certain amounts of data.
And you may be told, uh, for whatever reason, a legal reason.
There's certainly the legal hold reason to, you have to keep this
data, but there may also be a legal reason why, where you're told to
get rid of a certain set of data.
I can think of things like GDPR.
Most of the time in the case of GDPR, for example, and CCPA, if you have a business
reason to hold that data, you, you can.
But there still may be a scenario where you're told that you need to, um, you
know, get rid of a certain set of data.
And this is all very specific, um, compliance related things that.
You and I and people that think like you, and I don't necessarily think
about, uh, on a day-to-day basis.
Prasanna: No, I think that's important because that just goes back to why backup
and archive systems may not come together is because they are different teams
W. Curtis Preston: Yeah.
I.
Yeah, a backup person, you know, a storage management person, a, a
typical system administrator, can handle the u the a backup system.
They quite possibly are not qualified to handle a full, uh, archive
system, especially if it's one that's done for compliance purposes.
aNd I'll just, I'll just end this with telling my favorite.
Here's what can happen if you need an email archive and you don't have one.
Uh, I worked for a consulting company.
It was a big consulting company that had, I don't know, they had like a few
hundred, uh, consultants and we were hired by, um, a actually have two stories.
Prasanna: I, I, I, I think I know which story you're going to go to.
Yeah.
W. Curtis Preston: we were hired by a company that.
Needed.
They got, they got an e-discovery against their email, and what
they had was a weekly full backup from their, uh, email in exchange.
And we, basically, what that meant was because they, they
just had a weekly backup.
They didn't have an archive.
It ended up costing them well over a million dollars in consulting
time because we needed like this team of like 15 people.
It was like a three teams of five that were working.
24 hours a day, um, to, to be able to do these.
Because what it meant was you restore, exchange to this week, extract the
stuff you want, then you wipe that and you restore it to next week.
And it was just this very, very, uh, difficult process.
But the other one is this famous case.
And I don't wanna say the company because of what, I don't wanna
get the company wrong, but it was a large financial trading firm.
And the, what happened was it became infamous because they, they
didn't have an email archive system.
They had backup system and, and it wasn't well maintained.
And they, they changed things over time and so it took
them forever to satisfy the.
The electronic discovery request to get the emails that, that the
plaintiff in this case was looking for.
And then at some point you then tell the judge I.
We're done.
We, we, we, you know, we've satisfied the discovery request.
And then a little bit later they came back and they're like, sorry, judge.
Uh, we found this other box of tape.
Right?
And at that point, it had already gone on a really long time.
And then they'd said they were done and then they, it
turned out they weren't done.
The judge ended up.
Issuing what's called an adverse inference instruction, where where
they said basically whatever.
What it is, is it's an instruction from the judge that infers something
that is adverse to your case, hence the term adverse inference instruction.
So they basically said whatever the plaintiff says is on the tapes.
It's on the tapes because no one could possibly be this
bad at retrieving their data.
And so they must be doing it on purpose.
They're trying to hide something and boom, they lost a, it was like two billion
dollar lawsuit as a result of simply not having an email archive system.
W. Curtis Preston (2): Since recording this episode a month ago,
I actually learned about an 11 year old company that is built a nice
business that among other things.
Helps companies who used their backup systems as archive systems.
You know, the thing I tell you not to do.
So if you're trying to do e-discovery using your old backup tapes,
they'll do it for you as a service, saving you time and money on the
extraction and calling phases.
And reducing the amount of data that you have to give, whatever
e-discovery system that you're using.
All of those charged by the gigabytes.
So everything you can reduce there is, you know, goes in your favor.
They also have a service to significantly reduce, remove your iron mountain
bill while allowing you to search against any of those tapes at any time.
They call it the intelligent tape archive.
Also for those of you using Dell source one archive system.
That they are sundowning.
They've got a service for that as well.
They can directly extract cull and store that for you as well.
There are a very impressive company that really understands the
litigation and e-discovery worlds.
And they really surprised me with how easily they're able to extract
data directly from backup tapes without needing the original software.
Even if you managed to encrypt your backup tapes.
Their name is Sullivan Strickler and I'll put a link in the show notes.
Uh, in case you're interested.
W. Curtis Preston: So if you have the need for it, if you have a compliance reason.
Or you have other business reasons we gave you some in the early, you know,
this idea of do you, do you want to track who made what, you know, who
made what, when, in case you, you know, want to be able to sue them later?
Um, just realize a good archive is a double-edged sword in that
if you were doing something wrong.
It is the smoking gun.
And you, you know, uh, it will show everything that you were
saying, uh, to, to whom and when, and, you know, all this stuff.
So a good email archive is really only helpful if you are.
The type of company that does that, does the right thing.
But, but if you've got somebody in your company that's doing the wrong
thing, the email archive will prove that and you'll lose your case.
But honestly, uh, maybe you should anyway.
But, but here's the thing, if you are doing the right thing, a bad, like
using your backup system as the archive system, it can actually do you much
more damage than, uh, you could have trouble proving that you were right.
Even if you were, even if you were right.
And your company did nothing wrong.
Uh, you could lose the lawsuit.
Any final thoughts?
Prasanna: no, I think that's, yep.
Backup is not archive and archive is not backup.
W. Curtis Preston: Yeah, and there are two different types of archive, right?
There's sort of the storage management reason and there's the real time.
System that is for compliance reasons that make sure that you get a every copy
of everything that you can search against it, uh, for the purposes of eDiscovery.
And if you need that type, then uh, you'd better buy that type.
Because if you ever actually need it, you're really gonna
want that functionality.
All right.
Hopefully that's helped with your questions about archive.
Thanks for joining.
That is a wrap.
The backup wrap up is written, recorded and produced by me w Curtis Preston.
If you need backup or Dr.
Consulting content generation or expert witness work,
check out backup central.com.
You can also find links from my O'Reilly Books on the same website.
Remember, this is an independent podcast and any opinions that you
hear are those of the speaker.
And not necessarily an employer.
Thanks for listening.