you Found the backup wrap up your go-to podcast for all things
Speaker:backup recovery and cyber recovery.
Speaker:In this episode, we take a look at a tweet from Doge that claimed that the
Speaker:GSA saved a million dollars a year by converting 14,000 magnetic tapes.
Speaker:Something else.
Speaker:A lot of people reached out to me and said, Hey, what did I think about this?
Speaker:Is it actually possible to save that kind of money with that number of tapes?
Speaker:First, let me state, we will not be actually validating the claim.
Speaker:Why is that?
Speaker:Well, there just isn't enough information.
Speaker:We don't know what tapes they moved off of.
Speaker:We don't know what kind of storage that they moved to, and without
Speaker:that information, it's just simply not possible to actually validate.
Speaker:We're also not going to discuss politics at all.
Speaker:However, we will discuss whether or not it's even possible to save that kind of
Speaker:money by modernizing your tape storage.
Speaker:Plus you get an embarrassing story about me from the old days.
Speaker:By the way, if you don't know who I am, I'm w Curtis Preston, AKA, Mr.
Speaker:Backup, and I've been passionate about backup and recovery for
Speaker:over 30 years, ever since.
Speaker:I had to tell my boss that we had no backups of the production
Speaker:database that we had just lost.
Speaker:That's why I do this podcast.
Speaker:I don't want it to happen to me.
Speaker:I don't want it to happen to you.
Speaker:On this podcast, we turn unappreciated backup admins into Cyber Recovery Heroes.
Speaker:This is the backup wrap up.
Speaker:Welcome to the show.
Speaker:I. Hi, I am w Curtis Preston, AKA, Mr. Backup, and I have with me a guy that
Speaker:I think might be just as excited as I am about my upcoming window repair
Speaker:Prasanna Malaiyandi, how's it going?
Speaker:Prasanna
Speaker:I am good.
Speaker:Chris, did you get your sashes?
Speaker:got my, I got my, they're right here.
Speaker:Look at this.
Speaker:This beautiful window balance.
Speaker:So for
Speaker:Yeah.
Speaker:who may not be aware, in addition to this being an audio podcast, we
Speaker:also publish our videos on YouTube.
Speaker:On YouTube.
Speaker:the backup wrap up,
Speaker:Yeah, you can find it.
Speaker:Yeah,
Speaker:see all of our expressions and how we make funny faces, especially
Speaker:me when I have questions.
Speaker:Um,
Speaker:yeah,
Speaker:you could also see Curtis's window sashes.
Speaker:yeah.
Speaker:No, not well.
Speaker:I, I was calling East Sasha.
Speaker:It turns out that's the wrong term.
Speaker:This is, this is a win.
Speaker:This is a window balance.
Speaker:Um, basically what this is, is my window is roughly, uh, it's like three feet wide
Speaker:and, um, and two and a half feet high.
Speaker:It's a very big window, and it's a double pane window.
Speaker:It weighs like 30 pounds.
Speaker:And so without the, you know, again, with the video you can
Speaker:see there's these springs.
Speaker:Yeah.
Speaker:Um, what this does is it, is it.
Speaker:It's like the giant springs on your garage door.
Speaker:It makes, it, makes it, yeah,
Speaker:what was going to mind.
Speaker:It's like this
Speaker:yeah,
Speaker:you lift up heavy weights,
Speaker:yeah.
Speaker:And I, I, um, it's funny, like a lot of things, I had no idea all that magic
Speaker:was happening behind the scenes until, until it broke and then I couldn't
Speaker:lift this window up to save my life.
Speaker:Right.
Speaker:Um, but no, I got it out.
Speaker:I ordered the,
Speaker:The
Speaker:yeah.
Speaker:episode after we finished recording, I think you were trying to take it out
Speaker:Yeah.
Speaker:almost killed yourself.
Speaker:Yeah.
Speaker:And I could call a guy, right?
Speaker:I could very easily call the guy, but this would cost me like a
Speaker:thousand bucks to have the guy come.
Speaker:But this part, this part here in my hand was like, two of 'em was
Speaker:like, uh, 20 bucks, you know?
Speaker:So if I can fix it for 20 bucks, I'm, I'm in.
Speaker:And so I knew you'd be excited about that.
Speaker:if it was me, I would've just, uh, painted
Speaker:guy.
Speaker:no,
Speaker:Oh,
Speaker:over the gaps, or hired the guy to paint the gaps, so
Speaker:right, yeah, yeah.
Speaker:Because literally 20 feet that way, I do have a window
Speaker:that the window is shattered.
Speaker:It's cracked beyond repair.
Speaker:And uh, so I need to have a guy.
Speaker:And so it's literally, the window is like two feet by 16 inches,
Speaker:and the quote I got was $600.
Speaker:Wow.
Speaker:the glass in there.
Speaker:Yeah.
Speaker:Um, anyway, yeah.
Speaker:So this is why I do stuff, so, uh, here's what I wanted to talk about
Speaker:and, um, I'm, I'm, I'm gonna just warn our listeners that we're, we're gonna
Speaker:talk about a topic that, whichever wa, whichever side of the political.
Speaker:World they sit on, they're immediately gonna get excited one way or the other.
Speaker:We are not gonna do that.
Speaker:Okay?
Speaker:We are not gonna take a political position, um, about this topic.
Speaker:We're just gonna talk about the tech.
Speaker:Okay?
Speaker:So.
Speaker:Um, and what I'm referring to is, uh, doge had, uh, for those not in the us
Speaker:Doge is this new department of government, government efficiency run by Elon Musk.
Speaker:Um, and, their stated goal is to try to save the government
Speaker:money and they, um, uh.
Speaker:They tweeted last week.
Speaker:Here's the, I'm gonna quote the tweet, the U-S-G-S-A.
Speaker:So that's the Government Services Administration IT team just
Speaker:saved a million dollars a year by converting 14,000 magnetic
Speaker:tapes and parentheses, 70-year-old technology for information storage
Speaker:to permanent modern digital records.
Speaker:So a lot of people texted me and, and emailed me and, and said, you know,
Speaker:Curtis, what do you think of this?
Speaker:You know, and, and, and I'll be honest, my immediate reaction was incredulity.
Speaker:It's like, I was like, I, I, I was like, how, how would you save,
Speaker:you know, for, so first off, you know, 14,000 tapes is a lot, right?
Speaker:Um, and I, and I probably my first reaction.
Speaker:Was to the comment about 70-year-old information technology storage.
Speaker:so I think that is referring to, so how long has tape been around?
Speaker:70 years
Speaker:Okay,
Speaker:it was invented.
Speaker:Tape was invented in the fifties, right?
Speaker:Mag,
Speaker:but,
Speaker:rephrase.
Speaker:Magnetic tape was invented in the fifties.
Speaker:what we have today doesn't really look anything like what it
Speaker:No.
Speaker:years ago.
Speaker:No.
Speaker:Right.
Speaker:And so, and so perhaps that was just, it, it, it certainly seemed like they were
Speaker:trying to make it sound like it was old.
Speaker:Yes.
Speaker:Which,
Speaker:Uh, yeah.
Speaker:Which, which it is.
Speaker:It is old.
Speaker:But that's all, that's also like saying, you know, my car
Speaker:is a 70-year-old technology.
Speaker:Sure.
Speaker:You know?
Speaker:70, but, okay.
Speaker:Yeah.
Speaker:Yeah.
Speaker:Okay.
Speaker:Whatever.
Speaker:Um, how long have the cars been around?
Speaker:19. Oh, something.
Speaker:Okay, so over a hundred years, 120 years maybe.
Speaker:Um, so, uh, you know, and it's based on centuries old technology of the wheel.
Speaker:yeah.
Speaker:But, but so I think, right, given that this was a tweet, an
Speaker:x, whatever you call it these
Speaker:Right.
Speaker:um, there isn't a lot of information.
Speaker:So I think the first thing to call out is we don't know type of
Speaker:tape technology they were using.
Speaker:Right, right.
Speaker:And I, but I, I, what I wanted to do is sort of evaluate this tweet
Speaker:Yes.
Speaker:and, and just talk about it and see is this actually
Speaker:something that could be done?
Speaker:Is it possible to save a million dollars a year, assuming the, um,
Speaker:you know that you had 14,000 tapes.
Speaker:Right.
Speaker:Yep.
Speaker:um, I'm gonna make a big assumption right up front, and that is, I'm.
Speaker:I'm willing to bet, given that this is a government, um, department, it's one
Speaker:that's been around for a while, that what they're talking about is nine track tapes.
Speaker:and also to go along
Speaker:Yeah.
Speaker:runs on Cobolt.
Speaker:This is true.
Speaker:Right?
Speaker:Yeah.
Speaker:Thank you very much.
Speaker:Yeah.
Speaker:Most.
Speaker:Yeah.
Speaker:So again, what we're talking about is most likely mainframes running on cobol
Speaker:Yeah.
Speaker:mainframes often used.
Speaker:Nine track tapes now.
Speaker:Uh, so let for those of you, uh, you know, what, you know, for, for the, for
Speaker:the video audience, I'll put up a picture of what a nine track tape looks like for
Speaker:the, um, uh, for the, uh, r our role.
Speaker:That's A-U-R-A-L, uh, you know, uh, listeners.
Speaker:Basically, think of any old movie that you've seen computers in
Speaker:Yeah.
Speaker:from the sixties and seventies, even the eighties.
Speaker:And think about these giant refrigerator sized tape drives that had reel to reel.
Speaker:Uh, that's what a nine track tape drive looks like.
Speaker:to reel, think like movie projector.
Speaker:Yes.
Speaker:Yes.
Speaker:It, well, it's two big reels.
Speaker:That, and, and each reel, and, and again, just for, just for.
Speaker:Uh,
Speaker:Size
Speaker:know.
Speaker:Yeah.
Speaker:Size comparison.
Speaker:A nine track tape is about the size of a medium pizza.
Speaker:How's that for, for those that have, you know, just to get something in your head,
Speaker:What if they, what if you don't eat pizza?
Speaker:it's, who doesn't eat pizza?
Speaker:Get that person out here.
Speaker:That's what I'm gonna have to say.
Speaker:I mean, even vegetarians eat pizza, you know?
Speaker:Uh, even vegans eat pizza.
Speaker:Is there a group of people that are like anti pizza?
Speaker:For
Speaker:No.
Speaker:audience, they may not.
Speaker:The sizes are different.
Speaker:Okay, fine.
Speaker:I don't know what to tell the international people.
Speaker:But one thing though is I know we talk about mainframes and we don't know what
Speaker:technologies they use at the government services agency, and so it is hard to
Speaker:say are the mainframes they're running.
Speaker:That old, because you could still buy a modern mainframe.
Speaker:There are still companies that sell modern mainframes.
Speaker:Right.
Speaker:So it with modern tape drive technology and other things like that.
Speaker:So
Speaker:I, here, here's, here's where, here's where, but here's where I'm, here's
Speaker:why I've settled on nine track tape.
Speaker:We know that the Doge crew tends to be younger folk, right?
Speaker:And by younger folk, I mean people younger than me, right?
Speaker:So like you, you are younger folk, right?
Speaker:Um.
Speaker:I just think that they walked into the GSA and they saw these monstrosities and said,
Speaker:oh my God, we've got to get rid of this.
Speaker:And, and honestly, if I walked into a data center and saw nine track tape drives
Speaker:in production, I. I would be alarmed.
Speaker:Right.
Speaker:And if you're not even used to tape, right.
Speaker:'cause we, you know, we've talked about that, that not everybody has
Speaker:even seen tape in production period.
Speaker:If you walked into and you saw those things and you're like, what am I
Speaker:watching a movie from the fifties?
Speaker:Yeah.
Speaker:So I, I just think that that was the, the, if, if they walked in and they
Speaker:saw like a, a bunch of LTO drives, I don't think it would've had the effect.
Speaker:That resulted in the tweet.
Speaker:Yeah.
Speaker:So I think for the listeners,
Speaker:yeah.
Speaker:are considering for this exercise that
Speaker:Yeah.
Speaker:they are using mainframes.
Speaker:They are using nine track tapes
Speaker:Right.
Speaker:Yeah.
Speaker:there are 14,000 of them.
Speaker:14,000 of them.
Speaker:Okay.
Speaker:So let's talk about what that would look like.
Speaker:So again, it's
Speaker:tapes?
Speaker:what?
Speaker:Where would you store 14,000 tapes?
Speaker:Well, in a tape library.
Speaker:And by tape library, I don't mean.
Speaker:Robot, I mean literally a library where you would put tapes.
Speaker:Um, I, again, you know, I'm older than dirt, so I remember back when we had
Speaker:these things in production back at the bank, and the way you store nine
Speaker:track tapes is on a, it's like you've literally, they're, they're, again,
Speaker:they're about the size of a medium pizza you wanna store them vertically.
Speaker:You don't wanna store them.
Speaker:You don't want the tape to like
Speaker:I was
Speaker:flatten.
Speaker:Right?
Speaker:Uh, you, you could theoretical.
Speaker:The problem with that, like storing like a book is if they're not like
Speaker:a book and they could roll out.
Speaker:Hmm.
Speaker:So you want, you want something to sort of hold them in place so.
Speaker:Like, think about that for a minute.
Speaker:So they're about an inch wide.
Speaker:So a hundred tapes is, let's say 120 tapes is 120 inches, which is 10 feet, right?
Speaker:So we've got 14,000 of these things.
Speaker:So what is that?
Speaker:That's, let's do some math here.
Speaker:So
Speaker:1400,
Speaker:hang on.
Speaker:times.
Speaker:10. 1400 feet.
Speaker:So 1400 feet.
Speaker:So they're, they're about a, they're about, like, they're, they're a little
Speaker:over, they're about 13 inches tall.
Speaker:Right?
Speaker:So then you're gonna have some space around that for the rack.
Speaker:So let's say 18 inches, right?
Speaker:So we could do like four or five high, I think.
Speaker:So, um, it's gonna be, it's gonna be a pretty big, the, the
Speaker:point is it's gonna be a pretty big, um, physical room to hold.
Speaker:14,000 tapes.
Speaker:That's number one.
Speaker:The second thing is that, um, the actual drives, um, are monstrous.
Speaker:It's like the size of a fridge, right.
Speaker:You were mentioning
Speaker:I think it's 33 inches wide.
Speaker:That's a reasonably sized fridge.
Speaker:And then there are also really deep, because there's all kinds
Speaker:of stuff in there about, uh, like there's a vacuum, literally a vacuum
Speaker:to maintain tension on the tape.
Speaker:And, uh, it's just really, it, it's a very.
Speaker:Complicated series of, of mechanics to, to be able to, to have this tape
Speaker:do what it, what you want it to do.
Speaker:So, and being a government organization, you are going to have
Speaker:multiple of them because no one has made a nine track drive in years.
Speaker:So you're pro, you probably have a small collection of them in
Speaker:order to ha have enough running
Speaker:Yeah.
Speaker:Right.
Speaker:Uh, because you.
Speaker:I, I'm, well, there's a couple of possibilities here, right?
Speaker:You, you have a, a small number and then you have a very expensive service contract
Speaker:of somebody to come in and maintain them, that, that's highly possible.
Speaker:it's the
Speaker:Or you
Speaker:likely.
Speaker:Yeah, it's probably likely, right?
Speaker:So they probably have like three or four, and then they have a service
Speaker:contract so that at least one of them is running at all times.
Speaker:Right.
Speaker:Um, and, um, so just.
Speaker:The physical presence of these tape drives and their associated tapes
Speaker:are going to take up a significant amount of space, which has a cost.
Speaker:I don't know if it approaches a million dollars a year, but again,
Speaker:I've never built and maintained a government building before.
Speaker:Right.
Speaker:But as soon as we start talking about maintaining.
Speaker:Tape drives that haven't been made for 20, 30 years, then very quickly, that could,
Speaker:that could approach a million dollars.
Speaker:Thing on my mind though, so there are 14,000 tapes
Speaker:Uhhuh.
Speaker:and assuming they're nine track, that seems like a lot of data.
Speaker:But you're smiling, which
Speaker:I'm,
Speaker:Which means that you disagree with that statement.
Speaker:yeah.
Speaker:So how much data do you think, so we're gonna go, we're gonna,
Speaker:we're gonna go with, again, we're assuming all kinds of things.
Speaker:I'm gonna, for the purposes of discussion, assume that we're
Speaker:talking about the biggest, fastest.
Speaker:A tape drive that, you know, a a nine track tape drive that they made.
Speaker:How much data do you suppose is on such a tape?
Speaker:So given that it's probably from the seventies, eighties,
Speaker:Uhhuh?
Speaker:and everything was smaller, and I know we've talked about your experiences
Speaker:at the bank and how you had servers that were like a hundred gigs or less.
Speaker:Yeah.
Speaker:Right.
Speaker:I'm going to go with like 10 gigs.
Speaker:That's adorable.
Speaker:So at the latest available nine track tape drive, stored 150 megabytes,
Speaker:Wait, what?
Speaker:Gigabytes, megabytes.
Speaker:a megabytes 150 megabytes.
Speaker:Compressed or uncompressed.
Speaker:There was no compression.
Speaker:Oh,
Speaker:So 150 megabytes.
Speaker:Um, yeah.
Speaker:so, so, okay, so that's not a lot of data.
Speaker:So
Speaker:So, so it's both.
Speaker:It's it, it's so physically big.
Speaker:The drives are really physically big.
Speaker:The tapes are really physically big, physically big, but they don't,
Speaker:they don't hold squat in terms of data, like 150 megabytes get, you
Speaker:can't even buy like a thumb drive
Speaker:So,
Speaker:megabytes now.
Speaker:so what are they?
Speaker:So assuming that this is our backup infrastructure.
Speaker:I think it's their archive infrastructure.
Speaker:Okay.
Speaker:That's
Speaker:Yeah.
Speaker:I don't think this is back.
Speaker:I don't think they're making new tapes on this infrastructure.
Speaker:so this is all just, which also makes sense because.
Speaker:It's been
Speaker:I, by the way, I could be wrong.
Speaker:Yeah, yeah.
Speaker:No,
Speaker:I could totally be wrong.
Speaker:that they are only using it for archive, but that makes sense,
Speaker:Right, right.
Speaker:I.
Speaker:less data, less frequent data.
Speaker:You just write it out and you keep it stored because you need
Speaker:to put it somewhere for the
Speaker:Yeah.
Speaker:a hundred years.
Speaker:Yeah.
Speaker:Which, which brings up an interesting side issue that has nothing to do with cost.
Speaker:If they wrote these, uh, this data on these tapes, and these are archived tapes.
Speaker:These are archived tapes from 10, 20, 30 years ago, maybe even older.
Speaker:Those tapes are not made to hold data that long.
Speaker:Mm.
Speaker:Right.
Speaker:Modern LTO technology is, you know, uh, rated at holding data for 30 years.
Speaker:This stuff there, there is, there is relatively zero chance that there
Speaker:isn't bit rot on some of this data.
Speaker:Right.
Speaker:And that will only go, that will only
Speaker:worse.
Speaker:with time.
Speaker:Yeah.
Speaker:And so do you think that they've never touched these tapes?
Speaker:I think that some of these tapes have never been touched.
Speaker:Yeah.
Speaker:So if you never touch the tapes and no one knows about it, do you need to keep them?
Speaker:If, if bit Rod happens and no one is there to read it, did it really happen?
Speaker:Exactly.
Speaker:Well, that, well, you know, that is a, that is a topic that
Speaker:we talk about a lot, right?
Speaker:Uh, if you've got tapes.
Speaker:I,
Speaker:So the, the question is, do you know what's on those tapes?
Speaker:Right.
Speaker:Do you have a reason that I'm, I'm guessing, and I think it's
Speaker:a perfectly valid assumption.
Speaker:I'm assuming that, um, that they have a reason to keep
Speaker:the data that's on this tape.
Speaker:That this is historical GSA data, and that they have to keep it for, you know, it's
Speaker:literally their purpose in life, right?
Speaker:They have to be able to go back and, and, and pull up this stuff.
Speaker:I just don't think they do it very often.
Speaker:Yeah.
Speaker:Right.
Speaker:So meanwhile, and, and I'm, and I harken back, it's not the same technology,
Speaker:but I do harken back to what happened when they started pulling out all
Speaker:the old celluloid celluloid movies,
Speaker:Hmm.
Speaker:right?
Speaker:So, for example, when they went to, um, I, I, I, for those of you that
Speaker:are movie buffs, read the story of what happened when they went back
Speaker:to do a re-release of Star Wars.
Speaker:Star Wars was stored on the old celluloid stuff and it was a hot mess.
Speaker:Uh, they almost didn't get it right.
Speaker:, so basically the point is when you go to pull that stuff, um,
Speaker:you know, also they lost the original tapes of the moon landing,
Speaker:Hmm.
Speaker:Well, you know
Speaker:is, which is, which does not help for those who don't
Speaker:think it actually happened.
Speaker:But anyway, that's
Speaker:else is also, people are realizing they're losing their data for,
Speaker:what.
Speaker:uh, apparently I think it's Warner Brother movies from the early two
Speaker:Yeah, yeah, I did read about that.
Speaker:Yeah.
Speaker:They're no longer viewable, so.
Speaker:Yeah.
Speaker:So there, so, so that is a totally separate, uh, reason
Speaker:why they should potentially look at migrating this old data.
Speaker:But the question is, the claim was that they could save
Speaker:money by doing this migration.
Speaker:And most of my friends who reached out to me.
Speaker:You know, had friends, you know, claims like, you know, bs.
Speaker:You know, it's not true.
Speaker:It couldn't, there's no possible,
Speaker:so before we get into the second part, right, of
Speaker:yeah,
Speaker:so the tweet it said that they are using.
Speaker:Permanent or what was the phrase that they used?
Speaker:it, it said, um, it said permanent modern digital records.
Speaker:Okay, so
Speaker:I don't know what that means.
Speaker:I.
Speaker:does this sound like they're going to store it in some database or
Speaker:know.
Speaker:object store or something like that?
Speaker:Whatever.
Speaker:It's
Speaker:Yeah, the, the, it was, there was even less information about the second half
Speaker:than there was about the first half.
Speaker:Okay.
Speaker:But let's say whatever they pick, right?
Speaker:They're gonna
Speaker:Yeah.
Speaker:storage system.
Speaker:I, I'll just say this, I'm not even gonna try to justify or to explain
Speaker:what it is that they're doing.
Speaker:I'll just say if I were doing it.
Speaker:This is what I would do.
Speaker:would you do?
Speaker:Yeah.
Speaker:So first off, I would not, I would, well, there's a couple of different choices.
Speaker:But I'm saying if you were already using tape and you already have a
Speaker:system that knows how to talk to tape, I would probably continue to use tape.
Speaker:I just would move off of the old nine track tapes and onto
Speaker:something more modern like LTO.
Speaker:Right.
Speaker:I would probably migrate to LTO nine, which is the most recent, because we're
Speaker:talking about really long-term storage.
Speaker:Anyway, um, go ahead.
Speaker:data can you store in an LT nine?
Speaker:45 terabytes.
Speaker:And you said
Speaker:So,
Speaker:were one 50 megabytes.
Speaker:so, so, so here's the thing.
Speaker:You could fit all 14,000 tapes on less than about 20% of a single LTO nine
Speaker:cartridge, so they can make 10 copies.
Speaker:They would've spent a thousand dollars on media.
Speaker:Crazy.
Speaker:Yeah.
Speaker:Now I will just say this, um, it's, it is not gonna, it's not gonna be overnight.
Speaker:Right.
Speaker:That, that was one of the critiques that people had of the, of the tweet is
Speaker:they spoke about it in the past tense.
Speaker:There is zero possibility that they've done this already.
Speaker:I think they just sort of figured out that, you know, they're, you know,
Speaker:they're referring to it like that.
Speaker:So I think what you said makes total sense.
Speaker:Now, if I was dumb though, so I wanna play the other side,
Speaker:Okay.
Speaker:So if I was dumb and I said, okay, it's gonna be a disc
Speaker:space system, doesn't matter how
Speaker:Mm-hmm.
Speaker:Um, it's gonna be stored on disc.
Speaker:This is probably important data, which
Speaker:Mm-hmm.
Speaker:that I might need another copy somewhere for high availability
Speaker:Mm-hmm.
Speaker:Mm-hmm.
Speaker:Right.
Speaker:And disaster recovery.
Speaker:Mm-hmm.
Speaker:And then by the way, this is important data.
Speaker:I don't want to lose it, which means I probably need to take a backup of it.
Speaker:Right?
Speaker:So now I have three additional copies.
Speaker:I.
Speaker:In, in, in two different things and
Speaker:places
Speaker:to be,
Speaker:I have three additional copies on
Speaker:yeah.
Speaker:pieces of media, one of which is offsite, right?
Speaker:Our famous, the famous 3, 2, 1 rule.
Speaker:And all of this costs money, I'm probably running this in some data
Speaker:center, not under someone's desk,
Speaker:Right.
Speaker:that it's taking power and cooling and maintenance and upkeep, patching.
Speaker:Security issues of keeping everything online, all the rest of the stuff
Speaker:that we talk about on this podcast.
Speaker:And so they're saying that, that doing all of that cost is going to be
Speaker:saving them a million dollars versus even just keeping it on nine track.
Speaker:Forget about even moving to LTO.
Speaker:Yeah, so, so, so here, so, so yours.
Speaker:You are making an assumption there.
Speaker:Yes.
Speaker:That they would want to keep it on disc.
Speaker:And I'm saying I don't, I don't think they have a requirement to keep it online.
Speaker:I think that this is archival data and I think it's fine that it's on tape.
Speaker:I would have it on multiple tapes and I would have those
Speaker:tapes in multiple locations.
Speaker:I do not think there's a requirement to have one of those copies on disc,
Speaker:I don't think they need to, but I just, my interpretation of the tweet was they're
Speaker:probably storing it on disc somewhere.
Speaker:uh, already or.
Speaker:They're gonna archive it to this somewhere.
Speaker:I, those, those, those words, those words hurt my ears.
Speaker:know.
Speaker:two disc.
Speaker:But, um, I'm, I'm gonna take that out, uh, because I might wanna
Speaker:work for a company that does that.
Speaker:But, um, so I, I just don't, I don't think that's a requirement.
Speaker:Right.
Speaker:And, and this is about saving money.
Speaker:So I would, I would, assuming this, again, if I was running the show, I
Speaker:would say, what's our requirement?
Speaker:How often do we read this stuff?
Speaker:Well, basically never.
Speaker:So I would say, okay, well let's make sure we make it on
Speaker:multiple copies, multiple tapes.
Speaker:And, and, and here's, here's what's interesting is you could, even though
Speaker:purchasing 14,000 backup tapes, just that is gonna cost you over a million bucks.
Speaker:Yeah.
Speaker:Right.
Speaker:But it'd be one time purchase and then you'd be good for another 30 years.
Speaker:Right.
Speaker:Um, even if what you did was a one-to-one conversion of, of the be 'cause one
Speaker:of the problems is even though it'd be the most inefficiently stored LTO
Speaker:tape in history, right, you'd have 150 megabytes on a, assuming each of those
Speaker:tapes was full, which it probably isn't the case, you'd have 150 megabytes
Speaker:on a, you know, 45 terabyte tape.
Speaker:It'd be ridiculous.
Speaker:But one of the problems when we start talking about moving forward with
Speaker:tech like this is that you don't.
Speaker:You might not be able to consolidate the, the data onto
Speaker:from multiple tapes onto one tape.
Speaker:If you could, then that would be the way to go.
Speaker:Yeah.
Speaker:If you could figure out how to tell the mainframe, again, this is not, this
Speaker:is not the newest technology, right?
Speaker:If you could tell, if you could figure out how to tell the mainframe to take the
Speaker:a hundred, you know, the 14,000 tapes.
Speaker:And consolidate them onto one tape.
Speaker:And then once you make that one tape, then you make multiple copies of that one tape,
Speaker:then, then you, that would be amazing.
Speaker:But I, I'm not sure if that's going to be possible without
Speaker:a lot of specialized tech.
Speaker:Which then goes back to the, to the, the cost issue.
Speaker:But you're still gonna need some tech.
Speaker:Even if you move it to a digital modern, I can't remember the
Speaker:words, A modern digital format.
Speaker:Yeah.
Speaker:Right.
Speaker:You're still going to need that mainframe to read the old data
Speaker:Yes.
Speaker:out somewhere, which means you're probably gonna need
Speaker:But you,
Speaker:someone who
Speaker:oh, you, oh.
Speaker:to do this.
Speaker:will need those people, regardless of how you do it, you're gonna
Speaker:need some people that know what they're talking about, right?
Speaker:Like, you know, my employer, right?
Speaker:You know, S two data does this all the time.
Speaker:Yeah.
Speaker:You're going to need somebody that knows that that's good at mainframe tapes,
Speaker:you know, understands all of that stuff.
Speaker:If it's the better the person you get the.
Speaker:Better.
Speaker:The possibility that you might actually be able to consolidate tapes
Speaker:onto that would be the true goal.
Speaker:If you could consolidate those 14,000 tapes onto one LTO, that would be amazing.
Speaker:It's just, I'm not sure.
Speaker:Again, I'm not even as old as I am.
Speaker:I'm not a mainframe guy.
Speaker:I've seen mainframes.
Speaker:I've seen nine track drives.
Speaker:I've never actually have to drive one.
Speaker:Right.
Speaker:So you need someone older than me, right.
Speaker:We have, we have those people.
Speaker:And that's two data, right?
Speaker:We have one guy that's like, this is, this was his life, right?
Speaker:And, and you know, uh, and he, and he's brilliant.
Speaker:Um, but like, you gotta pull that guy outta retirement, you know,
Speaker:pay him lots of money to, to, to figure this out and see if there's
Speaker:a way to migrate those tapes.
Speaker:All of that will have a cost.
Speaker:Having said that.
Speaker:It's still possible that you could migrate all those tapes.
Speaker:You would have a one-time cost, you still would save a million dollars a year.
Speaker:It's just, we're not talking about until like next year,
Speaker:Yeah, I just
Speaker:maybe even two years.
Speaker:Okay.
Speaker:Go ahead.
Speaker:They just go to the local computer store or Amazon and
Speaker:they buy an external hard drive.
Speaker:I.
Speaker:By the way, you're killing me by the way.
Speaker:Let me just talk about another aspect of this.
Speaker:This is gonna take a minute.
Speaker:I. Yep.
Speaker:Right.
Speaker:So I, I ran some numbers and assuming the maximum transfer rate, assuming all of
Speaker:the tape drives are full, assuming that the tape drives all work all the time.
Speaker:Which, which is not a valid assumption every time I use old tape drives, you
Speaker:know, they, they, they don't, they don't work, you know, they, they,
Speaker:they fail on a pretty regular basis.
Speaker:Um, and assuming that you could do this 24 7, which you can't, and you
Speaker:And you have people there 24
Speaker:have people there 24 7, which you're not gonna do that, um,
Speaker:we're talking, um, it's, I think, I think it came out like 1100 hours.
Speaker:Right.
Speaker:When you factor in a nine to five work week, when you factor in,
Speaker:let's get three running tape drives.
Speaker:When you factor in the amount of project management that this is going to have to
Speaker:have, because anyone who's ever done any sort of government work knows that you've
Speaker:got prove and proven prove your process.
Speaker:You've gotta spend a ton of time proving your process before you can go do this
Speaker:because you're doing a one-way trip.
Speaker:Right.
Speaker:You know, to, to, to use must terms.
Speaker:We're going to Mars, right?
Speaker:You can, you can't.
Speaker:Oh, we forgot to put the tire on.
Speaker:You gotta sort it out before you've gotta do this one time, right.
Speaker:Um, so you just very quickly add some, so I came up with an estimate of six
Speaker:months, uh, uh, you know, of time during which you're going to be paying someone
Speaker:who's three, 300 to 500 bucks an hour.
Speaker:Yeah.
Speaker:Uh, so, you know, you're looking at, you're looking at $300,000 or so, and, and
Speaker:professional services, uh, that's just for one person and then you're gonna, they're
Speaker:probably gonna need some support people.
Speaker:There's a lot of moving around and moving tapes around.
Speaker:Um, so this is gonna take a while.
Speaker:It's not gonna.
Speaker:So there's, there's my opinion, there's, there's zero chance that
Speaker:this has happened already, but I don't think that makes the Tweet untrue.
Speaker:and this is, I think goes back to something we talked about before.
Speaker:If Bit Rod happens, notice it?
Speaker:Well, that's the other thing is you won't really know if
Speaker:bit rot happened, most likely
Speaker:Yeah.
Speaker:you go to transfer the data.
Speaker:and so I, another question is maybe they just ditch the old
Speaker:things and say, keep the old stuff on the old stuff moving forward.
Speaker:Yeah, it's not well, well, it's, they can, the que the po, but the
Speaker:point is that has a cost, right?
Speaker:There's an ongoing cost.
Speaker:This, this is the discussion that people get in.
Speaker:There's an ongoing cost to having those tape drives.
Speaker:You have to maintain them.
Speaker:They have to continue to work.
Speaker:Having those tape drives, having 14,000 tapes, you know, we just
Speaker:talked about the fact that having.
Speaker:What did I say?
Speaker:A hundred?
Speaker:A hundred of them having a hundred of them is 10 feet long.
Speaker:okay.
Speaker:know,
Speaker:assume that they
Speaker:there's power in, there's power in cooling for all of that.
Speaker:let's just assume they're gonna keep one tape drive.
Speaker:Okay.
Speaker:You can't, but go ahead.
Speaker:Or, okay, let's say they're gonna keep two and they're gonna keep all the tapes
Speaker:Okay.
Speaker:and they're not gonna touch any of the old stuff
Speaker:Okay.
Speaker:and they're gonna say, moving forward, we're gonna use a new form of archive.
Speaker:Uh huh.
Speaker:Right.
Speaker:That doesn't use those old nine track tapes.
Speaker:Yeah.
Speaker:Okay.
Speaker:Which they've probably already done all this, but go ahead.
Speaker:So going back to what you had said previously or earlier.
Speaker:Maintenance and upkeep and all the rest for these tape drives.
Speaker:Mm-hmm.
Speaker:Do you think that is where they're seeing, they're going to try to
Speaker:plus managing and swapping out the tapes and everything else?
Speaker:Mm-hmm.
Speaker:think that is potentially where they're saying they're getting
Speaker:a million dollar savings from?
Speaker:I think that's exactly what I'm saying.
Speaker:Okay.
Speaker:I think that, I think that they're saying that the cost, the, the physical
Speaker:footprint of how big these tape drives are, how big the tapes are.
Speaker:Just, just, just the physical presence of them has a cost.
Speaker:It's not a million dollars a year.
Speaker:I, at least, I don't think, but then there's a contract, there is a service
Speaker:contract to maintain those tape drives.
Speaker:Um, there's also.
Speaker:They got some person that looks like that, that looks like me, that
Speaker:understands how this stuff works.
Speaker:And tho they're not cheap.
Speaker:Right?
Speaker:So I think, I think that it's, it's totally true that the ex, just
Speaker:the mere existence, the continued existence of these systems is
Speaker:costing them a million bucks a year be because of the government aspect
Speaker:yeah,
Speaker:and the fact that this stuff really, really, really has to work.
Speaker:And so that, that, you know, that costs a lot.
Speaker:my question though, maybe I'm not stating it clearly, is.
Speaker:What if you left the old stuff around and just said old stuff?
Speaker:Leave it as it is.
Speaker:We'll pay the sunk cost and the ongoing cost of storing things
Speaker:Uh,
Speaker:and
Speaker:yeah, I, I, no, no, you, sorry to interrupt, but I get what you're
Speaker:saying, but, and I'm just saying.
Speaker:That just keeping the old stuff is costing you a million bucks a year.
Speaker:If you can spend a million bucks
Speaker:Hmm.
Speaker:something like that, and then the, and then your million bucks a year
Speaker:goes away, this is a beautiful thing.
Speaker:Right.
Speaker:That, that's what I'm saying.
Speaker:I, I guess what I'm saying is DOGE's claim doesn't sound astronomically impossible.
Speaker:I I do think it's impossible that they've done it already, but I think it's highly
Speaker:possible that they're gonna do it.
Speaker:Moving forward, it's gonna take them several months.
Speaker:Right.
Speaker:Because of the amount of project management and time and just
Speaker:the, just the sheer, it's 1100 hours to read 150 megabytes off
Speaker:of all tape drive, you know, off.
Speaker:Yeah.
Speaker:this is no different than companies that say are migrating
Speaker:from LT O five to LT O nine.
Speaker:Well, it's just a little bit more extreme.
Speaker:a little bit more extreme, but it's
Speaker:Yeah.
Speaker:similar.
Speaker:Like this is what
Speaker:Als Also, also, sorry to interrupt you.
Speaker:Zero level of automation here.
Speaker:That's true.
Speaker:There's no tape library for nine track drives.
Speaker:You need a person standing in front of.
Speaker:The tape drives to mount them, to feed the tape over, to do the thing, you
Speaker:know, and put it over on the, you know, it, it, it is a very manual process.
Speaker:Whereas you could absolutely buy, you could rent probably a tape
Speaker:library that you could fit 14,000.
Speaker:It'd be a big tape library, right?
Speaker:It'd be a couple hundred feet long,
Speaker:Yeah.
Speaker:hundred feet long, nah, it'd be like 50 feet long to put 14,000 tapes in it.
Speaker:You could push a button.
Speaker:And you could come back and it would be done.
Speaker:Yep.
Speaker:So it's, so, it's, it's, that's why I'm saying it's, it's an extreme
Speaker:version of what you're saying.
Speaker:No, that's something I did not think about.
Speaker:Yeah.
Speaker:Was that manual effort?
Speaker:Yeah.
Speaker:Yeah.
Speaker:Very, very manual.
Speaker:are you looking for a job in standing in front of those?
Speaker:Uh, tape.
Speaker:drives Curtis for the
Speaker:me, call me.
Speaker:Well, first I'd have to learn nine track tape drives.
Speaker:What's funny is I used to manage.
Speaker:Nine track.
Speaker:I used to manage people that mounted nine track tape drives, and I did used
Speaker:to have to mount them, but I never actually had to be the one on the
Speaker:keyboard doing, you know, doing the thing.
Speaker:Um, and um, and, and funny story, uh, that, that figures in that.
Speaker:So a hundred years ago when I was at the bank, and we had, so
Speaker:we were a mainframe shop, right?
Speaker:We were a credit card company.
Speaker:The mainframe was in Dallas.
Speaker:We were in Delaware, we had what are called channel
Speaker:attached tape drives, right?
Speaker:So basically the tape drives were in Delaware, the computers were in Dallas.
Speaker:You could cut a tape in Delaware from the data it was in that was in Dallas.
Speaker:And, um, and, and, and that, that, that was amazing to me back in the day.
Speaker:Right.
Speaker:The funny thing was to get data from upstairs to downstairs.
Speaker:We had to cut a tape and we had to bring that tape down to downstairs, right?
Speaker:And the, it was a really big operation to cut these tapes and to get them,
Speaker:because this was a credit card company and we absolutely played the float.
Speaker:What does that mean?
Speaker:The moment your money comes in, we're gonna play with it.
Speaker:We're gonna speculate on the market and all the, all the
Speaker:things that a bank would do.
Speaker:Play with your money.
Speaker:Then, and then credit it to your account like a couple days later.
Speaker:That, that's just the way it works.
Speaker:Right.
Speaker:If you didn't, if you weren't aware that that's happening,
Speaker:that's absolutely happening.
Speaker:And so it was, it was, I dunno, about a couple days, but it
Speaker:was definitely some time.
Speaker:And so it was a really big deal of getting these tapes downstairs.
Speaker:And, uh, one year we had a mainframe outage.
Speaker:Hmm.
Speaker:Our first ever, I was there for almost three years and our, it was
Speaker:our first ever mainframe outage.
Speaker:And we were in, uh, and, and we had, my memory was that we had three
Speaker:of these nine track tape drives.
Speaker:It was half, was like half of the space in our tape library, tape library,
Speaker:meaning the room where our tapes were.
Speaker:And, um, we had a team of people whose job it was to, to mount these tapes.
Speaker:It was well acknowledged that the moment the mainframe came back up,
Speaker:the moment the little light came on, on the channel attached tra drives.
Speaker:We were to start mounting tapes
Speaker:Yeah.
Speaker:because we want to play the float.
Speaker:So my boss's boss's, boss's boss's boss came by.
Speaker:I was already in there.
Speaker:I was like, guys, you understand?
Speaker:Here's the thing you see.
Speaker:I want you like standing at parade rest.
Speaker:You know, that's a military term, standing, you know, with your hands
Speaker:behind your back standing in front of this table library watching for that light.
Speaker:I heard the mainframes coming on.
Speaker:The moment that light turns red, you know, put the, or probably green, I don't know,
Speaker:Yeah.
Speaker:whatever that light comes on.
Speaker:Put the tape on, puts the, yeah, actually I think we
Speaker:probably prem mounted the tapes.
Speaker:The moment that light turns on, press the button, start spinning tapes and
Speaker:get those tapes down to payments.
Speaker:And, um, the boss's, boss's boss's boss.
Speaker:Tom Thamaides was his name, and he came in there to make sure that I
Speaker:knew, that I knew, so that they knew, you know, how important this was.
Speaker:Right.
Speaker:And so, and this was not, this did not happen every day.
Speaker:We did not get
Speaker:Yeah.
Speaker:the, you know,
Speaker:Yeah.
Speaker:basically the second in command at the bank Yeah.
Speaker:And is in there.
Speaker:And, um, um, and so I'm like, yeah, I got it.
Speaker:I got it.
Speaker:Mr. Mr. Thames, I've got it.
Speaker:Um, you see these guys, I literally, I am not, I literally had them
Speaker:standing in front of the tape drive.
Speaker:Three of them, three tape drives, three people.
Speaker:We're on it.
Speaker:And he's like, okay.
Speaker:So he goes over and I go to let him out of the tape, let him out of the room.
Speaker:And we had these, we had these mag lock, um, doors.
Speaker:And the way it would work is if you, if the, if the, if the door open, we had
Speaker:these, we had this piece of metal that, that would block a gap between the door
Speaker:so that you wouldn't be able to open the, um, the motion detector thing.
Speaker:Um, and, um, um.
Speaker:So there was a button to press to open up the mag lock if, if the
Speaker:door got opened wrong, and so I pressed the button to let him out.
Speaker:the wrong button.
Speaker:Yeah,
Speaker:Which button did you push, Curtis,
Speaker:the EPO button, the emergency,
Speaker:not
Speaker:the emergency power off,
Speaker:and how
Speaker:so
Speaker:from that?
Speaker:about three and a half hours.
Speaker:Uh,
Speaker:No one knew how to turn the power back on.
Speaker:So I extended the first ever mainframe outage by like three hours.
Speaker:Curtis did you still had a job there though, right?
Speaker:I did, I did not get fired.
Speaker:Uh oh.
Speaker:The funniest thing is, the funniest thing is, um, my boss, Susan
Speaker:Davidson, uh, great, great woman.
Speaker:She was normally, she was that kind of boss that you wanted to have,
Speaker:that basically, if you did something stupid, she would shield you from
Speaker:Yeah.
Speaker:the, like, this guy.
Speaker:But in this case, this guy was standing right there
Speaker:Yeah, exactly.
Speaker:saw me do the thing and um, Susan Davidson came in, she came in the door and she's
Speaker:like, it's, it's really quiet in here.
Speaker:And, and, and she goes, um.
Speaker:She said, um, what, what, what's going on?
Speaker:And I'm like, um, somebody pressed the a PO button and she's this person who would
Speaker:normally like protect me from such things.
Speaker:Who the fuck would press the a PO button right now?
Speaker:I.
Speaker:Oh,
Speaker:That would, that would be me.
Speaker:And like, actually the guy Tom, to me, he like pointed, he like pointed to me.
Speaker:oh, Curtis, do you hang your head in shame?
Speaker:I did.
Speaker:Uh, she did not fire me, but I did quit shortly after that.
Speaker:Anyway.
Speaker:So long story short, I think it's highly possible Doge may have found
Speaker:a way to save a million bucks.
Speaker:I think it's highly possible that this particular claim,
Speaker:uh, could be, could be true.
Speaker:Um, and I wish them the best of luck.
Speaker:Mm-hmm.
Speaker:Um, so anyway, uh, day in the life of tape drives.
Speaker:Day in the life of really old tape.
Speaker:How's that?
Speaker:All right, well thanks for, let me like, you know, think through this, you know?
Speaker:Yeah, no, it was good.
Speaker:Um, learned about nine track tapes more than I ever want to.
Speaker:By the way, have been around nine track tapes.
Speaker:Yeah.
Speaker:Yeah, my dad used to work for a company that did mainframes and every once in
Speaker:a while on the weekends, I would go in with him to just hang out and he'd be
Speaker:Yeah.
Speaker:and he'd let me into their labs and yeah, there were nine track tapes in there.
Speaker:So
Speaker:So this was like in the nineties.
Speaker:yeah, late
Speaker:you were, how old and how old were you in then?
Speaker:probably was, uh, probably eight, nine, something like that.
Speaker:You're killing me smalls.
Speaker:Yeah.
Speaker:Oh, hope.
Speaker:I hope everyone else listening feels as old as I do right now.
Speaker:Uh, thanks for listening.
Speaker:That is a wrap.
Speaker:The backup wrap up is written, recorded, and produced by me w Curtis Preston.
Speaker:If you need backup or Dr. Consulting content generation or expert witness
Speaker:work, check out backup central.com.
Speaker:You can also find links from my O'Reilly Books on the same website.
Speaker:Remember, this is an independent podcast and any opinions that
Speaker:you hear are those of the speaker and not necessarily an employer.
Speaker:Thanks for listening.