1 00:00:00,000 --> 00:00:03,540 In case you haven't heard, ransomware is after your backups. 2 00:00:04,050 --> 00:00:08,390 This encore episode is one of our most popular episodes from the last few years, 3 00:00:08,780 --> 00:00:14,390 and it covers exactly how to protect your backups from ransomware, why attackers 4 00:00:14,390 --> 00:00:19,120 are exfiltrating the data before they even try to encrypt it, uh, what real 5 00:00:19,139 --> 00:00:23,999 immutability looks like, uh, who should have root access, and the questions that 6 00:00:24,000 --> 00:00:29,669 you need to ask your backup vendor before you give them, uh, your data and trust 7 00:00:29,670 --> 00:00:31,350 them with your last line of defense. 8 00:00:31,830 --> 00:00:34,879 If this is your first time watching me or listening to me, I'm W. 9 00:00:34,889 --> 00:00:37,029 Curtis Preston, AKA Mr. Backup. 10 00:00:37,559 --> 00:00:42,630 I've been obsessing about backup and recovery for over 30 years now. 11 00:00:42,730 --> 00:00:44,649 If that's your bag, then I'm your guy. 12 00:00:45,099 --> 00:00:48,280 You're not gonna find anyone that cares about it more than me. 13 00:00:48,770 --> 00:00:52,699 Ever since 1993, when I had to tell my boss that there were no backups of 14 00:00:52,699 --> 00:00:54,349 the database that we had just lost. 15 00:00:54,849 --> 00:00:58,089 Now, I've written five O'Reilly books, a blog, and a podcast. 16 00:00:58,419 --> 00:01:02,340 Here, we turn unappreciated admins into cyber recovery heroes. 17 00:01:02,699 --> 00:01:04,640 This is the Backup Wrap-Up 18 00:01:19,038 --> 00:01:21,528 Hi, and welcome to Backup Central's podcast. 19 00:01:21,528 --> 00:01:23,118 I'm your host, W. Curtis Preston, 20 00:01:23,168 --> 00:01:23,518 Oh. 21 00:01:23,663 --> 00:01:28,938 a, Mr. Backup, and I with me, a guy whose only major flaw is that he 22 00:01:28,938 --> 00:01:30,858 won't go see scary movies with me. 23 00:01:31,218 --> 00:01:32,518 Prasanna Malaiyandi. 24 00:01:32,538 --> 00:01:33,078 How's it going? 25 00:01:33,078 --> 00:01:33,898 Prasanna. 26 00:01:34,113 --> 00:01:35,373 I am good, Curtis. 27 00:01:35,373 --> 00:01:41,403 I, yeah, I'm not a big fan of scary movies, especially on a large screen. 28 00:01:42,093 --> 00:01:45,753 Um, I don't like, I know some people like you love it, 29 00:01:45,948 --> 00:01:46,248 ah, 30 00:01:46,473 --> 00:01:50,043 but it's just I don't find joy in watching a scary movie. 31 00:01:50,048 --> 00:01:51,863 I do remember in college watching the Ring. 32 00:01:53,013 --> 00:01:53,883 On a big screen. 33 00:01:55,728 --> 00:01:57,318 That movie messed with my head. 34 00:01:58,698 --> 00:02:02,008 I remember coming home and asking someone to take my picture. 35 00:02:04,488 --> 00:02:06,318 And saying, is there a thing in the picture? 36 00:02:06,318 --> 00:02:08,208 You know, I was scared. 37 00:02:08,208 --> 00:02:09,378 I was scared to take the picture. 38 00:02:09,408 --> 00:02:09,648 Yeah. 39 00:02:09,783 --> 00:02:10,273 Yeah. 40 00:02:10,368 --> 00:02:11,808 was, that was a fun movie. 41 00:02:12,048 --> 00:02:13,818 Um, it really kind of messed 42 00:02:13,853 --> 00:02:14,343 Yeah. 43 00:02:14,493 --> 00:02:18,003 And you just saw a scary movie just recently. 44 00:02:18,528 --> 00:02:19,098 Yeah. 45 00:02:19,098 --> 00:02:19,708 It, it, 46 00:02:20,058 --> 00:02:20,658 it's not a 47 00:02:20,673 --> 00:02:21,213 know if I'd called 48 00:02:21,213 --> 00:02:21,663 Scarer. 49 00:02:21,663 --> 00:02:21,823 Yeah. 50 00:02:21,918 --> 00:02:23,898 per se, it's more like a thriller. 51 00:02:25,008 --> 00:02:29,568 Um, It's called the Menu and it stars. 52 00:02:29,598 --> 00:02:39,348 Uh, Ralph finds, um, a k a Voldemort, and uh, Anya, the 53 00:02:39,353 --> 00:02:39,913 Hello. 54 00:02:39,978 --> 00:02:40,758 Queens gambit. 55 00:02:41,983 --> 00:02:42,198 Yeah. 56 00:02:42,648 --> 00:02:47,208 Um, and some other, other character actors that you may be familiar with. 57 00:02:47,238 --> 00:02:53,268 Um, uh, it, it's, it's a movie that I, I enjoyed the movie. 58 00:02:53,358 --> 00:02:58,488 I will just say this, um, trigger alert for those who are bothered by such things. 59 00:02:58,488 --> 00:02:59,838 It does contain suicide. 60 00:03:00,508 --> 00:03:08,658 Um, having said that, within the context of the show, it, it was a 61 00:03:08,883 --> 00:03:09,228 fits in. 62 00:03:09,228 --> 00:03:10,098 part of the storyline. 63 00:03:10,638 --> 00:03:11,058 Mm-hmm. 64 00:03:11,208 --> 00:03:20,478 Um, and um, having said that, it was a really unique storyline. 65 00:03:20,478 --> 00:03:23,358 That's a commentary on Modern Day Society. 66 00:03:23,478 --> 00:03:26,838 You know, there's a class thing, there's a, there's 67 00:03:27,213 --> 00:03:27,363 Yeah. 68 00:03:27,798 --> 00:03:28,128 yeah. 69 00:03:28,203 --> 00:03:30,513 It's not often that you come out of a movie and be like, 70 00:03:30,543 --> 00:03:32,253 oh my God, that was so good. 71 00:03:32,808 --> 00:03:33,078 yeah, 72 00:03:33,093 --> 00:03:34,233 Like, I don't think, 73 00:03:34,308 --> 00:03:34,638 enjoyed 74 00:03:34,653 --> 00:03:34,893 yeah. 75 00:03:34,998 --> 00:03:37,308 like, oh, that was, I thought the ending. 76 00:03:38,913 --> 00:03:39,303 I don't know. 77 00:03:39,903 --> 00:03:45,333 think I would've done something slightly differently within the ending, but the um, 78 00:03:45,603 --> 00:03:47,313 That is why you are not the director, Curtis. 79 00:03:47,318 --> 00:03:48,003 We're the writer. 80 00:03:48,408 --> 00:03:50,718 Yeah, that's why I'm not the director or the writer. 81 00:03:50,718 --> 00:03:52,368 I, I was not the only one, by the way. 82 00:03:52,368 --> 00:03:55,728 I read some reviews and stuff, and a lot of people felt the ending 83 00:03:55,728 --> 00:03:58,188 was somewhat anti cli climactic. 84 00:03:58,188 --> 00:04:00,108 But it was one of those movies where you're like, what? 85 00:04:00,168 --> 00:04:01,158 What is over? 86 00:04:01,218 --> 00:04:03,918 Like, it's one of those kind of movies, but everything all the 87 00:04:03,923 --> 00:04:06,108 way up to that point I thought, I thought was really, really good. 88 00:04:06,348 --> 00:04:07,758 All the actors are really good. 89 00:04:07,938 --> 00:04:08,568 Um, 90 00:04:08,643 --> 00:04:10,353 never seen a Bollywood movie? 91 00:04:12,038 --> 00:04:13,548 yeah, what's 92 00:04:13,743 --> 00:04:16,083 Where it's like anti-climactic at the very end. 93 00:04:16,083 --> 00:04:20,793 It's like they literally, a lot of the movies boiled the ending down 94 00:04:20,793 --> 00:04:22,413 to like the last like five minutes. 95 00:04:24,198 --> 00:04:24,798 interesting. 96 00:04:25,578 --> 00:04:26,178 Um, 97 00:04:26,463 --> 00:04:26,643 Yeah. 98 00:04:27,093 --> 00:04:28,263 Shrek, oh, you would be proud. 99 00:04:28,263 --> 00:04:28,623 Shrek. 100 00:04:28,623 --> 00:04:31,023 And I watched while you were watching this movie, Shrek and 101 00:04:31,023 --> 00:04:32,253 I watched a Bollywood movie. 102 00:04:32,718 --> 00:04:33,278 Uhhuh, 103 00:04:33,783 --> 00:04:37,473 On Netflix and it was also suspenseful 104 00:04:37,848 --> 00:04:38,458 mm-hmm. 105 00:04:38,673 --> 00:04:44,883 of a comedy thriller ish, but I would say that they basically 106 00:04:44,883 --> 00:04:48,363 wrapped up all the entire movie in the last like five minutes. 107 00:04:49,353 --> 00:04:53,073 And then at the end, Shraddha and I were like, what did we just watch? 108 00:04:54,123 --> 00:04:54,813 Interesting. 109 00:04:55,668 --> 00:04:59,208 because we, because it left so many things open at the end and we're 110 00:04:59,208 --> 00:05:01,278 like, did we just watch all of that? 111 00:05:01,278 --> 00:05:03,798 Just for that last closing part? 112 00:05:04,623 --> 00:05:05,073 That's funny. 113 00:05:05,418 --> 00:05:07,578 it's like two and a half hours also, so, 114 00:05:08,968 --> 00:05:11,958 Interesting thing about this movie is, you know, how the movie's gonna 115 00:05:11,958 --> 00:05:13,938 end about halfway into the movie. 116 00:05:14,208 --> 00:05:14,658 Hmm. 117 00:05:14,838 --> 00:05:16,728 Like they tell you how the movie's gonna end. 118 00:05:17,658 --> 00:05:19,128 it's just how it gets there. 119 00:05:19,208 --> 00:05:20,568 It, it, it's just was, was 120 00:05:20,718 --> 00:05:21,078 Yeah. 121 00:05:21,328 --> 00:05:21,818 Nice. 122 00:05:21,948 --> 00:05:23,658 Um, so I recommend it. 123 00:05:23,838 --> 00:05:27,198 Um, and, uh, you know, the, the 124 00:05:27,243 --> 00:05:27,633 are you and 125 00:05:27,928 --> 00:05:28,768 saga continues. 126 00:05:29,043 --> 00:05:29,313 yeah. 127 00:05:29,583 --> 00:05:33,003 Are you and Jeff gonna talk about it on your next, on the other podcast, 128 00:05:33,138 --> 00:05:36,408 know, he keeps threatening to record another episode and then he doesn't do it. 129 00:05:36,408 --> 00:05:38,088 But I'll , I'll take 130 00:05:38,193 --> 00:05:40,213 I think Curtis is calling you out here. 131 00:05:43,018 --> 00:05:46,728 Jeff and I actually recorded a new episode this week, so look for it at, 132 00:05:46,758 --> 00:05:50,478 The Things That Entertain Us podcast, wherever you listen to podcasts. 133 00:05:50,534 --> 00:05:55,664 Today we're going, we're continuing in our backup to basic series, and 134 00:05:55,664 --> 00:06:00,824 we're covering, we're covering the book, modern Data Protection on Hold. 135 00:06:01,649 --> 00:06:06,449 those of you watching this on backup central.com where you can watch the video 136 00:06:06,689 --> 00:06:09,389 of Prasanna and I waxing philosophical. 137 00:06:11,504 --> 00:06:18,314 Um, and, uh, you know, what we're doing is we're sort of going through the book 138 00:06:19,039 --> 00:06:22,994 just discussing, you know, some important topics that I, that I found in the book. 139 00:06:24,164 --> 00:06:28,354 and this next one is about protecting backup and archive data, which 140 00:06:29,084 --> 00:06:30,854 think we can all agree is important. 141 00:06:31,834 --> 00:06:35,334 If you would like to download a free e-book copy of Modern Data Protection, 142 00:06:35,364 --> 00:06:37,914 you can do so at druva.com  slash ebook. 143 00:06:38,114 --> 00:06:41,174 Uh, we have to protect it from a couple of different things, right? 144 00:06:42,434 --> 00:06:47,444 We have to, mainly what I'm talking about in here is protecting 145 00:06:47,444 --> 00:06:50,414 it against attacks, right? 146 00:06:52,244 --> 00:06:59,624 both attacks like a, um, like a disaster type type attack. 147 00:06:59,624 --> 00:07:04,184 But, but mainly what I'm talking about is protecting it from cyber attacks, 148 00:07:06,794 --> 00:07:09,824 Um, I'm gonna have these same conversations over and over, so 149 00:07:09,824 --> 00:07:13,964 sometimes I forget what we say here versus what I say other places, but 150 00:07:15,914 --> 00:07:25,544 it, it is a well acknowledged fact that some of the largest of the ransomware 151 00:07:25,544 --> 00:07:35,594 bad actors are specifically targeting backup systems in order to do the. 152 00:07:36,359 --> 00:07:40,769 The extortion style of ransomware attack, and let me specify what I'm saying there. 153 00:07:41,369 --> 00:07:47,609 So, you know, a, a bunch of companies started realizing that, well, if I just 154 00:07:47,609 --> 00:07:52,109 get a decent backup in recovery and disaster recovery system, I can say, you 155 00:07:52,114 --> 00:07:55,259 know, pound sand the ransomware folks. 156 00:07:55,634 --> 00:07:55,914 Mm-hmm. 157 00:07:55,979 --> 00:07:59,249 the ransomware folks then said, oh, well, we'll show you. 158 00:07:59,459 --> 00:08:03,869 And so they developed this, this exfiltration style, right. 159 00:08:04,834 --> 00:08:12,209 and the um, The idea there is that they find databases and file 160 00:08:12,209 --> 00:08:16,349 systems and spreadsheets and whatnot that contains sensitive data. 161 00:08:16,354 --> 00:08:17,579 That could be two different things. 162 00:08:17,584 --> 00:08:19,049 That could be your company secrets. 163 00:08:19,049 --> 00:08:20,399 It could be your company plans. 164 00:08:20,399 --> 00:08:25,584 It could be your 11 herbs and spices or your, your, um, You know, your 165 00:08:25,584 --> 00:08:31,434 corporate plans for the next year, you know, um, but it could also be 166 00:08:31,524 --> 00:08:34,404 secrets that are like embarrassing. 167 00:08:34,764 --> 00:08:37,224 Uh, I think the best example of that, and I know we've talked 168 00:08:37,224 --> 00:08:38,514 about it, is the Sony attack 169 00:08:39,284 --> 00:08:43,874 This is the one where someone broke in to the movie studios and got a 170 00:08:43,874 --> 00:08:49,634 bunch of emails talking about scripts and actors and all the rest of that. 171 00:08:49,874 --> 00:08:52,424 and very, some, some, and not very 172 00:08:53,324 --> 00:08:53,684 Yeah. 173 00:08:53,864 --> 00:08:55,364 flattering terms. 174 00:08:55,364 --> 00:08:55,664 Yeah. 175 00:08:55,964 --> 00:08:57,374 So it could be that, right? 176 00:08:58,094 --> 00:09:04,514 the point is, there is no way to protect against that once the data's out. 177 00:09:05,054 --> 00:09:05,384 Yep. 178 00:09:05,669 --> 00:09:06,959 It's not like the other attack. 179 00:09:06,959 --> 00:09:10,769 If you get the, the old school attack, if you had a good backup system, you're good. 180 00:09:10,889 --> 00:09:13,289 It doesn't matter how good of a backup system you have, if the 181 00:09:13,289 --> 00:09:19,289 data is exfiltrated, you're gonna be forced to make some major, uh, 182 00:09:19,554 --> 00:09:19,794 Yep. 183 00:09:19,889 --> 00:09:20,219 right? 184 00:09:21,479 --> 00:09:30,029 So, um, what, what these, uh, ransomware groups have started realizing is, 185 00:09:32,399 --> 00:09:36,029 they, uh, instead the, the people are starting to harden 186 00:09:36,029 --> 00:09:37,979 their primary environments. 187 00:09:38,259 --> 00:09:38,539 Mm-hmm. 188 00:09:39,269 --> 00:09:45,389 And so, but one, you know, you, you, you've heard me talk, did 189 00:09:45,394 --> 00:09:49,709 I say one thing that has never changed in backup and recovery in 190 00:09:49,709 --> 00:09:50,999 the world of backup and recovery? 191 00:09:50,999 --> 00:09:52,079 Do you, do you know what it is? 192 00:09:52,079 --> 00:09:54,419 I often say like 193 00:09:54,509 --> 00:09:54,959 It's 194 00:09:55,169 --> 00:09:56,129 that has never changed, 195 00:09:57,299 --> 00:10:02,099 that the junior person is always responsible for backups 196 00:10:02,099 --> 00:10:02,939 and recovery. 197 00:10:03,224 --> 00:10:06,974 that is technically a cause of the thing the thing that 198 00:10:06,974 --> 00:10:08,244 No one who cares about 199 00:10:08,474 --> 00:10:10,634 no one wants to do the backups, right? 200 00:10:11,354 --> 00:10:15,434 one wants to raise their hand and say that they want to be the backup guy. 201 00:10:15,704 --> 00:10:16,484 So why does that matter? 202 00:10:16,484 --> 00:10:23,174 Well, it's because it means that, that it ends up being an an or and ignored system. 203 00:10:23,474 --> 00:10:23,894 Yep. 204 00:10:24,974 --> 00:10:32,849 and as a result, it that it gets ignored by everyone, including 205 00:10:32,849 --> 00:10:34,139 the cybersecurity folks. 206 00:10:34,289 --> 00:10:34,679 Right? 207 00:10:35,159 --> 00:10:37,289 They're like, ah, that's just a systematic in the corner. 208 00:10:37,289 --> 00:10:38,099 It's fine. 209 00:10:38,369 --> 00:10:41,549 We don't care about it except it contains all of our data. 210 00:10:42,389 --> 00:10:42,929 Yeah. 211 00:10:43,629 --> 00:10:49,409 And depending on how, how you handle, things like encryption, 212 00:10:50,189 --> 00:10:52,179 they may be able to unen encrypt. 213 00:10:52,649 --> 00:10:56,189 Said backups and then, you know, get the keys to the kingdom. 214 00:10:56,414 --> 00:11:00,854 Yeah, and like you were saying earlier, Curtis, the fact that a lot 215 00:11:00,854 --> 00:11:04,574 of people focus on production systems harden them, but they may not always 216 00:11:04,579 --> 00:11:06,164 be considering these backup systems. 217 00:11:07,499 --> 00:11:08,519 Right, right, right. 218 00:11:08,804 --> 00:11:09,524 Easy target. 219 00:11:10,349 --> 00:11:11,189 Easy target. 220 00:11:11,189 --> 00:11:13,009 And so you need to. 221 00:11:14,069 --> 00:11:15,554 not be such an easy target, right? 222 00:11:15,704 --> 00:11:16,194 Yeah. 223 00:11:16,304 --> 00:11:19,994 the old joke about the guy, two guys crossing a field and they see, 224 00:11:20,174 --> 00:11:24,644 they see a bull and the guy says, you know, can you outrun that bull? 225 00:11:24,644 --> 00:11:26,054 He said, Nope, I don't have to. 226 00:11:26,294 --> 00:11:26,894 What are you talking about? 227 00:11:26,924 --> 00:11:28,214 All I have to do is outrun you, 228 00:11:28,379 --> 00:11:28,499 You 229 00:11:29,804 --> 00:11:34,574 So, so you just want to be less of an interesting target 230 00:11:34,574 --> 00:11:36,114 than the person next to you. 231 00:11:37,724 --> 00:11:41,744 So the, the first thing that we talk about in terms of protecting 232 00:11:41,744 --> 00:11:43,304 backup data is encryption, 233 00:11:43,889 --> 00:11:44,789 Yeah, which 234 00:11:44,984 --> 00:11:45,194 uh, 235 00:11:45,479 --> 00:11:46,499 everyone should be doing. 236 00:11:46,499 --> 00:11:51,239 Like if no, if you aren't doing encrypting your backups, I don't 237 00:11:51,239 --> 00:11:52,409 know what's wrong with you. 238 00:11:52,409 --> 00:11:55,599 Like even if it's not encrypted, honestly, 239 00:11:55,604 --> 00:11:56,819 like that should be your default. 240 00:11:56,924 --> 00:11:57,584 Persona. 241 00:11:58,559 --> 00:12:02,909 no, because encryption isn't some heavyweight thing that it used to be. 242 00:12:02,909 --> 00:12:03,239 Right. 243 00:12:03,239 --> 00:12:08,459 You don't have to go physically encrypt your data as it comes out. 244 00:12:08,459 --> 00:12:10,409 You could do disc level encryption, right? 245 00:12:10,739 --> 00:12:11,189 There's so. 246 00:12:11,804 --> 00:12:16,124 Ways you could leverage encryption to fit into your environment. 247 00:12:16,124 --> 00:12:19,334 It should be a no-brainer that all your data at rest is encrypted. 248 00:12:20,174 --> 00:12:22,034 And in a lot of companies, right? 249 00:12:22,034 --> 00:12:25,124 If you wanna fit, be sort of like, uh, what is it? 250 00:12:25,124 --> 00:12:29,054 If you wanna look at HIPAA com, uh, readiness or other things, right? 251 00:12:29,059 --> 00:12:31,184 Other regulatory obligations. 252 00:12:31,454 --> 00:12:34,454 A lot of 'em say All your data at at rest should be encrypt. 253 00:12:35,189 --> 00:12:35,799 Mm-hmm. 254 00:12:36,209 --> 00:12:41,159 . Well, I, um, do think, I do think backup page backup data should be 255 00:12:41,159 --> 00:12:43,289 encrypted for this very reason. 256 00:12:43,349 --> 00:12:43,709 Right? 257 00:12:44,129 --> 00:12:47,909 And I'll give you different reasons for the different things, right? 258 00:12:47,909 --> 00:12:51,719 So if it's, if you got a backup server on-prem and you're storing 259 00:12:51,719 --> 00:12:57,239 your backups on a. File system, you know, an NFS mounted file system. 260 00:12:58,169 --> 00:13:01,829 Then they're unencrypted, you know 261 00:13:01,979 --> 00:13:02,969 Everyone has access. 262 00:13:03,029 --> 00:13:03,329 Yeah. 263 00:13:03,789 --> 00:13:09,659 Everyone has access to those backups and a lot of these PR backup products they offer 264 00:13:10,859 --> 00:13:16,829 to, to, to, so some of the backup products store the backups, not in a backup format. 265 00:13:16,829 --> 00:13:19,799 They're in native format, so you could just mount a disc, right? 266 00:13:19,979 --> 00:13:20,469 Yeah. 267 00:13:20,729 --> 00:13:22,769 backup products, put it in a backup format. 268 00:13:23,759 --> 00:13:26,709 but they offer an ability to read that backups like it, 269 00:13:26,709 --> 00:13:28,229 it's, it's for DR purposes. 270 00:13:28,229 --> 00:13:32,009 So, so if you're not encrypting them, then you're just, you're 271 00:13:32,009 --> 00:13:33,119 just asking for trouble. 272 00:13:33,284 --> 00:13:33,584 Yeah. 273 00:13:33,634 --> 00:13:34,439 the disc side. 274 00:13:34,444 --> 00:13:36,749 The second side will be the tape side. 275 00:13:36,754 --> 00:13:39,959 So if you are, and believe it or not, many people are still making backup 276 00:13:39,959 --> 00:13:45,299 tapes and they're making 'em primarily, I think for offsite, uh, storage. 277 00:13:45,494 --> 00:13:45,914 Yep. 278 00:13:46,709 --> 00:13:52,739 And so, And this is the, this is the one where I'm like, if you are ma, if you are 279 00:13:52,739 --> 00:13:59,879 putting your company's like crown jewels onto a little, you know, plastic tape and 280 00:13:59,884 --> 00:14:01,709 then you're handing it to a man in a van 281 00:14:02,159 --> 00:14:02,324 Yep. 282 00:14:02,464 --> 00:14:05,069 you are not at this point encrypting that data, 283 00:14:07,229 --> 00:14:09,869 you know, professional malfeasance at this 284 00:14:09,869 --> 00:14:10,139 point. 285 00:14:10,309 --> 00:14:16,604 and I wonder, it's almost as if that should be like a default, you know, 286 00:14:17,114 --> 00:14:22,784 like you can't write data out without encrypting it, but I guess you have to 287 00:14:22,784 --> 00:14:26,609 deal with key management and all the rest of that, and so, , do you have 288 00:14:26,609 --> 00:14:28,349 flexibility to shoot yourself in the foot? 289 00:14:29,594 --> 00:14:30,674 yeah, exactly. 290 00:14:32,114 --> 00:14:36,854 but I will also say this encryption is only good, is only as good as the 291 00:14:36,854 --> 00:14:40,424 authentication and authorization and key management system that is connected to it. 292 00:14:40,589 --> 00:14:40,919 Yep. 293 00:14:41,234 --> 00:14:43,604 And if you can easily defeat. 294 00:14:44,219 --> 00:14:50,459 That, uh, system then, you know, and, and, and I mentioned, so for example, 295 00:14:50,459 --> 00:14:55,589 I know we mentioned it on the podcast, please do not make your backup server 296 00:14:55,589 --> 00:14:57,719 part of your, uh, active directory. 297 00:14:58,499 --> 00:14:58,989 What, 298 00:14:59,084 --> 00:14:59,504 Domain. 299 00:14:59,549 --> 00:14:59,609 it, 300 00:14:59,924 --> 00:15:00,344 Domain. 301 00:15:01,544 --> 00:15:01,964 domain. 302 00:15:02,504 --> 00:15:02,994 Yeah. 303 00:15:03,479 --> 00:15:06,209 Um, Just please don't do that. 304 00:15:06,359 --> 00:15:12,059 so the next term is one that we can debate. 305 00:15:13,369 --> 00:15:13,719 No 306 00:15:14,134 --> 00:15:15,749 I, honestly, I don't even need you. 307 00:15:15,899 --> 00:15:19,199 I can debate it myself, right? 308 00:15:19,199 --> 00:15:23,789 It's one of those where I go back and forth, you know, and that's the term air. 309 00:15:25,439 --> 00:15:26,809 What is like air gap? 310 00:15:26,809 --> 00:15:27,129 Curtis? 311 00:15:27,794 --> 00:15:29,474 well, well, there 312 00:15:29,519 --> 00:15:30,379 Traditionally? 313 00:15:30,479 --> 00:15:30,869 Yeah. 314 00:15:31,289 --> 00:15:31,439 Yeah. 315 00:15:33,494 --> 00:15:37,064 It's, and it is where it, it is from once we get the term. 316 00:15:37,789 --> 00:15:44,484 the idea was to put a gap of error between the protection copy and the protected 317 00:15:44,534 --> 00:15:44,744 Yeah. 318 00:15:45,434 --> 00:15:46,184 And for 319 00:15:46,214 --> 00:15:48,554 you're dealing with plumbing right now, since you're dealing 320 00:15:48,554 --> 00:15:50,084 with plumbing right now, right? 321 00:15:50,084 --> 00:15:52,514 You have an air gap normally, right? 322 00:15:52,514 --> 00:15:54,404 When you think about your dishwasher 323 00:15:54,689 --> 00:15:56,699 Now you made me think about the dishwasher that 324 00:15:56,699 --> 00:15:57,269 has made my 325 00:15:57,314 --> 00:15:57,644 or, 326 00:15:57,644 --> 00:15:57,914 fine. 327 00:15:58,424 --> 00:15:58,724 Fine. 328 00:15:58,904 --> 00:16:00,044 Or a toilet. 329 00:16:00,254 --> 00:16:00,604 How about that? 330 00:16:01,889 --> 00:16:03,604 that's actually technically a water gap, but 331 00:16:05,594 --> 00:16:06,494 Don't, you're killing me. 332 00:16:06,524 --> 00:16:08,384 Don't be, don't be bringing plumbing into this. 333 00:16:09,554 --> 00:16:10,454 anyway, yeah. 334 00:16:10,454 --> 00:16:15,194 So you, you had to separate the, it meant, again, it meant the, it 335 00:16:15,194 --> 00:16:16,994 meant the man in the van, right? 336 00:16:16,999 --> 00:16:21,644 You, you, you, you put some tapes in a box and you handed that box to a, you 337 00:16:21,649 --> 00:16:24,854 know, a man or a woman, and then they got into a van and they took it someplace. 338 00:16:25,034 --> 00:16:30,724 And in order to get to that thing, needed to physically, Um, access it. 339 00:16:30,724 --> 00:16:30,994 Right? 340 00:16:31,169 --> 00:16:31,439 Yep. 341 00:16:32,224 --> 00:16:35,374 that if you were a hacker and you wanted to get to it, it, it 342 00:16:35,374 --> 00:16:36,784 would be virtually impossible. 343 00:16:36,784 --> 00:16:38,344 Not completely impossible. 344 00:16:38,344 --> 00:16:38,914 There's no, 345 00:16:39,479 --> 00:16:39,659 It's 346 00:16:39,814 --> 00:16:40,024 cuz 347 00:16:40,164 --> 00:16:41,959 penetration rather than cyber. 348 00:16:42,184 --> 00:16:48,094 yeah, remember for example, we would, uh, once a year at an 349 00:16:48,094 --> 00:16:51,694 unannounced time, we would attempt to circumvent the security of. 350 00:16:52,484 --> 00:16:53,744 Uh, storage fender 351 00:16:54,314 --> 00:16:54,514 Mm. 352 00:16:55,724 --> 00:16:59,954 we would, uh, use really good like liars and whatnot. 353 00:17:00,194 --> 00:17:03,674 And more specifically, we would use someone like me that they 354 00:17:03,674 --> 00:17:05,174 knew, they knew the person. 355 00:17:06,044 --> 00:17:09,914 And so they would go over there and they would, and we would just have 356 00:17:09,914 --> 00:17:13,604 this really, really like elaborate. 357 00:17:14,099 --> 00:17:15,059 Scenario 358 00:17:15,229 --> 00:17:15,404 yeah. 359 00:17:15,449 --> 00:17:21,599 that basically would want, would want the, it would make the person 360 00:17:21,599 --> 00:17:26,189 that works there want to leave us alone in the vault with the tapes. 361 00:17:27,449 --> 00:17:29,579 And at least once we were able to do that, 362 00:17:30,394 --> 00:17:30,514 Hmm. 363 00:17:31,274 --> 00:17:35,914 Was a person go, or did the person have to go through training, I guess. 364 00:17:36,764 --> 00:17:39,824 Yeah, I don't, I, you know, honestly, I was so long ago, I don't remember 365 00:17:39,824 --> 00:17:43,064 what happened to that, to that person who left us alone in the vault. 366 00:17:43,784 --> 00:17:45,014 But, so that's what I'm saying. 367 00:17:45,014 --> 00:17:47,384 It's not, it, it was never 100%. 368 00:17:47,834 --> 00:17:48,344 Right. 369 00:17:48,434 --> 00:17:53,084 Um, the, um, you've seen the movies, you 370 00:17:53,309 --> 00:17:53,729 Yep. 371 00:17:54,344 --> 00:17:56,864 I've seen Oceans 11 and 12 and 13. 372 00:17:58,034 --> 00:18:03,344 The um, but the problem is most of us don't use tape. 373 00:18:03,719 --> 00:18:03,989 Mm-hmm. 374 00:18:04,709 --> 00:18:08,639 And most of us, or many of us don't even use a data center anymore. 375 00:18:08,984 --> 00:18:09,164 Yep. 376 00:18:09,599 --> 00:18:16,469 So this idea of putting a, an actual air gap between A and B is quite difficult. 377 00:18:16,474 --> 00:18:16,589 And 378 00:18:16,724 --> 00:18:17,144 Yep. 379 00:18:17,669 --> 00:18:20,129 we've sort of adopted or adapted 380 00:18:22,139 --> 00:18:24,719 the, you know, the virtual air gap. 381 00:18:24,779 --> 00:18:25,199 Right. 382 00:18:25,619 --> 00:18:35,969 Um, the, um, Uh, so this is in as many ways as we can. 383 00:18:35,999 --> 00:18:36,269 I don't know. 384 00:18:36,329 --> 00:18:37,279 You want to talk about 385 00:18:37,439 --> 00:18:41,699 Yeah, and I think we talked about this on the episode about backups, 386 00:18:41,699 --> 00:18:43,139 or maybe it was restores, but. 387 00:18:44,174 --> 00:18:50,684 It's even though they are available online, right, that second copy, 388 00:18:50,684 --> 00:18:53,954 so it's not the same as having that tape in a vault somewhere. 389 00:18:54,134 --> 00:18:57,454 You wanna make that, make it as close to that as possible. 390 00:18:57,914 --> 00:18:59,144 And we totally get that. 391 00:18:59,144 --> 00:19:01,184 You still have to have connectivity, right? 392 00:19:01,184 --> 00:19:03,104 It's still gonna be up, but can you lock it down? 393 00:19:03,514 --> 00:19:08,744 Can you basically make that target as inaccessible as possible? 394 00:19:10,424 --> 00:19:14,804 Uh, to prevent it from being attacked by say, someone, um, gaining access 395 00:19:14,804 --> 00:19:16,634 to your backup source, right? 396 00:19:16,634 --> 00:19:18,074 And now they have access to your vault. 397 00:19:18,554 --> 00:19:22,904 Uh, some of the examples are if you're backing up in the cloud, right, put 398 00:19:22,909 --> 00:19:27,524 it in a separate account that no one has access to other than whatever 399 00:19:27,524 --> 00:19:29,214 is transferring the data, right? 400 00:19:29,354 --> 00:19:30,944 Make it in a different region. 401 00:19:30,944 --> 00:19:32,924 So it's not easily accessible. 402 00:19:33,344 --> 00:19:35,924 And there are many, many, many other things you could do as well. 403 00:19:35,924 --> 00:19:37,174 But those are just some of the example. 404 00:19:38,474 --> 00:19:41,894 Yeah, I like if we're talking cloud, I like the idea of putting it in a 405 00:19:41,894 --> 00:19:43,664 different region and a different account. 406 00:19:44,204 --> 00:19:44,624 Right. 407 00:19:44,684 --> 00:19:47,114 Um, because we're not just talking about hacking. 408 00:19:47,114 --> 00:19:52,214 We're also, we also need to think about like, Disaster recovery and things 409 00:19:52,274 --> 00:19:52,694 Yep. 410 00:19:52,814 --> 00:19:53,264 right? 411 00:19:53,354 --> 00:19:56,804 So again, O V H Cloud, we don't want to have that happen to us. 412 00:19:56,804 --> 00:20:01,394 So we want to make sure that a copy of the backup data is, is well far away 413 00:20:01,394 --> 00:20:02,984 from the thing that it's protecting. 414 00:20:04,094 --> 00:20:09,194 And yeah, separate account a much more limited, an account 415 00:20:09,199 --> 00:20:10,964 with much more limited access, 416 00:20:11,309 --> 00:20:11,549 Yep. 417 00:20:11,954 --> 00:20:13,904 and you can. 418 00:20:14,309 --> 00:20:17,609 Um, you can consider using immutable storage, but we're 419 00:20:17,609 --> 00:20:19,139 gonna talk about that later. 420 00:20:19,139 --> 00:20:25,319 But essentially, this is the, we, we separated as much as This 421 00:20:25,319 --> 00:20:25,739 Yep. 422 00:20:27,329 --> 00:20:32,579 um, you know, one of your previous employers data domain, right? 423 00:20:32,639 --> 00:20:36,509 And, and, and, All of those things, they would always talk about 424 00:20:36,509 --> 00:20:38,219 replicating to another data domain. 425 00:20:38,969 --> 00:20:43,709 And I always, I always remember thinking, well, if I'm replicating to another data 426 00:20:43,709 --> 00:20:47,939 domain, it's essentially like in order to do that, I gotta have it in the same 427 00:20:48,104 --> 00:20:48,464 So, 428 00:20:48,569 --> 00:20:50,099 like land. 429 00:20:50,249 --> 00:20:50,699 Right? 430 00:20:50,759 --> 00:20:51,299 Or, or at 431 00:20:51,434 --> 00:20:52,034 or Van. 432 00:20:52,174 --> 00:20:53,039 like it's in the land. 433 00:20:53,084 --> 00:20:53,324 Yeah. 434 00:20:53,624 --> 00:20:53,894 Well, 435 00:20:54,059 --> 00:20:54,569 Um, 436 00:20:55,184 --> 00:20:58,574 that's why they actually introduced a feature, specifically talking 437 00:20:58,574 --> 00:21:00,104 about data domain, right? 438 00:21:00,104 --> 00:21:02,204 Their cyber recovery solution. 439 00:21:02,719 --> 00:21:03,329 Mm-hmm. 440 00:21:03,364 --> 00:21:08,279 that allowed for writing to a lockdown data domain that didn't 441 00:21:08,279 --> 00:21:09,779 have access from anywhere else. 442 00:21:09,899 --> 00:21:13,619 And going back to the point you were talking about, Curtis, okay, is it 443 00:21:13,624 --> 00:21:16,679 on the network and I can connect to it then, doesn't that make it open? 444 00:21:16,679 --> 00:21:20,309 And so what it would actually do is shut down network ports right 445 00:21:20,309 --> 00:21:24,089 outside of your backup windows, such that it's not available, or 446 00:21:24,089 --> 00:21:25,589 at least your attack surface is. 447 00:21:27,489 --> 00:21:30,089 Could you delete the backups? 448 00:21:30,089 --> 00:21:30,719 Was it. 449 00:21:31,349 --> 00:21:35,219 Could you delete the backups once you send it to it via the backup interface? 450 00:21:36,089 --> 00:21:36,809 Typically no. 451 00:21:37,709 --> 00:21:38,069 Okay. 452 00:21:38,249 --> 00:21:40,349 Because that, cuz that would be, that would be an important 453 00:21:40,439 --> 00:21:40,649 Yeah. 454 00:21:41,099 --> 00:21:43,679 It would also replicate like your backup environment, right? 455 00:21:43,679 --> 00:21:46,799 So you'd have your own backup server in that vault, if you will. 456 00:21:46,799 --> 00:21:50,069 So there's no connectivity to anything outside, because if there 457 00:21:50,069 --> 00:21:51,479 is, then you're just open to risk. 458 00:21:52,754 --> 00:21:53,444 Right, right. 459 00:21:54,044 --> 00:21:59,534 Um, so just like, so I've got a list here on virtual air gap here. 460 00:22:00,259 --> 00:22:04,104 talk about disabling or impairing R D P. Why does that matter? 461 00:22:05,069 --> 00:22:08,369 Ah, so many ransomware issues have happened. 462 00:22:09,314 --> 00:22:09,974 Rdp, which 463 00:22:09,974 --> 00:22:10,844 is Remote Desktop 464 00:22:10,924 --> 00:22:11,124 the way? 465 00:22:11,204 --> 00:22:11,864 Protocol, 466 00:22:12,004 --> 00:22:12,124 you, 467 00:22:12,554 --> 00:22:12,794 right? 468 00:22:12,794 --> 00:22:16,574 Which is how most people connect remotely to a Windows client. 469 00:22:17,354 --> 00:22:22,214 Um, most people forget to turn it off or they leave it on because it's just easier. 470 00:22:22,214 --> 00:22:23,714 They don't need to physically connect to it. 471 00:22:24,144 --> 00:22:29,339 Uh, The only downside is it's a big attack surface, and there are 472 00:22:29,339 --> 00:22:31,079 a lot of exploits using R D P. 473 00:22:31,079 --> 00:22:35,819 And so when you enable R D P, you're letting yourself or leaving yourself 474 00:22:35,819 --> 00:22:39,599 open to hackers and other exploits coming in, which could then move 475 00:22:39,599 --> 00:22:43,199 laterally across your network and take out other pieces of your environment. 476 00:22:44,219 --> 00:22:48,389 Yeah, there is a way to automate sort of disabling and enabling R 477 00:22:48,389 --> 00:22:51,359 D P, um, like on a large scale. 478 00:22:51,839 --> 00:22:56,939 Um, I also talk about putting it in a different operating system, right? 479 00:22:56,944 --> 00:22:57,209 Not 480 00:22:57,329 --> 00:22:57,629 Yep. 481 00:22:57,869 --> 00:23:00,179 operating system for your backups in your primary, 482 00:23:00,449 --> 00:23:00,869 Yep. 483 00:23:01,079 --> 00:23:02,039 that's possible. 484 00:23:02,099 --> 00:23:05,219 I know that there's a lot of big window shops and they use windows. 485 00:23:06,239 --> 00:23:07,889 Uh, backup servers. 486 00:23:08,789 --> 00:23:10,109 I don't think that's a good idea. 487 00:23:10,199 --> 00:23:16,904 Um, but you know, Um, and by the way, I, I forgot to throw out our disclaimer. 488 00:23:16,904 --> 00:23:19,254 I'll throw out, um, Prasanna. 489 00:23:19,524 --> 00:23:20,454 Prasanna. 490 00:23:20,474 --> 00:23:21,944 And I work for different companies. 491 00:23:22,304 --> 00:23:23,204 I work for Druva. 492 00:23:23,209 --> 00:23:24,194 He works for Zoom. 493 00:23:25,094 --> 00:23:26,684 this is not a podcast of either company. 494 00:23:26,684 --> 00:23:30,584 This is an independent podcast and the opinions that you hear are ours. 495 00:23:31,214 --> 00:23:34,964 And, uh, be sure to rate us@ratethispodcast.com slash restore. 496 00:23:36,104 --> 00:23:39,744 um, if you'd like to join the conversation, you want to talk. 497 00:23:40,189 --> 00:23:41,599 What's going on in your world? 498 00:23:42,049 --> 00:23:44,899 Uh, you know, whether you're, uh, you know, an end user. 499 00:23:44,899 --> 00:23:48,049 If you're an end user, we really want to have you, you know, real 500 00:23:48,054 --> 00:23:51,199 people that actually do backups out there in the trenches. 501 00:23:51,199 --> 00:23:51,499 Weed. 502 00:23:51,499 --> 00:23:51,799 You're, 503 00:23:52,339 --> 00:23:52,689 Do. 504 00:23:52,694 --> 00:23:53,549 Do you miss it, Curtis 505 00:23:54,859 --> 00:23:55,399 No 506 00:23:59,399 --> 00:24:07,619 I do, you know, as soon as I think about, um, Back when I used to be the 507 00:24:07,619 --> 00:24:13,259 person in the trenches, like when I was a consultant, and I, and I, I flashback 508 00:24:13,259 --> 00:24:22,049 to this one time where we were completely redoing the backups of, I'll just say 509 00:24:22,049 --> 00:24:33,359 a large television station, like, like a television station, and remember. 510 00:24:35,114 --> 00:24:41,054 I remember that time when the, when the, the, the of it basically said, 511 00:24:42,284 --> 00:24:44,714 going home until this is done. 512 00:24:45,044 --> 00:24:50,264 He literally, I mean, there's a long story following up to this, but he basically, 513 00:24:50,834 --> 00:24:57,194 um, held us hostage in the data center said, no one's going home until this, 514 00:24:57,284 --> 00:24:59,744 this thing you said is gonna happen. 515 00:24:59,744 --> 00:25:00,854 You said it's gonna happen. 516 00:25:01,094 --> 00:25:03,044 I'm like, yeah, but it's not gonna finish until. 517 00:25:03,779 --> 00:25:09,329 10 30 at night, like, yeah, so we're not going home until it's done. 518 00:25:10,259 --> 00:25:15,659 And I'm like, okay, well this is kidnapping, but whatever. 519 00:25:15,839 --> 00:25:19,409 Um, you know, uh, so I think about stuff like that, right? 520 00:25:19,444 --> 00:25:19,934 Yeah. 521 00:25:20,099 --> 00:25:21,089 moments of terror. 522 00:25:21,089 --> 00:25:24,749 But yeah, I, you know, I wanna live vicariously through those who 523 00:25:24,749 --> 00:25:25,979 have been through those moments of 524 00:25:25,999 --> 00:25:26,114 Yeah. 525 00:25:26,954 --> 00:25:32,414 Well, and it's also interesting to learn like what the more recent challenges 526 00:25:32,414 --> 00:25:34,934 are in environments, because otherwise 527 00:25:34,934 --> 00:25:35,384 how do you 528 00:25:35,414 --> 00:25:35,714 Yeah, 529 00:25:36,439 --> 00:25:38,744 otherwise, how do you build great products, you know, or build 530 00:25:38,744 --> 00:25:39,584 products to solve 531 00:25:39,749 --> 00:25:40,439 exactly. 532 00:25:41,399 --> 00:25:44,729 So what, what I was thinking about saying, which is what reminded me to do the 533 00:25:44,729 --> 00:25:50,129 disclaimer, is just that, um, you know, one way to have a different operating 534 00:25:50,129 --> 00:25:52,109 system is to use a SaaS provider. 535 00:25:52,409 --> 00:25:56,519 Druva is not the only one, but there are SaaS providers 536 00:25:56,524 --> 00:25:58,229 that are cloud native or use. 537 00:25:58,589 --> 00:26:03,509 Non windows, uh, tools um, you know, you do that. 538 00:26:03,509 --> 00:26:06,029 Also, you, you gain the second thing that I list here, which 539 00:26:06,029 --> 00:26:07,319 is about separating the storage. 540 00:26:07,404 --> 00:26:07,884 Yeah, 541 00:26:08,129 --> 00:26:11,369 don't, um, you know, we talked, we talked about that already, 542 00:26:11,729 --> 00:26:14,339 the, these replication, right? 543 00:26:14,729 --> 00:26:16,909 Uh, but um, 544 00:26:18,194 --> 00:26:18,974 but it's also 545 00:26:19,079 --> 00:26:19,409 you're just. 546 00:26:19,499 --> 00:26:20,414 use nfs. 547 00:26:21,644 --> 00:26:22,604 Yeah, exactly. 548 00:26:22,604 --> 00:26:22,904 Yes. 549 00:26:22,904 --> 00:26:26,564 Don't use NFS as a way to back up to the server use. 550 00:26:26,774 --> 00:26:30,284 There are ways to back up to, for example, a data domain box in other 551 00:26:30,284 --> 00:26:34,004 boxes exposing the backups via nfs. 552 00:26:34,364 --> 00:26:36,054 Uh, I also talk about using object storage. 553 00:26:36,074 --> 00:26:38,354 I'm a huge fan of using object storage. 554 00:26:38,684 --> 00:26:42,164 Now, some of you go, well, object storage is too slow, to which I 555 00:26:42,169 --> 00:26:44,294 say, then you're using it wrong. 556 00:26:44,339 --> 00:26:44,729 Yep. 557 00:26:45,284 --> 00:26:45,884 Right. 558 00:26:46,319 --> 00:26:46,499 Yep. 559 00:26:47,444 --> 00:26:48,884 Druva uses object storage. 560 00:26:48,884 --> 00:26:52,514 All our backups are on object storage, and we are not too slow. 561 00:26:52,514 --> 00:26:56,714 In fact, we get into competitions all the time with these big 562 00:26:56,834 --> 00:26:58,574 on-prem companies and we win. 563 00:26:59,054 --> 00:27:01,394 And here we are, we're a copy in the cloud and they're an on-prem 564 00:27:01,399 --> 00:27:03,584 appliance, and we win that restore test. 565 00:27:03,654 --> 00:27:08,234 Object storage is not too stor slow, but if you, if you treat it like file 566 00:27:08,234 --> 00:27:10,004 system storage, I think then it is, 567 00:27:10,144 --> 00:27:11,124 And it's a challenge. 568 00:27:11,129 --> 00:27:11,324 Yeah. 569 00:27:11,939 --> 00:27:16,484 all your backups in one object, Tonight's, tonight's backup of 570 00:27:16,484 --> 00:27:22,664 server X is an image, and that image becomes an object on, uh, s3. 571 00:27:22,844 --> 00:27:23,174 Yeah. 572 00:27:23,324 --> 00:27:24,944 Then it's gonna be slow, right? 573 00:27:25,994 --> 00:27:29,024 Um, we talk about using aut storage. 574 00:27:29,029 --> 00:27:30,494 We're gonna come back to that in a minute. 575 00:27:30,554 --> 00:27:32,714 Um, that, that's an option that we talked 576 00:27:32,789 --> 00:27:33,239 And then, 577 00:27:33,314 --> 00:27:33,944 S3 578 00:27:34,139 --> 00:27:35,309 yeah, and then your. 579 00:27:36,614 --> 00:27:39,344 And then your favorite topic is using tape. 580 00:27:39,614 --> 00:27:42,974 I know, Curtis, you always like to throw out the disclaimer. 581 00:27:42,974 --> 00:27:46,274 Actually, I should thank you because before this podcast, 582 00:27:46,279 --> 00:27:47,834 I knew very little about tape. 583 00:27:47,839 --> 00:27:49,994 Now I know just a bit more about tape, 584 00:27:50,114 --> 00:27:50,504 yeah, 585 00:27:50,744 --> 00:27:53,024 the number of experts who've come on in talking to you. 586 00:27:53,029 --> 00:27:54,824 But yeah, it's 587 00:27:55,014 --> 00:27:55,434 not 588 00:27:55,514 --> 00:27:57,824 there's no, there's no beating the tape when it comes. 589 00:27:57,824 --> 00:27:59,384 That's an actual air gap. 590 00:27:59,714 --> 00:28:00,074 Yep. 591 00:28:00,464 --> 00:28:00,809 right? 592 00:28:01,069 --> 00:28:05,309 If, if you, if you want a real air gap, I'm, I know there's challenges with 593 00:28:05,309 --> 00:28:09,599 it, you know, uh, but there is, you know, there is, there is an air gap 594 00:28:09,899 --> 00:28:10,229 Yep. 595 00:28:10,709 --> 00:28:12,809 Uh, and I already talked about using a backup service. 596 00:28:12,809 --> 00:28:15,479 So let's talk about immutability. 597 00:28:15,899 --> 00:28:16,199 Yep. 598 00:28:16,439 --> 00:28:17,639 What is immutability? 599 00:28:18,329 --> 00:28:19,349 Is it a made up word? 600 00:28:19,409 --> 00:28:21,089 It sounds a little like a made up word, 601 00:28:22,529 --> 00:28:26,879 you know, this is another one of those things where we have to go back in time. 602 00:28:28,664 --> 00:28:38,894 Historically immutability wa it, it, it was like a legal term that you needed 603 00:28:38,894 --> 00:28:45,254 to be able to prove that the thing you were presenting in court had not changed. 604 00:28:45,314 --> 00:28:46,364 It was immutable. 605 00:28:46,369 --> 00:28:50,984 That's all, you know, immutable just means cannot be changed. 606 00:28:51,144 --> 00:28:51,564 Yep. 607 00:28:51,794 --> 00:28:53,274 when you're, when you. 608 00:28:54,519 --> 00:28:57,674 uh, you pres, you, you know, you have to preserve chain of custody and you 609 00:28:57,674 --> 00:29:01,094 have to, you have to be able to say, or it's really helpful if you're able 610 00:29:01,094 --> 00:29:05,894 to say, this email that I'm showing you is exactly the same email as the email 611 00:29:05,894 --> 00:29:08,462 that we got five years ago from Steve. 612 00:29:08,669 --> 00:29:13,389 Yep, and nothing changed and no one had the ability to change it yet. 613 00:29:14,594 --> 00:29:17,564 That's where we used to talk about immutability. 614 00:29:18,344 --> 00:29:22,904 Now we talk a lot about it in terms of cyber attacks. 615 00:29:23,829 --> 00:29:29,099 And I would add to that, we also talk about it in terms of things like bit rot. 616 00:29:29,129 --> 00:29:33,659 We've talked about bit rot on this podcast where this is 617 00:29:33,659 --> 00:29:35,549 silent data corruption, right? 618 00:29:35,549 --> 00:29:38,819 That bits just flip underneath magnetic storage devices. 619 00:29:38,819 --> 00:29:40,139 It's just the way they work. 620 00:29:41,234 --> 00:29:46,359 and the, you know, and if, if you've got, if you've got a bunch of petabytes 621 00:29:46,359 --> 00:29:48,249 of data, you got flipped bits in there. 622 00:29:48,489 --> 00:29:50,619 It's just a matter of, it's just 623 00:29:51,574 --> 00:29:53,234 when you're gonna run across it, right? 624 00:29:53,294 --> 00:29:53,684 yeah. 625 00:29:54,069 --> 00:29:54,489 Yeah. 626 00:29:54,759 --> 00:29:55,509 Um, 627 00:29:57,789 --> 00:30:02,579 so, when we talk about immutability, we say, well, we, if somebody 628 00:30:02,584 --> 00:30:03,849 says they have immutable. 629 00:30:04,559 --> 00:30:07,379 Backups or immutable storage. 630 00:30:07,769 --> 00:30:12,329 What you're saying is stuff that gets put here can't be changed. 631 00:30:12,744 --> 00:30:13,354 Mm-hmm. 632 00:30:15,749 --> 00:30:16,889 here's the thing. 633 00:30:18,179 --> 00:30:21,179 Depending on how you look at it, it's complete nonsense. 634 00:30:22,289 --> 00:30:23,959 Because everything's just software. 635 00:30:24,279 --> 00:30:24,869 Anything can be cheap. 636 00:30:24,899 --> 00:30:25,979 is changeable. 637 00:30:26,039 --> 00:30:26,279 Yeah. 638 00:30:26,369 --> 00:30:28,319 me a torch, I'll change that. 639 00:30:28,669 --> 00:30:29,909 S o b, right? 640 00:30:30,239 --> 00:30:30,899 Give me phy. 641 00:30:30,929 --> 00:30:34,034 So All bets are off. 642 00:30:34,534 --> 00:30:42,764 Everyone knows that, um, if it's something on a server that you can, that, that, 643 00:30:42,884 --> 00:30:48,554 that is in your data center, that also, in my opinion, is not really immutable. 644 00:30:49,584 --> 00:30:54,299 Um, because if someone has root, uh, on that server, they can wipe the server. 645 00:30:55,199 --> 00:30:58,409 it's even built, like, you know, when we talk about immutable storage on 646 00:30:58,414 --> 00:31:00,149 Lennox, it's not really that immutable. 647 00:31:00,149 --> 00:31:01,499 It's it's immutable ish. 648 00:31:01,799 --> 00:31:05,249 So, so why do I say, like, why do I talk about it if it's bs? 649 00:31:06,119 --> 00:31:07,109 Well, here's the point. 650 00:31:07,319 --> 00:31:09,989 Nothing was ever a hundred percent immutable, 651 00:31:10,904 --> 00:31:13,064 Even that document right, that you were talking about. 652 00:31:13,634 --> 00:31:14,894 Yeah, exactly right. 653 00:31:14,894 --> 00:31:20,114 As long as like if, if, if you put something on a a, a, a, right once 654 00:31:21,074 --> 00:31:25,364 cd, uh, optical platter, right? 655 00:31:25,364 --> 00:31:29,684 A worm tape, it's immutable to a point. 656 00:31:30,794 --> 00:31:33,884 If you have physical access, it's no longer immutable. 657 00:31:33,969 --> 00:31:34,209 Yeah. 658 00:31:34,574 --> 00:31:39,104 You could, you could say from a legal standpoint, you could later, if you have 659 00:31:39,304 --> 00:31:42,584 the same disc and you've preserved chain of custody, and you could say, we can 660 00:31:42,584 --> 00:31:44,714 show that this disc was not destroyed, 661 00:31:44,719 --> 00:31:45,209 Yeah. 662 00:31:45,404 --> 00:31:48,254 we can show via these check sums and whatnot, that the stuff that we're giving 663 00:31:48,254 --> 00:31:49,574 you is the stuff that we have before. 664 00:31:50,909 --> 00:31:54,869 Saying that you've got a storage system that can't ever, ever, ever be changed. 665 00:31:55,109 --> 00:31:58,169 Um, I would argue it's just nonsense. 666 00:31:58,199 --> 00:31:58,529 Right. 667 00:31:58,529 --> 00:32:02,249 So it's, it's a, it's a, it's a, it's a degree. 668 00:32:02,249 --> 00:32:03,869 It's a what, what did we call it? 669 00:32:03,869 --> 00:32:04,319 Like a, 670 00:32:04,709 --> 00:32:05,189 Spectrum. 671 00:32:05,189 --> 00:32:06,479 like a spectrum. 672 00:32:06,484 --> 00:32:06,809 Thank you. 673 00:32:06,809 --> 00:32:06,869 I 674 00:32:06,869 --> 00:32:07,079 Yeah. 675 00:32:07,174 --> 00:32:07,949 was a word in there. 676 00:32:08,489 --> 00:32:09,029 And even 677 00:32:09,029 --> 00:32:10,469 prism is what was coming to mind. 678 00:32:10,979 --> 00:32:14,549 and even for those storage systems that have immutability, right, 679 00:32:14,549 --> 00:32:17,399 that are used for like financial records and everything else, 680 00:32:17,729 --> 00:32:18,339 Mm-hmm. 681 00:32:18,359 --> 00:32:22,289 of that is certified by the storage vendors saying they have all the checks 682 00:32:22,294 --> 00:32:24,119 in place and all the rest, right? 683 00:32:24,119 --> 00:32:27,169 In the end it's all just software, right? 684 00:32:27,299 --> 00:32:30,749 So they've built in the, into the software, those checks to make 685 00:32:30,749 --> 00:32:32,519 sure that data cannot be deleted, 686 00:32:33,209 --> 00:32:33,539 right. 687 00:32:33,544 --> 00:32:34,199 is present. 688 00:32:35,459 --> 00:32:39,719 And the, and you know, we're doing all this, like we're gonna, I'm 689 00:32:39,719 --> 00:32:43,949 slamming it and then I'm gonna talk about how important it's . But the 690 00:32:44,009 --> 00:32:51,479 other is, um, Uh, I'd say the, one that I like the best right now is 691 00:32:51,479 --> 00:32:54,779 like the, the object lock and s3. 692 00:32:55,229 --> 00:32:58,949 And what I like about that is because it's, it is at least 693 00:32:58,949 --> 00:33:00,609 physically separated from. 694 00:33:01,484 --> 00:33:02,054 You. 695 00:33:02,204 --> 00:33:05,624 Now, that doesn't mean that somebody can't go crazy in the AWS data 696 00:33:05,624 --> 00:33:10,004 center and, and blow it up, but even that, it's built into that. 697 00:33:10,454 --> 00:33:14,114 So it, it would have to be like, it would have to be a really concerted 698 00:33:14,119 --> 00:33:19,244 attack to be able to attack multiple locations of S3 to be able to do damage. 699 00:33:20,114 --> 00:33:20,984 it's, it's not only 700 00:33:21,089 --> 00:33:21,284 the 701 00:33:21,414 --> 00:33:21,574 multiple 702 00:33:21,644 --> 00:33:21,824 is 703 00:33:21,974 --> 00:33:22,254 location. 704 00:33:22,514 --> 00:33:23,324 as what's 705 00:33:23,359 --> 00:33:23,384 Yeah. 706 00:33:23,804 --> 00:33:29,324 It's not just multiple locations of s3, but also they have to tack your primary 707 00:33:29,324 --> 00:33:33,464 site as well, so everything needs to be coordinated across multiple vendors and 708 00:33:33,464 --> 00:33:36,944 corporations, which will probably have their own security practices, et cetera. 709 00:33:37,169 --> 00:33:41,069 this is, this is that, this is that, uh, spectrum, right? 710 00:33:41,114 --> 00:33:41,264 Yeah. 711 00:33:41,279 --> 00:33:45,929 I'd put, I'd put that and write, you know, worm tape, write once, 712 00:33:45,934 --> 00:33:49,499 read, write once, read many tape. 713 00:33:50,129 --> 00:33:52,859 Um, and then, and, and optical plat. 714 00:33:52,889 --> 00:33:54,719 I put that on one end of the spectrum. 715 00:33:56,124 --> 00:33:59,999 on the other put at the end of the spectrum is I would have unencrypted 716 00:34:01,019 --> 00:34:09,119 backup stored on a NFS mounted storage system behind a Windows backup server. 717 00:34:09,779 --> 00:34:10,079 Yep. 718 00:34:12,299 --> 00:34:13,199 that's your spectrum. 719 00:34:13,349 --> 00:34:16,709 So we wanna be closer to this end than that 720 00:34:16,829 --> 00:34:17,069 Yep. 721 00:34:17,639 --> 00:34:18,149 Right? 722 00:34:18,269 --> 00:34:18,719 Um, 723 00:34:19,019 --> 00:34:19,529 But there are some 724 00:34:19,679 --> 00:34:19,919 ever 725 00:34:19,939 --> 00:34:20,219 downsides. 726 00:34:20,699 --> 00:34:20,969 Yeah. 727 00:34:21,059 --> 00:34:24,539 And there are some downsides to immutability too, right? 728 00:34:24,659 --> 00:34:29,039 Uh, you can't, like once the data gets written, you can't delete it 729 00:34:29,044 --> 00:34:31,139 before that time period typically. 730 00:34:31,139 --> 00:34:33,479 So there are some challenges as well. 731 00:34:34,844 --> 00:34:36,224 So, um, 732 00:34:38,444 --> 00:34:43,349 so what we need to do, Is we need to, I, I think 733 00:34:43,499 --> 00:34:43,789 I. 734 00:34:44,069 --> 00:34:48,779 multiple things there is that, um, we need to protect against access. 735 00:34:49,139 --> 00:34:53,009 We need to protect against, you know, change deletion, corruption. 736 00:34:53,669 --> 00:34:57,299 Um, uh, was I, what was I talking about? 737 00:34:57,329 --> 00:34:59,339 Um, bit rot, 738 00:34:59,729 --> 00:34:59,999 Yep. 739 00:35:01,409 --> 00:35:03,779 So there are vendors out there. 740 00:35:06,089 --> 00:35:16,559 That, they're like, well, we have a, append only file system, and, we, 741 00:35:17,699 --> 00:35:20,819 and we have data lock the backups. 742 00:35:21,479 --> 00:35:23,009 And so they say, we're immutable. 743 00:35:24,659 --> 00:35:26,039 and I'm like, okay. 744 00:35:26,039 --> 00:35:29,789 And, and they say because they're saying that like, you can't encrypt the backups 745 00:35:29,789 --> 00:35:31,169 with a, with a ransomware attack. 746 00:35:31,214 --> 00:35:31,514 Yep. 747 00:35:32,259 --> 00:35:33,524 and I, and I go, that's good. 748 00:35:33,524 --> 00:35:37,544 That's farther, that's, that's closer to the other one than the other ones. 749 00:35:37,784 --> 00:35:39,434 But it's like, well, what about other things? 750 00:35:39,824 --> 00:35:40,124 Right. 751 00:35:40,124 --> 00:35:44,534 What about other types of attacks that attack the operating system itself? 752 00:35:44,954 --> 00:35:45,344 Right. 753 00:35:47,474 --> 00:35:49,424 You gain privileged access to that server, 754 00:35:50,774 --> 00:35:51,074 yeah. 755 00:35:51,914 --> 00:35:56,264 And I wanna clarify though, what you're talking about is someone 756 00:35:56,444 --> 00:36:00,344 who's written their own file system or is leveraging a file 757 00:36:00,344 --> 00:36:03,934 system on top of a standard server. 758 00:36:05,294 --> 00:36:05,804 Yes. 759 00:36:06,449 --> 00:36:10,319 Not a storage appliance, because I think that's a little bit more, 760 00:36:10,679 --> 00:36:14,429 that's like further along in the spectrum, I would say like a 761 00:36:14,444 --> 00:36:15,254 Yeah, 762 00:36:15,554 --> 00:36:17,504 storage appliance entire too. 763 00:36:17,534 --> 00:36:17,954 So, 764 00:36:18,374 --> 00:36:18,704 yeah. 765 00:36:18,944 --> 00:36:24,494 So, So, so, we got a couple different types of backup servers here, right? 766 00:36:24,499 --> 00:36:24,614 So 767 00:36:24,644 --> 00:36:24,854 Yeah. 768 00:36:24,884 --> 00:36:27,644 the purpose-built backup appliance of various 769 00:36:28,304 --> 00:36:30,554 shades, shades flavors. 770 00:36:30,559 --> 00:36:31,034 Sure. 771 00:36:31,814 --> 00:36:35,134 Um, and I would put them. 772 00:36:36,404 --> 00:36:38,684 More immutable, right? 773 00:36:38,684 --> 00:36:40,754 So immutable is supposed to be a binary 774 00:36:40,864 --> 00:36:41,354 Yeah. 775 00:36:41,864 --> 00:36:43,274 I don't see it as that, right? 776 00:36:43,274 --> 00:36:49,124 So, I put it more immutable than I bought a box. 777 00:36:49,184 --> 00:36:52,664 I installed Windows or Linux, and I install my favorite backups or 778 00:36:53,099 --> 00:36:53,429 Yeah, 779 00:36:53,954 --> 00:36:54,914 Why? 780 00:36:54,974 --> 00:36:56,684 Why, why do you feel that way? 781 00:36:56,684 --> 00:36:57,434 I agree with you. 782 00:36:57,434 --> 00:36:58,304 Why do you feel that way? 783 00:36:58,769 --> 00:37:03,569 I feel that way because when it comes to that appliance, typically 784 00:37:03,689 --> 00:37:05,519 there's more things locked down. 785 00:37:05,519 --> 00:37:08,279 There's less configuration options, right? 786 00:37:08,284 --> 00:37:10,209 It's kind of purpose built for that reason. 787 00:37:10,859 --> 00:37:13,889 Versus when you're rolling your own, you have to worry 788 00:37:13,889 --> 00:37:15,149 about all those dependencies. 789 00:37:15,149 --> 00:37:16,919 What flavor of the OS are you taking? 790 00:37:17,129 --> 00:37:19,289 Are you running through all the security patches? 791 00:37:19,289 --> 00:37:22,589 Is it, are there any nuances in the way it's being deployed 792 00:37:22,589 --> 00:37:25,349 today that leads to security vulnerabilities and things like that? 793 00:37:26,264 --> 00:37:27,074 Yeah, exactly. 794 00:37:27,074 --> 00:37:32,624 And, and when you update those appliances, you update an 795 00:37:32,624 --> 00:37:34,844 image which updates everything. 796 00:37:34,949 --> 00:37:35,369 Yep. 797 00:37:36,374 --> 00:37:41,084 if I have a a box, a Windows box, I gotta update Windows or Linux. 798 00:37:41,759 --> 00:37:42,509 And then the backup 799 00:37:42,509 --> 00:37:43,289 software. 800 00:37:43,364 --> 00:37:45,104 new security, then you gotta update the backup 801 00:37:45,209 --> 00:37:45,419 Yep. 802 00:37:45,434 --> 00:37:45,824 So, yeah. 803 00:37:45,824 --> 00:37:48,854 And these are, and I would say that's, so I would say that the 804 00:37:48,854 --> 00:37:50,774 appliances are more immutable than. 805 00:37:51,944 --> 00:37:53,594 Than the build drill in box. 806 00:37:53,909 --> 00:37:54,269 Yep. 807 00:37:55,304 --> 00:38:03,284 and the, uh, but I, but I'm just gonna say that if you, if we're 808 00:38:03,284 --> 00:38:09,014 talking physical access, I still, I'm gonna put a service like S3 or a 809 00:38:09,014 --> 00:38:13,334 service like Druva that's up in the cloud, farther down the immutability 810 00:38:13,339 --> 00:38:18,614 spectrum than a survey, than a server that is sitting in your data center. 811 00:38:18,839 --> 00:38:19,199 Yep. 812 00:38:19,979 --> 00:38:25,139 And I think it's important because everyone is now doing virtualized, right? 813 00:38:25,139 --> 00:38:27,629 It doesn't matter if it's a physical server or a virtual server. 814 00:38:28,009 --> 00:38:30,389 Anything that's running on your infrastructure or in your 815 00:38:30,389 --> 00:38:34,919 infrastructure is less secure in that immutability spectrum. 816 00:38:36,539 --> 00:38:37,019 Yeah. 817 00:38:39,689 --> 00:38:44,699 the, uh, yeah, I put in the book a lot of things are mislabeled, immutable, 818 00:38:45,509 --> 00:38:49,439 uh, I just, again, it's a spectrum, but. 819 00:38:51,194 --> 00:38:56,114 If, if all you're doing is writing backups to a Linux file 820 00:38:56,114 --> 00:39:00,344 system with the immutability flag turned on, that is not immutable. 821 00:39:00,344 --> 00:39:06,404 I mean, it's, it is more immutable than not But if I, but if someone 822 00:39:06,404 --> 00:39:10,904 with root can and, and you gotta have someone with root, right? 823 00:39:11,144 --> 00:39:11,474 Yep. 824 00:39:11,624 --> 00:39:15,584 if a person with root can go in and unset all those immutability flag, 825 00:39:16,184 --> 00:39:16,754 It don't matter. 826 00:39:17,849 --> 00:39:18,689 That's, I don't know. 827 00:39:18,869 --> 00:39:21,569 That's just not, it's be, it's better than nothing. 828 00:39:21,844 --> 00:39:22,264 Yep. 829 00:39:22,499 --> 00:39:22,889 Right? 830 00:39:22,889 --> 00:39:28,289 So I, I don't want to be the, the, you know, is the enemy of good or 831 00:39:28,289 --> 00:39:30,929 whatever, but I don't need perfect. 832 00:39:30,929 --> 00:39:36,119 And so I don't wanna say that that's crap, but I, I think it's, it's not as immutable 833 00:39:36,119 --> 00:39:37,679 as those appliances that we talked about. 834 00:39:37,679 --> 00:39:41,909 And I still think that a service nobody gets access to your servers 835 00:39:41,909 --> 00:39:43,379 is more immutable than that. 836 00:39:43,619 --> 00:39:43,949 Yeah. 837 00:39:45,619 --> 00:39:47,669 and I still want everybody to back up everything to Tape 838 00:39:50,069 --> 00:39:51,329 Oh, Curtis, you and 839 00:39:51,329 --> 00:39:52,019 your tapes. 840 00:39:52,049 --> 00:39:54,479 at, I work at a tapeless backup company. 841 00:39:54,899 --> 00:39:56,849 Um, so 842 00:39:59,009 --> 00:39:59,339 here's 843 00:39:59,354 --> 00:39:59,564 here's 844 00:40:00,449 --> 00:40:00,569 Go 845 00:40:00,734 --> 00:40:04,004 one thing I wanna ask, and I know we've covered this on prior podcasts. 846 00:40:05,894 --> 00:40:11,894 What are things that an admin can do to understand where on that spectrum, 847 00:40:12,314 --> 00:40:17,024 when someone says, when a vendor says immutability, what are questions 848 00:40:17,024 --> 00:40:18,584 that they should be asking to be able 849 00:40:18,584 --> 00:40:19,034 to determine? 850 00:40:20,429 --> 00:40:24,239 Who has root or admin, right? 851 00:40:24,604 --> 00:40:24,614 Determined. 852 00:40:24,869 --> 00:40:29,429 Who has that and what controls are placed over that? 853 00:40:30,869 --> 00:40:37,079 So, um, if it's an appliance, so like I, I know of like one company. 854 00:40:39,929 --> 00:40:43,889 They do have root, they have a they have the root password on your 855 00:40:43,889 --> 00:40:48,179 system, or they have the password to an account that has a u i D of zero 856 00:40:48,619 --> 00:40:49,039 Yep. 857 00:40:49,199 --> 00:40:49,529 of you. 858 00:40:50,189 --> 00:40:56,429 Um, is essentially the same thing, but in order to log into that account, they 859 00:40:56,459 --> 00:40:58,049 can't log into that account remotely. 860 00:40:58,109 --> 00:40:58,979 You have to. 861 00:40:59,359 --> 00:41:00,889 You have to do an SSH 862 00:41:01,194 --> 00:41:01,634 yeah. 863 00:41:01,639 --> 00:41:02,179 stuff, right? 864 00:41:02,179 --> 00:41:04,339 You have to open up a door for them to log in. 865 00:41:05,089 --> 00:41:07,879 Um, you know, what kind of protections are put against that. 866 00:41:09,409 --> 00:41:14,659 Uh, if the answer is it's just a local box and you're the one that has route, 867 00:41:14,869 --> 00:41:20,269 or there's unprotected route access from someone, um, I just, I worry well. 868 00:41:21,174 --> 00:41:27,824 you know, so if, if, if you or anyone in your ministry in your place has root, not 869 00:41:27,824 --> 00:41:29,774 very immutable, it's better than nothing, 870 00:41:30,114 --> 00:41:30,534 Yep. 871 00:41:30,619 --> 00:41:32,234 not very immutable, right? 872 00:41:35,264 --> 00:41:40,394 you know root, like if, if, if you normally never log into 873 00:41:40,394 --> 00:41:42,644 the system as root, right? 874 00:41:42,674 --> 00:41:48,464 No one in your, you only at you, you go to a ui, you log it as you, right? 875 00:41:48,584 --> 00:41:50,204 Um, and. 876 00:41:51,194 --> 00:41:54,344 There isn't direct route access by anyone. 877 00:41:54,644 --> 00:41:59,744 And the only way you can get route access is you can, um, you do the SSH 878 00:41:59,804 --> 00:42:01,994 tunnel thing that's more immutable. 879 00:42:01,994 --> 00:42:02,984 I like that better. 880 00:42:03,944 --> 00:42:07,004 Um, you still, but again, if you don't know this already, you 881 00:42:07,004 --> 00:42:08,804 still have to do physical security 882 00:42:08,844 --> 00:42:09,334 Yeah. 883 00:42:09,434 --> 00:42:09,764 Yep. 884 00:42:09,764 --> 00:42:10,574 Exactly. 885 00:42:10,604 --> 00:42:11,564 do every other box. 886 00:42:11,564 --> 00:42:11,954 Right. 887 00:42:12,794 --> 00:42:19,239 Um, and then, You know, you ask a lot of que just ask a lot of questions. 888 00:42:19,244 --> 00:42:24,569 What happens if so, for example, I'm currently wondering because I haven't 889 00:42:24,569 --> 00:42:30,719 found a good answer online, I'm currently wondering what happens with Amazon S3 890 00:42:31,019 --> 00:42:34,049 object lock if I delete my account. 891 00:42:35,459 --> 00:42:36,629 I don't think they allow you to. 892 00:42:36,629 --> 00:42:40,619 It depends on what type of object lock you're using, because there are 893 00:42:40,619 --> 00:42:40,739 two. 894 00:42:40,889 --> 00:42:42,719 what, I know what you're saying, right. 895 00:42:42,929 --> 00:42:45,239 Um, but if I'm u is it the compliance mode? 896 00:42:45,239 --> 00:42:46,229 It's the more, 897 00:42:46,904 --> 00:42:47,294 Yeah. 898 00:42:48,059 --> 00:42:48,389 yeah. 899 00:42:48,629 --> 00:42:50,099 So assuming I'm using compliance 900 00:42:50,354 --> 00:42:50,654 Oh wait. 901 00:42:50,684 --> 00:42:52,694 Compliance is R L C R, lg. 902 00:42:52,724 --> 00:42:57,884 I think compliance is less in governance is more, or is it the 903 00:42:57,884 --> 00:42:58,424 other way around? 904 00:42:58,469 --> 00:42:58,629 it 905 00:42:58,694 --> 00:42:58,904 Yeah. 906 00:42:59,054 --> 00:42:59,444 Whichever way 907 00:42:59,444 --> 00:42:59,654 it's. 908 00:42:59,699 --> 00:43:00,599 think it's the other way around. 909 00:43:00,599 --> 00:43:03,719 But anyway, so I'm using the more strict one. 910 00:43:04,794 --> 00:43:06,629 and my credit card stops working. 911 00:43:08,879 --> 00:43:09,269 Right. 912 00:43:09,299 --> 00:43:11,549 It's not like they're gonna keep my storage forever. 913 00:43:12,029 --> 00:43:14,009 You, when you say they won't let you delete it. 914 00:43:14,729 --> 00:43:15,839 Well, I don't have to delete it. 915 00:43:15,839 --> 00:43:17,909 I'm just gonna have my credit card not work anymore. 916 00:43:18,164 --> 00:43:18,554 Yeah. 917 00:43:18,869 --> 00:43:19,259 Right. 918 00:43:20,159 --> 00:43:20,549 What 919 00:43:20,654 --> 00:43:23,354 like you said, it's not protected in all scenarios, right? 920 00:43:24,149 --> 00:43:24,629 Right. 921 00:43:24,869 --> 00:43:26,519 I don't know what the answer is. 922 00:43:27,149 --> 00:43:27,779 I am curious. 923 00:43:27,779 --> 00:43:30,179 I actually bought, I, I keep forgetting to do this, but I want to 924 00:43:30,179 --> 00:43:32,559 go create an Amazon account separate. 925 00:43:32,794 --> 00:43:33,404 Mm-hmm. 926 00:43:34,109 --> 00:43:35,819 Put some object lock stuff in there. 927 00:43:36,179 --> 00:43:39,869 Not a whole lot, just like days or something, 928 00:43:40,139 --> 00:43:40,559 Yeah. 929 00:43:40,589 --> 00:43:45,089 like a couple of gigabytes or something, and then go delete my account and see what 930 00:43:45,149 --> 00:43:45,989 And See what happens. 931 00:43:46,019 --> 00:43:47,189 let me delete the account. 932 00:43:47,759 --> 00:43:48,089 Yeah. 933 00:43:48,899 --> 00:43:53,789 I did read something somewhere that, that there is this like 30, 60 day timeframe. 934 00:43:54,379 --> 00:43:54,539 Hmm. 935 00:43:55,289 --> 00:44:00,929 So again, that's still better because I'm assuming that if you're using Amazon S3 936 00:44:00,929 --> 00:44:01,859 You don't want it to go by. 937 00:44:02,454 --> 00:44:05,489 or any of the other folks you're gonna, you're gonna know that 938 00:44:05,489 --> 00:44:07,259 somebody deleted your account 939 00:44:07,424 --> 00:44:07,724 Yeah. 940 00:44:08,729 --> 00:44:09,209 Right. 941 00:44:09,869 --> 00:44:20,729 Um, and I know what I would want you to ask is what happens if, what happens 942 00:44:20,759 --> 00:44:23,099 if someone gets through someone? 943 00:44:23,279 --> 00:44:29,699 So I was just seeing a thing that said that it's like more than half of hacks 944 00:44:29,699 --> 00:44:31,379 are through compromised credentials. 945 00:44:31,859 --> 00:44:34,709 So what happens if someone, you know, you're a backup service? 946 00:44:35,039 --> 00:44:42,089 What happens if someone ahold of my admin credentials and is able to circumvent mfa? 947 00:44:42,569 --> 00:44:42,929 Yep. 948 00:44:44,159 --> 00:44:44,609 What 949 00:44:44,674 --> 00:44:45,239 damage can they 950 00:44:45,239 --> 00:44:45,509 do 951 00:44:45,509 --> 00:44:45,899 you have 952 00:44:45,989 --> 00:44:46,079 Yeah. 953 00:44:46,294 --> 00:44:47,879 Yeah, how much damage they can do. 954 00:44:48,059 --> 00:44:51,629 Can you undo any of that damage, et cetera, et cetera, et cetera. 955 00:44:51,629 --> 00:44:57,749 I know that go back in time, let's say two years, I don't think Druva 956 00:44:57,749 --> 00:44:59,729 had good answers to that question. 957 00:45:00,119 --> 00:45:00,329 Yeah. 958 00:45:00,389 --> 00:45:00,779 Right. 959 00:45:00,779 --> 00:45:04,229 I remember being in those meetings and saying, Hey, we need to address this. 960 00:45:04,379 --> 00:45:05,579 We have addressed that. 961 00:45:05,579 --> 00:45:08,849 I think we now have really good answers to those questions. 962 00:45:11,009 --> 00:45:15,089 but does your, does you know, if you're not using Druva, 963 00:45:15,089 --> 00:45:16,079 does your vendor have good 964 00:45:16,249 --> 00:45:16,739 Yeah. 965 00:45:17,029 --> 00:45:17,229 questions? 966 00:45:17,234 --> 00:45:18,269 I don't know. 967 00:45:18,719 --> 00:45:19,049 Right. 968 00:45:19,049 --> 00:45:22,859 So, yeah, so come up with worst case scenarios and that one. 969 00:45:23,804 --> 00:45:28,694 compromised admin credentials, whether you're talking a service or 970 00:45:29,714 --> 00:45:34,904 so, like for example, I know that like Druva has data lock, you know, 971 00:45:34,934 --> 00:45:36,884 rubric and cohesive have data locks. 972 00:45:37,154 --> 00:45:41,914 I know that with them, if you, if you compromise, uh, credentials 973 00:45:42,734 --> 00:45:48,554 and you log into them and try to delete, if you, if you enable 974 00:45:48,704 --> 00:45:50,294 datalock, you will not be able to do 975 00:45:50,444 --> 00:45:51,054 Mm-hmm. 976 00:45:51,644 --> 00:45:55,889 So, , what is your vendor's answer to that question? 977 00:45:57,059 --> 00:46:02,639 Um, and um, and then also ask 'em about bit rot. 978 00:46:03,569 --> 00:46:04,979 What are they doing about bit rot? 979 00:46:05,279 --> 00:46:09,629 If they're using object storage, I feel better object storage will automatically 980 00:46:09,629 --> 00:46:12,359 detect bit rot happening underneath. 981 00:46:12,494 --> 00:46:12,984 Yeah. 982 00:46:13,169 --> 00:46:17,219 cuz it will change the hash and somebody will be like, Hey, well not 983 00:46:17,219 --> 00:46:19,229 somebody, a program will figure it out. 984 00:46:20,679 --> 00:46:22,724 Um, I think Biro is rare. 985 00:46:23,024 --> 00:46:23,804 Just for the record. 986 00:46:23,954 --> 00:46:24,704 I think it's rare. 987 00:46:25,844 --> 00:46:26,194 Um, 988 00:46:27,119 --> 00:46:27,989 It's rare. 989 00:46:28,729 --> 00:46:29,594 bit error. 990 00:46:30,089 --> 00:46:30,449 Yeah. 991 00:46:30,554 --> 00:46:30,914 But, 992 00:46:31,649 --> 00:46:32,939 But it's still bad 993 00:46:32,944 --> 00:46:33,659 when it happens. 994 00:46:34,139 --> 00:46:34,349 Yeah. 995 00:46:34,349 --> 00:46:36,539 And the problem is when it's silent, that's the worst. 996 00:46:37,394 --> 00:46:37,994 Exactly. 997 00:46:38,114 --> 00:46:39,284 That's the worst. 998 00:46:39,914 --> 00:46:40,274 All right. 999 00:46:40,274 --> 00:46:41,834 Well enough about protecting backup data. 1000 00:46:41,834 --> 00:46:43,094 You got any final thoughts? 1001 00:46:43,369 --> 00:46:45,059 Well, I think we covered it all. 1002 00:46:45,389 --> 00:46:47,669 I really like virtual air gaps, but that's just me 1003 00:46:51,884 --> 00:46:55,604 I like the old school air gaps, but it's really hard to do those these days. 1004 00:46:56,654 --> 00:46:57,164 All right. 1005 00:46:57,164 --> 00:46:59,894 Well thanks for listening to us. 1006 00:47:02,858 --> 00:47:07,558 The Backup Wrap-Up is written, recorded, and produced by me, W. Curtis Preston. 1007 00:47:08,148 --> 00:47:12,908 If you need backup or DR consulting, content generation, or expert witness 1008 00:47:12,908 --> 00:47:15,708 work, check out backupcentral.com. 1009 00:47:16,218 --> 00:47:19,278 You can also find links for my O'Reilly books on the same website. 1010 00:47:20,008 --> 00:47:23,978 Remember, this is an independent podcast, and any opinions that 1011 00:47:23,978 --> 00:47:27,948 you hear are those of the speaker and not necessarily an employer. 1012 00:47:28,818 --> 00:47:29,478 Thanks for listening