1 00:00:00,030 --> 00:00:03,640 There could be a hacker working their way into your backup server right 2 00:00:03,640 --> 00:00:08,590 now, not because your firewall failed, but because nobody ever even bothered 3 00:00:08,600 --> 00:00:11,480 to change the password on the box. 4 00:00:12,190 --> 00:00:16,210 Today, Prasanna and I sit down with Mike Saylor, uh, to talk about backup 5 00:00:16,210 --> 00:00:21,550 security best practices, the stuff that most teams skip, mainly because 6 00:00:21,580 --> 00:00:23,840 backups are invisible until they're not. 7 00:00:24,430 --> 00:00:28,510 Uh, we're talking about things like default passwords on backup hardware, 8 00:00:28,840 --> 00:00:33,520 service accounts that nobody's looking at, uh, admin logins at 2:00 in the 9 00:00:33,520 --> 00:00:36,360 morning, and why passkeys beat MFA. 10 00:00:37,100 --> 00:00:41,840 There's a real story in here about a red teamer who broke into a company through 11 00:00:41,840 --> 00:00:47,170 their backup system and walked out the front door with the domain controller. 12 00:00:48,240 --> 00:00:50,880 If this is your first time watching or listening to me, I'm 13 00:00:50,880 --> 00:00:53,400 W. Curtis Preston, aka Mr. Backup. 14 00:00:53,810 --> 00:00:56,940 I've been obsessing about backup, recovery, and now cyber 15 00:00:56,940 --> 00:00:59,370 recovery for over 30 years. 16 00:00:59,740 --> 00:01:01,470 If that's your bag, then I'm your guy. 17 00:01:01,750 --> 00:01:05,430 You're not gonna find anyone that cares about this topic more than me. 18 00:01:05,910 --> 00:01:10,310 Ever since 1993 when I had to tell my boss that there were no backups of 19 00:01:10,310 --> 00:01:12,100 the database that we had just lost. 20 00:01:12,650 --> 00:01:15,030 Now I've written five O'Reilly books, a blog, and a podcast. 21 00:01:16,270 --> 00:01:20,210 Here we turn unappreciated admins into cyber recovery heroes. 22 00:01:20,350 --> 00:01:22,400 This is the Backup Wrap Up. 23 00:01:34,326 --> 00:01:35,976 Hey, welcome to the Backup Wrap Up. 24 00:01:35,976 --> 00:01:40,916 I'm your host, Debbie Curtis Preston, and today I have my two besties with me. 25 00:01:41,316 --> 00:01:47,016 We've got Prasanna, getting more gray hair every day, and Mike, 26 00:01:47,056 --> 00:01:49,096 my fellow gray-haired enthusiast. 27 00:01:49,096 --> 00:01:49,986 How's it going, Mike? 28 00:01:52,049 --> 00:01:52,639 It's going great. 29 00:01:52,919 --> 00:01:54,379 Thanks for having me, and good to catch up with you guys 30 00:01:54,404 --> 00:01:54,934 Yeah, 31 00:01:55,011 --> 00:01:56,081 have a question, Curtis 32 00:01:56,424 --> 00:01:56,704 What's that? 33 00:01:58,229 --> 00:02:04,339 Is it better to have natural gray or is it better to dye your hair? 34 00:02:04,409 --> 00:02:07,839 That's always been, like, the constant debate, and I think men typically 35 00:02:08,319 --> 00:02:10,319 don't dye versus women, right? 36 00:02:10,319 --> 00:02:11,379 I'm just going. 37 00:02:11,476 --> 00:02:11,496 All 38 00:02:11,629 --> 00:02:14,079 That's probably generalization 39 00:02:14,336 --> 00:02:14,876 Prasanna. 40 00:02:14,936 --> 00:02:17,476 Two little stories on, to answer that question. 41 00:02:17,816 --> 00:02:20,286 clearly I've gone with natural, clearly. 42 00:02:20,656 --> 00:02:26,096 and there was a minute there where I tried, some, like the kind that 43 00:02:26,126 --> 00:02:29,766 is just supposed to subtly dye your hair, before I went completely gray. 44 00:02:30,026 --> 00:02:32,686 And I did it, and my wife didn't even notice. 45 00:02:32,756 --> 00:02:36,826 So I was like, then, what's even the point?" And then the other thing was 46 00:02:36,836 --> 00:02:40,116 I was talking to her one day and I was like, my wife is Filipino for those 47 00:02:40,126 --> 00:02:43,336 that don't know, and I said, one good thing about Filipino, you guys, like 48 00:02:43,336 --> 00:02:44,856 your hair like stays black forever." 49 00:02:44,856 --> 00:02:47,996 And she looked at me and she goes, know we dye it, right?" 50 00:02:48,176 --> 00:02:49,256 It's like, "No, I had no idea." 51 00:02:52,081 --> 00:02:53,051 Oh, Curtis. 52 00:02:53,496 --> 00:02:54,346 typical dude. 53 00:02:54,526 --> 00:02:55,056 Anyway. 54 00:02:55,386 --> 00:02:55,916 All right. 55 00:02:56,226 --> 00:03:03,236 So we are back to our, series, of, g- basically working through our new book, 56 00:03:03,456 --> 00:03:07,846 that, Mike and I came out with, which is Learning Ransomware Response and Recovery. 57 00:03:07,856 --> 00:03:11,506 For those of you watching on YouTube, you can see a giant version 58 00:03:11,506 --> 00:03:13,086 of that over my shoulder there. 59 00:03:13,456 --> 00:03:15,606 And, if you're not checking us out on YouTube, you should 60 00:03:15,606 --> 00:03:16,766 definitely check out our channel. 61 00:03:16,786 --> 00:03:20,016 And of course, we also have the shorts version. 62 00:03:20,326 --> 00:03:25,006 we do 30 second to a minute and a half short clips, which are doing really well. 63 00:03:25,206 --> 00:03:28,236 and we get a lot of, get a lot of, commentary on them. 64 00:03:28,286 --> 00:03:34,106 sometimes, not good, but sometimes people have no problem with, expressing 65 00:03:34,106 --> 00:03:37,626 their opinion, to a random stranger on the internet because they're generally 66 00:03:37,626 --> 00:03:39,656 people that don't, know who we are. 67 00:03:40,036 --> 00:03:45,396 And, so they're just, they just see me making a random claim and, and then 68 00:03:45,396 --> 00:03:47,786 they wanna argue with me, which is fine. 69 00:03:48,186 --> 00:03:49,086 You want discussion? 70 00:03:49,446 --> 00:03:49,696 Yeah 71 00:03:50,228 --> 00:03:54,128 yeah, more interaction, And, any comment, even if the comment is, "You're a 72 00:03:54,158 --> 00:03:56,378 freaking idiot," it builds engagement. 73 00:03:57,018 --> 00:03:59,178 So comments are good. 74 00:03:59,538 --> 00:04:00,548 but today, 75 00:04:00,684 --> 00:04:02,164 co-host has amazing hair 76 00:04:03,064 --> 00:04:03,494 yeah. 77 00:04:03,524 --> 00:04:04,504 Yeah, I-- we did 78 00:04:04,714 --> 00:04:05,504 You talking about me? 79 00:04:05,662 --> 00:04:05,772 Yeah. 80 00:04:05,834 --> 00:04:06,144 sorry 81 00:04:07,832 --> 00:04:10,692 We did get a, we did get a comment about your hair is gorgeous. 82 00:04:10,792 --> 00:04:12,402 and I knew he was talking about you. 83 00:04:12,672 --> 00:04:17,792 but, speaking of the book, there is a, there's a quote in there from one of our 84 00:04:17,802 --> 00:04:20,132 podcast guests, which was Duane Lafleur. 85 00:04:20,992 --> 00:04:24,502 Duane is a, he's a red team person, right? 86 00:04:24,512 --> 00:04:29,092 he hacks companies on purpose for, as requested by the companies. 87 00:04:29,352 --> 00:04:31,922 I once again will mention the movie Sneakers. 88 00:04:32,152 --> 00:04:34,312 If you haven't seen it, go watch it. 89 00:04:34,502 --> 00:04:38,592 it's not, obviously it- it's movies, but it does a pretty good job 90 00:04:38,612 --> 00:04:41,332 of showing what, red teaming is. 91 00:04:41,382 --> 00:04:46,212 And, and there's a few late great actors in there, specifically, Robert Redford. 92 00:04:46,492 --> 00:04:46,632 y- 93 00:04:46,682 --> 00:04:46,792 IP 94 00:04:47,229 --> 00:04:48,459 great just, yeah. 95 00:04:48,829 --> 00:04:55,399 but, it, Duane's, Duane talked about how he loved backups, except 96 00:04:55,459 --> 00:04:57,539 not for the reason I love backups. 97 00:04:57,799 --> 00:05:02,239 And that was he loved them because, one of the things that we talk about a lot 98 00:05:02,249 --> 00:05:05,779 is that backups, because of what they are, where, you're either invisible 99 00:05:05,779 --> 00:05:11,089 or you're in trouble, because of what they are, they often go, ignored from 100 00:05:11,089 --> 00:05:12,789 a cybersecurity perspective, right? 101 00:05:12,809 --> 00:05:13,819 For a couple of reasons. 102 00:05:13,829 --> 00:05:17,349 One is nobody wants to, get their hands in there because once you 103 00:05:17,349 --> 00:05:19,759 start sticking around in the backups, somebody might ask you actually 104 00:05:19,759 --> 00:05:21,589 to be the backup person, right? 105 00:05:21,629 --> 00:05:25,789 And so there's that, and then because of that, it's often the junior person. 106 00:05:25,799 --> 00:05:29,039 So you have the junior person running backups, and the junior person is the one 107 00:05:29,039 --> 00:05:30,649 who knows the least about cybersecurity. 108 00:05:31,299 --> 00:05:34,969 And today we're gonna talk about things that you need to be doing to 109 00:05:34,969 --> 00:05:38,749 your backup system, and specifically we're gonna talk about accounts 110 00:05:38,749 --> 00:05:41,369 today that are around all the time. 111 00:05:41,369 --> 00:05:43,769 We're gonna talk about service accounts, we're gonna talk about passwords. 112 00:05:44,139 --> 00:05:51,969 And he talked about, where he was able to penetrate a system, a company via their 113 00:05:51,969 --> 00:05:58,139 backup system because what he did was he used some lackadaisical security to, take 114 00:05:58,139 --> 00:06:04,679 control of the backup system and then used that control to restore, the backup 115 00:06:04,679 --> 00:06:10,799 server to a, com- the storage completely outside of the company's, firewall. 116 00:06:11,039 --> 00:06:14,699 It was a backup of the domain controller that he restored to outside of this, 117 00:06:14,749 --> 00:06:19,039 their world where he then had complete control over that server and he was able 118 00:06:19,039 --> 00:06:20,849 to extract all kinds of information. 119 00:06:20,849 --> 00:06:24,319 And this is the kind of thing that we talk about a lot in the book where 120 00:06:24,319 --> 00:06:29,479 it's like your backup server, only does it need to be protected because 121 00:06:29,499 --> 00:06:36,119 you need it for recovery when you get hit from, a ransomware, but also 122 00:06:36,119 --> 00:06:41,035 because it is an exfiltration Source, a potential exfiltration source. 123 00:06:41,385 --> 00:06:45,445 And so there are unfortunately a lot of default passwords. 124 00:06:45,475 --> 00:06:47,065 This is one of the things we're gonna talk about. 125 00:06:47,295 --> 00:06:50,645 There's a lot of default passwords that are in, especially I'm 126 00:06:50,645 --> 00:06:52,155 gonna say backup hardware. 127 00:06:52,435 --> 00:06:55,675 there are some default passwords in some of the backup software, but 128 00:06:55,675 --> 00:06:57,765 I think it's less of a, a problem. 129 00:06:58,195 --> 00:07:01,795 But, so that's what we're gonna talk about in this episode. 130 00:07:02,035 --> 00:07:03,505 Prasanna, it sounded like you, you had 131 00:07:03,680 --> 00:07:04,640 I had a question. 132 00:07:04,680 --> 00:07:04,940 Yeah. 133 00:07:05,240 --> 00:07:08,660 I know you were just talking about default passwords, but I don't think 134 00:07:08,660 --> 00:07:10,820 it applies only for the backup space. 135 00:07:10,870 --> 00:07:15,400 I don't know if you've seen all the issues people have had with network 136 00:07:15,400 --> 00:07:19,880 routers being compromised, being used as botnets for attacks in other places 137 00:07:19,880 --> 00:07:22,250 or as resident, residential proxies. 138 00:07:23,160 --> 00:07:26,960 But all of this stems from, people not changing the default password 139 00:07:26,960 --> 00:07:28,360 when they buy a router, right? 140 00:07:28,720 --> 00:07:32,390 You go buy a wireless access point and you are like, "Oh, I'll just 141 00:07:32,390 --> 00:07:33,570 leave the defaults as it is." 142 00:07:34,159 --> 00:07:34,929 Yeah, agreed 143 00:07:34,939 --> 00:07:38,659 Everything that we're gonna say in this episode would also apply to any other 144 00:07:38,669 --> 00:07:43,359 network or storage infrastructure, except that this podcast isn't called 145 00:07:43,359 --> 00:07:45,949 The Network Schnetwork, it's called The Backup Wrap-Up, and so we're 146 00:07:45,949 --> 00:07:47,799 gonna focus on the backup systems. 147 00:07:48,029 --> 00:07:52,539 But yeah, Prasanna, you're completely, correct that, that there are a lot of, 148 00:07:52,689 --> 00:07:57,899 default passwords out there, especially on network equipment that, the good news 149 00:07:57,899 --> 00:08:02,169 is, at least with network equipment, there are people that actually want to 150 00:08:02,179 --> 00:08:07,639 be network admins, and so they actually study it, and they tend to be, they often 151 00:08:07,659 --> 00:08:12,179 tend to be cybersecurity leaning, and so they tend to be a little better at 152 00:08:12,179 --> 00:08:14,329 doing this, although clearly not perfect. 153 00:08:14,958 --> 00:08:18,538 Or have been doing it for a longer amount of time than the backup admins have been 154 00:08:19,329 --> 00:08:23,669 Mike, the first thing I wanna talk about is this idea of service accounts. 155 00:08:24,019 --> 00:08:26,589 and w- that they're invisible, that they're this thing that's 156 00:08:26,589 --> 00:08:31,409 happening and it's a thing that you can then use to, to, attack things. 157 00:08:31,409 --> 00:08:32,979 You wanna talk about what service accounts are? 158 00:08:34,920 --> 00:08:36,940 About the service accounts Yeah the funny thing about service accounts a lot of 159 00:08:36,940 --> 00:08:42,960 times when we red team an organization even the security services that you're 160 00:08:42,960 --> 00:08:47,600 running are susceptible to compromise So a lot of times we take advantage of 161 00:08:47,610 --> 00:08:52,970 like the antimalware service account or like a Qualys vulnerability scanner It's 162 00:08:52,970 --> 00:08:56,670 a service account that's got privileges it's not protected they're assuming the 163 00:08:56,820 --> 00:09:00,440 endpoint is protected And so when you compromise that account now you have 164 00:09:00,450 --> 00:09:04,550 privilege across all of the endpoints where that agent or service is deployed 165 00:09:05,420 --> 00:09:10,110 it is often overlooked and it's overlooked for a couple of reasons One typically 166 00:09:10,110 --> 00:09:14,170 when you deploy a technology that requires a service account you give it privilege 167 00:09:14,190 --> 00:09:18,420 cause you want it to work shortest path to getting things operational we don't have 168 00:09:18,420 --> 00:09:22,550 to troubleshoot restricted access Just give it all the access and it'll work And 169 00:09:22,550 --> 00:09:27,200 then we'll we'll pull the access back as needed but they forget that part or often 170 00:09:27,200 --> 00:09:30,980 it's overlooked or we ran out of time or we ran out of budget or we've got this 171 00:09:30,980 --> 00:09:34,890 other fire that came up we need to go put out and it just gets pushed off the plate 172 00:09:34,898 --> 00:09:35,128 Mike, 173 00:09:35,178 --> 00:09:36,088 and yeah it's very 174 00:09:36,136 --> 00:09:40,086 and Mike, just real quickly, could you help y- listeners who may not 175 00:09:40,086 --> 00:09:42,236 know, like what is a service account? 176 00:09:42,236 --> 00:09:45,766 Because maybe not everyone is familiar with that aspect 177 00:09:47,136 --> 00:09:51,616 they take a couple of different flavors The the probably the most common one 178 00:09:51,616 --> 00:09:55,066 is just it's just a it's like another user account on the network So there's 179 00:09:55,096 --> 00:10:00,826 mikecompanycom and then there's backupagentcompanycom so it's another 180 00:10:00,826 --> 00:10:04,876 user account in the network and you just you have to know the credentials to log 181 00:10:04,876 --> 00:10:08,966 into it and use it Sometimes that's just filed away somewhere in a password vault 182 00:10:08,966 --> 00:10:13,026 or somebody's desk and you don't have to log into it because once it's running it 183 00:10:13,026 --> 00:10:14,906 just runs until you need to make a change 184 00:10:15,019 --> 00:10:18,329 and some of those accounts, Mike, some of those accounts, if they're created 185 00:10:18,339 --> 00:10:23,409 by the backup software itself, they could indeed have like default passwords 186 00:10:23,409 --> 00:10:25,519 that the backup software, put in there. 187 00:10:26,920 --> 00:10:31,070 Certainly and that lead that leads it to the next type and that's more of an 188 00:10:31,080 --> 00:10:36,400 agentbased software So you install a piece of software on the computer it has its own 189 00:10:36,400 --> 00:10:41,230 credentials and you authenticate it back to a console like your backup console your 190 00:10:41,240 --> 00:10:47,320 antivirus console and it just it runs on that that endpoint with those credentials 191 00:10:47,320 --> 00:10:51,260 and often back to the initial comment often it's a privileged privileged account 192 00:10:51,515 --> 00:10:55,465 Yeah, because if you think about backups, in order for backups to do their job, 193 00:10:55,465 --> 00:10:57,495 they have to have superuser access. 194 00:10:57,495 --> 00:11:02,785 They have to be able to access all files in order to both, first to be able to 195 00:11:02,785 --> 00:11:06,665 res- to back them up, but also, just as importantly, to be able to restore them. 196 00:11:06,665 --> 00:11:11,575 You have to have write-level access to all the accounts, and that does bring me back. 197 00:11:11,575 --> 00:11:14,865 I'll pick on, what back in the day was my favorite product was NetBackup, 198 00:11:15,815 --> 00:11:19,125 and, they had a tool called BPGP. 199 00:11:19,175 --> 00:11:22,755 they did, I do, I did find out they definitely eventually, got rid of this 200 00:11:22,775 --> 00:11:26,045 tool, but, it was originally called BPCP. 201 00:11:26,095 --> 00:11:29,165 BP was the Backup Plus, the original name of the product. 202 00:11:29,585 --> 00:11:35,015 And if you were on the backup server, you could use BPCP to read or write any 203 00:11:35,015 --> 00:11:40,635 file transfer from any file from any client where that daemon, was running. 204 00:11:41,055 --> 00:11:43,365 and so it ju- it just made it really easy. 205 00:11:43,365 --> 00:11:47,485 even with the software, with most backup software products, you can use 206 00:11:47,485 --> 00:11:52,465 the software to back up a file, then restore it locally, but BPCP made it 207 00:11:52,515 --> 00:11:54,205 possible to just do it in one step. 208 00:11:54,525 --> 00:11:56,895 and that just gives you an idea of the kind of thing that 209 00:11:56,915 --> 00:11:58,715 you could do if you have this. 210 00:11:58,765 --> 00:12:03,455 Certainly and if I could add two two comments real quick One since you brought 211 00:12:03,455 --> 00:12:05,235 it up is it pronounced demon or daemon 212 00:12:05,851 --> 00:12:07,371 I say demon, 213 00:12:07,947 --> 00:12:08,727 I say Damon. 214 00:12:08,863 --> 00:12:10,123 you do you're backup guy 215 00:12:10,337 --> 00:12:10,557 Yeah 216 00:12:10,581 --> 00:12:12,611 yeah, it is spelled daemon. 217 00:12:13,441 --> 00:12:14,161 I don't know. 218 00:12:14,611 --> 00:12:20,461 by the way, it's also pronounced L- Linux, based on the fact that it came from Linus 219 00:12:21,927 --> 00:12:23,147 I will never pronounce it 220 00:12:23,173 --> 00:12:26,393 I will never pronounce it that way either, but it is based on the name 221 00:12:26,393 --> 00:12:31,253 of the guy who wrote it initially, whose name is pronounced Linus. 222 00:12:31,633 --> 00:12:32,103 whatever. 223 00:12:32,213 --> 00:12:32,643 Anyway, 224 00:12:33,083 --> 00:12:36,993 the last thing I'll add the last thing I'll add on default passwords there 225 00:12:37,003 --> 00:12:43,023 there's another problem or a a a variation of the default password situation and 226 00:12:43,023 --> 00:12:47,323 that's coincidental password So maybe the password has been changed but 227 00:12:47,323 --> 00:12:51,473 it's the same password that's used for a lot of other things So this admin 228 00:12:51,473 --> 00:12:55,753 account password is the same as that admin password But then also when you 229 00:12:55,813 --> 00:13:01,393 get to managed service providers so maybe you outsource IT management that 230 00:13:01,733 --> 00:13:05,413 IT management company may be using the same backup password for your 231 00:13:05,413 --> 00:13:09,243 environment that as the same password they're using at another company's 232 00:13:09,243 --> 00:13:09,813 environment 233 00:13:10,072 --> 00:13:11,092 my head to think of that 234 00:13:11,453 --> 00:13:16,993 world very similarly bad guys are looking for the shortest path with the least 235 00:13:17,043 --> 00:13:21,363 with the most value So if I'm gonna steal if I know to look for backups 236 00:13:21,363 --> 00:13:25,433 and that's where all the data is gonna look at managed service providers cause 237 00:13:25,433 --> 00:13:26,853 that's where all the passwords are 238 00:13:27,678 --> 00:13:27,868 Yeah. 239 00:13:27,963 --> 00:13:31,373 So if I compromise one managed service provider I likely have 240 00:13:31,373 --> 00:13:34,383 access to many client environments 241 00:13:34,526 --> 00:13:38,136 a thing, this was several years ago, but there was a dentist 242 00:13:38,523 --> 00:13:40,113 Post office, yeah 243 00:13:40,336 --> 00:13:44,386 that they hacked the MSP, and as a result they were able to hack, 244 00:13:44,466 --> 00:13:46,196 hundreds of dentists around the country 245 00:13:47,595 --> 00:13:48,005 That's right 246 00:13:48,079 --> 00:13:52,829 So for the service account, Mike's, because when it gets deployed to 247 00:13:52,839 --> 00:13:55,999 your environment, you're going to have multiple agents deployed across 248 00:13:56,009 --> 00:13:59,629 all of these systems in order to be able to back up and restore them. 249 00:14:00,289 --> 00:14:04,029 Are each of these service accounts across all of the individual machines 250 00:14:04,039 --> 00:14:08,279 uniquely protect- or do they have unique passwords, or is it typically 251 00:14:08,769 --> 00:14:13,069 a single service account which might be local to a particular machine might 252 00:14:13,079 --> 00:14:17,799 share a common password with another service account on a different machine? 253 00:14:20,469 --> 00:14:25,279 Typically and I would say typically putting that in the 90 percentile range 254 00:14:25,299 --> 00:14:29,769 the passwords are the same I think there are some highly regulated and 255 00:14:30,109 --> 00:14:34,869 possibly highly more secure environments where you could assign unique passwords 256 00:14:34,869 --> 00:14:36,609 but that's definitely the exception 257 00:14:36,787 --> 00:14:40,087 more management, more complexity, all the rest of that 258 00:14:40,611 --> 00:14:41,051 For sure 259 00:14:41,501 --> 00:14:41,631 For 260 00:14:41,902 --> 00:14:45,562 so it's bad enough if we, are able to compromise a single account. 261 00:14:45,852 --> 00:14:49,032 what then, let's talk about privilege escalation, Mike. 262 00:14:50,362 --> 00:14:54,902 what is privilege escalation and what could someone do, once they achieve that? 263 00:14:56,967 --> 00:15:01,347 So privilege escalation is taking whatever access you you're able to gain 264 00:15:01,347 --> 00:15:08,377 at whatever level normal user guest super user admin global admin and get to that 265 00:15:08,377 --> 00:15:12,937 next level and sometimes we talk about privilege escalation but sometimes you can 266 00:15:12,937 --> 00:15:19,957 actually justify demotion for a particular objective so maybe I wanna pretend I'm 267 00:15:19,957 --> 00:15:24,857 Curtis So I'm global admin I have that ability I can now give myself access to 268 00:15:24,857 --> 00:15:31,227 a a demoted account like Curtis normal user in order to use Curtis's identity 269 00:15:31,837 --> 00:15:36,927 to achieve whatever but the promotion part or the escalation part is I am 270 00:15:36,927 --> 00:15:42,897 Curtis and I wanna be an admin so I'm using Curtis's local permissions on his 271 00:15:42,917 --> 00:15:48,737 computer or his network permissions to find and recon is always the first phase 272 00:15:49,877 --> 00:15:53,287 Find those service accounts find those accounts that have default passwords 273 00:15:53,317 --> 00:15:58,287 and then escalate to that and that whether that's straight to admin or an 274 00:15:58,287 --> 00:16:03,157 administrative privileged account or some series of stepping stones to get there 275 00:16:03,574 --> 00:16:07,784 the thing from a backup perspective that I want people to understand is that almost 276 00:16:07,784 --> 00:16:11,334 every backup software product that I worked with has the idea of a pre and a 277 00:16:11,334 --> 00:16:16,784 post script, So if you can put a script in the appropriate place, the backup 278 00:16:16,784 --> 00:16:21,274 software will run that script as the privileged account that backup runs at. 279 00:16:21,534 --> 00:16:25,924 So if you basically create a script, it creates a user that gives you the 280 00:16:26,014 --> 00:16:30,084 permission that you want, the backup, and put it in the right place, backup will 281 00:16:30,084 --> 00:16:34,534 then use that, will run that script for you, and then you can put in the script 282 00:16:34,544 --> 00:16:36,594 to clean up, behind yourself, right? 283 00:16:36,924 --> 00:16:39,154 So this is just something you need to be aware of. 284 00:16:39,324 --> 00:16:43,654 You should be looking for these scripts, that you should be checking if there 285 00:16:43,654 --> 00:16:46,074 aren't any of these scripts, you should be looking to make sure that 286 00:16:46,074 --> 00:16:48,234 they don't magically appear, right? 287 00:16:48,284 --> 00:16:51,664 and then also if you are using these scripts, you should make sure that 288 00:16:51,674 --> 00:16:53,704 the, that they don't, change on you 289 00:16:54,403 --> 00:17:01,293 Do you know, Curtis or Mike, if any of these backup systems actually look 290 00:17:01,353 --> 00:17:05,893 to see if a script has been changed, like outside… I'm just imagining 291 00:17:05,893 --> 00:17:08,833 a case that someone goes, like a script is defined, a pre-script. 292 00:17:09,173 --> 00:17:11,113 They go behind the scenes to the file system. 293 00:17:11,113 --> 00:17:12,553 They change the script. 294 00:17:12,913 --> 00:17:14,843 They haven't changed the name of it, right? 295 00:17:14,873 --> 00:17:19,303 But is there any backup software that actually does like a verification 296 00:17:19,683 --> 00:17:24,573 to ensure that like a script has not changed between when the admin 297 00:17:24,573 --> 00:17:26,343 actually configured it versus now? 298 00:17:26,372 --> 00:17:26,982 Yeah. 299 00:17:27,042 --> 00:17:29,469 I, I don't, I certainly don't know of any that do that. 300 00:17:29,479 --> 00:17:32,639 But I tell you what, we've got a lot of listeners, and a lot 301 00:17:32,639 --> 00:17:34,259 of them are vendor, people. 302 00:17:34,449 --> 00:17:39,259 So I… If you're aware of a product that does do that, that checks the 303 00:17:39,269 --> 00:17:43,039 scripts that, you know, that… It would be… What would be really nice 304 00:17:43,039 --> 00:17:46,948 is to make sure that the first time a new script is run, it, we send up 305 00:17:46,948 --> 00:17:50,718 some authentication and say, "Hey, is this what you intended to do?" and you 306 00:17:50,718 --> 00:17:54,688 do four eyes, authentication, which is where you have to have two people, 307 00:17:54,868 --> 00:17:56,728 to authenticate That would be nice. 308 00:17:56,728 --> 00:17:59,998 It would also be nice to do what you're suggesting, which is, checking 309 00:18:00,008 --> 00:18:02,238 to make sure that the script that we're running is the same as this. 310 00:18:02,268 --> 00:18:04,448 You could do that by fingerprinting and all that kind of stuff. 311 00:18:04,838 --> 00:18:07,048 But, so this is just something to look into. 312 00:18:07,138 --> 00:18:08,968 and then, let's talk about… What? 313 00:18:08,978 --> 00:18:09,388 Go ahead. 314 00:18:09,592 --> 00:18:13,362 real real quick on that I yeah I'm not familiar with a backup solution that does 315 00:18:13,362 --> 00:18:16,832 that but there are thirdparty tools that do file integrity monitoring so they'll 316 00:18:16,832 --> 00:18:20,762 look for changes in the hash changes in the configuration Tripwire used to do that 317 00:18:21,472 --> 00:18:24,972 I don't know what they're called today But then a lot of security monitoring 318 00:18:24,972 --> 00:18:30,312 tools you can point at an object whether that's a file or a folder or metadata and 319 00:18:30,362 --> 00:18:32,132 it'll alert you when something changes 320 00:18:32,274 --> 00:18:39,084 because all those, the, the ones that I'm aware of, the, would put the script 321 00:18:39,084 --> 00:18:40,934 in a particular predictable place. 322 00:18:41,344 --> 00:18:43,304 and, and so you could monitor for that. 323 00:18:44,164 --> 00:18:47,364 So let's talk about something that I talk a lot about, Mike, and that 324 00:18:47,364 --> 00:18:54,084 is this idea that the backup system should not share any, credentials with 325 00:18:54,084 --> 00:18:55,974 the production environment, right? 326 00:18:56,004 --> 00:19:00,174 that we don't put it in the, Active Directory domain, for example. 327 00:19:00,194 --> 00:19:01,454 why is that the case? 328 00:19:03,360 --> 00:19:07,380 Well which it just it goes back to that idea or the concept of putting as many 329 00:19:07,380 --> 00:19:14,840 layers of security and obfuscation and really just extra effort as you can to 330 00:19:14,870 --> 00:19:20,830 not only deter and potentially hinder but also give you time to identify when 331 00:19:21,100 --> 00:19:26,350 weird things are happening So for example if you've got your backup your backup 332 00:19:26,350 --> 00:19:31,710 system on a separate network like a like a management network so all your servers 333 00:19:31,770 --> 00:19:36,460 you'd have two two network interfaces one for production and one for backup and 334 00:19:36,460 --> 00:19:42,220 management all day long your production network interface is just super busy 335 00:19:42,220 --> 00:19:46,080 and it's dynamic there's just tons of activity and you that's needle in a hayst 336 00:19:46,400 --> 00:19:51,890 needle in a haystack looking for bad actors But on your management interface 337 00:19:52,090 --> 00:19:56,750 that's like you can baseline that You know exactly what's going on there when to 338 00:19:56,750 --> 00:20:01,600 expect backups what bandwidth looks like connections and all those things So much 339 00:20:01,630 --> 00:20:07,340 less dynamic traffic on that interface and much easier to identify when weird things 340 00:20:07,340 --> 00:20:12,040 are happening So there's that then on the Active Directory side let's just take the 341 00:20:12,040 --> 00:20:16,490 example that you brought up where a bad guy was able to to get a copy of a domain 342 00:20:16,490 --> 00:20:21,110 controller then the domain controller has all the privileged accounts and some of 343 00:20:21,110 --> 00:20:26,600 those are you just have to have like your global admin and Office 365 exchange admin 344 00:20:26,610 --> 00:20:30,140 All those things are in your normal Active Directory and they need to be but some of 345 00:20:30,140 --> 00:20:33,790 these others like your backup admin maybe your your cybersecurity tools some of 346 00:20:36,200 --> 00:20:40,600 those other things recovery accounts So if someone does compromise your 347 00:20:40,600 --> 00:20:44,900 domain you've got this other interface with separate credentials that you 348 00:20:44,900 --> 00:20:49,150 might be able to re reacquire some of those servers or part of your network 349 00:20:49,538 --> 00:20:51,198 Yeah, I li- I like that separate network. 350 00:20:51,198 --> 00:20:55,118 I've always been a fan of the separate network, for backup traffic, more 351 00:20:55,118 --> 00:20:58,828 so back when that really meant that the backup system could get 352 00:20:58,828 --> 00:21:00,358 a predictable amount of bandwidth. 353 00:21:00,718 --> 00:21:04,978 But now, And what's really nice about it now is that we can do this, this is 354 00:21:04,988 --> 00:21:07,228 part of infrastructure as code, right? 355 00:21:07,228 --> 00:21:11,348 That you can just literally, as you're creating your VMs, in the cloud, you 356 00:21:11,348 --> 00:21:14,738 can create them this way, and you say, "All backup traffic goes through 357 00:21:14,738 --> 00:21:18,908 this, port, and production traffic goes through that port." Prasanna, 358 00:21:19,044 --> 00:21:22,704 actually see that in production today, Mike? 359 00:21:23,094 --> 00:21:23,164 oh, 360 00:21:23,287 --> 00:21:26,937 I wonder if a lot of this like sort of… 'Cause I know like ideally, 361 00:21:26,937 --> 00:21:31,077 it is best practice to have like separate management network, backup 362 00:21:31,097 --> 00:21:32,377 network, and production network. 363 00:21:32,977 --> 00:21:37,317 But that's also like time-consuming to set up and manage over time and everything 364 00:21:37,317 --> 00:21:41,837 else, And also in the world of virtual machines where it could be easier, 365 00:21:41,837 --> 00:21:43,517 but it's also still a bit of a pain. 366 00:21:43,977 --> 00:21:48,017 And so are you actually seeing people continue to have these like 367 00:21:48,327 --> 00:21:50,207 isolated networks that they use? 368 00:21:52,071 --> 00:21:55,251 I haven't seen any new ones A lot of the ones that are out there have been there 369 00:21:55,251 --> 00:22:02,061 for a while mostly in telecom And y your backup strategy your management strategy's 370 00:22:02,061 --> 00:22:06,051 gonna drive that architecture in the environments where I've seen the separate 371 00:22:06,061 --> 00:22:11,401 interface their backup jobs were huge and even having the separate interface 372 00:22:11,401 --> 00:22:15,161 and better bandwidth they still struggled with backup jobs completing before the 373 00:22:15,161 --> 00:22:19,531 next business day so that was primarily the driver for the strategy for those 374 00:22:19,531 --> 00:22:24,521 organizations is how do we make our backup strategy effective instead of thinking we 375 00:22:24,521 --> 00:22:29,851 need a new strategy and then probably more recently today they're all mostly VLANs 376 00:22:29,851 --> 00:22:32,301 instead of physical network interfaces 377 00:22:32,392 --> 00:22:33,332 Yeah, that's a good point. 378 00:22:33,382 --> 00:22:35,932 do you wanna, just cover what a VLAN is real quick for those 379 00:22:35,932 --> 00:22:36,942 that are not familiar with that? 380 00:22:38,837 --> 00:22:44,617 Yeah So a a VLAN is j it's a virtual network but it's run off of one appliance 381 00:22:44,717 --> 00:22:50,417 And so if you think about a a switch with let's just 32 16 32 ports that you 382 00:22:50,417 --> 00:22:55,877 would plug in your network cables to when you log into that that switch the 383 00:22:55,877 --> 00:23:01,507 interface for that allows you to assign or create these virtual networks so that 384 00:23:01,567 --> 00:23:09,107 network jack Ethernet jack one through six is VLAN one now you can create access 385 00:23:09,107 --> 00:23:14,367 lists and restrictions whether it's computer in VLAN one cannot talk to any 386 00:23:14,367 --> 00:23:19,687 other computers and vice versa so it's isolated but you can also restrict the 387 00:23:19,697 --> 00:23:24,877 type of traffic we don't allow people in VLAN one to receive internet traffic we 388 00:23:24,877 --> 00:23:30,947 restrict that both normal HTML and the encrypted 443 ports and those protocols 389 00:23:30,947 --> 00:23:35,587 You can do a lot with VLANs from a a another layer of network management 390 00:23:35,587 --> 00:23:41,457 and security And then a computer a a server again if you had two network 391 00:23:41,457 --> 00:23:44,597 interfaces or even a virtual network interface now that we're talking about 392 00:23:44,597 --> 00:23:51,477 virtual machines you can connect that one computer to one to many virtual networks 393 00:23:52,127 --> 00:23:54,487 through through that VLaning switch 394 00:23:54,590 --> 00:23:59,270 is a way we can segregate the traffic even though we only have one actual interface. 395 00:24:00,687 --> 00:24:01,067 Right 396 00:24:01,189 --> 00:24:05,539 about most consumer Wi-Fi devices or routers these days, right? 397 00:24:05,539 --> 00:24:08,039 You have a guest network and you have your normal, right? 398 00:24:08,349 --> 00:24:11,489 And that is a form of s- network segregation. 399 00:24:11,749 --> 00:24:14,299 I'm sure they're using VLANs or something under the covers as well. 400 00:24:15,596 --> 00:24:16,006 All right. 401 00:24:16,766 --> 00:24:22,126 So let's just talk about, let's review, the basically the things that people 402 00:24:22,126 --> 00:24:25,356 can do, to help with this problem. 403 00:24:25,646 --> 00:24:30,086 and the first one I'm gonna talk about is the easy one, which is go look for and 404 00:24:30,086 --> 00:24:32,436 change all the default passwords, right? 405 00:24:32,616 --> 00:24:36,326 think about all of these backup, especially backup service accounts. 406 00:24:36,596 --> 00:24:38,716 Prasanna, you mentioned network infrastructure. 407 00:24:38,716 --> 00:24:40,246 I'd say storage infrastructure. 408 00:24:40,576 --> 00:24:44,696 servers don't tend to have, 'cause unless it's a, an appliance 409 00:24:44,706 --> 00:24:47,846 server that you bought, they don't tend to have default passwords. 410 00:24:47,856 --> 00:24:51,966 Although I suppose like a, particular distribution might have that. 411 00:24:52,426 --> 00:24:55,466 any other default passwords, Mike, that you can think of that people 412 00:24:55,466 --> 00:24:57,756 should go be look, looking to change? 413 00:25:02,901 --> 00:25:07,031 Just about anything comes with a default password whether it's blank or password 414 00:25:07,171 --> 00:25:14,791 admin or the name of the product and there there may also be a support account so if 415 00:25:14,791 --> 00:25:18,271 you need help you're gonna call a number or get on a website and they're gonna 416 00:25:18,281 --> 00:25:25,011 be able to remote connect in using their support account My recommendation to all 417 00:25:25,011 --> 00:25:31,301 of those support situations turn it off until you need help Don't just leave it 418 00:25:31,301 --> 00:25:36,801 on cause those are just like home security systems you A contractor now it's not 419 00:25:36,801 --> 00:25:39,931 even the company that installs it now they hire contractors that come out and 420 00:25:39,931 --> 00:25:46,581 install your home security system and the default you know key or password is 1111 421 00:25:46,581 --> 00:25:54,001 or what have you And you can put in your new code 1234 But unless you delete or 422 00:25:54,001 --> 00:25:56,331 disable the original code it's still there 423 00:25:57,542 --> 00:25:57,872 Yeah, 424 00:25:58,099 --> 00:26:00,792 But it… One, one thing though. 425 00:26:00,812 --> 00:26:06,072 So I have a sprawling enterprise network, right? 426 00:26:06,072 --> 00:26:07,242 With a whole bunch of devices. 427 00:26:08,462 --> 00:26:12,182 How am I supposed to figure out, what has default passwords and what doesn't? 428 00:26:12,182 --> 00:26:16,022 is there a tool out there that can ki- that can look across my entire 429 00:26:16,022 --> 00:26:18,742 network and they say, "Hey, this is the type of device you have. 430 00:26:18,742 --> 00:26:20,282 Okay, let me try its default"? 431 00:26:21,772 --> 00:26:26,322 A vulnerability scanner like like Nessus You can you can You There's an open source 432 00:26:26,322 --> 00:26:31,432 version of Nessus it doesn't have all the bells and whistles but it's good enough 433 00:26:31,596 --> 00:26:31,636 Yep. 434 00:26:31,746 --> 00:26:32,466 Spell that, Mike 435 00:26:32,652 --> 00:26:40,392 make sure you read the NESSUS Nessus make sure you read the manual or watch 436 00:26:40,392 --> 00:26:44,612 some some YouTube videos before you run it cause it can it can be disruptive if 437 00:26:44,612 --> 00:26:49,852 you don't configure it well But there are plugins for or switches to turn 438 00:26:49,852 --> 00:26:54,112 on to test for default passwords and there's other So those are traditional 439 00:26:54,112 --> 00:27:01,502 network devices like switches routers servers some software and then for OT 440 00:27:01,552 --> 00:27:05,942 operational technology related hardware like SCADA devices and thermostats and 441 00:27:06,442 --> 00:27:12,282 printers and smart boards there are other tools that are more current on 442 00:27:12,492 --> 00:27:14,422 configurations and passwords for those 443 00:27:14,478 --> 00:27:14,738 Yeah. 444 00:27:14,778 --> 00:27:19,228 to go back to your support account, I can, give a shout-out to Rubrik here. 445 00:27:19,238 --> 00:27:22,118 One of the things I know, and maybe there's other products that do this, 446 00:27:22,118 --> 00:27:26,538 but I just know in, in the case of Rubrik, they're not able to connect 447 00:27:26,548 --> 00:27:28,378 from the outside to the support account. 448 00:27:28,388 --> 00:27:30,838 They require you to do an SSH tunnel. 449 00:27:31,108 --> 00:27:34,758 the, the server is set up or the, the appliance is set up in such a way that 450 00:27:35,088 --> 00:27:39,518 you can't connect to the outside on that account, and it would be… If 451 00:27:39,518 --> 00:27:43,068 it's possible for you to do that, And by the way, if you guys are aware of 452 00:27:43,068 --> 00:27:47,268 other backup software or backup hardware products that, that do that same 453 00:27:47,268 --> 00:27:51,758 concept, I like that as well, that the account is set up in such a way that you 454 00:27:51,798 --> 00:27:53,708 cannot connect to it from the outside. 455 00:27:54,038 --> 00:27:55,168 the, I like that a lot. 456 00:27:55,558 --> 00:27:56,788 another thing… Go ahead. 457 00:27:57,540 --> 00:27:57,890 What's that? 458 00:27:58,273 --> 00:28:02,983 that thought and enhancing my previous comment about turning stuff off when you 459 00:28:02,983 --> 00:28:10,113 don't need it that includes publicfacing services so if a vendor does need to SSH 460 00:28:10,113 --> 00:28:14,643 or VPN in to address help you address a problem those services should only be 461 00:28:14,643 --> 00:28:16,803 enabled and turned on when they're needed 462 00:28:16,986 --> 00:28:17,476 Agreed. 463 00:28:17,493 --> 00:28:18,453 not just all 464 00:28:18,596 --> 00:28:19,176 Agreed. 465 00:28:19,236 --> 00:28:21,426 Especially my favorite service, RDP, the, the 466 00:28:23,526 --> 00:28:25,246 Ransomware Deployment Protocol. 467 00:28:25,626 --> 00:28:27,066 Two two other things I'd like to add 468 00:28:27,080 --> 00:28:27,450 sure 469 00:28:27,816 --> 00:28:35,786 is one of the other situations that we run into a lot is backup admin is running 470 00:28:35,906 --> 00:28:40,066 daytoday operations as backup admin So he's checking his email he's surfing the 471 00:28:40,066 --> 00:28:47,146 internet and that's the admin account And so good best practice is Bob backup admin 472 00:28:47,156 --> 00:28:51,956 has a b a normal Bob account that's not an admin and that's what he's doing daytoday 473 00:28:51,956 --> 00:28:58,806 stuff in and then he's gotta log into Bob the backup admin to do backup admin stuff 474 00:28:59,010 --> 00:28:59,190 Like 475 00:28:59,346 --> 00:29:00,326 That's painful, Mike. 476 00:29:02,500 --> 00:29:06,640 And then the last thing I'll mention because if this happened your your 477 00:29:06,640 --> 00:29:10,360 red team example earlier would have been more difficult if not preventable 478 00:29:10,760 --> 00:29:14,540 encrypt your backups and have different credentials for the decryption part 479 00:29:14,596 --> 00:29:14,866 Yeah. 480 00:29:14,996 --> 00:29:15,496 Agreed. 481 00:29:15,596 --> 00:29:16,076 Agreed. 482 00:29:16,286 --> 00:29:22,076 Now, in the case of this particular red team, he was able to actually take control 483 00:29:22,076 --> 00:29:26,016 of the backup server, and so encryption doesn't help at that point, right? 484 00:29:26,056 --> 00:29:30,316 he didn't steal backups, he stole the back- he stole the backup server, right? 485 00:29:30,506 --> 00:29:33,606 and I think that's a mi a a common misconception for people that 486 00:29:33,626 --> 00:29:36,966 think my data's encrypted it's only encrypted when you're not using it 487 00:29:37,039 --> 00:29:37,349 Right 488 00:29:37,456 --> 00:29:39,946 you can't work on encrypted data We're not that smart 489 00:29:40,085 --> 00:29:40,505 yeah, by the 490 00:29:40,616 --> 00:29:43,796 when you log in or authenticate to a system that's encrypted it becomes 491 00:29:43,806 --> 00:29:45,656 unencrypted so that you can use it 492 00:29:45,843 --> 00:29:49,923 By the way, along the same lines as, admin also, if your account, if 493 00:29:49,923 --> 00:29:54,823 your admin account doesn't need to be named admin, change that, right? 494 00:29:54,843 --> 00:29:58,433 Again, don't make it the default username and the default password, right? 495 00:29:58,823 --> 00:29:59,323 admin? 496 00:29:59,925 --> 00:30:01,675 yeah, admin, yeah, exactly. 497 00:30:01,905 --> 00:30:04,565 and then we can also just talk about just hardening everything, but 498 00:30:04,565 --> 00:30:08,125 especially the management plane and especially backup stuff, and that is 499 00:30:08,345 --> 00:30:14,725 if you're not using either MFA or, pass keys on your, security infrastructure 500 00:30:14,725 --> 00:30:17,935 and your backup infrastructure, it's time to do that, right? 501 00:30:17,985 --> 00:30:21,445 Prasanna, you wanna remind us what MFA is and why we care? 502 00:30:21,453 --> 00:30:21,763 Yeah. 503 00:30:22,183 --> 00:30:24,233 MFA is multi-factor authentication. 504 00:30:24,243 --> 00:30:26,713 So you see this everywhere, right? 505 00:30:26,713 --> 00:30:29,543 It's like you log in using your password and it's "Please 506 00:30:29,583 --> 00:30:31,923 touch your keypad," right? 507 00:30:31,923 --> 00:30:34,703 The biometric sensor in order to know it's you or look at the face 508 00:30:34,703 --> 00:30:39,953 ID so it knows it's you, and so this way you have multiple ways to 509 00:30:39,953 --> 00:30:41,253 authenticate that it's just you. 510 00:30:41,253 --> 00:30:45,343 It's not just the password you know, but it's also, who you are. 511 00:30:45,693 --> 00:30:47,743 And so that's multi-factor authentication. 512 00:30:48,293 --> 00:30:52,523 Passkey is, I don't know what I would call it. 513 00:30:52,523 --> 00:30:55,293 It's like passwords on steroids because you don't really need to 514 00:30:55,293 --> 00:30:57,663 know the password anymore, right? 515 00:30:57,663 --> 00:31:04,273 It's the ability to tie a particular device to a particular website such that 516 00:31:04,273 --> 00:31:09,273 you no longer necessarily need to use a password whenever you are accessing that. 517 00:31:09,683 --> 00:31:14,103 It's tied… And once again, it uses sort of MFA or, biometrics 518 00:31:14,823 --> 00:31:18,543 in order to do the initial establishing of the authentication. 519 00:31:18,813 --> 00:31:19,953 But after that, it's automatic. 520 00:31:20,509 --> 00:31:21,009 Yeah, so you 521 00:31:21,093 --> 00:31:21,583 How'd I do? 522 00:31:21,739 --> 00:31:25,309 yourself to the device and the device authe- authenticates itself. 523 00:31:25,709 --> 00:31:30,239 Mike, just curious, one thing I've noticed lately has been random 524 00:31:30,239 --> 00:31:33,749 companies that I interface with and they've done away with passwords, 525 00:31:33,749 --> 00:31:38,679 but they've replaced it with, "We're going to email you a one-time code." 526 00:31:38,739 --> 00:31:42,879 And I'm like, "This isn't better." w- your thoughts on that? 527 00:31:44,887 --> 00:31:48,657 No I agree given enough reconnaissance and time bad guys are gonna figure that 528 00:31:48,657 --> 00:31:54,147 out for that organization And if it also depends on whether are you emailing it to 529 00:31:54,147 --> 00:31:58,547 my personal account Are you emailing it to my work account are you requiring me to 530 00:31:58,557 --> 00:32:03,257 go set up a new account like that I don't use for anything else there's ways of 531 00:32:03,457 --> 00:32:10,477 doing that and each one of them have pros and cons but long story short MFA really 532 00:32:10,477 --> 00:32:16,487 needs to be a wwhat would be considered outofbounds communication so not normal so 533 00:32:16,487 --> 00:32:20,877 like a an authenticator app on your phone is probably the better So you've gotta 534 00:32:20,923 --> 00:32:21,183 Yeah 535 00:32:21,307 --> 00:32:26,737 and set it up with layers also So my authenticator app requires a PIN to get 536 00:32:26,737 --> 00:32:31,047 into the authenticator app in order to get the code to use as MFA to get into 537 00:32:31,047 --> 00:32:36,617 whatever site But preempting all of that making sure that policy does not allow 538 00:32:36,627 --> 00:32:42,837 me to bypass MFA because I now trust this device or remember me or any of those 539 00:32:42,837 --> 00:32:48,687 other things that would then store an MFA token in a browser or on this computer to 540 00:32:48,697 --> 00:32:54,667 be trusted again in the future MFA has to happen every time in order to be effective 541 00:32:54,859 --> 00:32:59,789 again, Mike, I, I so often learn random stuff from you that, and this is the one 542 00:32:59,789 --> 00:33:05,199 today, and that is I didn't know the, my, my particular authenticator app does not 543 00:33:05,319 --> 00:33:08,639 require a pin, and I like that idea a lot. 544 00:33:08,689 --> 00:33:11,079 I'm gonna go check to see if it supports it, and then perhaps 545 00:33:11,079 --> 00:33:12,289 I just haven't turned it on. 546 00:33:12,749 --> 00:33:15,989 Similar to back in the day when you talked about having a separate browser 547 00:33:15,989 --> 00:33:20,709 for, or a separate browser session, and the way I did it was I take my accounts 548 00:33:20,709 --> 00:33:25,179 that matter, and I run it on a completely different browser, and then I t- I use 549 00:33:25,179 --> 00:33:29,469 Chrome as my, regular browser, and then I use a different browser for that stuff. 550 00:33:29,769 --> 00:33:35,159 And I actually installed a Chrome plugin that if I go to any of those accounts that 551 00:33:35,219 --> 00:33:38,809 matter, it says, "Hey, you're not supposed to be over here. You're supposed to be 552 00:33:38,809 --> 00:33:40,119 over there in the other place," right? 553 00:33:40,489 --> 00:33:42,209 So I love it when I… I've been 554 00:33:42,345 --> 00:33:43,525 Learn tidbits. 555 00:33:43,529 --> 00:33:45,259 it when I learn st- stuff new. 556 00:33:45,679 --> 00:33:48,099 th- this particular old dog, new tricks. 557 00:33:48,129 --> 00:33:48,819 What was that, Mike? 558 00:33:49,897 --> 00:33:53,707 similar to your plugin, on the company network, if you've-- if 559 00:33:53,707 --> 00:34:00,217 you're diligent about making sure that your admins are using the admin 560 00:34:00,217 --> 00:34:05,307 account only when they're doing admin stuff, you can monitor, like alert. 561 00:34:05,357 --> 00:34:08,087 Every time an admin logs in, someone should know. 562 00:34:08,127 --> 00:34:11,677 It gets logged, and then over time, you've got a baseline. 563 00:34:11,687 --> 00:34:15,987 Like admins, 99% of the time, they're only logging into the admin account 564 00:34:15,987 --> 00:34:19,517 Monday through Friday, eight to five, maybe Saturday to do some work. 565 00:34:20,067 --> 00:34:23,297 But so if you can baseline that activity along with having 566 00:34:23,297 --> 00:34:26,887 visibility into it, again, you can start to look for anomalies. 567 00:34:26,897 --> 00:34:30,547 Someone, admin logged in at 2:00 a.m., we should call the admin 568 00:34:30,547 --> 00:34:31,597 and see if that's really them. 569 00:34:31,627 --> 00:34:35,077 And that's, if it's a one-off thing, then that's, that's easy to manage. 570 00:34:35,367 --> 00:34:39,017 then on the audit or even incident response side, if you're looking 571 00:34:39,027 --> 00:34:43,377 backwards about, in, in time for activity about an, an admin account, you 572 00:34:43,377 --> 00:34:46,347 should be able to tie admin logged in. 573 00:34:46,677 --> 00:34:48,607 is there a ticket associated with that? 574 00:34:49,007 --> 00:34:51,047 was there a, a change in the backup system? 575 00:34:51,047 --> 00:34:55,587 Was there a break fix, a new install, a new config? 576 00:34:55,617 --> 00:34:56,637 Is there a ticket for that? 577 00:34:56,637 --> 00:35:00,057 So that goes back to general controls and are we actually doing, these good, 578 00:35:00,627 --> 00:35:04,257 am I following the change management or patch management policy or what have you? 579 00:35:04,657 --> 00:35:06,317 so from an audit perspective, there's that. 580 00:35:06,327 --> 00:35:12,287 then incident response also, if it's an admin and we're logging those activities, 581 00:35:12,287 --> 00:35:16,477 that's gonna help us build the picture for how this happened and when it happened and 582 00:35:16,921 --> 00:35:17,161 Yeah, 583 00:35:17,161 --> 00:35:17,481 all that 584 00:35:17,494 --> 00:35:23,364 and you can also, a very common thing to do is to prevent the ability to log in 585 00:35:23,364 --> 00:35:26,724 directly as the, the admin account, right? 586 00:35:26,794 --> 00:35:28,304 you can say, you can establish that. 587 00:35:28,334 --> 00:35:31,954 You have to es- you have to start with establishing that as a policy, right? 588 00:35:31,984 --> 00:35:34,884 And then there are technological things that you can do to simply 589 00:35:34,884 --> 00:35:38,684 prevent a person from logging in, directly to the admin account. 590 00:35:38,694 --> 00:35:41,664 They have to log in as themselves and then become the admin account. 591 00:35:42,004 --> 00:35:46,734 and that even if you can't prevent it, you can, again, 592 00:35:46,744 --> 00:35:48,244 like you said, monitor for that. 593 00:35:48,254 --> 00:35:52,024 So if they, if somebody, if anybody does directly monitor into the admin 594 00:35:52,024 --> 00:35:54,394 account, that's an event, right? 595 00:35:54,404 --> 00:35:56,184 That you need to go research, which… 596 00:35:56,324 --> 00:35:59,804 And maybe it's just your employee, it's 2:00 in the morning and they're half 597 00:35:59,804 --> 00:36:03,635 asleep and they don't know what they're doing, but, maybe it's a, bad guy. 598 00:36:03,685 --> 00:36:07,705 I had the opposite, situation go where we were seeing failed login attempts to 599 00:36:07,705 --> 00:36:09,675 an admin account every Thursday morning. 600 00:36:10,455 --> 00:36:12,715 And, we would escalate that to the client. 601 00:36:12,725 --> 00:36:15,045 They would go look at it, they'd close the ticket, and, but 602 00:36:15,045 --> 00:36:16,305 we wouldn't get any feedback. 603 00:36:16,765 --> 00:36:19,855 But every Thursday morning, there were these failed admin account logins, 604 00:36:19,885 --> 00:36:24,195 and it was because their admin was going through a tough patch and would 605 00:36:24,195 --> 00:36:28,645 go get drunk every Wednesday night and come in Thursday morning and have 606 00:36:28,665 --> 00:36:30,065 trouble remembering his password. 607 00:36:31,215 --> 00:36:33,965 So eventually we heard the whole story, but, that's 608 00:36:35,113 --> 00:36:38,413 Yeah, b- Curtis, one of the things you write, you talked about MFA 609 00:36:38,413 --> 00:36:38,973 and passkeys at the beginning. 610 00:36:39,241 --> 00:36:39,591 Yeah 611 00:36:39,863 --> 00:36:43,573 Do you know of any backup systems that use passkeys? 612 00:36:45,671 --> 00:36:48,161 I know that there are some that support it, yeah. 613 00:36:48,941 --> 00:36:49,251 yeah. 614 00:36:49,321 --> 00:36:55,391 and I think most… what some of them do is they outsource it to, 615 00:36:55,401 --> 00:36:57,211 if you're gonna use an OTP, right? 616 00:36:57,211 --> 00:37:00,491 You're gonna, you're gonna use something like Okta, right? 617 00:37:00,501 --> 00:37:02,211 They're gonna use some sort of SSO, right? 618 00:37:02,211 --> 00:37:05,111 Single sign-on package, and then they outsource it to that. 619 00:37:05,371 --> 00:37:08,981 but I am aware of a handful that, that directly support passkeys, and this 620 00:37:08,981 --> 00:37:12,691 is the thing, like if you're not, if you're not looking into passkeys now, 621 00:37:12,721 --> 00:37:16,131 it's the best thing that's available, from a security perspective, from 622 00:37:16,131 --> 00:37:18,201 a login pers- security perspective. 623 00:37:18,201 --> 00:37:19,381 It's better than MFA. 624 00:37:19,721 --> 00:37:23,141 if you're using MFA, we're actually gonna be doing, the, an upcoming 625 00:37:23,141 --> 00:37:27,541 episode on we're gonna be talking about phishing-resistant MFA why that's a thing 626 00:37:27,541 --> 00:37:29,561 and why you need it and, and what to do. 627 00:37:29,791 --> 00:37:33,531 But again, you could just skip it and go straight to passkeys. 628 00:37:33,801 --> 00:37:34,971 and that's a good thing. 629 00:37:35,671 --> 00:37:35,851 right. 630 00:37:35,911 --> 00:37:37,011 any final thoughts, Mike? 631 00:37:39,777 --> 00:37:43,357 Be diligent, read the manual, change your passwords 632 00:37:43,573 --> 00:37:44,013 There you go. 633 00:37:44,073 --> 00:37:44,553 Prasanna 634 00:37:47,493 --> 00:37:51,023 I got nothing, but I do miss our, podcast recordings, so looking 635 00:37:51,023 --> 00:37:52,643 forward to the upcoming ones 636 00:37:53,443 --> 00:37:54,083 Yeah. 637 00:37:54,133 --> 00:37:54,533 Yeah. 638 00:37:54,533 --> 00:37:56,143 It was, we took a little break there. 639 00:37:56,243 --> 00:37:59,483 and, so hopefully the people are enjoying the, or did enjoy the, the 640 00:37:59,483 --> 00:38:01,373 encore episodes that we put out. 641 00:38:01,393 --> 00:38:03,973 There were some really good episodes in there, so I hope they enjoyed those. 642 00:38:04,203 --> 00:38:06,173 and I'm glad that we're now, recording again. 643 00:38:06,533 --> 00:38:08,063 all right, folks, thanks for listening. 644 00:38:08,083 --> 00:38:09,313 You are why we do this. 645 00:38:09,483 --> 00:38:10,513 That is a wrap 646 00:38:13,520 --> 00:38:18,220 The Backup Wrap Up is written, recorded, and produced by me, W. Curtis Preston. 647 00:38:18,810 --> 00:38:23,570 If you need backup or DR consulting, content generation, or expert witness 648 00:38:23,570 --> 00:38:26,370 work, check out backupcentral.com. 649 00:38:26,880 --> 00:38:29,940 You can also find links for my O'Reilly books on the same website. 650 00:38:30,670 --> 00:38:34,640 Remember, this is an independent podcast, and any opinions that 651 00:38:34,640 --> 00:38:38,610 you hear are those of the speaker and not necessarily an employer. 652 00:38:39,480 --> 00:38:40,140 Thanks for listening