Speaker:

I am super excited about this episode.

Speaker:

We have a networking expert coming on and we talk about what to do in your network,

Speaker:

when you get a ransomware attack, I bet you've been wanting to know that answer.

Speaker:

So stay tuned.

W. Curtis Preston:

Hi and welcome to Backup Central's Restore All podcast.

W. Curtis Preston:

I'm your host, W.

W. Curtis Preston:

Curtis Preston, aka a Mr.

W. Curtis Preston:

Backup and have with me possibly my Pex consultant Prasanna Malaiyandi.

W. Curtis Preston:

How's it going?

W. Curtis Preston:

Prasanna,

Prasanna Malaiyandi:

am.

Prasanna Malaiyandi:

I'm good Curtis.

Prasanna Malaiyandi:

And just for people that's p e x, not P E C K S.

W. Curtis Preston:

Yeah, this is the piping, the, uh, the modern

W. Curtis Preston:

piping alternative to copper, which I think is far superior.

W. Curtis Preston:

And, uh, You know what?

W. Curtis Preston:

Just, just for those that are watching this on on video, which is only a

W. Curtis Preston:

handful of you, but I'm gonna tilt my camera up and this is what my office

W. Curtis Preston:

looks like right now because I got yet another pinhole leak in my, um, second

W. Curtis Preston:

story bathroom water supply, which happens to be right above my office.

W. Curtis Preston:

And yesterday I was just sitting here at my desk and I get this

W. Curtis Preston:

drip drip on my face and I'm like,

Prasanna Malaiyandi:

you're like, am I sweating profusely?

W. Curtis Preston:

Yeah.

W. Curtis Preston:

And the pipe, the pipe is actually over there.

W. Curtis Preston:

The, the, the joint that's leaking, it's actually over there, but you know,

W. Curtis Preston:

the water finds its way, you know,

W. Curtis Preston:

along a drywall seam and then it just sort of drips

W. Curtis Preston:

down onto my face.

Prasanna Malaiyandi:

you know what though, Curtis, I have to say

Prasanna Malaiyandi:

congratulations on finally finishing your other project, which we should

Prasanna Malaiyandi:

tell our

W. Curtis Preston:

my other pro.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

For those who have been following along my other project is, is, I

W. Curtis Preston:

mean, it, it's still, you know, at this point it's 98% done.

W. Curtis Preston:

But, you know, I have put the, the stair, you know, the, the,

W. Curtis Preston:

the flooring on the stairs.

W. Curtis Preston:

Um, it looks really good.

W. Curtis Preston:

Uh, it looks way better than the ceiling in this room, I will say that.

W. Curtis Preston:

Um, and now there's the official first mess on the new floor

W. Curtis Preston:

. There's just, as I

Prasanna Malaiyandi:

At some point it's gonna happen, you know?

W. Curtis Preston:

yeah.

W. Curtis Preston:

Well, apparently that some point is today.

W. Curtis Preston:

Uh, but yeah, so the plumber, who is a good guy, uh, he's been here before,

W. Curtis Preston:

um, he, um, he's talking to me about, he knows a guy that does, uh, PEX repiping

W. Curtis Preston:

so, we'll,

Prasanna Malaiyandi:

Our listeners will learn all about water piping

Prasanna Malaiyandi:

soon in the next few episodes as we go

W. Curtis Preston:

More, more than you ever wanted to know.

W. Curtis Preston:

Uh, and by the way, I did check they can go through the attic.

W. Curtis Preston:

So, um, that is a, that is a real possibility for the, um, and, um, yeah,

W. Curtis Preston:

so anyway, um, and I just realized that, uh, my son-in-law is home today.

W. Curtis Preston:

He's not normally home.

W. Curtis Preston:

I just heard him making noise.

W. Curtis Preston:

I hope he gets his bath out of the way before the plumber gets here.

W. Curtis Preston:

I didn't, I didn't warn him.

W. Curtis Preston:

Uh, I actually hear, I hear a bath going on right now, . So,

W. Curtis Preston:

so that answer's that question.

W. Curtis Preston:

Way too much information going on at depress pressing

W. Curtis Preston:

household.

W. Curtis Preston:

Well, listen.

Prasanna Malaiyandi:

okay.

W. Curtis Preston:

Yeah, we wanna bring on our guest.

W. Curtis Preston:

Uh, he is both, I would say, a friend of the pod.

W. Curtis Preston:

He's also been an enemy of the pod at once.

W. Curtis Preston:

You may recall that we had an episode where basically we just argued

W. Curtis Preston:

with Tom without his per, without him being here to defend himself.

W. Curtis Preston:

Um, that was over a blog post that he said, uh, something about, uh, backup

W. Curtis Preston:

people reporting to security people.

W. Curtis Preston:

And, uh, I

W. Curtis Preston:

think I had an issue with that or something.

W. Curtis Preston:

Tom has been in the industry about 20 years and he is an

W. Curtis Preston:

event lead over at Gestalt.

W. Curtis Preston:

It the, uh, what would you call it?

W. Curtis Preston:

The makers of the Tech Field Day series, which, uh, uh, my

W. Curtis Preston:

employer has used quite a bit.

W. Curtis Preston:

And, um, uh, we're glad to have him on the podcast.

W. Curtis Preston:

Welcome, Tom Hollingsworth.

Tom Hollingsworth:

Well, thank you for having me on Curtis.

Tom Hollingsworth:

It was, uh, it was fascinating to listen to an episode where I was, I was arguing

Tom Hollingsworth:

with somebody and it wasn't even here.

Tom Hollingsworth:

But, uh, I, I, I love, I love listening to you guys, and I've learned quite a bit.

Tom Hollingsworth:

In fact, uh, the very first time that Curtis and I ever met at Tech Field Day

Tom Hollingsworth:

back in 2011, he was teaching me about data de-duplication, and I was trying to

Tom Hollingsworth:

convince him that IP V6 was important.

Tom Hollingsworth:

And I can tell you which one of those things panned out a lot better than the.

W. Curtis Preston:

Uh, well, you know, is it, is that the thing where

W. Curtis Preston:

you do the nat behind the thing?

W. Curtis Preston:

That's what I recall really learning from you was that you gotta do

Tom Hollingsworth:

if you want me to come crashing through your roof

Tom Hollingsworth:

like the Kool-Aid man, just keep

Tom Hollingsworth:

it up my.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

I wanted to bring on somebody that actually understood networking

W. Curtis Preston:

far better than me, right?

W. Curtis Preston:

Which, which is basically many people in the world.

W. Curtis Preston:

With ransomware attacks.

W. Curtis Preston:

One of the things that we talk about is once you've, um, you

W. Curtis Preston:

know, figured out that you actually have a ransomware attack, you,

W. Curtis Preston:

you want to isolate the network.

W. Curtis Preston:

And there there's a discussion, you know, I've been talking with with CISOs lately

W. Curtis Preston:

and, and, and what, what appears to be the reality is that that few environments

W. Curtis Preston:

actually do the, the actual full.

W. Curtis Preston:

Like we just we're just shutting everything off.

W. Curtis Preston:

Right.

Prasanna Malaiyandi:

Go grab the cable, pull it out quick, quick,

W. Curtis Preston:

Speaker:

They're actually, I know.

W. Curtis Preston:

Speaker:

Tom, did you ever watch, uh, alias when it was on

Tom Hollingsworth:

I've seen a couple of

W. Curtis Preston:

and.

W. Curtis Preston:

Okay, well there's an episode in there when they were having a cyber

W. Curtis Preston:

attack and the, uh, what's his name?

W. Curtis Preston:

Um, uh, Marshall Flank man comes running into the data center and he just literally

W. Curtis Preston:

starts flipping, flipping power switches.

W. Curtis Preston:

He's like, they're downloading all the files off the server and he down.

W. Curtis Preston:

He just flips all the power switches off.

W. Curtis Preston:

And so, you know, on one end there is the complete.

W. Curtis Preston:

like networking, shutdown, like literally both internal and external, right?

W. Curtis Preston:

Um, because, you know, once the, once the ransomware is inside, it's gonna try to

W. Curtis Preston:

crawl around and, and make things worse.

W. Curtis Preston:

So that's one way.

W. Curtis Preston:

And then there are, you know, and, and then there's the, those that go, well,

W. Curtis Preston:

well, I'm just going to turn it off, or I'm gonna unplug the cable at the

W. Curtis Preston:

one server or the three servers that appear to be infected and I'm not gonna

W. Curtis Preston:

worry about the rest of the network.

W. Curtis Preston:

And somewhere in the, between those two extremes is what everybody else does.

Prasanna Malaiyandi:

And maybe we should also talk about basics

Prasanna Malaiyandi:

of networking before we jump into this to talk about the detail.

Prasanna Malaiyandi:

Because just

W. Curtis Preston:

Go ahead.

W. Curtis Preston:

Go ahead.

W. Curtis Preston:

Prasanna,

W. Curtis Preston:

what, what do you think we should be talking about first?

Prasanna Malaiyandi:

no, I think it's sort of, because what you just

Prasanna Malaiyandi:

mentioned, Curtis, like everyone might think, oh, all computers

Prasanna Malaiyandi:

are plugged into the same network.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

I think it's important to talk about some of the best practices from networking,

Prasanna Malaiyandi:

Tom, if you could, about sort of network isolation, BLANs, other things like that.

Prasanna Malaiyandi:

Before we get into sort of the other side of things,

W. Curtis Preston:

Yeah, so please explain all networking technology, period before

Tom Hollingsworth:

the good news.

Tom Hollingsworth:

,you've already talked a little bit about it because it's just

Tom Hollingsworth:

a series of tubes, pipes, if you will, that we send things through.

Tom Hollingsworth:

Uh, now the, the, the important thing to realize when you're trying to think

Tom Hollingsworth:

about how ransomware propagates through a network is to realize that, um, the

Tom Hollingsworth:

way that networks have traditionally been built is we have this perimeter

Tom Hollingsworth:

on the outside, you know, it's probably bounded by firewalls and a bunch of

Tom Hollingsworth:

other stuff, and it looks really, really imposing on the castle walls,

Tom Hollingsworth:

but inside of the network, it's a whole lot easier to get around.

Tom Hollingsworth:

And that's just due to the nature of the way that that networks operate.

Tom Hollingsworth:

I mean, ethernet is effectively like trying to shout out somebody's order

Tom Hollingsworth:

number at a fast food restaurant and hoping that you get the right one.

Tom Hollingsworth:

Everybody's gonna hear the message, but if it's not meant for you,

Tom Hollingsworth:

we're just gonna ignore it.

Tom Hollingsworth:

But the problem is, is that that allows you to propagate a lot of information

Tom Hollingsworth:

very quickly, and that's what ransomware is trying to take, uh, advantage of

Tom Hollingsworth:

whenever it's, it's trying to, uh, do almost like, you know, reconnaissance

Tom Hollingsworth:

lateral movement in the network.

Tom Hollingsworth:

So I'm, I'm looking for a whole bunch of, um, , potentially vulnerable servers

Tom Hollingsworth:

going all the way back, you know, to the beginning of my professional IT career,

Tom Hollingsworth:

I was actually working on a help desk, uh, when the S SQL slammer worm came out.

Tom Hollingsworth:

And boy, you'd be surprised how many people had that port open to the internet,

Tom Hollingsworth:

uh, because everything shut down.

Tom Hollingsworth:

And it was really weird to see that.

Tom Hollingsworth:

And you're like, well, you know, at the time I'm, I'm kind of

Tom Hollingsworth:

freshly minted in my career.

Tom Hollingsworth:

And I'm like, well, how could that happen?

Tom Hollingsworth:

And, and now all these years later, I look at it and go, oh my God,

Tom Hollingsworth:

these people were stupid because you're not supposed to do that.

Tom Hollingsworth:

But that's one of the things that people want to take advantage of because the,

Tom Hollingsworth:

the systems want to talk to each other.

Tom Hollingsworth:

They want to be able to exchange information.

Tom Hollingsworth:

That's the purpose of a network.

Tom Hollingsworth:

You actually have to do extra work to prevent them from talking to each other.

W. Curtis Preston:

Right.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

I think that's, you know, I, I, and the, the number of times I went in and out

W. Curtis Preston:

of data centers, uh, over the years, I remember only one, uh, where they had

W. Curtis Preston:

very solid internal firewalls, basically.

W. Curtis Preston:

Right.

W. Curtis Preston:

That, that it was very difficult to do, to traverse laterally

W. Curtis Preston:

within the organization.

W. Curtis Preston:

And that was actually Intuit, uh, right.

W. Curtis Preston:

And, and it's because of what they felt they had.

W. Curtis Preston:

Right.

W. Curtis Preston:

They had all of this very sensitive personal data, thanks to their, you know,

W. Curtis Preston:

they, they had QuickBooks, they have TurboTax, they have all of that stuff.

W. Curtis Preston:

And so they had to basically firewall off systems between each other to

W. Curtis Preston:

prevent that lateral movement that you're right by design in most networks, you

W. Curtis Preston:

buy a switch, you buy, well, a bunch of switches, you plug everything in.

W. Curtis Preston:

And everything talk, everything can talk to everything.

W. Curtis Preston:

Um, and unless you do something to prevent it, a lot of those ports that

W. Curtis Preston:

you talked about, right, just like the SQL, uh, issue, a lot of those

W. Curtis Preston:

ports are visible to the internet.

W. Curtis Preston:

Right.

W. Curtis Preston:

I, I think a, another one would be a, a vCenter Right.

W. Curtis Preston:

And Hyper V, the, that, those ports being visible to the internet, I suppose

W. Curtis Preston:

you hear about that a lot as well.

Tom Hollingsworth:

Yeah.

Tom Hollingsworth:

I usually do.

Tom Hollingsworth:

Whenever there's some kind of, uh, a vulnerability that comes out and

Tom Hollingsworth:

everyone's like, I hope you don't have these exposed to the internet, and

Tom Hollingsworth:

you can literally hear the scrabbling as people run into their keyboards

Tom Hollingsworth:

to figure out if that's the case.

Tom Hollingsworth:

But, you know, as,

Tom Hollingsworth:

as Prasanna mentioned, I mean, we have ways to kind of like segment

Tom Hollingsworth:

networks away from each other.

Tom Hollingsworth:

And it's funny that you bring up that, that Intuit had kind of a, a rigorous

Tom Hollingsworth:

internal firewall structure because in my experience, um, companies or organizations

Tom Hollingsworth:

that are very, uh, heavily regulat.

Tom Hollingsworth:

Have much more strict internal structure.

Tom Hollingsworth:

And the reason for that is because they need the ability to say

Tom Hollingsworth:

for a fact, Curtis cannot see anything on this network because he

Tom Hollingsworth:

hasn't been authorized to see it.

Tom Hollingsworth:

Now, you can do that through software constructs.

Tom Hollingsworth:

I mean, VLANs, virtual local area networks are kind of the, the most common

Tom Hollingsworth:

way to do it, where we, we effectively divide some, uh, uh, partition on the

Tom Hollingsworth:

switch and we say, this port belongs to this vlan, so it can only talk to

Tom Hollingsworth:

other ports that are on that vlan.

Tom Hollingsworth:

Uh, but that's not even good enough for some organizations.

Tom Hollingsworth:

And, and the, the one that everybody always thinks of is Mission Impossible,

Tom Hollingsworth:

the Tom Cruise movie with the, the machine that's in a vault that's

Tom Hollingsworth:

not connected to anything else.

Tom Hollingsworth:

We would call that an air gap system.

Tom Hollingsworth:

Or you can have an air gap network a lot of times things like, um, HVAC or

Tom Hollingsworth:

management systems are air gap from the rest of the network because they

Tom Hollingsworth:

have different controls and different needs, but I also don't trust those

Tom Hollingsworth:

people to, um, secure their stuff.

Tom Hollingsworth:

So I'm gonna build a wall in front of that air gap or just completely

Tom Hollingsworth:

isolate it, uh, itself so that I don't have to worry about securing it.

Tom Hollingsworth:

And if, uh, you, you say hvac, you say things like, you know, uh, um,

Tom Hollingsworth:

environmental control systems and any security people listening to this

Tom Hollingsworth:

podcast are immediately thinking, man, those are back doors that I

Tom Hollingsworth:

can use to get into the system.

Tom Hollingsworth:

Because no matter what, they're still gonna have to be connected

Tom Hollingsworth:

to the network somehow.

Tom Hollingsworth:

And that just increases your, um, you know, your threat profile.

W. Curtis Preston:

Right.

Prasanna Malaiyandi:

Yeah, it's interesting because I think most people

Prasanna Malaiyandi:

who think about home networks, right?

Prasanna Malaiyandi:

Everything's typically flat in a home, right?

Prasanna Malaiyandi:

Everything can talk to everything, every single iot device out there, right?

Prasanna Malaiyandi:

And they're not always thinking about, Hey, I got this smart light bulb.

Prasanna Malaiyandi:

Isn't it great?

Prasanna Malaiyandi:

Isn't it awesome?

Prasanna Malaiyandi:

And then realizing that's on my network, everything is now exposed and could

Prasanna Malaiyandi:

be potentially exposed if there's a security issue with that single device,

Tom Hollingsworth:

Those devices are, you know, they obviously have an IP address,

Tom Hollingsworth:

they have some kind of a control system.

Tom Hollingsworth:

You would hope that most of them have some kind of a security function that

Tom Hollingsworth:

allows them to, to securely communicate back to whatever controls them.

Tom Hollingsworth:

But multiply that by a factor of 10 for all of the devices that could be

Tom Hollingsworth:

on your average enterprise network.

Tom Hollingsworth:

And when you start saying things like, you know, access controls for those devices,

Tom Hollingsworth:

or port security like network engineering and, and operations folks, like, they

Tom Hollingsworth:

just start breaking out into hives.

Tom Hollingsworth:

because like the, the, just the amount of work that it takes to create that

Tom Hollingsworth:

level of security is its own monster.

Tom Hollingsworth:

I mean, anyone who's ever deployed a technology like 8 0 2 0.1 x, which

Tom Hollingsworth:

is effectively, I am only gonna allow authorized devices to be plugged into

Tom Hollingsworth:

this port, knows that like there's this whole enrollment process and

Tom Hollingsworth:

are you on the authorized users list?

Tom Hollingsworth:

And what happens if you're using a different device today?

Tom Hollingsworth:

And it's just, it's maddening and it, it drives people to insane to the

Tom Hollingsworth:

point where, and that's the normal people who know what they're doing.

Tom Hollingsworth:

Could you imagine an executive plugging their laptop into a network port one

Tom Hollingsworth:

day and going, this doesn't work.

Tom Hollingsworth:

And you tell 'em, oh, it's doing that on purpose because we

Tom Hollingsworth:

want to keep everything secure.

Tom Hollingsworth:

What do you think is gonna happen?

Tom Hollingsworth:

The executive's probably gonna look at you and go, I don't care.

Tom Hollingsworth:

Make

Prasanna Malaiyandi:

Turn it off.

Prasanna Malaiyandi:

Exactly.

Prasanna Malaiyandi:

. We don't need that.

Tom Hollingsworth:

It's getting

Tom Hollingsworth:

in my way.

Prasanna Malaiyandi:

Yeah,

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Well, I know that when we, when we had, um, you know, we had a, a, a security

W. Curtis Preston:

person on and they had a list of things that they wanted people to do that they

W. Curtis Preston:

felt were common sense, that were, um, ways to prevent basically, sort of,

W. Curtis Preston:

I, I think the proper thing today when we talk about ransomware is to just

W. Curtis Preston:

assume something in your, in your world is going to get ransomware, right?

W. Curtis Preston:

It's just, it is, I think it's just impossible to, to,

W. Curtis Preston:

to stop it 100% of the time.

W. Curtis Preston:

So just assume that's going to happen.

W. Curtis Preston:

So then there's all about.

W. Curtis Preston:

How to prevent it from activating itself, from talking to the command and control

W. Curtis Preston:

servers and also the lateral movement.

W. Curtis Preston:

Right?

W. Curtis Preston:

So he

Prasanna Malaiyandi:

reducing the black

W. Curtis Preston:

Speaker:

lateral movement, right?

W. Curtis Preston:

Speaker:

So what's that?

Prasanna Malaiyandi:

Limiting the blast

W. Curtis Preston:

radius.

W. Curtis Preston:

So, so Tom, what, what kinds of things besides VLANs?

W. Curtis Preston:

Because even VLANs, you know, we have the, we have the VLAN

W. Curtis Preston:

for this and the VLAN for that.

W. Curtis Preston:

Still all the servers within that VLAN can talk to each other.

W. Curtis Preston:

What else can companies do, uh, with modern networking equipment to prevent

W. Curtis Preston:

lateral movement or to basically prevent it from everything and then, and then, uh,

W. Curtis Preston:

selectively allow it for certain servers.

Tom Hollingsworth:

Well, the first thing you have to do is you have to

Tom Hollingsworth:

realize that a completely flat network.

Tom Hollingsworth:

Is not a stable network.

Tom Hollingsworth:

I mean, there is a limit to the amount of chatter that a network can tolerate

Tom Hollingsworth:

before it starts running into problems.

Tom Hollingsworth:

Um, ethernet is not a, uh, a medium that allows for a large number of hosts because

Tom Hollingsworth:

eventually they're gonna, it, you know, it's like recording a podcast eventually

Tom Hollingsworth:

with too many guests on the podcast.

Tom Hollingsworth:

You're all gonna wanna talk over the top of each other,

Tom Hollingsworth:

and ethernet doesn't like that.

Tom Hollingsworth:

So once you had a certain boundary, you kind of have to divide it

Tom Hollingsworth:

up into these little domains.

Tom Hollingsworth:

Um, collision domains are what we call them, and that's one

Tom Hollingsworth:

of the things that a VLAN is.

Tom Hollingsworth:

But as we've learned over the years about what we really should be doing,

Tom Hollingsworth:

we've kind of built a super set of that.

Tom Hollingsworth:

And anyone out there who has been reading any kind of the tech press recently, or

Tom Hollingsworth:

been to any trade show in the last couple of years, probably heard of something

Tom Hollingsworth:

like Zero Trust Network Architecture or, or, you know, just Zero Trust in general.

Tom Hollingsworth:

It's a buzzword.

Tom Hollingsworth:

I'm, I'll be the first to admit it, but the principles behind it are fairly sound.

Tom Hollingsworth:

what you do is you take the tools that you've already been given, those ones

Tom Hollingsworth:

that I told you, make your network team break out in hives, and you try to

Tom Hollingsworth:

implement them in such a way as to reduce the complexity of the implementation.

Tom Hollingsworth:

And think about like, you know, think about a teenager and they

Tom Hollingsworth:

want a, a list of, uh, things that they can do when they get a car.

Tom Hollingsworth:

Are you gonna tell them you can do anything you want, but

Tom Hollingsworth:

you can't do this and you can't do that and you can't do this?

Tom Hollingsworth:

Or are you gonna be more explicit?

Tom Hollingsworth:

You can only do these things and if it's not on that list, you can't do it.

Tom Hollingsworth:

Well, most people would say, well, I'm only go, I'm gonna do the second

Tom Hollingsworth:

thing because I want to make sure that they're only going to school and to

Tom Hollingsworth:

work into this one friend's house.

Tom Hollingsworth:

But we don't build networks that way.

Tom Hollingsworth:

I mean, we, we typically allow as much as possible because of the

Tom Hollingsworth:

situations we find ourselves in where something doesn't work right.

Tom Hollingsworth:

And we don't know why.

Tom Hollingsworth:

So we will put a little catchall at the bottom of the, the access

Tom Hollingsworth:

list and go permit everything else.

Tom Hollingsworth:

and then we leave it.

Tom Hollingsworth:

And that's the worst thing that you can do.

Tom Hollingsworth:

And what Zero Trust Network architectures try to do is they try

Tom Hollingsworth:

to say, okay, that server over there is running our backup software.

Tom Hollingsworth:

What should it, what should communicate with it?

Tom Hollingsworth:

And how should it be communicated with, you know, maybe it only needs to accept

Tom Hollingsworth:

connections on these three or four ports.

Tom Hollingsworth:

Maybe it only accepts connections from these authorized users.

Tom Hollingsworth:

And you're effectively creating an isolation for that unit.

Tom Hollingsworth:

And if something needs to access it and you're having problems with it, the

Tom Hollingsworth:

software usually allows you to kind of dig into that a little bit and go, oh, it

Tom Hollingsworth:

looks like that this program did an update and it now needs to communicate over this

Tom Hollingsworth:

port, uh, and I need to allow that port.

Tom Hollingsworth:

But you're doing it in a, in a way that allows you to kind of control that access.

Tom Hollingsworth:

But more importantly, what happens is that when something tries to operate

Tom Hollingsworth:

outside of that access control, it slams it shut and hopefully will send

Tom Hollingsworth:

you some kind of a warning, you know, Hey, we just noticed that this server

Tom Hollingsworth:

over here is trying to communicate with the rest of the network on Port 4 45.

Tom Hollingsworth:

and I know it shouldn't be doing that.

Tom Hollingsworth:

You need to take a look at it.

Tom Hollingsworth:

And so limiting that blast radius, that broadcast capability tends

Tom Hollingsworth:

to prevent lateral movement.

Tom Hollingsworth:

And like you said, people who are going to attack you are, are

Tom Hollingsworth:

going to be dedicated in doing it.

Tom Hollingsworth:

Either they're gonna be dedicated to looking for a very specific exploit and

Tom Hollingsworth:

just kind of hauling in whatever they can do, or they're gonna be looking to

Tom Hollingsworth:

attack you, you specifically, however they can get to you that second kind

Tom Hollingsworth:

of attacker, very difficult to block.

Tom Hollingsworth:

It's like a door lock, a dedicated burglar is gonna get into your house.

Tom Hollingsworth:

You're looking to prevent more of the first one where it's like, oh, we were

Tom Hollingsworth:

able to get in through your HVAC system and boy, we're gonna turn this thing loose

Tom Hollingsworth:

and see what open file shares you've got out there and what we can do with them.

Tom Hollingsworth:

You, you need to create structure in the organization that does not allow

Tom Hollingsworth:

people to move laterally that that prevents them from accessing things.

Tom Hollingsworth:

Or worse yet, alerts you when things start doing a lot of scanning across

Tom Hollingsworth:

your network, looking for those kinds of things because the, the rest of the group

Tom Hollingsworth:

that's trying to get into your network doesn't know that stuff's there either.

Tom Hollingsworth:

They're gonna have to go looking and just like the burglars that are casing the

Tom Hollingsworth:

joint, you need to look for those people.

Prasanna Malaiyandi:

So multiple things popped up in my head,

Prasanna Malaiyandi:

Tom, as you were talking.

Prasanna Malaiyandi:

So the first is, as you're talking about the burglar example, I'm gonna bring this

Prasanna Malaiyandi:

up again for the second week, but Curtis had recommended reading The Cuckoo's Egg.

Prasanna Malaiyandi:

I don't know if you've read that book.

Prasanna Malaiyandi:

Tom.

Prasanna Malaiyandi:

Highly recommend you read it.

Prasanna Malaiyandi:

It's basically, 1980s, a hacker gets into a mainframe and starts moving

Prasanna Malaiyandi:

laterally across all these like military networks and science networks

Prasanna Malaiyandi:

because everything was connected.

Prasanna Malaiyandi:

And like you said, that example was go and try all the door locks and he

Prasanna Malaiyandi:

would try default passwords and some of these systems, like the mainframes,

Prasanna Malaiyandi:

people would not change the defaults.

Prasanna Malaiyandi:

And so he got in and it was just that lateral movement across

Prasanna Malaiyandi:

everything in the environment.

Prasanna Malaiyandi:

So that's like the first thing that came to mind as you were talking.

Prasanna Malaiyandi:

Um, the other thing that also came to mind is I totally get the reason to have

Prasanna Malaiyandi:

like that zero trust and only enables services that, and patterns that are known

Prasanna Malaiyandi:

to be valid and disable everything else.

Prasanna Malaiyandi:

Uh, my question.

Prasanna Malaiyandi:

As a network engineer or operations person, how do

Prasanna Malaiyandi:

you manage that at the scale?

Prasanna Malaiyandi:

Because there's so many applications, so many servers, it's hard to predict

Prasanna Malaiyandi:

what's going to talk with what, um, and coming up with, because

Prasanna Malaiyandi:

like, everything's all connected.

Prasanna Malaiyandi:

Like in my mind I think about like Facebook and graphs, right?

Prasanna Malaiyandi:

Everything is connected in the world, right?

Prasanna Malaiyandi:

And so everything in your network to some extent is probably

Prasanna Malaiyandi:

connected in some form or fashion.

Prasanna Malaiyandi:

So how do you sort of go about even coming up with, okay, these things

Prasanna Malaiyandi:

are the things that should be talking to the backup server in your example.

Tom Hollingsworth:

So it takes a lot of teamwork because as a network person,

Tom Hollingsworth:

I don't care what's running over my network, I just need to make sure that

Tom Hollingsworth:

these two things can talk to each other.

Tom Hollingsworth:

And so in a way, like if you've ever deployed a server, um, you, you have a

Tom Hollingsworth:

list, okay, it needs to communicate, uh, using this protocol over these ports or,

Tom Hollingsworth:

you know, uh, think about, uh, opening something like, I need to open HTTPS to

Tom Hollingsworth:

the server, but not http because I don't want it to ever communicate over http.

Tom Hollingsworth:

And that's actually one of the things that we've noticed a lot recently is that a

Tom Hollingsworth:

lot of protocols that used to have their own dedicated ports have now just started

Tom Hollingsworth:

writing over, uh, HTTP and https s.

Tom Hollingsworth:

Because it's just easier.

Tom Hollingsworth:

Uh, bit Torrent was actually one of the first ones to start doing this because

Tom Hollingsworth:

they're like, well, eighty's gonna be open anyway, which is the port for http.

Tom Hollingsworth:

So we'll just ride on that because most people fire, most people's firewalling

Tom Hollingsworth:

systems just allow that by default, because that's what the web uses.

Tom Hollingsworth:

And so it gets kind of insidious and you almost have to think at a higher level.

Tom Hollingsworth:

So what.

Tom Hollingsworth:

it crack open any networking textbook in the world, and they're gonna

Tom Hollingsworth:

give you this seven layer model.

Tom Hollingsworth:

It's like a seven layer dip from Taco Bell, but there's no refried

Tom Hollingsworth:

beans in the seven layer OSI model.

Tom Hollingsworth:

But we play a lot in the bottom of that, where the physical connections

Tom Hollingsworth:

happen, where the IP addresses allow systems to talk to each other.

Tom Hollingsworth:

Once we get above a certain level, that's where the applications take over.

Tom Hollingsworth:

And as networking people, we're not as concerned about that.

Tom Hollingsworth:

But boy, the server people are because, oh, you know, I need to be able to have

Tom Hollingsworth:

these two devices talking to each other.

Tom Hollingsworth:

I need to make sure this is all un impeded.

Tom Hollingsworth:

And the first thing that happens when two servers can't talk to each other is you

Tom Hollingsworth:

gotta find the network people, people.

Tom Hollingsworth:

And you're like, you need to tell me what's going on here.

Tom Hollingsworth:

And then invariably, like the security team gets drawn in because like, oh no,

Tom Hollingsworth:

we told him that he had to block that because nobody should ever be using that.

Tom Hollingsworth:

And, and you, you really do have to pull those people together.

Tom Hollingsworth:

I mean, think of, you know, think of a book like, uh, gene Kim's Phoenix project.

Tom Hollingsworth:

Like you can't work in isolation anymore.

Tom Hollingsworth:

As much as we might like to.

Tom Hollingsworth:

Because so many things are so inter interdependent now.

Tom Hollingsworth:

It's like, you know, the, the old joke is, is what does the server do?

Tom Hollingsworth:

I don't know.

Tom Hollingsworth:

Unplug the cable and we'll see who screams the loudest.

Tom Hollingsworth:

You wanna figure out what people, uh, what port is being used.

Tom Hollingsworth:

Let's block it and see who comes to yell at us.

Tom Hollingsworth:

Like, that's kind of the way you have to do some of these things.

Tom Hollingsworth:

Cuz

Tom Hollingsworth:

the other thing, and we, we all know that nobody, nobody ever

Tom Hollingsworth:

skips documentation, right?

Tom Hollingsworth:

I realized while editing this episode, that we forgot to

Tom Hollingsworth:

throw out our disclaimer.

Tom Hollingsworth:

Uh, Prasanna and I work for different companies.

Tom Hollingsworth:

He works for zoom.

Tom Hollingsworth:

I work for Druva.

Tom Hollingsworth:

This is not a podcast of either company.

Tom Hollingsworth:

It is an independent podcast.

Tom Hollingsworth:

So the opinions that you hear are ours.

Tom Hollingsworth:

Also, if you'd like to join the podcast, please reach out to me

Tom Hollingsworth:

at w Curtis Preston on, uh, at Gmail or at WC Preston on Twitter.

Tom Hollingsworth:

Or linkedin.com/i N slash Mr.

Tom Hollingsworth:

Backup.

Tom Hollingsworth:

And you'll find me, uh, we'd love to have you join and also be sure to

Tom Hollingsworth:

rate us at your favorite podcatcher.

Tom Hollingsworth:

Thanks a lot.

Tom Hollingsworth:

Now onto my silly story.

W. Curtis Preston:

You brought up an old memory of mine.

W. Curtis Preston:

Literally like my first months in being a cis admin, we were trying to decommission,

W. Curtis Preston:

um, uh, the, you know, the, the, the first computer designed to run Unix was the

W. Curtis Preston:

three BK and the at and t had a three BK I think it was like a three B 1000.

W. Curtis Preston:

And it was their attempt at a multiprocessor architecture.

W. Curtis Preston:

And we had this beast and we were trying to decommission it.

W. Curtis Preston:

And, uh, we had gotten down to, we had fi you know, and, and we had gotten down

W. Curtis Preston:

to that phase where it's like, well, we're just gonna turn it off and whoever

W. Curtis Preston:

yells will be the one that we missed.

W. Curtis Preston:

Right.

W. Curtis Preston:

But I remember the, um, We had, uh, stripped it, all of, all of

W. Curtis Preston:

its regular networking cable.

W. Curtis Preston:

I don't exactly remember exactly why, but I remember that there was one cable

W. Curtis Preston:

left and it was running across the floor and we were doing the last like

W. Curtis Preston:

download of, of whatever it was off of this server onto something else.

W. Curtis Preston:

And the manager for that cost center was in there and he

W. Curtis Preston:

kept stepping on the cable.

W. Curtis Preston:

And, um, we told him that he was slowing down the download whenever

W. Curtis Preston:

he would step on the cable.

W. Curtis Preston:

And, um, we actually caught him, we left him into data center.

W. Curtis Preston:

We actually caught him like watching the monitor and like the throughput speed

W. Curtis Preston:

and sort of stepping on and stepping up and off and off on the cable.

W. Curtis Preston:

Anyway.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Good, good stories.

W. Curtis Preston:

I, so the question I want to ask you about, all of the things you just talked

W. Curtis Preston:

about, is this something built into modern networking equipment or is this, um,

W. Curtis Preston:

you know, are these extra applications that I'm buying that then configure

W. Curtis Preston:

that networking equipment for me?

Tom Hollingsworth:

So it can be both.

Tom Hollingsworth:

The, the basics of being able to isolate hosts and configure systems

Tom Hollingsworth:

has been built in for years.

Tom Hollingsworth:

I mean, anyone can write an a c L, right?

Tom Hollingsworth:

The thing is, is that scaling that across a large organization

Tom Hollingsworth:

is where it typically falls down.

Tom Hollingsworth:

Eventually, your security team can't keep up with all the changes.

Tom Hollingsworth:

They throw their hands up in the air and it lies fallow for as long as

Tom Hollingsworth:

it takes for you to get infected.

Tom Hollingsworth:

So the additional tools that are basically being brought to market and are, are

Tom Hollingsworth:

popular now, kind of organize that system.

Tom Hollingsworth:

They put a, a, a shiny.

Tom Hollingsworth:

UI on it, if you will, to, to go in and say, okay, I, I want to enable port

Tom Hollingsworth:

security on these ports because back when I started this port security was,

Tom Hollingsworth:

if it isn't being used, shut it off.

Tom Hollingsworth:

Just like shut down the port.

Tom Hollingsworth:

And then if somebody plugs into it and it doesn't work, well then now we know

Tom Hollingsworth:

we need to enable that port and we need to know who's trying to use it.

Tom Hollingsworth:

But now you have the ability to like have somebody plug in a device,

Tom Hollingsworth:

whether it's an IOT system or what have

Tom Hollingsworth:

you, and this, the device will like register with the system.

Tom Hollingsworth:

It'll say, Hey, I need access.

Tom Hollingsworth:

And then the system can come back and say, Hey, it looks like somebody

Tom Hollingsworth:

plugged in an S thermostat over here.

Tom Hollingsworth:

Well, that's actually a bad example cause they don't use wires, but you know, a

Tom Hollingsworth:

laptop or some other kind of device, you need to go, like check it out.

Tom Hollingsworth:

Or you can even set a policy that says, I'm going to allow you for

Tom Hollingsworth:

now, but I have the ability to just cut it off if I need to.

Tom Hollingsworth:

Or if it's one of these recognized device classes or something like that.

Tom Hollingsworth:

So for smaller systems, , you know, for, for smaller organizations, if

Tom Hollingsworth:

your IT department isn't already completely overworked, you can't

Tom Hollingsworth:

implement some of this by hand.

Tom Hollingsworth:

It's just a matter of if it works really well, that means you're gonna

Tom Hollingsworth:

be spending a lot of time tuning that system to keep working effectively.

Tom Hollingsworth:

And once you get past a certain point, the, uh, the solutions

Tom Hollingsworth:

that do this are reassuringly expensive because they're worth it.

W. Curtis Preston:

Right.

W. Curtis Preston:

Oh, I understood cuz that, that they would help you save the, the, the labor.

W. Curtis Preston:

And is there a category of these types of tools that, that a

W. Curtis Preston:

category name that we give to them?

Tom Hollingsworth:

Uh, there's, there's a bunch of different ones.

Tom Hollingsworth:

Uh, access management is typically one that you, you see, um, honestly,

Tom Hollingsworth:

tools like Aruba ClearPass or uh, Cisco ice, uh, ise, uh, integrated

Tom Hollingsworth:

services engine, or integrated security engine, I forget which one it is.

Tom Hollingsworth:

But they're, they're, they're not identity and access management,

Tom Hollingsworth:

although they can be integrated that.

Tom Hollingsworth:

There are some smaller ones that, that have these capabilities.

Tom Hollingsworth:

A lot of it is, is mostly figuring out what you need because there's

Tom Hollingsworth:

different, you know, some systems are, are configured so that you're

Tom Hollingsworth:

controlling access to devices.

Tom Hollingsworth:

I only wanna authorize people to be able to log into this

Tom Hollingsworth:

device and make changes to it.

Tom Hollingsworth:

Well, that's different than I want to change the way that people

Tom Hollingsworth:

in my network are accessing data like that is a different kind of

Tom Hollingsworth:

identity and access management.

Tom Hollingsworth:

So you need to do a little bit of investigative work to make sure that you

Tom Hollingsworth:

are, uh, properly using the right tool.

Tom Hollingsworth:

Cause if you spend a lot of money on one that doesn't give you what you want or

Tom Hollingsworth:

does a, a, a terrible job of it, then not only are you gonna be upset, but the

Tom Hollingsworth:

people that are or authorizing your budget are not gonna be very happy with you.

Prasanna Malaiyandi:

Now a lot of these changes, if I think about an

Prasanna Malaiyandi:

enterprise environment, things are easier to a fair extent to control, right?

Prasanna Malaiyandi:

If you're looking at servers or virtualization, other things like that.

Prasanna Malaiyandi:

But then I go to think about other environments like a school, right, where

Prasanna Malaiyandi:

you have students coming and going, right?

Prasanna Malaiyandi:

Or a stadium or a conference center, right?

Prasanna Malaiyandi:

Does it get significantly more difficult to do what you talked

Prasanna Malaiyandi:

about, Tom, in those environments?

Prasanna Malaiyandi:

Or can the same tools apply there as well?

Tom Hollingsworth:

Yes and no.

Tom Hollingsworth:

Um, I, I, I'm, I'm the typical IT nerd.

Tom Hollingsworth:

The answer is, it depends for whatever question you ask, but I'll tell you

Tom Hollingsworth:

that in some ways, um, schools and other places where your user base

Tom Hollingsworth:

is not employed directly by you.

Tom Hollingsworth:

Can have a slightly easier time if you're willing to, um, sacrifice a little bit.

Tom Hollingsworth:

So I know that there are a lot of colleges out there that treat their student

Tom Hollingsworth:

dorm networks like the wild, wild west.

Tom Hollingsworth:

We don't care what goes on out there, but we're not gonna keep an eye on it either.

Tom Hollingsworth:

So like, if

Tom Hollingsworth:

there's a, you know, a piece of ransomware or something that's running rampant

Tom Hollingsworth:

through the system, all we did is tell you that you had to have your antivirus

Tom Hollingsworth:

up to date to be able to join our network.

Tom Hollingsworth:

So,

Tom Hollingsworth:

so what, uh, the stadiums are actually a, a really interesting, uh, problem

Tom Hollingsworth:

too, because not only do you have a a, a group of users that are outside of

Tom Hollingsworth:

your control, they're very transient, um, in a lot of those places.

Tom Hollingsworth:

Like they, they actually have, uh, wireless networks that are set up

Tom Hollingsworth:

so that, um, they can only talk.

Tom Hollingsworth:

, like they block all device to device communication, which

Tom Hollingsworth:

is something that you can do.

Tom Hollingsworth:

It's a little bit more complicated, but it effectively treats, um, the stadium

Tom Hollingsworth:

itself like a demilitarized zone in a, uh, in a, in a security structure.

Tom Hollingsworth:

So for most people that are, that are familiar with it, you know, you've got the

Tom Hollingsworth:

outside internet, which is big and scary.

Tom Hollingsworth:

You've got your inside network, which is soft and, and you know,

Tom Hollingsworth:

uh, you don't want it to get hurt.

Tom Hollingsworth:

And then in the middle you have the dmz, which is basically the moat where

Tom Hollingsworth:

you're like, I'm gonna put everything that I don't care if it gets attacked

Tom Hollingsworth:

out there so that if it breaks, it can't get back into my network.

Tom Hollingsworth:

And so, but the otherwise, the other thing there is I only allow

Tom Hollingsworth:

certain traffic to come back through.

Tom Hollingsworth:

So if something bad were to happen, I can just basically cut it off and

Tom Hollingsworth:

sink it into the moat and I'm done.

W. Curtis Preston:

Yeah, I think, uh, hotels have a similar model, right?

W. Curtis Preston:

Where the base, uh, I know having, having plugged in multiple devices that

W. Curtis Preston:

needed to talk to each other in hotel networks, they don't like that very much.

W. Curtis Preston:

Uh, and you end up having to bring basically your own router if, if

W. Curtis Preston:

that's something that you want to do.

W. Curtis Preston:

Right?

W. Curtis Preston:

Um, so the, so it sounded like, um, if I understood you correctly,

W. Curtis Preston:

the access management part is this sort of basic security thing, that

W. Curtis Preston:

there are tools that do just that, and then there's also this identity

W. Curtis Preston:

access, which is a, a bigger pain.

W. Curtis Preston:

I would, I would imagine.

W. Curtis Preston:

But those that want that, and it sounds like when we put those

W. Curtis Preston:

two together, that's what, what we call a SEIM tool, right?

W. Curtis Preston:

Is uh, uh, identity and access management.

W. Curtis Preston:

But it sounds like there's just an access management.

W. Curtis Preston:

That, for those that need just that there, there's smaller and less

W. Curtis Preston:

expensive than a full SEIM tool.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Not, not inexpensive, but just less expensive.

Tom Hollingsworth:

Well, and it, it also matters as to what you're spending

Tom Hollingsworth:

your resources on, because there are tools that will do this for free.

Tom Hollingsworth:

But they are not supported at all by anybody other than people on a forum.

Tom Hollingsworth:

And they'll be glad to tell you that you misconfigured something

Tom Hollingsworth:

and go figure it out yourself.

Tom Hollingsworth:

Like we, we've dealt with that.

Tom Hollingsworth:

And I'm not really crapping on the open source community because

Tom Hollingsworth:

they do an amazing job of this.

Tom Hollingsworth:

I'm crapping on the fact that open source communities are not as well supported

Tom Hollingsworth:

as the bigger players in these markets.

Tom Hollingsworth:

And that's honestly where the expensive part comes from.

Tom Hollingsworth:

You're not paying for the software, although you, you

Tom Hollingsworth:

kind of are in some ways.

Tom Hollingsworth:

You're paying for somebody to answer the phone when somebody is like breathing

Tom Hollingsworth:

down your neck because something won't work or something won't come online.

Tom Hollingsworth:

And so a and a and you're also trying to get to that point where

Tom Hollingsworth:

it's, it's not automated as much as it is as low friction as possible.

Tom Hollingsworth:

Because what you want in situations is people to just

Tom Hollingsworth:

be able to get on the network.

Tom Hollingsworth:

That's, that's the thing.

Tom Hollingsworth:

If you've ever tried to log into a wifi network that has a captive portal

Tom Hollingsworth:

that requires you to like accept a whole bunch of licensing agreements

Tom Hollingsworth:

and type your room number in and all the other stuff, you know that it's not

Tom Hollingsworth:

the most frustrating thing, but it's definitely not what you want to hear.

Tom Hollingsworth:

As opposed to like, oh, this device has already been pre-authorized cuz you logged

Tom Hollingsworth:

in with your active directory username.

Tom Hollingsworth:

Well, we'll just let it on the network.

Tom Hollingsworth:

That's completely frictionless.

Tom Hollingsworth:

But the amount of effort that it takes to make it frictionless is where your time

Tom Hollingsworth:

and resource invests gonna come from.

Prasanna Malaiyandi:

Tom, I know we started this all out with Curtis asking,

Prasanna Malaiyandi:

how do you prevent lateral movement in networks right from ransomware?

Prasanna Malaiyandi:

Just given the fact that ransomware does move laterally in a lot of networks?

Prasanna Malaiyandi:

Does this mean people are not using these tools or have not

Prasanna Malaiyandi:

configured the networks correctly?

Prasanna Malaiyandi:

Because it seems like if you did all the things that we just talked about, it

Prasanna Malaiyandi:

should have prevented a lot of the lateral movement that we see in ransomware today.

Tom Hollingsworth:

Well, Prasanna, I'm gonna tell you something

Tom Hollingsworth:

that my dad always tell me, and you have to understand.

Tom Hollingsworth:

My dad grew up in the country.

Tom Hollingsworth:

If a frog had wings, he wouldn't bump his ass every time he hopped.

Tom Hollingsworth:

So yes, if you turn on all of these tools, you will cut down on a lot of this stuff.

Tom Hollingsworth:

But does that mean your network's not working correctly?

Tom Hollingsworth:

No.

Tom Hollingsworth:

It just means that we didn't enable all these extra features that we have

Tom Hollingsworth:

to keep track of because I can get four, uh, four network ports on a.

Tom Hollingsworth:

and plug four devices in there and they're gonna work is the

Tom Hollingsworth:

best way for them to work.

Tom Hollingsworth:

Absolutely not, but I also don't have to do a whole lot of extra configuration.

Tom Hollingsworth:

A lot of people are looking at this from the perspective of, I need to

Tom Hollingsworth:

make sure that everything is, is able to communicate with everything else.

Tom Hollingsworth:

They're not looking at it like you, like the example you had earlier, Curtis, when

Tom Hollingsworth:

you log into the hotel wifi and I can't talk to anything else on the hotel wifi.

Tom Hollingsworth:

They're not thinking in a, in an isolation mode.

Tom Hollingsworth:

And we're, we're that ship's turning because a lot of people are now

Tom Hollingsworth:

realizing that, that that traditional idea of having a very stiff, crunchy

Tom Hollingsworth:

perimeter with a very soft internal network doesn't work so well.

Tom Hollingsworth:

Because what ends up happening is, is that once people get through

Tom Hollingsworth:

the perimeter, they have free reign to do whatever they want.

Tom Hollingsworth:

You, you do have to build these controls in place to effectively

Tom Hollingsworth:

slow them down or to herd them to places that you want them to go.

Tom Hollingsworth:

And that's what a lot of people have spent time developing and working on.

Tom Hollingsworth:

And there's varying degrees of success to make that work.

Tom Hollingsworth:

It has to shift the mindset though.

Tom Hollingsworth:

Um, you know, application people are just turn on all the ports

Tom Hollingsworth:

and I'll turn them off later.

Tom Hollingsworth:

When I tell you which ones I don't need, you won't, because you'll

Tom Hollingsworth:

get busy doing something else.

Tom Hollingsworth:

It's like developers, they're like, I'm gonna load everything I can possibly

Tom Hollingsworth:

think of in the memory so that I know the library that I need is there.

Tom Hollingsworth:

And then you wonder why your, your, uh, application is consuming

Tom Hollingsworth:

like three terabytes of ram.

Tom Hollingsworth:

It's like, uh, maybe you need to pa pair back a little bit on that.

W. Curtis Preston:

Yeah.

W. Curtis Preston:

So it, it sounds like these tools are there.

W. Curtis Preston:

Uh, I think a lot of people do use them, but you talked about, like in the very

W. Curtis Preston:

beginning, you, you said that people's heads are gonna start spinning or

W. Curtis Preston:

whatever, because there is a lot of work involved in implementing these things.

W. Curtis Preston:

And the moment you flip that switch from, you know, per, you know, from

W. Curtis Preston:

everything is permitted to only the things that are permitted or permitted, uh,

W. Curtis Preston:

you're gonna get 5,000 tickets, right?

W. Curtis Preston:

I can't do this and I can't do that.

W. Curtis Preston:

And they, they see that.

W. Curtis Preston:

They see that very real worry.

W. Curtis Preston:

Uh, and I, and I think it stops many people from implementing this

W. Curtis Preston:

because they just see it as the amount of work they're gonna have

W. Curtis Preston:

to do to initially implement it.

W. Curtis Preston:

Um, they're, and they're not seeing the risk of what's gonna happen when

W. Curtis Preston:

they get a ransomware infection, and then it just goes crazy.

Tom Hollingsworth:

Most tools that are, are set up like this.

Tom Hollingsworth:

Uh, they have a learning mode where they will, you could put 'em in place and

Tom Hollingsworth:

they just sit there and they watch for at

Tom Hollingsworth:

least the first, you know, week or two.

Tom Hollingsworth:

And they're mapping out all of these application dependencies.

Tom Hollingsworth:

So, you know, the backup system needs to receive traffic on this port for

Tom Hollingsworth:

this application from this subnet.

Tom Hollingsworth:

And then it allows you to carefully craft that rule so that only devices

Tom Hollingsworth:

from this subnet can talk to that server on these ports and nothing else.

Tom Hollingsworth:

And if you let the tool go long enough, you'll be able to like, suss

Tom Hollingsworth:

out exactly what you need to know.

Tom Hollingsworth:

But yeah, that first day you click the switch to from, you know, allow list to

Tom Hollingsworth:

deny a list is just like you're, you're staring at the ticket queue because

Tom Hollingsworth:

you're like, oh, what happens if I, if this machine hadn't been turned on for a

Tom Hollingsworth:

week or what, you know?

W. Curtis Preston:

yeah.

Tom Hollingsworth:

Yeah.

Tom Hollingsworth:

It just, it, it, it is, it's maddening because you're always gonna wonder if you

Tom Hollingsworth:

didn't get the right stuff, but like you said, would you rather be worried about

Tom Hollingsworth:

one machine that can't talk to another?

Tom Hollingsworth:

Or would you be worrying about the fact that you're getting a phone call from

Tom Hollingsworth:

the CIO saying, uh, yeah, the database has just got encrypted by this new flavor

Tom Hollingsworth:

of malware that we haven't seen yet.

Tom Hollingsworth:

Uh, why did that?

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Another thing I want to ask you, I wanna sort of move forward into

W. Curtis Preston:

the, the ransomware part here.

W. Curtis Preston:

Although Prasanna, I'm so glad you basically told us to go backwards.

W. Curtis Preston:

You always, you're really good at that, you know, you're really good at

W. Curtis Preston:

making me go backwards.

W. Curtis Preston:

Uh, anyway, uh, I wanted, so one of the things, so we talked about

W. Curtis Preston:

trying to limit lateral movement.

W. Curtis Preston:

Another thing that was suggested was to not permit, uh, new, new either

W. Curtis Preston:

new domains, like domains that just recently were created, or domains

W. Curtis Preston:

that w got recently active, right.

W. Curtis Preston:

From a DNS perspective, is that, is that still fall under the networking purview?

W. Curtis Preston:

Um, or is that like, is that another world?

Tom Hollingsworth:

It, it tend, anything that involves names and not

Tom Hollingsworth:

numbers tends to float up towards the application team or the security

Tom Hollingsworth:

team.

Tom Hollingsworth:

Uh, and the reason for that is because, like you said, like one of the things

Tom Hollingsworth:

that, that we see a lot in security now is it's this idea that you wanna black hole

Tom Hollingsworth:

things that are, that are relatively new.

Tom Hollingsworth:

Like why is this machine suddenly starting to communicate over a d n

Tom Hollingsworth:

s name that I've never seen before?

Tom Hollingsworth:

But it also

Tom Hollingsworth:

requires that your devices have the intelligence to be able to resolve that

Tom Hollingsworth:

because, you know, application layer firewalls will see, oh, you are trying to

Tom Hollingsworth:

access this service that I don't recognize on a domain that I've never seen before.

Tom Hollingsworth:

Whereas a, a lower level, almost like a packet filtering firewall will say,

Tom Hollingsworth:

oh, well that's an IP address connection on this port from here to there.

Tom Hollingsworth:

Uh, I don't see a reason why I shouldn't be using that.

Tom Hollingsworth:

And so, You, You, kind of have to integrate those two things together

Tom Hollingsworth:

because like you said, you know, something doesn't look right here

Tom Hollingsworth:

because why would it be contacting a brand new DNS name that it should, it

Tom Hollingsworth:

has no reason to contact or worse yet?

Tom Hollingsworth:

Uh uh.

Tom Hollingsworth:

You can ask the people over at SolarWinds.

Tom Hollingsworth:

Why is this DLL suddenly talking to .ru addresses?

W. Curtis Preston:

right?

W. Curtis Preston:

Yeah.

W. Curtis Preston:

Well, when he says new domain names, he actually means domain names that were

W. Curtis Preston:

like recently registered, not just domain names that are new to your network.

W. Curtis Preston:

And then also ones that, that were, they were registered but they had, they hadn't

W. Curtis Preston:

been active or something like that.

W. Curtis Preston:

So that sounds like that's a d n s uh, you know, there's a d I world, right?

W. Curtis Preston:

Um, we had, we had somebody on from that.

W. Curtis Preston:

I think we need to have some, because this is, I think that's, , if you can

W. Curtis Preston:

reasonably do that, where you could basically push a button, just sort of like

W. Curtis Preston:

the, the, the deny the allowed deny thing.

W. Curtis Preston:

If you can reasonably say, I, I don't want, I don't want anybody

W. Curtis Preston:

talking to domain names that were registered 24 hours ago.

W. Curtis Preston:

Right.

W. Curtis Preston:

I, I If you could, if you could do something like that, it will of course

W. Curtis Preston:

also create some trouble, uh, tickets.

W. Curtis Preston:

But I'm thinking far less.

W. Curtis Preston:

And if you could do that, it stops to command and control, uh, you

W. Curtis Preston:

know, the, the ransomware from reaching out at command and control,

W. Curtis Preston:

um,

Tom Hollingsworth:

slows the

Tom Hollingsworth:

process down.

Tom Hollingsworth:

But the one thing I will say there though, is that you need to make sure

Tom Hollingsworth:

that your users are expecting that change.

Tom Hollingsworth:

Because if it requires you to go out and check a list or, uh, get some kind of

Tom Hollingsworth:

una authorization to go to this domain name, even if it adds one second to the

Tom Hollingsworth:

resolution time, that's one extra second that people are going to complain about

Tom Hollingsworth:

and you know who they're gonna complain.

Tom Hollingsworth:

the networking team, because the network isn't working.

Tom Hollingsworth:

Not the d n s block list checker or the application that has this built into it.

Tom Hollingsworth:

Oh, no, no.

Tom Hollingsworth:

It's the network's fault because the packets aren't

Tom Hollingsworth:

going where they're supposed to.

W. Curtis Preston:

As we used to say back, back when I was, you know,

W. Curtis Preston:

when I first said that we, we would say the problem's under the floor.

W. Curtis Preston:

Right?

W. Curtis Preston:

Uh, meaning, meaning it was a networking problem.

W. Curtis Preston:

Um, go ahead, Prasanna.

Prasanna Malaiyandi:

So moving on.

Prasanna Malaiyandi:

So we talked about how to prevent lateral movement, how to detect these, uh,

Prasanna Malaiyandi:

rogue, uh, servers that are coming up.

Prasanna Malaiyandi:

One thing I wanted to ask is, so say you do get hit by ransomware, right?

Prasanna Malaiyandi:

They're able to move laterally.

Prasanna Malaiyandi:

What happens next from a networking perspective?

Prasanna Malaiyandi:

Well, I guess two questions.

Prasanna Malaiyandi:

One is how do you, how would you go about bringing down your network or sort

Prasanna Malaiyandi:

of isolating what needs to be isolated?

Prasanna Malaiyandi:

Like how do you actually figure out what's going on in your network?

Prasanna Malaiyandi:

And then the second question is, okay, now that you've sort of

Prasanna Malaiyandi:

identified that, how do you slowly recover from those situations?

Tom Hollingsworth:

Incident response is never fun because

Tom Hollingsworth:

it's a whole lot of cleanup.

Tom Hollingsworth:

And, uh, and, and the first thing you have to do is you have to,

Tom Hollingsworth:

you have to get people out of your network because there's, you know,

Tom Hollingsworth:

there's obviously, there's the tools that kind of run on their own.

Tom Hollingsworth:

And there are tools that kind of have to be piloted by people.

Tom Hollingsworth:

So you have to create, uh, limits on the, on the system to be able to stop that.

Tom Hollingsworth:

And fingers crossed that you're not in a situation where your entire

Tom Hollingsworth:

network has been taken down by whatever is causing the problem.

Tom Hollingsworth:

Because I've seen that before too, where not only does it try to laterally move to

Tom Hollingsworth:

infect systems, it also throws up enough extra garbage that you are, it's Inca,

Tom Hollingsworth:

you're capable of logging into any of your

Tom Hollingsworth:

management networks.

Tom Hollingsworth:

So we're lesson number one.

Tom Hollingsworth:

Make sure all your management networks are kind of isolated so that you

Tom Hollingsworth:

always have the ability to use those.

Tom Hollingsworth:

But the first thing that I would.

Tom Hollingsworth:

As I would cut off outside access immediately, I would

Tom Hollingsworth:

lock the firewall in place.

Tom Hollingsworth:

I mean, you don't have to like run through the data center screaming with

Tom Hollingsworth:

your hair on fire and start yanking cables out like the alias episode.

Tom Hollingsworth:

But you need to be able to lock all of those connections down.

Tom Hollingsworth:

And specifically you need to look for ones that, you know, could be like, you know,

Tom Hollingsworth:

from really weird external addresses, or worse yet ones that are coming in.

Tom Hollingsworth:

Once you've blocked that external access in and out, you gotta do it

Tom Hollingsworth:

in both directions because obviously you don't want anything getting out

Tom Hollingsworth:

because the two things that I can think of are command and control traffic.

Tom Hollingsworth:

If some kind of tool that's being, uh, um, orchestrated or data exfiltration

Tom Hollingsworth:

and, and you're like, oh, well I can stop those file transfers.

Tom Hollingsworth:

Yeah, look up oil rig.

Tom Hollingsworth:

It was, uh, it was able to exfiltrate data through DNS queries.

Tom Hollingsworth:

Like that's the kind of crap you have to worry about.

Tom Hollingsworth:

So you've gotta lock it down.

Tom Hollingsworth:

Then you have to isolate because that's

Tom Hollingsworth:

the other thing too.

Prasanna Malaiyandi:

But, but before you move on,

W. Curtis Preston:

I stop you there?

W. Curtis Preston:

Uh, so how, how do you do that, right?

W. Curtis Preston:

Is this, is this something where you have to create.

W. Curtis Preston:

A button to press up, you know, because this sounds like a lot of little steps

W. Curtis Preston:

you probably need to do to do this manually, or is there something I can

W. Curtis Preston:

do upfront that says, in the event of a ransomware attack, push this button.

W. Curtis Preston:

Hey, gum.

W. Curtis Preston:

Shut up.

W. Curtis Preston:

Anyway, uh, in the event of a ransomware attack, press this button and it

W. Curtis Preston:

does the 10 things I need to do.

W. Curtis Preston:

Uh, what, what do you think

Tom Hollingsworth:

Some of them do have a big red button press here to, to like

Tom Hollingsworth:

terminate all firewall connections.

Tom Hollingsworth:

But most of the time you're gonna have to create like a checklist or, or have

Tom Hollingsworth:

a system of like, okay, I'm gonna go into these rules and I'm gonna uncheck

Tom Hollingsworth:

these five boxes and then I'm gonna hit the terminate connections button to make

Tom Hollingsworth:

sure that no new connections can be made.

Tom Hollingsworth:

Also, if you have a rule at the bottom of your firewall list that

Tom Hollingsworth:

says Permit ip, any, any, take it out

Tom Hollingsworth:

now because it's not doing you any good.

Tom Hollingsworth:

But, but more importantly, you, you have to, you know, uh, all

Tom Hollingsworth:

kill switches have to be wired.

Tom Hollingsworth:

, there's no such thing as a magical switch that you can just hit, even if it's one

Tom Hollingsworth:

that the, that the provider has given you investigate what it actually does.

Tom Hollingsworth:

Does it dump the rules completely?

Tom Hollingsworth:

Does it just like suspend the rules until you go in and manually add them?

Tom Hollingsworth:

Remember that that could also cut off your connection to the firewall, so

Tom Hollingsworth:

you need to have another way to get into it just in case that happens.

Tom Hollingsworth:

Another reason for an isolated management network, but the, the idea is, is that

Tom Hollingsworth:

you, you, you need to investigate what your options are because God help you

Tom Hollingsworth:

if you really do have to run down to the data center and yank the cables

Tom Hollingsworth:

out, and if that is a case and, and hey, it's just as valid as anything else.

Tom Hollingsworth:

Can you make sure that you have the right keys, that you know which

Tom Hollingsworth:

firewall you're yanking out of?

Tom Hollingsworth:

Are there any other exits off of your network?

Tom Hollingsworth:

Because that's another problem that you may run into.

Tom Hollingsworth:

What happens if someone has created another exit off of your network,

Tom Hollingsworth:

either accidentally or on purpose?

Tom Hollingsworth:

And what happens then?

Tom Hollingsworth:

Because you know it's just as easy for me to plug something into your network.

Tom Hollingsworth:

And if there's another way off of it, I'm gonna find it.

Prasanna Malaiyandi:

Yeah.

Prasanna Malaiyandi:

The one other thing though, I know you talked about, and it totally

Prasanna Malaiyandi:

makes sense to kill all incoming and outcoming traffic, but just thinking a

Prasanna Malaiyandi:

step forward, like when you're dealing with incident response, like doesn't

Prasanna Malaiyandi:

that also take out like your chat channels, your slack channels, your

Prasanna Malaiyandi:

video conferencing, everything else, like what do you do at that point?

Prasanna Malaiyandi:

Is it just hope you have everyone's cell phone numbers?

Tom Hollingsworth:

you need to have a plan for out of band incident

Tom Hollingsworth:

response because y it's, it, it's just like any crime scene.

Tom Hollingsworth:

I need to figure out what's, what's been hit and I need to figure out

Tom Hollingsworth:

how much of it is going to spread.

Tom Hollingsworth:

And you're thinking to yourself like, I can't shut my network down

Tom Hollingsworth:

permanently because you know it's gonna cost me X amount of dollars.

Tom Hollingsworth:

Yes, but it's also gonna cost you x plus whatever amount of

Tom Hollingsworth:

dollars when the next system gets

Tom Hollingsworth:

hit, when it uncovers a device that no, nobody's patched it in years.

Tom Hollingsworth:

Um, I'm not gonna lie.

Tom Hollingsworth:

Incident response can work over iMessage text threads for a good couple of

Tom Hollingsworth:

hours while you try to figure that out.

Tom Hollingsworth:

Or, you know, buy your incident response team like those little, you

Tom Hollingsworth:

know, hotspots or enable the data plans on their phone so that they can join

Tom Hollingsworth:

their laptop there and join a Slack instance outside of your network.

Tom Hollingsworth:

because that way nothing is working internal to your network.

Tom Hollingsworth:

Because that's the other thing too.

Tom Hollingsworth:

If you, if this is something that's particularly insidious on a window

Tom Hollingsworth:

system and your incident responders are using Windows systems and they join

Tom Hollingsworth:

the network to be able to do incident response and their laptops get compromised

Tom Hollingsworth:

because they join the network again, you're gonna feel really, really dumb.

Tom Hollingsworth:

It's like, uh, the professional, when they blew up the bomb squad truck, it's like,

Tom Hollingsworth:

come on guys, what were you expecting?

W. Curtis Preston:

You just reminded me of the, there's a, there's a series

W. Curtis Preston:

of commercials and there's one where the commercial is like, it's like a

W. Curtis Preston:

horror movie and the, there's a bunch of kid, it's like the, you know, I got

W. Curtis Preston:

the guy with the, the, the ax murderers looking for the group of kids, and

W. Curtis Preston:

they're like, why don't we go hang out?

W. Curtis Preston:

Why don't we go hide in that shed over there with all the, uh, with all

W. Curtis Preston:

the, uh, machetes or something like

Tom Hollingsworth:

Yeah.

W. Curtis Preston:

Um, so, so we talked about blocking external traffic.

W. Curtis Preston:

What about blocking internal traffic?

W. Curtis Preston:

You know, uh, basically the lateral traffic, uh, be due to the, we

W. Curtis Preston:

know we have ransomware and we know it's gonna try to crawl.

W. Curtis Preston:

What about blocking that, uh, access?

Tom Hollingsworth:

So that's where you hope that your management

Tom Hollingsworth:

networks are, um, isolated because the first thing I would do going

Tom Hollingsworth:

into a router is shut down the route.

Tom Hollingsworth:

Tables prevent, um, traffic from being passed across network boundaries.

Tom Hollingsworth:

Um, what you're effectively doing in there is you are

Tom Hollingsworth:

containing the damage to one area.

Tom Hollingsworth:

Now, yeah, you're gonna take things down, but if you can isolate that network as

Tom Hollingsworth:

the location for wherever the problem is, you can then bring other networks

Tom Hollingsworth:

back online and be relatively certain that they're not gonna be infected.

Tom Hollingsworth:

I really hope that you're not using like, just regular routing, that you

Tom Hollingsworth:

have some kind of a security boundary there, because that makes it a whole lot.

Tom Hollingsworth:

But you, you've got to think in, in phases.

Tom Hollingsworth:

Obviously, you know, using the kill switch is gonna take everything

Tom Hollingsworth:

down, but then you have to start, you know, can I bring this back online?

Tom Hollingsworth:

Is this going to be infected?

Tom Hollingsworth:

What would I be looking for?

Tom Hollingsworth:

Um, so I actually have a, a story about this, uh, this happened

Tom Hollingsworth:

last year to my children.

Tom Hollingsworth:

Uh, one of 'em goes to the public high school here, uh, and I got a rocket

Tom Hollingsworth:

text message from their IT department saying, please turn off all public school

Tom Hollingsworth:

issue devices until further notice.

Tom Hollingsworth:

And I'm like, uhoh, somebody got hit with something fun.

Tom Hollingsworth:

And this was like the last day before Christmas break or something.

Tom Hollingsworth:

So we went in and we turned off my kid's MacBook, right?

Tom Hollingsworth:

So now, immediately I, because I know what the, the thing was, I don't

Tom Hollingsworth:

want anybody to like phone home and get infected and then like infect

Tom Hollingsworth:

the parents networks or whatever.

Tom Hollingsworth:

Okay, no problem.

Tom Hollingsworth:

We just shut it off.

Tom Hollingsworth:

But then I'm like, I wonder what it could.

Tom Hollingsworth:

like I, I'm kind of curious and, and they've, to this day, they've never

Tom Hollingsworth:

disclosed what it was, but you would get an email like the next week,

Tom Hollingsworth:

oh, if you're using like a, a, a, a corporate phone or if you're using

Tom Hollingsworth:

a MacBook, you can turn it back on.

Tom Hollingsworth:

Well, that automatically kind of lowers the horizon of, it has to

Tom Hollingsworth:

be something that's focused on Windows or something like that.

Tom Hollingsworth:

So then you start running through your head of what it could possibly be.

Tom Hollingsworth:

Well, an incident response, you have to do the same thing.

Tom Hollingsworth:

What server got hit?

Tom Hollingsworth:

Oh, well, it was the database server and it was running this version of,

Tom Hollingsworth:

uh, you know, windows or SQL server.

Tom Hollingsworth:

Okay.

Tom Hollingsworth:

Does that mean that Max can get on the network?

Tom Hollingsworth:

Do I want them on the network?

Tom Hollingsworth:

Is it a situation where even though they can't be infected, they could

Tom Hollingsworth:

propagate something to another location?

Tom Hollingsworth:

Like there's a lot that you have to go into because obviously the executives are

Tom Hollingsworth:

gonna be like, when can we do back up and.

Tom Hollingsworth:

and if you're a publicly traded company, oh God, the stockholders are like outdoors

Tom Hollingsworth:

with pitchforks and torches and they wanna know when they can get their dividends.

Tom Hollingsworth:

And you're like, uh, when I figure out how much of this data got encrypted

Tom Hollingsworth:

or stolen, and you're always gonna be fighting that tension and you can't

Tom Hollingsworth:

just shut everything off forever.

Tom Hollingsworth:

So that's part of incident response is you've got one team working on figuring

Tom Hollingsworth:

out how to stop whatever infected you, but you've got another team figuring

Tom Hollingsworth:

out how to bring things back online.

Tom Hollingsworth:

That's why we call it business continuity now.

Tom Hollingsworth:

Right.

Prasanna Malaiyandi:

It is interesting about the incident response.

Prasanna Malaiyandi:

How have you seen cases?

Prasanna Malaiyandi:

Like how do you actually, well, two questions I have.

Prasanna Malaiyandi:

How do you figure out like that, this segment, going back to what

Prasanna Malaiyandi:

you said, you kill all the routes.

Prasanna Malaiyandi:

How do you figure out that this segment is safe or not?

Prasanna Malaiyandi:

And then I guess that, yeah, that's actually only one question.

Tom Hollingsworth:

Well, so typically what, and, and you're, you're

Tom Hollingsworth:

effectively, when you create these boundaries, it's, it's like looking

Tom Hollingsworth:

for the hot potato effectively, because unless you, like in the alias episode,

Tom Hollingsworth:

just go click all the switches off.

Tom Hollingsworth:

Those devices can still communicate to each other at layer two.

Tom Hollingsworth:

Now, where you don't wanna have a problem is, is that it's in the data.

Tom Hollingsworth:

because if you isolate the layer two data center, now you've got a real problem.

Tom Hollingsworth:

Because if those servers, if if it's looking for servers, those

Tom Hollingsworth:

servers can still get infected.

Tom Hollingsworth:

That's why it's actually better to have like a, you know, a host route or

Tom Hollingsworth:

something like that, or something that, that kind of isolates that per unit thing.

Tom Hollingsworth:

I mean, honestly, like a V switch is perfect for this because like,

Tom Hollingsworth:

if it's not bound for that host, I'm not gonna let it go any further.

Tom Hollingsworth:

But effectively what you have to do is you have to look for chatter

Tom Hollingsworth:

that's still going on in the network.

Tom Hollingsworth:

Like you, you, I've shut all this down.

Tom Hollingsworth:

and I told my users to like disable their machines or, or turn them off or

Tom Hollingsworth:

whatever, what's still trying to talk.

Tom Hollingsworth:

And then you go take that on a case by case basis.

Tom Hollingsworth:

Oh, this device is still sending traffic that it's, but

Tom Hollingsworth:

it's looking for this server.

Tom Hollingsworth:

Okay, well I'm, I, I can shut it off because I know that it's probably safe.

Tom Hollingsworth:

But then you run into something like, oh, this thing is chattering

Tom Hollingsworth:

an awful lot and it's chattering on a way that it shouldn't be chattering.

Tom Hollingsworth:

Like that's how I've gone and found hosts that have been infected, but not

Tom Hollingsworth:

by ransomware, but by early malware because they just kept hammering the

Tom Hollingsworth:

firewall with these outbound requests.

Tom Hollingsworth:

And I'm like, you shouldn't

Tom Hollingsworth:

be doing that.

Tom Hollingsworth:

So it's, it's almost like a little bit of detective work.

Tom Hollingsworth:

The good news is, is that even though the network devices are kind of like

Tom Hollingsworth:

dumb from the perspective of I don't care what application is trying to talk,

Tom Hollingsworth:

where they're really good at telling you that things are still generating traffic.

Tom Hollingsworth:

It's like, oh, this port is still sending a ton of packets f bound

Tom Hollingsworth:

for this address on this location.

Tom Hollingsworth:

And so then you're like, oh, I think something might be up here.

Prasanna Malaiyandi:

Do you ever see cases where people.

Prasanna Malaiyandi:

, almost do a, like, create a black hole on the device itself to sort

Prasanna Malaiyandi:

of sync the packets there so it doesn't go out, rather than having

Prasanna Malaiyandi:

to necessarily do it on the switch.

Tom Hollingsworth:

Um, you can, uh, that's actually a really great way to

Tom Hollingsworth:

determine what it's trying to contact is to create like a null route on the system.

Tom Hollingsworth:

Uh, uh, going all the way back like three or four years.

Tom Hollingsworth:

Like Mark Marcus Hutchins, that's how he actually stopped a major outbreak of

Tom Hollingsworth:

malware, uh, for all the good it did, and he got arrested by the FBI later.

Tom Hollingsworth:

But he basically black hole the dns.

Tom Hollingsworth:

He bought the domain black hole it because if that domain name was

Tom Hollingsworth:

active, then it would stop propagating.

Tom Hollingsworth:

And so he figured that out by saying, oh, I wonder where this is

Tom Hollingsworth:

going and I wonder what it's doing.

Tom Hollingsworth:

You can do that.

Tom Hollingsworth:

And it's actually the next step in incident response, which you've isolated

Tom Hollingsworth:

the system, is I wanna see how it behaves and what it's trying to do.

Tom Hollingsworth:

Cuz that could give me a clue as to what I got hit with and

Tom Hollingsworth:

what they could be looking for.

Tom Hollingsworth:

And that gives you, you know, a, a little bit of opportunity, but that's

Tom Hollingsworth:

a little bit more of an advanced tool that you would, you would want to use.

Tom Hollingsworth:

Uh, just because black holding traffic on a, on a device takes

Tom Hollingsworth:

a little bit of setup, especially if you're fighting against people

Tom Hollingsworth:

who don't want you to do that.

Prasanna Malaiyandi:

Yeah.

W. Curtis Preston:

Speaker:

Yeah, so it sounds like.

W. Curtis Preston:

Speaker:

A, a lot of the things that you talked about in the last couple of minutes, they

W. Curtis Preston:

Speaker:

would be a lot easier to do again, if we segmented the network in the first place,

W. Curtis Preston:

Speaker:

right?

W. Curtis Preston:

Speaker:

We put people with Windows laptops on one network.

W. Curtis Preston:

Speaker:

We put people with Mac laptops on a network, another network.

W. Curtis Preston:

Speaker:

We put the, the, the phones right?

W. Curtis Preston:

Speaker:

That are doing the wifi.

W. Curtis Preston:

Speaker:

We put them on another network.

W. Curtis Preston:

Speaker:

Um, and we put servers on a different network.

W. Curtis Preston:

Speaker:

We put, maybe we put servers of a different type on, on a different network.

W. Curtis Preston:

Speaker:

So that way you could basically say you don't have to tell the,

W. Curtis Preston:

Speaker:

the, the users to not do anything.

W. Curtis Preston:

Speaker:

You can just say shut off the, the laptop, uh, network.

W. Curtis Preston:

Speaker:

Right?

W. Curtis Preston:

Speaker:

Um, and you, you shut off the laptop network and so on.

W. Curtis Preston:

Speaker:

And, and all the networks that where we don't currently,

W. Curtis Preston:

Speaker:

what we're not looking at.

W. Curtis Preston:

Speaker:

And then, okay, who's trying to talk?

W. Curtis Preston:

Speaker:

Who's trying to talk?

W. Curtis Preston:

Speaker:

Why is this server surfing?

W. Curtis Preston:

Speaker:

The web

Tom Hollingsworth:

Yeah.

W. Curtis Preston:

Speaker:

There's nobody over there.

W. Curtis Preston:

Speaker:

Why is this server going over report 80?

Tom Hollingsworth:

Well, a lot of places already kind of have this by

Tom Hollingsworth:

default, even if they didn't realize they were doing it because you have

Tom Hollingsworth:

different classes of devices that you wanna treat them differently.

Tom Hollingsworth:

Like for example, the uh, um, the server network, we want to have a

Tom Hollingsworth:

little bit more security in there.

Tom Hollingsworth:

Maybe a little less host to host East to west traffic kind of thing.

Tom Hollingsworth:

The wireless network where all the laptops and the devices connect.

Tom Hollingsworth:

I'm a little less careful about that because I actually have identity

Tom Hollingsworth:

management in place that validates the users when they try to log in.

Tom Hollingsworth:

Maybe I have a guest wireless network for my, for people that come into the lobby.

Tom Hollingsworth:

That one's wide open to the internet outbound only.

Tom Hollingsworth:

So I don't need to worry about that quite as much.

Tom Hollingsworth:

And then, you know, like phones and printers and things like that, that

Tom Hollingsworth:

have very specific things like, you know, I wouldn't enable Bonura in my

Tom Hollingsworth:

internal network, but maybe for the printer vlan I would, because I want

Tom Hollingsworth:

people to be able to find a printer.

Tom Hollingsworth:

Open up their laptop.

Tom Hollingsworth:

So they've already created these segments.

Tom Hollingsworth:

You just have to know where the buttons are to shut them off.

Tom Hollingsworth:

So maybe the example is I wanna isolate the servers from the rest

Tom Hollingsworth:

of the network, cuz I think there's something in there, but I can still

Tom Hollingsworth:

leave the wireless network up.

Tom Hollingsworth:

Maybe have everybody join the guest access network and force them all

Tom Hollingsworth:

out to the internet to do, you know, incident response or chat channels

Tom Hollingsworth:

or something like that where I'm, you know, but I'm creating these bounds so

Tom Hollingsworth:

that traffic flows one direction only, or it prevents certain things inside

Tom Hollingsworth:

of other areas because, you know, there's nothing to say like the, you

Tom Hollingsworth:

know, the, the, uh, s IDs that are on printers that are like, you know, set up,

Tom Hollingsworth:

uh, set me up or something like that can't be compromised.

Tom Hollingsworth:

And then if they can get into your printer network, it's like,

Tom Hollingsworth:

oh crap, where can they go from?

W. Curtis Preston:

Yeah.

W. Curtis Preston:

And, and Bonjour of course would be the, um, I, I don't know how would

W. Curtis Preston:

I define

Prasanna Malaiyandi:

file sharing.

Tom Hollingsworth:

It, it is, it's almost like an auto configuration announcement,

Tom Hollingsworth:

uh, setting where, uh, it, it, and you can thank Steve Jobs for this.

Tom Hollingsworth:

He's like, I hate setting up printers.

Tom Hollingsworth:

And so basically what he did is he set up a system so that the printers

Tom Hollingsworth:

can announce that they exist.

Tom Hollingsworth:

And your laptop is constantly listening for these.

Tom Hollingsworth:

Bonura is another one of those protocols that is extra chatty and you kinda

Tom Hollingsworth:

wanna put bounds on it so that like you don't have the Apple TV four hallways

Tom Hollingsworth:

down announcing itself to the people in accounting because one, it's annoying.

Tom Hollingsworth:

And two, you never know when you're gonna do something you're not supposed to.

Prasanna Malaiyandi:

Interesting.

Prasanna Malaiyandi:

So yeah, I guess a lot of these are really around setting up

Prasanna Malaiyandi:

that initial network properly.

Prasanna Malaiyandi:

So then when you do have these issues, you can recover quickly and

Prasanna Malaiyandi:

identify and then recover quickly.

Prasanna Malaiyandi:

Right?

Prasanna Malaiyandi:

But if you don't have that initial setup done, then you're

Prasanna Malaiyandi:

in for a world of hurt, I guess.

Tom Hollingsworth:

and not just initial setup.

Tom Hollingsworth:

You actually do have to treat the network like a living, breathing organism.

Tom Hollingsworth:

I can't think of a single server admin out there that installs,

Tom Hollingsworth:

you know, windows server.

Tom Hollingsworth:

What are we up now?

Tom Hollingsworth:

20 20, 20 23 Windows, server X, I don't know, installs it

Tom Hollingsworth:

and then never patches it.

Tom Hollingsworth:

Never

Tom Hollingsworth:

touches it again.

Tom Hollingsworth:

Like, like you people are probably just shaking, even thinking.

Tom Hollingsworth:

, you cannot configure a network and then just leave it alone.

Tom Hollingsworth:

You do have to go in and, and tweak things and move things and change things.

Tom Hollingsworth:

And, you know, not just when you're trying to fix a broken thing,

Tom Hollingsworth:

either, you have to like, okay, is this subnet big enough for the

Tom Hollingsworth:

number of hosts that are in it?

Tom Hollingsworth:

Should I create routes over here?

Tom Hollingsworth:

It looks like there's a lot of extra traffic going on over this direction.

Tom Hollingsworth:

Maybe I need to disallow that because it looks like it's something

Tom Hollingsworth:

that shouldn't be happening.

Tom Hollingsworth:

Like, if you're not constantly pruning back what you are working on then,

Tom Hollingsworth:

and that's the problem that a lot of the, the, uh, ransomware writers have

Tom Hollingsworth:

figured out, like a lot of, a lot of their secrets, if you wanna call them,

Tom Hollingsworth:

that are just inadequate it support.

Tom Hollingsworth:

Like, we're gonna hope that you had left this on by default and we're

Tom Hollingsworth:

gonna take advantage of it and use it.

Tom Hollingsworth:

And if you did, I'm sorry, but like, you know, if any best practices

Tom Hollingsworth:

guide out there says, shut that off, and you didn't shut it off,

Tom Hollingsworth:

are you in that big of a hurry?

W. Curtis Preston:

Yeah, well we're, we're living in a world

W. Curtis Preston:

where, uh, you know, people don't even change their default password.

W. Curtis Preston:

So, um, listen, here's the thing, Tom, my plumber's here, so, uh, I, I, you

W. Curtis Preston:

know, I got a tradesman that actually showed up at two o'clock when he said

W. Curtis Preston:

he was gonna be here at two o'clock.

W. Curtis Preston:

So I gotta , we gotta shut this baby down.

W. Curtis Preston:

Uh, Tom, this has been, this has been a great conversation.

W. Curtis Preston:

Um, so thanks, thanks a lot.

Tom Hollingsworth:

Well, thanks for having me.

Tom Hollingsworth:

It's, it's been fun to talk about networking with, uh, with some folks

Tom Hollingsworth:

that coming at it from a slightly different perspective and understanding,

Tom Hollingsworth:

you know, what are we trying to accomplish with it, and in some

Tom Hollingsworth:

cases, what are we trying to disallow?

Prasanna Malaiyandi:

Hmm,

W. Curtis Preston:

Absolutely.

W. Curtis Preston:

Thanks again, Prasanna, once again, making me go backwards,

Prasanna Malaiyandi:

I, you know me, I try, you take one step back, two steps

Prasanna Malaiyandi:

forward or something like that, right?

W. Curtis Preston:

something like that.

W. Curtis Preston:

I

W. Curtis Preston:

like that.

W. Curtis Preston:

All right.

W. Curtis Preston:

And thanks again to our listeners.

W. Curtis Preston:

Remember to subscribe so that you can restore it all.