Snorkel42 00:00:00
can I make an admission real quick?
Snorkel42 00:00:02
I had no idea what that karma meant.
W. Curtis Preston:Yeah.
Snorkel42 00:00:06
I did not know what that number translated.
W. Curtis Preston:Hi, and welcome to Backup Central's Restore it All podcast.
W. Curtis Preston:I'm your host, W.
W. Curtis Preston:Curtis Preston.
W. Curtis Preston:AKA Mr.
W. Curtis Preston:Backup.
W. Curtis Preston:And I have with me, my fellow photo-shoot model, Prasanna Malaiyandi.
W. Curtis Preston:going, Prasanna?
Prasanna Malaiyandi:I'm good.
Prasanna Malaiyandi:That was a very interesting experience, but I had fun and
Prasanna Malaiyandi:it was good to see you too.
Prasanna Malaiyandi:What's funny is that's the first time I've been to an office in the last two years.
Prasanna Malaiyandi:It just wasn't my office.
W. Curtis Preston:Right, Yeah.
W. Curtis Preston:yeah, w it is actually, Druva's new office.
W. Curtis Preston:We moved from, we were over on California avenue in Sunnyvale,
W. Curtis Preston:and now we're over on.
W. Curtis Preston:What is it?
Prasanna Malaiyandi:mission college
W. Curtis Preston:college in Santa Clara across from the Intel museum?
W. Curtis Preston:I believe,
Prasanna Malaiyandi:yeah, it's one of Intel's campuses
W. Curtis Preston:Yeah.
W. Curtis Preston:and, so I wanted to do some photos podcast.
Prasanna Malaiyandi:Curtis tired of his face being on the homepage
Prasanna Malaiyandi:or the title art for the podcast.
Prasanna Malaiyandi:But I liked your photo.
W. Curtis Preston:what's that.
Prasanna Malaiyandi:I liked the picture of you on the podcast.
W. Curtis Preston:have you seen the new, the the bearded contemplated one?
Prasanna Malaiyandi:Yes, I have.
Prasanna Malaiyandi:That one's also good too.
Prasanna Malaiyandi:Yeah.
W. Curtis Preston:I got lots of compliments on that photo,
W. Curtis Preston:but I wanted your picture.
W. Curtis Preston:I wanted this giant mane of yours to be captured because I think one
W. Curtis Preston:day you're going to come to your senses and maybe cut it all off and
W. Curtis Preston:I want to capture It for posterity.
W. Curtis Preston:And so we had a photo shoot and, we got to, we got some, it was.
W. Curtis Preston:a lot of fun doing that,
Prasanna Malaiyandi:It was.
Prasanna Malaiyandi:Thank you, Alex, if you're listening to this, but that was awesome.
Prasanna Malaiyandi:Thank you for the help.
W. Curtis Preston:Alex did a great job and there were, just a couple
W. Curtis Preston:of dudes taking photos in an office.
W. Curtis Preston:It wasn't awkward at any time.
Prasanna Malaiyandi:No, not at all.
W. Curtis Preston:Not at all.
W. Curtis Preston:Yeah.
W. Curtis Preston:Look longingly into each other's eyes.
W. Curtis Preston:Yeah, By the time you hear this, hopefully those photos or some
W. Curtis Preston:of those photos will be up on the website, at backupcentral.com.
W. Curtis Preston:And, before I bring on our guest, I'll go ahead and throw out our usual disclaimer.
W. Curtis Preston:I mentioned, the Druva office I work for Druva.
W. Curtis Preston:Prasanna happens to work for zoom.
W. Curtis Preston:This is not a podcast of either company.
W. Curtis Preston:And the opinions that you hear are all ours.
W. Curtis Preston:Be sure to rate us at ratethispodcast.com/restore
W. Curtis Preston:and subscribe to the podcast.
W. Curtis Preston:You can either do it in your favorite pod catcher, or you can
W. Curtis Preston:go over to backupcentral.com and subscribe to our mailing list.
W. Curtis Preston:And we'll let you know, every time we come out with an episode and, we also are
W. Curtis Preston:always looking for interesting guests.
W. Curtis Preston:We have discovered a good one this week.
W. Curtis Preston:I'm very excited to bring him on.
W. Curtis Preston:And, if you're interested in the things we're interested, which is data
W. Curtis Preston:protection, disaster recovery, backup and recovery security, beer, barbecue.
Prasanna Malaiyandi:barbecue.
Prasanna Malaiyandi:Definitely.
Prasanna Malaiyandi:Yes.
Prasanna Malaiyandi:To barbecue.
W. Curtis Preston:Definitely.
W. Curtis Preston:Yes.
W. Curtis Preston:To barbecue.
W. Curtis Preston:And, then, just reach out to me @wcpreston on Twitter or wcurtispreston@gmail.com.
W. Curtis Preston:Today I wanted to talk about a little something has
W. Curtis Preston:happened in the InfoSec world.
W. Curtis Preston:Isn't that right, Prasanna?
Prasanna Malaiyandi:Yep.
Prasanna Malaiyandi:just a tiny little something, nothing big.
W. Curtis Preston:I'm flashing back to the matrix.
W. Curtis Preston:I don't remember which one.
W. Curtis Preston:I think it was the third one, the key master.
W. Curtis Preston:you got the guy with all the keys.
W. Curtis Preston:So we're talking of course, about the Okta compromise that went wide this week.
W. Curtis Preston:So this guest Prasanna,
Prasanna Malaiyandi:You're excited.
Prasanna Malaiyandi:Aren't you Curtis?
Prasanna Malaiyandi:Aren't you?
W. Curtis Preston:I, am.
W. Curtis Preston:He's shrouded in mystery.
W. Curtis Preston:we've had guests on before where we've used pseudonyms.
W. Curtis Preston:but I knew those people.
W. Curtis Preston:And so I knew their actual names.
W. Curtis Preston:And then I had to pretend not to know their names and refer to them
W. Curtis Preston:by Harry Potter and Ron Weasley.
W. Curtis Preston:And you may recall that during the.
W. Curtis Preston:Recording.
W. Curtis Preston:Occasionally I would slip up and call
Prasanna Malaiyandi:Yes.
Prasanna Malaiyandi:I remember that.
Prasanna Malaiyandi:Lots of editing for you.
Prasanna Malaiyandi:Lots of listening to it.
W. Curtis Preston:But in this case, I have no idea who this guy actually is.
W. Curtis Preston:I just know that he knows his stuff.
W. Curtis Preston:He has been in IT for 25 years in the InfoSec space for about 20.
W. Curtis Preston:He has are you ready for this?
W. Curtis Preston:He has a karma rating on Reddit of 33,000.
W. Curtis Preston:I'm excited that I've got like 600.
W. Curtis Preston:He's got 33,000, which means that at least 33,000 times someone has upvoted him.
W. Curtis Preston:That's impressive in and of itself because Reddit is a crazy place
W. Curtis Preston:where if you say things that people don't like, even if you're correct,
W. Curtis Preston:they'll vote you down anyway.
W. Curtis Preston:So he's managed to convince 33,000 people somewhere to click, on what he wrote.
W. Curtis Preston:Welcome to the podcast.
W. Curtis Preston:Snorkel42.
Snorkel42 00:05:26
Hello, can I make an admission real quick?
Snorkel42 00:05:29
I had no idea what that karma meant.
W. Curtis Preston:Yeah.
Snorkel42 00:05:33
I did not know what that number translated.
W. Curtis Preston:Every time you get an up vote, you get karma and
W. Curtis Preston:every time someone downvotes you, your karma is subtracted from.
Snorkel42 00:05:44
Okay, now I know
W. Curtis Preston:Yeah.
Snorkel42 00:05:47
me
W. Curtis Preston:I post in a number of non it, subreddits where very
W. Curtis Preston:much you can say the right thing, the correct thing, and they will
W. Curtis Preston:get mad at you and downvote you, and then you just lost a bunch of karma.
W. Curtis Preston:And since I'm, I'm a relatively, especially compared to you,
W. Curtis Preston:I'm a relatively new Redditor, would that be the right word?
W. Curtis Preston:And I'm trying to,
Prasanna Malaiyandi:Reddit poster.
W. Curtis Preston:It helps you bubble up into threads and things.
W. Curtis Preston:it gives your posts more weight.
W. Curtis Preston:So yeah.
W. Curtis Preston:The fact that you have 33,000 is a BFD.
Snorkel42 00:06:25
I'll get a t-shirt made.
W. Curtis Preston:Why don't you describe what happened at Okta.
Snorkel42 00:06:31
Yeah, I guess to start, I don't have any insider knowledge here.
Snorkel42 00:06:36
All I have is what's been announced and what little Okta has
Snorkel42 00:06:40
been able to slip through their marketing and legal teams, but
W. Curtis Preston:We're going to get
Snorkel42 00:06:45
to right.
Snorkel42 00:06:46
but basically they contract out their customer support to a third
Snorkel42 00:06:50
party located in Costa Rica.
Snorkel42 00:06:52
and sometime around the end of January.
Snorkel42 00:06:56
A attacker from the lapses group gained access to one of those support
Snorkel42 00:07:01
engineer's laptops, supposedly over RDP, which is something else to get into.
Snorkel42 00:07:05
and for about five days they had access to, that laptop and were able to
Snorkel42 00:07:10
monitor if not interact the privileged access that support engineer had.
Snorkel42 00:07:14
So Okta has a, an internal application that they call a super user,
Snorkel42 00:07:18
which by the way, what a terrible name doesn't mean that you have,
Snorkel42 00:07:21
super user God mode necessarily.
Snorkel42 00:07:24
It's just the administrative interface to.
Snorkel42 00:07:26
Um, but it doesn't necessarily mean that, Hey, I'm going to go in and reset
Snorkel42 00:07:30
everyone's password to something I know, but it did give them the ability
Snorkel42 00:07:33
to reset passwords, to send, links to customers, to go reset their passwords
Snorkel42 00:07:38
and to reset their multifactor.
Snorkel42 00:07:40
When Okta started off with, their CSO saying no customers impacted.
Snorkel42 00:07:44
We detected this back in January, or we detected an unsuccessful attempt.
Snorkel42 00:07:49
Which is the other thing to get into here, back in January and shut it down.
Snorkel42 00:07:53
And then the second blog post, they added a little bit more.
Snorkel42 00:07:56
And then the third blog posts like, 2.5% of our customers,
Snorkel42 00:07:59
about 366 people impacted,
Prasanna Malaiyandi:Yeah, and it's interesting because it's
Prasanna Malaiyandi:366 people, but it's really 366 customers, which could be of any size.
Snorkel42 00:08:09
and we certainly know from the screenshots that CloudFlare was one
Snorkel42 00:08:12
of those customers and CloudFlare CEO was fairly vocal on Twitter about his
Snorkel42 00:08:16
dissatisfaction with Okta right now.
Prasanna Malaiyandi:And CloudFlare, I thought published a pretty
Prasanna Malaiyandi:good blog as well about sort of their analysis and what they did.
W. Curtis Preston:fact, a lot of people felt that Cloudflare's analysis was
W. Curtis Preston:better than what Octa provided themselves.
Snorkel42 00:08:35
me included.
Snorkel42 00:08:36
Yeah.
Prasanna Malaiyandi:Okay.
W. Curtis Preston:Yeah.
W. Curtis Preston:and by the way, there, there were a couple of people you put on a, another
W. Curtis Preston:Reddit thread that was about this.
W. Curtis Preston:Maybe more than one.
W. Curtis Preston:I don't know, but I know I was reading one of them.
W. Curtis Preston:And more than one former employee of Okta logged in and, and replied and
W. Curtis Preston:explained basically what the super-user, which I agree it's a really bad name,
W. Curtis Preston:but, I don't know where to start, but the thing about the resetting passwords
W. Curtis Preston:and stuff, like they, it didn't appear that they had the ability to do anything,
W. Curtis Preston:to be able to access an account.
W. Curtis Preston:They could reset a password.
W. Curtis Preston:They could reset or even remove MFA, but that, but anything towards
W. Curtis Preston:that would be sent to the customer.
W. Curtis Preston:So at best, they would only be able to do that if they had access to
W. Curtis Preston:that customer in the first place.
W. Curtis Preston:So it didn't look like they would have been able to use this to
W. Curtis Preston:actually access any customers.
W. Curtis Preston:does that sound about right?
W. Curtis Preston:Yeah.
Snorkel42 00:09:44
It sounds about right to me for a few reasons.
Snorkel42 00:09:46
Look to me, the biggest issue here has been the way Octa has been communicating
Snorkel42 00:09:52
to customers and the lack of transparency.
Snorkel42 00:09:54
And the reason why it's important is I find myself in my own back of
Snorkel42 00:09:59
my mind saying, if we believe Okta.
Snorkel42 00:10:02
Every time I think about this.
Snorkel42 00:10:04
If we believe them, and the only reason I have that question for a
Snorkel42 00:10:06
company that their entire existence hinges on, if we believe Okta.
Snorkel42 00:10:11
Like they provide, maybe we should start that off of what does Okta do.
Snorkel42 00:10:15
So Okta provides authentication services and identity management to corporations,
Snorkel42 00:10:19
particularly around single sign-on And, multi-factor, and the way, you can place
Snorkel42 00:10:22
them in your infrastructure can get as far as deep as: through the Okta portal,
Snorkel42 00:10:27
I can reset your active directory, your on-prem active directory password.
Snorkel42 00:10:31
so you know, it's a company that holds a lot of keys and if you
Snorkel42 00:10:35
can't trust them, my God, how do you keep them in your enterprise?
Snorkel42 00:10:40
And so since to your point of, yeah, it doesn't look like they had that
Snorkel42 00:10:44
kind of access to really take over the accounts of Okta customers.
Snorkel42 00:10:49
And I believe that to be true, not so much because of what Okta said, but mainly
Snorkel42 00:10:53
because lapses didn't appear to use it.
Snorkel42 00:10:55
I think if they would have had that access, we would have seen a
Snorkel42 00:10:57
much bigger impact, but I don't.
W. Curtis Preston:I'm a hundred percent on board with everything that
W. Curtis Preston:you're saying about, that the biggest issue here has been Okta's response.
W. Curtis Preston:I just wanted to for anyone who's listening to this for the
W. Curtis Preston:first time, it sounds horrible.
W. Curtis Preston:And screenshots of customer data sounds horrible, but it, I do want
W. Curtis Preston:to at least say it does look based on the information that we have most of
W. Curtis Preston:which did not come from Okta, that they wouldn't have been able to actually
W. Curtis Preston:access any customer's environment.
W. Curtis Preston:They might have been able to annoy some customers, Change your
W. Curtis Preston:passwords and things like that.
W. Curtis Preston:but they.
W. Curtis Preston:I completely agree with you that from the get-go, like from the very beginning from
W. Curtis Preston:message one and all the way up to message three their verbiage is really weird,
Snorkel42 00:11:47
Yeah.
Prasanna Malaiyandi:I wondering if this has to do with any of, the new laws going
Prasanna Malaiyandi:through Congress, around data breaches, or just public perception with everything
Prasanna Malaiyandi:going on in the world right now that they just did a PR blunder, if you will.
Snorkel42 00:12:06
It could be that.
Snorkel42 00:12:08
Yeah.
Snorkel42 00:12:08
and certainly they are a publicly traded company and that has certainly been
Snorkel42 00:12:12
called out a number of forums of, they, they may be tied as to what they can say.
Snorkel42 00:12:17
That one of the things that really sticks out to me is from Cloudflare's response,
Snorkel42 00:12:22
it was very clear that they learned about this along with everyone else.
Snorkel42 00:12:25
So one day they woke up and saw screenshots on Twitter from an
Snorkel42 00:12:30
attack group of their information.
Snorkel42 00:12:32
And Okta has a publicly disclosed, privacy and security policy that - i can even name
Snorkel42 00:12:39
the sections 20 and 21- talk about when they will alert customers of a breach.
Snorkel42 00:12:44
And I think you would have to do some pretty fancy legal footwork
Snorkel42 00:12:50
to explain why CloudFlare did not know about this in January 20.
Snorkel42 00:12:53
To me that is the real big takeaway from, do I trust this company anymore?
Snorkel42 00:12:57
The fact that the customers that we do know were impacted clearly
Snorkel42 00:13:00
didn't find out until the rest of the world found out as well.
Snorkel42 00:13:03
Um, and that's an example of Okta not following their own policies
Snorkel42 00:13:06
and that's troubling to me.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:And especially because these are in legal contracts, You could be held
Prasanna Malaiyandi:liable for it and you're losing the trust of your customers, right?
Prasanna Malaiyandi:Who would trust Okta the next time something happens?
Snorkel42 00:13:17
yeah.
Snorkel42 00:13:17
And then of course the timing, all of it is very interesting in that Okta is
Snorkel42 00:13:21
saying that we couldn't respond until we got the security incident report
Snorkel42 00:13:26
from the company that this third party hired, and we just got that conveniently.
Snorkel42 00:13:30
As the screenshots got posted.
Snorkel42 00:13:31
but even then it's so you had a third party contractor to get compromised
Snorkel42 00:13:35
and what kind of access they had.
Snorkel42 00:13:37
You just sat on it for three months.
Snorkel42 00:13:39
are you telling me that Okta did not actually get involved in
Snorkel42 00:13:41
that incident response at all?
Snorkel42 00:13:44
so just a lot of things not adding up and it certainly doesn't
Snorkel42 00:13:47
paint a pretty picture for Okta.
W. Curtis Preston:yeah.
W. Curtis Preston:I forgot.
W. Curtis Preston:I w I was scrolling through the post right now, the verbiage where it says,
W. Curtis Preston:you made a comment that, that it seemed that, marketing was involved in these,
W. Curtis Preston:the statements that went out because like it did start out with even in the third
W. Curtis Preston:one, even in the third message, they referred to it as, an attempt to access.
Snorkel42 00:14:15
and unsuccessful.
W. Curtis Preston:This was not an attempt.
W. Curtis Preston:This was a hack.
W. Curtis Preston:I don't know what the proper term is.
Snorkel42 00:14:20
no, I think a hack is absolutely the right term And
Snorkel42 00:14:23
so they also have a timeline.
Snorkel42 00:14:25
And I think if you marry those blog posts, where they talk about
Snorkel42 00:14:29
unsuccessful attempts and the timeline.
Snorkel42 00:14:30
You, you see where they're getting to the unsuccessful attempt that I derive is.
Snorkel42 00:14:35
where Okta caught em.
Snorkel42 00:14:36
which honestly, we need to take a step back and really give Okta security, some
Snorkel42 00:14:40
kudos in the detection to begin with.
Snorkel42 00:14:42
So where Okta caught them was the attacker apparently attempted to add
Snorkel42 00:14:47
a second MFA token to the support engineer's account, so that they could
Snorkel42 00:14:51
start approving from what I've been told.
Snorkel42 00:14:53
Okta internally uses MFA a lot.
Snorkel42 00:14:56
So apparently Okta is huge on every step you take within their networks MFA.
Snorkel42 00:15:01
So it looks like the attacker tried to add their own MFA token.
Snorkel42 00:15:04
So while they had RDP access and the support engineer was
Snorkel42 00:15:08
away, they could start moving around and start respond to MFA.
Snorkel42 00:15:11
And Okta caught that.
Snorkel42 00:15:13
The addition of an MFA token from a weird location.
Snorkel42 00:15:16
Which is fantastic, like really great job and darn it.
Snorkel42 00:15:20
So I think what, Okta is referring to when they say, there was an unsuccessful
Snorkel42 00:15:25
attempt, was the unsuccessful attempt to add that MFA token.
Snorkel42 00:15:28
But boy is that some marketing wordplay there that say, oh Yeah.
Snorkel42 00:15:32
it's an unsuccessful attempt to take over an engineer laptop.
Snorkel42 00:15:35
no, they took it over.
Prasanna Malaiyandi:They took it over.
Prasanna Malaiyandi:They had access to your network.
Prasanna Malaiyandi:They got in, they just weren't able to.
Snorkel42 00:15:44
Yeah.
W. Curtis Preston:And for five days they could do, because they were
W. Curtis Preston:controlling that laptop via RDP, which is the remote desktop protocol, which
W. Curtis Preston:should not be exposed to the internet.
Prasanna Malaiyandi:Curtis' second favorite topic, I think.
Snorkel42 00:15:56
and so it is a really good point.
Snorkel42 00:16:00
And I think this is what's way more interesting about this.
Snorkel42 00:16:03
Is w was it exposed to the internet?
Snorkel42 00:16:05
We don't know if it wasn't then it would sure seem, the lapses actually have access
Snorkel42 00:16:13
to this third party's network first, and then managed to get RDP access to this
Snorkel42 00:16:18
contractors laptop, the screenshots show Global Protect of which Global Protect
Snorkel42 00:16:22
is a VPN product from Palo Alto Networks.
Snorkel42 00:16:25
So it was, this person worked from home.
Snorkel42 00:16:26
possibly and was their home network breached.
Snorkel42 00:16:29
So it does, from an InfoSec standpoint, certainly this screams to me.
Snorkel42 00:16:32
you should have RDP locked down on your workstations, does your laptop need
Snorkel42 00:16:36
to be able to accept RDP connections?
Snorkel42 00:16:38
Absolutely not.
Snorkel42 00:16:39
But, yeah.
Snorkel42 00:16:39
So I think that RDP side though is really a big topic because it has a lot of, fine
Snorkel42 00:16:48
read between the lines there of, what does it mean that they had RDP access?
Snorkel42 00:16:51
How did they even reach it over RDP?
Prasanna Malaiyandi:And it's something we may never really find out.
Prasanna Malaiyandi:Unless.
Snorkel42 00:16:59
Yeah.
Prasanna Malaiyandi:because noone's really going to bring that up.
Prasanna Malaiyandi:Noone's going to talk about that.
Prasanna Malaiyandi:They're just going to say, yeah, we stopped the attack
Prasanna Malaiyandi:or whatever the breach was.
Prasanna Malaiyandi:They'll focus on the Okta side.
Prasanna Malaiyandi:Everyone talks about okay, we found the issue or we saw what they were
Prasanna Malaiyandi:trying to do and we stopped it.
Prasanna Malaiyandi:End of story.
Prasanna Malaiyandi:Not necessarily.
Prasanna Malaiyandi:How do they really get in the first place?
Prasanna Malaiyandi:And what does that look like and how do we prevent that from happening.
Snorkel42 00:17:20
Yeah, and the thing to keep in mind too.
Snorkel42 00:17:22
When we think about this is a third-party contractor that specializes in 24/7
Snorkel42 00:17:27
customer support for larger enterprises.
Snorkel42 00:17:29
If lapses, which is a attack group, that's setting the world on fire Right.
Snorkel42 00:17:33
now had access to their network.
Snorkel42 00:17:36
What else was going on?
Snorkel42 00:17:38
Okta may be the tip the iceberg for them.
W. Curtis Preston:Yeah.
W. Curtis Preston:somebody, one of the commenters, they felt that the whole like throwing out the.
W. Curtis Preston:Screenshots from Okta was actually an attempt at subterfuge on the part
W. Curtis Preston:of lapses to throw away attention from the fact that the real problem
W. Curtis Preston:is the access you just mentioned.
W. Curtis Preston:Somebody said, maybe that's why they threw out all the Okta information because.
W. Curtis Preston:we're not talking about Sykes.
W. Curtis Preston:We're talking about Okta.
W. Curtis Preston:The technically the hack was actually a Sykes.
W. Curtis Preston:Okta was just the customer in this case.
Snorkel42 00:18:07
And Octa is very quick to call that out.
Snorkel42 00:18:10
They say third party, as often as they possibly can as if that's,
Snorkel42 00:18:14
oh, wash our hands of that.
Snorkel42 00:18:15
Sure, we gave that third-party access to go reset your passwords
Snorkel42 00:18:18
and MFAs, but that's their problem.
W. Curtis Preston:There were people, a handful of people in the comments
W. Curtis Preston:and there, as there always is on the internet, there were a handful of people
W. Curtis Preston:that came to Octa's defense regarding that they get thousands of attacks a day.
W. Curtis Preston:And that they're saying no customer systems were accessed.
W. Curtis Preston:and so was there really a duty to report back in January?
W. Curtis Preston:What do you think about that?
Snorkel42 00:18:50
The title of the post I had made was am I overreacting?
Snorkel42 00:18:53
cause I is probably coming through in this podcast.
Snorkel42 00:18:55
I'm still quite upset with them.
Snorkel42 00:18:57
and I agree with a lot of what the, what those folks were saying.
Snorkel42 00:19:00
yes, I'm sure Okta is attacked repeatedly.
Snorkel42 00:19:04
Now the question is what does that look like if they are attacked so often to the
Snorkel42 00:19:09
point that people have access to the super user program that it's such a non-event
Snorkel42 00:19:13
for them, like that's an everyday event
Snorkel42 00:19:15
then holy cow, right?
Snorkel42 00:19:16
Like seriously, if that's every day for Okta, then we had something
Snorkel42 00:19:19
way bigger to talk about here.
Snorkel42 00:19:20
I'm guessing that's not the case.
Snorkel42 00:19:22
I'm guessing this.
Snorkel42 00:19:23
This was a significant event for, um, and downplaying that I don't buy.
Snorkel42 00:19:28
I think this was a significant event and Okta was very happy to.
Snorkel42 00:19:32
keep it under wraps and hope that it never came out.
Snorkel42 00:19:35
and I think I, as we said at the start, yeah, I don't think anyone
Snorkel42 00:19:39
really, I don't think there was any actual breach of a customer's account.
Snorkel42 00:19:43
I think what we saw on the screenshots was pretty much all that happened,
Snorkel42 00:19:47
but I think if you were to get CloudFlare CEO on those podcasts, he
Snorkel42 00:19:51
would tell you that was significant.
Snorkel42 00:19:52
and the fact that he didn't know about it until a couple of days
Snorkel42 00:19:56
ago, when it was posted on Twitter, Was not acceptable for him.
Snorkel42 00:19:59
And I'll be really surprised if CloudFlare isn't looking at moving
Snorkel42 00:20:03
to another provider right now.
Prasanna Malaiyandi:It reminds me.
W. Curtis Preston:If the Cloudflare CEO wants to come on this podcast.
W. Curtis Preston:he's he or she is more than welcome.
Prasanna Malaiyandi:This kind of reminds me like how bad it could be
Prasanna Malaiyandi:of, I don't know if you both recalled the RSA hack that happened many years
Prasanna Malaiyandi:ago where the root key was compromised.
Prasanna Malaiyandi:Because that's almost what could have happened to Okta, except in the case of
Prasanna Malaiyandi:Okta, there is no hardware fobs, right?
Snorkel42 00:20:29
so again, I think Okta does absolutely deserve some praise here.
Snorkel42 00:20:34
despite giving the super user application a really stupid name,
Snorkel42 00:20:38
this tier two support engineer, didn't have the ability to reset the password
Snorkel42 00:20:42
to something that he would know.
Snorkel42 00:20:43
if that were a scenario, if the attacker could have gone in and made
Snorkel42 00:20:46
the password password, then this would have been a much bigger deal.
W. Curtis Preston:
Speaker:subsequently deactivating MFA.
Snorkel42 00:20:54
yeah, absolutely.
W. Curtis Preston:Change the password to what you want and
W. Curtis Preston:subsequently deactivating MFA.
W. Curtis Preston:You're in.
Snorkel42 00:21:00
Yeah.
Snorkel42 00:21:00
and if you start thinking about how many customers Okta has and what Octa
Snorkel42 00:21:03
actually does and where I'm walking into.
Snorkel42 00:21:08
If they had access to the fed ramp, if they were able to get
Snorkel42 00:21:10
into government systems that way.
Snorkel42 00:21:12
But, Zoom, for example, if they were able to get into the zoom
Snorkel42 00:21:14
Okta page, what applications would they be able to get into?
Snorkel42 00:21:17
I can tell you from my company, it's pretty much domain admin.
Snorkel42 00:21:21
you have access to everything, right?
Snorkel42 00:21:23
definitely kudos to Okta for having those controls.
Snorkel42 00:21:26
And again, I really do praise their security team for
Snorkel42 00:21:29
catching it that quickly.
Snorkel42 00:21:30
that was an excellent detection on their part, especially for
Snorkel42 00:21:32
a third party in Costa Rica.
Snorkel42 00:21:34
Having that kind of logging.
Snorkel42 00:21:35
Fantastic.
Snorkel42 00:21:36
but Yeah.
Snorkel42 00:21:36
to your point, it could have been massive.
W. Curtis Preston:Yeah, this, I do think I, I liked that even though, we
W. Curtis Preston:agree it was a weird, it's a weird name.
W. Curtis Preston:It does appear that.
W. Curtis Preston:That th that there, what they did employ the concept of least privilege, right?
W. Curtis Preston:There's a reason that they have that, that they have the ability for
W. Curtis Preston:a support person to do the password reset because sometimes customers get
W. Curtis Preston:locked out of their accounts there's no other way to do that, but they at
Snorkel42 00:22:10
I need to interrupt because I disagree.
W. Curtis Preston:you disagree?
Snorkel42 00:22:14
I do.
Snorkel42 00:22:15
so here's the thing, right?
Snorkel42 00:22:16
So they had the, these Okta support engineers, or honestly,
Snorkel42 00:22:19
these, what did we say?
Snorkel42 00:22:20
A Sykes?
Snorkel42 00:22:21
I can't remember which company came first, but they, these contractors
Snorkel42 00:22:25
have the ability to reset the passwords of their customers, of
Snorkel42 00:22:30
every user within their customers.
Snorkel42 00:22:32
And if Octa is your identity provider, that is where your accounts live.
Snorkel42 00:22:35
They are the source of truth.
Snorkel42 00:22:37
Then may.
Snorkel42 00:22:40
But if Okta is just your single sign on solution, your SAML solution or
Snorkel42 00:22:43
something along those lines for active directory, do you, if you signed up for
Snorkel42 00:22:48
Okta as a customer, do you expect that there's some third-party company and
Snorkel42 00:22:51
Costa Rica that can reset your active directory passwords of your CEO right now?
Snorkel42 00:22:56
I wouldn't.
Snorkel42 00:22:56
And I would expect them to give me access my admin access back to my dashboard if I
Snorkel42 00:23:01
happen to walk myself out at that point.
Snorkel42 00:23:03
Yeah.
Snorkel42 00:23:04
But down to the individual user level, that certainly caught me off guard.
Snorkel42 00:23:08
I did not expect that.
Prasanna Malaiyandi:Because that should go through your normal it process,
Prasanna Malaiyandi:which is owned by your company and driving it through active directory.
Prasanna Malaiyandi:That way.
Prasanna Malaiyandi:way
Snorkel42 00:23:16
And it calls out another thing that Okta publishes a document
Snorkel42 00:23:20
of their subcontractors, who they use.
Snorkel42 00:23:23
and this company is on that list and our 24/7 customer support.
Snorkel42 00:23:27
And in the notes, it says something along the lines of, they have no data centers.
Snorkel42 00:23:34
They simply have access to our Salesforce and AWS, that's it.
Snorkel42 00:23:38
So if you are doing your due diligence as a customer and doing your vendor reviews
Snorkel42 00:23:42
you're gonna look at their subcontractors.
Snorkel42 00:23:43
You see this and like, all right, I don't care if they have access to Salesforce
Snorkel42 00:23:46
and how AWS that can mean anything.
Snorkel42 00:23:49
I don't think any reasonable human can read that and go, oh, this third-party
Snorkel42 00:23:53
contractor has the ability to reset my active directory passwords.
W. Curtis Preston:Yeah, good point.
W. Curtis Preston:And so I'll take back my comment.
W. Curtis Preston:I forgot about that part.
W. Curtis Preston:and there was a comment again from the former Okta employees or people
W. Curtis Preston:claiming to be former Okta employees.
W. Curtis Preston:And what they said was the practice and the policy is that you do
W. Curtis Preston:not use this power to do that, but that power is still there.
Prasanna Malaiyandi:that's probably the mistake, right?
Prasanna Malaiyandi:That you can't trust people to have the power and not use it.
W. Curtis Preston:you, what you would do, I could see edge
W. Curtis Preston:cases where maybe that's needed.
W. Curtis Preston:I don't, I can't imagine it right now, but let's just say those edge cases
W. Curtis Preston:to me, those would be edge cases and they would require additional MFA.
W. Curtis Preston:For example, if you're going to do the thing that, we don't think
W. Curtis Preston:should normally be done, then that should require a MFA or MPA.
W. Curtis Preston:If you're going to reset, a password that deep, then it should have
W. Curtis Preston:to come from multiple people.
Snorkel42 00:24:50
Yeah.
Snorkel42 00:24:51
I just going to say, if it's an edge case, it would be an escalation.
Snorkel42 00:24:54
right?
Snorkel42 00:24:55
There's one person on a floor.
W. Curtis Preston:But, and so yeah, you're right.
W. Curtis Preston:So in one sense, they did separate, but they didn't, it sounds like you're saying
W. Curtis Preston:they could do, they could have done the least privilege concept a little better.
Snorkel42 00:25:07
Yeah.
Snorkel42 00:25:07
I think more it's and it goes back to exactly what I was railing
Snorkel42 00:25:11
against at the very beginning.
Snorkel42 00:25:13
It's a communication issue that I'm struggling with Okta right
Snorkel42 00:25:16
now, as an Okta customer, I did not know they had that capability.
Snorkel42 00:25:21
And after doing due diligence and I happen to know, but sub-process
Snorkel42 00:25:24
or document, cause I happened to have it on my computer.
Snorkel42 00:25:27
When this happened, I went, wait a minute.
Snorkel42 00:25:29
Was that even disclosed?
Snorkel42 00:25:30
And I went and looked he's huh, no, not at all.
Snorkel42 00:25:33
And the word that really stuck out to me was the simply they simply have the
Snorkel42 00:25:39
ability to access Salesforce and AWS wow.
Snorkel42 00:25:42
so how does a company.
Snorkel42 00:25:43
Who's trying to do the right thing.
Snorkel42 00:25:45
He's trying to do their due diligence, trying to make sure that they're
Snorkel42 00:25:47
onboarding vendors, that aren't going to open them up for security woes.
Snorkel42 00:25:52
What do you do in that situation?
Snorkel42 00:25:54
When a company, as big as Okta is frankly, at least being awfully
Snorkel42 00:26:01
liberal with their definitions.
W. Curtis Preston:Yeah, that's that is, I would feel much better if they
W. Curtis Preston:notified of what happened in January.
W. Curtis Preston:And they said, listen, there was this thing happen.
W. Curtis Preston:I can understand why they might not want to, but I can see, this
W. Curtis Preston:is what, this is what happened.
W. Curtis Preston:We're not sure of the extent we're studying it, et cetera, whatever,
W. Curtis Preston:but just a simple notification.
W. Curtis Preston:And that would allow people to go, just do a quick.
W. Curtis Preston:did anybody change their passwords?
W. Curtis Preston:Did anybody lose her MFA?
W. Curtis Preston:W whatever.
W. Curtis Preston:just let me go as a user go.
W. Curtis Preston:Just do a quick check that, that everything seems fine, but if you're not
W. Curtis Preston:even told that you're not going to go do it, you're not going to go do a check.
Snorkel42 00:26:48
and honestly, it could have been a great success story for
Snorkel42 00:26:51
them, as I've said several times now I have nothing but praise for
Snorkel42 00:26:54
the security team that caught it.
Snorkel42 00:26:56
And if they would've came out in a reasonable time and said, Hey,
Snorkel42 00:26:59
heads up, here's what happened.
Snorkel42 00:27:01
Here's where we detected and stopped and the controls that
Snorkel42 00:27:03
we had in place to realize it.
Snorkel42 00:27:05
And here's what we're going to do to make sure that RDP is an accessible on,
Snorkel42 00:27:08
there's, third-party support, contractors, laptops and things along those lines.
Snorkel42 00:27:12
I think people would have been, wow.
Snorkel42 00:27:14
Yeah, that could have been real bad, but kudos to Okta, they got their fingers
Snorkel42 00:27:18
on the pulse and they know what's up instead this is a PR nightmare for them.
Snorkel42 00:27:23
To what end?
Snorkel42 00:27:23
I don't know.
Snorkel42 00:27:24
I don't know what they were gaining from trying to keep this quiet and then
Snorkel42 00:27:28
doing this frankly, pathetic attempt at wordsmithing, their official message.
Prasanna Malaiyandi:and I think that's the danger because a lot of
Prasanna Malaiyandi:people in the InfoSec community, right?
Prasanna Malaiyandi:You guys know the difference.
Prasanna Malaiyandi:Like you can smell, The lack of transparency, That something is
Prasanna Malaiyandi:fishy, something doesn't sound right.
Snorkel42 00:27:45
right.
Prasanna Malaiyandi:And you lose trust in them.
Snorkel42 00:27:48
Yeah.
Snorkel42 00:27:49
And again, for a company like Okta, that's not a company you can lose
Snorkel42 00:27:51
trust in and it's a shame too.
Snorkel42 00:27:53
It's a shame to see because it would've been such an easy message
Snorkel42 00:27:56
for them to deal with properly.
W. Curtis Preston:Yeah.
W. Curtis Preston:their stock price went from 166 to 145 in the last two days.
W. Curtis Preston:it could have been it's a 10% loss could have been, could have been better.
W. Curtis Preston:and I agree with you.
W. Curtis Preston:I think that companies that are Okta customers.
W. Curtis Preston:are going to reevaluate their, their trust that they have
W. Curtis Preston:placed in this huge company.
W. Curtis Preston:And Okta's, and the thing that really is the Octa's the default, right?
W. Curtis Preston:Okta's everywhere
Snorkel42 00:28:27
yeah.
W. Curtis Preston:and, yeah, that's just.
Prasanna Malaiyandi:you have to have a good reason not to pick Okta.
Prasanna Malaiyandi:Like you won't lose your job for recommending Okta and your company.
Prasanna Malaiyandi:gotten to that stage for Okta, or it was that way for Okta.
Snorkel42 00:28:41
It's to the point where if you are implementing a new product
Snorkel42 00:28:44
and you look up their documentation of how do I implement SAML chances
Snorkel42 00:28:47
are, it's going to say, oh, if you're an Okta customer, just click here
Snorkel42 00:28:49
and they'll have those screenshots.
Snorkel42 00:28:51
And then there's the everyone else.
Prasanna Malaiyandi:Yeah.
W. Curtis Preston:Okta screenshots means something very different now.
Snorkel42 00:28:57
Right?
W. Curtis Preston:Oh, that's tough.
W. Curtis Preston:so w I can, I assume now that the 2.5% of customers have been notified.
Snorkel42 00:29:11
this again goes back to if we take Okta's word for it.
Snorkel42 00:29:16
Yes.
Snorkel42 00:29:16
that, that was the other side of this, of the blog post, right?
Snorkel42 00:29:19
The comment of.
Snorkel42 00:29:20
after we, after doing a thorough analysis over the last 24 hours and scanning, I
Snorkel42 00:29:25
think it was like 125,000 log entries.
Snorkel42 00:29:27
that's a significant number as logs go.
Snorkel42 00:29:29
we now know that 2.5% of our customers were impacted, which again, you take
Snorkel42 00:29:33
just half a step back and you think about that, so for three months you did nothing.
Snorkel42 00:29:38
Then these screenshots came out and in 24 hours you looked at what do
Snorkel42 00:29:42
you think 125,000 logs are to Okta.
Snorkel42 00:29:46
An eighth of their log.
Snorkel42 00:29:47
Yeah.
Snorkel42 00:29:48
It's nothing.
Snorkel42 00:29:48
And it's such a disingenuous comment, too., you look at it like, oh,
Snorkel42 00:29:51
so you had like skilled security engineers looking at those logs.
Snorkel42 00:29:55
No, you ran some greps on them.
Snorkel42 00:29:57
you didn't actually look at them.
W. Curtis Preston:I was going to throw a grep.
W. Curtis Preston:I was going to say they, grepped some stuff.
Snorkel42 00:30:02
like I tell you this, I'm sure the, their CTO isn't the guy
Snorkel42 00:30:06
actually or CSO, I forget what his title is, is actually writing these things.
Snorkel42 00:30:09
I'm sure it's going through all kinds of legal marketing, but man, every
Snorkel42 00:30:13
time he posts it's get the popcorn out.
Snorkel42 00:30:14
Cause he's just keeps making it worse.
W. Curtis Preston:I don't know any advice for Okta customers?
W. Curtis Preston:What is it?
Snorkel42 00:30:18
with regards to this breach, I truly don't think it's
Snorkel42 00:30:22
significant in terms of customer impact.
Snorkel42 00:30:24
always do your IR.
Snorkel42 00:30:25
Always do your due diligence.
Snorkel42 00:30:26
Go look at your logs from that timeframe.
Snorkel42 00:30:28
See if there were any strange resets.
Snorkel42 00:30:31
but I, that breach is not keeping me up at night.
Snorkel42 00:30:35
What my advice is.
Snorkel42 00:30:37
And certainly what I've been doing is contact your Okta reps and make
Snorkel42 00:30:40
a stink and try to drive the message that I'm not concerned about.
Snorkel42 00:30:45
This breach.
Snorkel42 00:30:46
I am concerned about how you handled it and the fact that you waited
Snorkel42 00:30:51
three months and for the attackers to give up the evidence for you to say
Snorkel42 00:30:54
something makes me have to question.
Snorkel42 00:30:57
What else are you sitting on?
Snorkel42 00:30:58
What other security incidents have occurred that you have
Snorkel42 00:31:01
conveniently not reported?
Snorkel42 00:31:04
And I'm not saying there aren't any, I don't know, but that's the problem that I
Snorkel42 00:31:08
don't, I no longer trust them to tell me.
Snorkel42 00:31:11
and I think that's what Okta needs to hear and, be nice to your rep, right?
Snorkel42 00:31:16
Like they're caught up in those too, but make sure you're saying, Hey,
Snorkel42 00:31:18
escalate this to your executive level.
Snorkel42 00:31:20
Cause this is just not an acceptable way for a company like Okta to be off.
Snorkel42 00:31:23
Yeah.
Snorkel42 00:31:23
that's my main comment to all Okta customers is now's the time to take
Snorkel42 00:31:28
off the gloves and raise some stink.
W. Curtis Preston:send a WTF.
Snorkel42 00:31:32
Yeah.
W. Curtis Preston:Your Octa rep for the record.
W. Curtis Preston:It's two months, not three months just throwing that out there.
Snorkel42 00:31:39
End of January.
Snorkel42 00:31:40
It's end of March.
Snorkel42 00:31:41
Okay.
Snorkel42 00:31:42
Fair enough.
W. Curtis Preston:Once I got the math, right.
Prasanna Malaiyandi:I know for once Curtis.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:Curtis math, they're not
W. Curtis Preston:always off by an order of magnitude usually or something I'm
W. Curtis Preston:like, I think that was 10% and was like,
Snorkel42 00:31:55
There's 1%.
Snorkel42 00:31:56
2, 3.
Snorkel42 00:31:57
That's how it works.
W. Curtis Preston:Exactly.
Snorkel42 00:32:02
Fair enough.
Snorkel42 00:32:02
Yeah,
W. Curtis Preston:yeah.
W. Curtis Preston:thanks for coming on to talk about the, the Okta situation.
Snorkel42 00:32:09
it was a pleasure.
Snorkel42 00:32:11
I don't know if that's the right word.
W. Curtis Preston:is never fun is it Prasanna?,
Prasanna Malaiyandi:No, it's not.
Prasanna Malaiyandi:Yeah.
Prasanna Malaiyandi:I hope at least you had a chance to vent snorkel.
Snorkel42 00:32:21
Yeah, it was cathartic.
Snorkel42 00:32:22
Yes.
Snorkel42 00:32:23
Thank you.
W. Curtis Preston:Yeah.
W. Curtis Preston:Yeah, this is not a, one of my favorite words is the German word
W. Curtis Preston:schaudenfreude, which means taking joy in the misfortunes of others.
W. Curtis Preston:this is not that, this is anger, right?
W. Curtis Preston:This is.
W. Curtis Preston:I agree with you.
W. Curtis Preston:Like how is this, it's the whole like, oh, now you're telling us
W. Curtis Preston:after the screenshots came out and would we ever even heard of anything?
Snorkel42 00:32:47
They should have saw it coming, This is what this group does.
Snorkel42 00:32:49
They're an extortion group.
Snorkel42 00:32:50
They stole the screenshots and I would be, I'd be willing to bet money that
Snorkel42 00:32:55
they were holding that up to Okta, send us money, or we're going to disclose.
Snorkel42 00:32:59
I could not have been surprised when those screenshots finally appeared
Snorkel42 00:33:02
the whole thing.
Snorkel42 00:33:03
Just You have to wonder what they were thinking.
Prasanna Malaiyandi:It's, it sounds a little staged if you will, or
Prasanna Malaiyandi:planned, They knew it was coming.
Prasanna Malaiyandi:They weren't paying up or whatever, and
Snorkel42 00:33:13
Which you would think that they would take the other approach and
Snorkel42 00:33:17
let's be the ones to control that message.
Snorkel42 00:33:18
Then let's be the ones to disclose that this happened.
Snorkel42 00:33:20
Cause it really, if we take them at their word, it wasn't that big of a deal.
Snorkel42 00:33:27
They could have controlled that message instead.
Snorkel42 00:33:28
it's a big deal.
W. Curtis Preston:it's not the crime.
W. Curtis Preston:It's the cover up.
Prasanna Malaiyandi:Yeah.
Snorkel42 00:33:33
No.
W. Curtis Preston:same old.
W. Curtis Preston:All right.
W. Curtis Preston:thanks to our listeners and, be sure to subscribe so that you can restore it all.