1
00:00:00,410 --> 00:00:03,802
On this episode of data driven Frank and Andy interview

2
00:00:03,866 --> 00:00:07,306
stephen Oren, the CTO of Intel Federal

3
00:00:07,498 --> 00:00:11,326
yes. Intel, the computer chip company. Because if you want

4
00:00:11,348 --> 00:00:15,182
to train your AI models in a reasonable amount of time, you need better

5
00:00:15,236 --> 00:00:18,974
hardware. Well, it turns out that intel has developed new

6
00:00:19,012 --> 00:00:22,410
CPU instructions to accelerate AI workloads

7
00:00:22,490 --> 00:00:25,670
FPGAs allow for faster development in custom

8
00:00:25,740 --> 00:00:29,458
applications with specific needs. Speaking of intel,

9
00:00:29,554 --> 00:00:33,186
you have to check out an upcoming intel and Red Hat webinar

10
00:00:33,378 --> 00:00:36,854
link in the show notes. Tell them Bailey sent you.

11
00:00:36,972 --> 00:00:38,380
Now on with the show.

12
00:00:41,950 --> 00:00:45,766
Hello and welcome to Data Driven, the podcast where we explore the emergent fields

13
00:00:45,798 --> 00:00:48,906
of data science, data engineering, and of course,

14
00:00:49,008 --> 00:00:52,746
artificial intelligence. As with me, I always have Andy

15
00:00:52,778 --> 00:00:55,920
Leonard, my most favorite data engineer in the world.

16
00:00:57,570 --> 00:01:01,386
And today we have a special guest, Steve Oren, who is the federal

17
00:01:01,418 --> 00:01:04,734
CTO of intel. Yes, that's right, intel, the chip

18
00:01:04,782 --> 00:01:08,050
company. And although they do a lot more stuff

19
00:01:08,120 --> 00:01:11,694
now. So welcome to the show, Steve.

20
00:01:11,822 --> 00:01:15,326
Thank you and glad to be here. Frank and Andy cool.

21
00:01:15,448 --> 00:01:19,222
So one of the things that I think people have not realized, people

22
00:01:19,276 --> 00:01:22,902
think that AI is a software story, right?

23
00:01:22,956 --> 00:01:26,200
Primarily. But quickly, once you get into it,

24
00:01:27,130 --> 00:01:30,582
everyone goes gaga for things like Chat GPT or

25
00:01:30,716 --> 00:01:34,266
well, no one's really gone gaga for Barred just yet. We're going to give that

26
00:01:34,288 --> 00:01:37,660
a few more time for the paint to dry on that.

27
00:01:38,670 --> 00:01:42,302
But quickly, I think when people start

28
00:01:42,356 --> 00:01:46,014
becoming builders of AI tools, the

29
00:01:46,052 --> 00:01:49,886
number one restriction, aside from kind of what your data engineering pipeline looks

30
00:01:49,908 --> 00:01:53,742
like, is how quick you can train these models. And

31
00:01:53,796 --> 00:01:57,060
obviously, I'm pretty sure intel has a thing or two to say about

32
00:01:57,990 --> 00:02:01,726
hardware. Absolutely. And as you've as you've

33
00:02:01,758 --> 00:02:05,330
alluded to AI, and all the things that make up

34
00:02:05,400 --> 00:02:09,086
AI rely heavily on the infrastructure that you're

35
00:02:09,118 --> 00:02:12,838
training you're inferencing. But even before you get to the fun stuff, how do you

36
00:02:12,844 --> 00:02:16,534
do the data curation? How do you suck in the data? The ingestion get the

37
00:02:16,572 --> 00:02:20,360
large multi node data sets that these large language models are

38
00:02:20,990 --> 00:02:24,666
trained against. There's a lot of hardware and infrastructure that has to make

39
00:02:24,688 --> 00:02:28,426
that happen. And then when you get to the important phase with how do

40
00:02:28,448 --> 00:02:32,266
you train those in a timely fashion, hardware is

41
00:02:32,288 --> 00:02:35,834
the answer. And what we're seeing in a lot of these spaces,

42
00:02:35,882 --> 00:02:39,546
especially we start looking at things like large language models and transformers

43
00:02:39,658 --> 00:02:43,438
as well as looking at other approaches that are coming out,

44
00:02:43,604 --> 00:02:46,782
is that not only does the hardware matter, but the type of hardware

45
00:02:46,846 --> 00:02:50,450
matters. If you think about it, it's not a one size

46
00:02:50,520 --> 00:02:54,034
fits all. It's a heterogeneous architecture to make sure you have the right

47
00:02:54,072 --> 00:02:57,638
hardware for your workload. One great example. So

48
00:02:57,724 --> 00:03:01,542
large language models in graph analytics requires not just

49
00:03:01,596 --> 00:03:05,266
heavy duty hardware but the right memory architecture to keep those nodes

50
00:03:05,298 --> 00:03:08,550
in place while you're training. And what you find is that often

51
00:03:08,620 --> 00:03:12,294
doesn't fit well. Intel just a classic GPU only kind of mode,

52
00:03:12,342 --> 00:03:16,154
which is what the classic AIS leveraged, just the sheer number

53
00:03:16,192 --> 00:03:19,562
of cores that you would have in a GPU. And so what we're seeing is

54
00:03:19,616 --> 00:03:23,434
optimizing the hardware for the kind of workload is the answer to getting

55
00:03:23,472 --> 00:03:27,038
timely training. And especially when you start doing more. That sort of iterative. And

56
00:03:27,044 --> 00:03:30,446
feedback training, it's not a one and done, it's an ongoing process. So you need

57
00:03:30,468 --> 00:03:34,094
that to be quick enough and powerful enough and robust enough to handle those

58
00:03:34,132 --> 00:03:37,826
workloads. And then the other side where hardware really starts to matter is on the

59
00:03:37,848 --> 00:03:41,614
inferencing, you want to be able to ask the question and get a response fairly

60
00:03:41,662 --> 00:03:45,198
quickly, if not near real time. If you're in a car and it's

61
00:03:45,294 --> 00:03:48,386
autonomous driving, you want it real time. You want to know that's a tree and

62
00:03:48,408 --> 00:03:52,246
not a shadow. If you're talking about online and doing some

63
00:03:52,268 --> 00:03:54,966
fun stuff with chat GBT, you still don't want to wait 20 minutes for your

64
00:03:54,988 --> 00:03:58,726
response. And so inferencing matters, training matters, and

65
00:03:58,748 --> 00:04:02,230
the kind of hardware and infrastructure that support it. And that's why intel

66
00:04:02,390 --> 00:04:05,578
and our ecosystem are looking at providing a

67
00:04:05,584 --> 00:04:09,418
heterogeneous set of architectures. So our classic CPU, so the Xeon and

68
00:04:09,424 --> 00:04:13,226
the server and CPU and the client core, but also FPGA

69
00:04:13,258 --> 00:04:17,006
based logic AI accelerators like our Habana chips in

70
00:04:17,028 --> 00:04:20,666
the cloud and our targeted edge AI

71
00:04:20,698 --> 00:04:24,286
chips like Movidius for video processing and the like. But then

72
00:04:24,388 --> 00:04:27,874
really, besides the hardware, it's that software infrastructure layer. How do you

73
00:04:27,912 --> 00:04:31,646
optimize your code? Because most AI developers are not hardware

74
00:04:31,678 --> 00:04:35,266
experts, nor do I want them necessarily to be. So a lot of it is

75
00:04:35,288 --> 00:04:39,130
about building out those abstraction layers that optimize your code, that's

76
00:04:39,150 --> 00:04:42,998
doing your hugging face or whatever, to take full advantage of the

77
00:04:43,004 --> 00:04:46,726
hardware underneath you, without you having to know what hardware is underneath you so

78
00:04:46,748 --> 00:04:49,898
that you can provision your workload where it needs to go and not have to

79
00:04:49,904 --> 00:04:53,690
worry about the hardware infrastructure. And that's part of our overall strategy. And working

80
00:04:53,760 --> 00:04:57,340
with the broader ecosystem, the open source community, the

81
00:04:57,950 --> 00:05:01,706
commercial providers, and the software frameworks to give them the

82
00:05:01,728 --> 00:05:05,070
tools to get the best performance out of their AI and their

83
00:05:05,220 --> 00:05:09,038
data science, right? And I think you hit the nail on the head. I

84
00:05:09,044 --> 00:05:12,582
think we're at an inflection point. Not so much in engineering,

85
00:05:12,746 --> 00:05:16,580
right, but more in the perception, right? Because whenever you think, oh,

86
00:05:20,150 --> 00:05:23,906
we have a large workload we got to do, let's throw some GPU at

87
00:05:23,928 --> 00:05:27,686
it, right? And it's a little more nuanced than that. I think

88
00:05:27,708 --> 00:05:31,478
people are finding out that you need more than just a

89
00:05:31,484 --> 00:05:35,240
bunch of GPU. And I was on a call

90
00:05:35,610 --> 00:05:39,382
and I want to get your thoughts on this, because he said something very similar

91
00:05:39,436 --> 00:05:43,126
to what you said. You ever have these moments

92
00:05:43,158 --> 00:05:46,666
when you're on a call and somebody smart says something, you're like, I don't know

93
00:05:46,688 --> 00:05:50,426
about that, right? And it's kind of like what they did

94
00:05:50,448 --> 00:05:54,286
in World War Z and where there was like the 10th Man Rule, where no

95
00:05:54,308 --> 00:05:58,046
matter how ridiculous it sounds at first, you kind of want to

96
00:05:58,068 --> 00:06:01,886
investigate it. And that's why I was glad when your

97
00:06:01,908 --> 00:06:04,306
name popped up in the feed because I'm like, yeah, I want to talk to

98
00:06:04,328 --> 00:06:06,900
you about this. Because he was basically saying that

99
00:06:08,230 --> 00:06:11,426
GPU usage is

100
00:06:11,448 --> 00:06:15,140
overrated and that where the real advantage is going to be

101
00:06:16,070 --> 00:06:19,430
is going to be in software acceleration and on

102
00:06:19,500 --> 00:06:23,190
CPU kind of optimization too, which sounds

103
00:06:23,260 --> 00:06:26,806
a lot like what you said. And when I first heard that, my first thought

104
00:06:26,828 --> 00:06:30,646
was, I don't know about that, but this guy's plugged in. He's a

105
00:06:30,668 --> 00:06:34,506
big shot at Red Hat, right? He's plugged in, he knows a lot. And I

106
00:06:34,528 --> 00:06:38,074
was like, I didn't want to just dismiss that. Like, if my cousin said that,

107
00:06:38,112 --> 00:06:41,340
I'd be like, yeah, okay, but if this guy says it,

108
00:06:42,030 --> 00:06:45,834
whether or not he's right, maybe yet to be determined,

109
00:06:45,882 --> 00:06:49,662
but the fact that he believes it means that there's a trail there to follow.

110
00:06:49,716 --> 00:06:53,486
So I've been kind of poking around at stuff. Tell me

111
00:06:53,508 --> 00:06:57,146
about that. It sounds like there's some weight

112
00:06:57,178 --> 00:07:00,978
behind that opinion. So Frankie, you hit it on

113
00:07:00,984 --> 00:07:04,654
the head there. It's not that GPUs aren't important, it's just GPUs

114
00:07:04,702 --> 00:07:08,546
aren't the only and best solution for all aspects of AI. And there are

115
00:07:08,568 --> 00:07:12,054
certain vendors that want, again, for a variety of reasons, want GPU to be the

116
00:07:12,092 --> 00:07:15,542
foundation for all of your AI activities. Like if you're a GPU based

117
00:07:15,596 --> 00:07:19,078
hardware company. Exactly makes sense. But

118
00:07:19,244 --> 00:07:22,218
when you actually go look at the benchmarks across multiple and here's the key thing,

119
00:07:22,224 --> 00:07:25,610
across multiple AI types. So different

120
00:07:25,760 --> 00:07:29,498
algorithmic models as well as the flow, so there's different stages. So the

121
00:07:29,504 --> 00:07:33,078
inference versus training, ingestion and curation

122
00:07:33,174 --> 00:07:36,846
versus the training, versus the feedback training, what you'll find is

123
00:07:36,868 --> 00:07:40,414
that GPUs will rock for certain things and they are important for certain things,

124
00:07:40,452 --> 00:07:43,646
both from that vendor as well as from a variety of other vendors. GPUs do

125
00:07:43,668 --> 00:07:47,518
play a key role, but when you look at the breadth of AI activities

126
00:07:47,614 --> 00:07:51,234
and the benchmarks associated, you actually find that a lot of really

127
00:07:51,272 --> 00:07:54,994
good work just happens on standard commercial off the shelf CPU. And

128
00:07:55,032 --> 00:07:58,774
actually most of the inferencing, I mean, we're talking in the 70% to 80%

129
00:07:58,812 --> 00:08:02,226
of inferencing happens best on CPU

130
00:08:02,338 --> 00:08:06,006
and areas like large language model and graph analytic based

131
00:08:06,108 --> 00:08:09,222
approaches. The numbers really show very

132
00:08:09,276 --> 00:08:12,970
clearly that it's not a core bound problem,

133
00:08:13,040 --> 00:08:16,746
it's a memory bound problem. And so having efficient in

134
00:08:16,768 --> 00:08:20,118
and out of memory, which is what you get from a CPU or an accelerator

135
00:08:20,214 --> 00:08:23,914
with ample memory on board, is actually much more powerful

136
00:08:23,962 --> 00:08:27,566
for training those types of data sets because the GPU you're dealing with that

137
00:08:27,588 --> 00:08:31,406
latency across the bus. And that actually starts to matter when you're

138
00:08:31,428 --> 00:08:34,414
talking about billions or trillions of node graph

139
00:08:34,462 --> 00:08:38,174
analytics. So I wouldn't say that GPUs

140
00:08:38,222 --> 00:08:41,906
are a dying breed. That is absolutely not the case. And there's going to be

141
00:08:41,928 --> 00:08:45,474
a huge market for GPUs or GPU like

142
00:08:45,512 --> 00:08:48,198
functionality. I want to be careful about that because you don't have to have a

143
00:08:48,204 --> 00:08:51,894
discrete card. The reality is you can have GPU capabilities embedded in your

144
00:08:51,932 --> 00:08:54,854
processor. We've already seen from intel and from other

145
00:08:54,892 --> 00:08:58,642
architectures. The real interesting thing is making sure that

146
00:08:58,716 --> 00:09:02,422
whatever your workload is can be optimized, like your friend said, optimized

147
00:09:02,486 --> 00:09:06,074
through software to that hardware. So that if you are

148
00:09:06,112 --> 00:09:09,850
running a large language model, that you're actually

149
00:09:09,920 --> 00:09:13,646
running it on the right hardware, and that the hardware and your software know how

150
00:09:13,668 --> 00:09:17,214
to work together to give you the best performance if you're working

151
00:09:17,252 --> 00:09:21,038
on. I'm seeing a lot of really cool things right now around graph based

152
00:09:21,124 --> 00:09:24,622
approaches in the memory intensive side of that

153
00:09:24,676 --> 00:09:27,922
and the switching back and forth between that. Those

154
00:09:27,976 --> 00:09:31,714
latencies can really come to bear when you're talking about cross bus

155
00:09:31,832 --> 00:09:35,666
kind of communication. So having high amount of memory available directly to

156
00:09:35,688 --> 00:09:38,910
the CPU to be able to do those training, keep all that data in flight

157
00:09:38,990 --> 00:09:41,990
so you can train, is going to be one of the key

158
00:09:42,060 --> 00:09:45,766
differentiators of how you can take those large angle models, apply them to

159
00:09:45,788 --> 00:09:48,870
more than just writing cool essays by Shakespeare.

160
00:09:49,790 --> 00:09:53,514
I think what we're going to see is things like chat, GPT, and that whole

161
00:09:53,552 --> 00:09:57,274
category of transformer based approaches applied to just about everything, not

162
00:09:57,312 --> 00:10:00,166
just chat, but prediction

163
00:10:00,278 --> 00:10:04,062
approaches. And it's really about getting it the training sets to become

164
00:10:04,116 --> 00:10:07,754
smart on those very vertical domains.

165
00:10:07,882 --> 00:10:11,418
That's going to be a resource intensive process and it's not going to be throwing

166
00:10:11,434 --> 00:10:14,606
a bunch of GPU or it's going to be a lot of cloud scaling and

167
00:10:14,628 --> 00:10:18,050
it's going to be a lot of memory intensive activities. And like your friend

168
00:10:18,120 --> 00:10:21,966
highlighted, the software is going to really matter, that it's taking full advantage

169
00:10:21,998 --> 00:10:25,826
of the hardware to get you those performance report. Well, this reminds me a

170
00:10:25,848 --> 00:10:29,350
lot of just patterns I've seen over the decades of

171
00:10:29,420 --> 00:10:33,170
being in computing as a hobbyist and then a profession

172
00:10:33,330 --> 00:10:36,982
is you see a lot of things come into the

173
00:10:37,036 --> 00:10:40,710
fore as being very monolithic, and then people

174
00:10:40,780 --> 00:10:44,390
realize, wait, that's really a team effort.

175
00:10:44,470 --> 00:10:47,146
And I think about it as a baseball team, right? You don't want to put

176
00:10:47,168 --> 00:10:50,734
the pitcher, the person who's skilled at pitching in center field, can they

177
00:10:50,772 --> 00:10:54,302
perform there? Well, gosh, yeah, but you're wasting them,

178
00:10:54,356 --> 00:10:57,982
right? They are tuned their whole body, their

179
00:10:58,036 --> 00:11:01,674
desires, their motivations. They love being pitchers.

180
00:11:01,722 --> 00:11:05,386
So put that person on the pitchers mound and you see this

181
00:11:05,428 --> 00:11:09,026
happen. And it's in all sorts of places. We saw it, frank and I have

182
00:11:09,048 --> 00:11:12,802
seen it over the years when the unicorns were the big

183
00:11:12,936 --> 00:11:16,774
deal, the data science unicorns who could do data engineering and everything

184
00:11:16,812 --> 00:11:20,550
that we've kind of broken out now into other fields.

185
00:11:21,050 --> 00:11:24,850
And we're seeing it now in the hardware

186
00:11:25,010 --> 00:11:28,646
and in the distribution of the separation of

187
00:11:28,668 --> 00:11:32,506
concerns and the distribution of concerns, getting every component to do

188
00:11:32,528 --> 00:11:36,042
what it's best at. And along with that, and I'll shut up after

189
00:11:36,096 --> 00:11:39,706
this, is this whole idea that it's moving so

190
00:11:39,808 --> 00:11:43,646
fast that the hardware that's going to perform

191
00:11:43,748 --> 00:11:47,450
the task first sometimes isn't even identified

192
00:11:47,530 --> 00:11:51,326
yet because some new approach popped into the equation. If

193
00:11:51,348 --> 00:11:54,986
somebody tested something and went, this is great. Now whether I run it

194
00:11:55,108 --> 00:11:58,866
and you just see that and it's on a scale now where it

195
00:11:58,888 --> 00:12:02,642
used to be measured in years and moved to months, it's now weeks

196
00:12:02,696 --> 00:12:06,406
and sometimes days. It's just amazing how fast this

197
00:12:06,428 --> 00:12:10,258
is going. And not that long ago, people were predicting

198
00:12:10,434 --> 00:12:12,600
an AI intel. Right.

199
00:12:14,810 --> 00:12:18,466
I think Dolly kind of and the whole generative artwork

200
00:12:18,578 --> 00:12:21,866
stuff, I think kind of like, wait a minute, there's something here. Then Dolly came

201
00:12:21,888 --> 00:12:25,446
out and then OpenAI did the one two punch of here's

202
00:12:25,478 --> 00:12:29,322
Dolly a couple of months later, here's Chachi BT. Now you're just seeing like

203
00:12:29,376 --> 00:12:32,746
it's on fire. Like it's not just AI summer, it's an AI heat

204
00:12:32,778 --> 00:12:36,606
wave. Yeah, exactly. It is. It's a full El

205
00:12:36,628 --> 00:12:40,318
Nino. I like that. That's the

206
00:12:40,324 --> 00:12:41,520
quotable, for sure.

207
00:12:44,550 --> 00:12:48,100
I think one of the things I think people realized is,

208
00:12:49,030 --> 00:12:52,734
and a lot of the thinking was that AI

209
00:12:52,782 --> 00:12:56,434
winter was coming because we're hitting processor or

210
00:12:56,472 --> 00:13:00,226
hardware kind of upper barriers. And I think we're

211
00:13:00,258 --> 00:13:03,878
finding out, I think much to what you said is that it's not just about

212
00:13:04,044 --> 00:13:07,510
throw this many GPUs at it. It's right. The entire story, the entire

213
00:13:07,580 --> 00:13:11,418
bus matters. Right. So the shortstop matters using the

214
00:13:11,424 --> 00:13:15,274
baseball analogy. Right. The outfielders. Right. You can't really win

215
00:13:15,312 --> 00:13:18,874
a lot of baseball games if not everybody on the team is

216
00:13:18,912 --> 00:13:22,746
playing at their best. Absolutely. And just to take that metaphor

217
00:13:22,778 --> 00:13:26,334
all the way, the turf matters, too. The infrastructure that you're running

218
00:13:26,372 --> 00:13:29,454
those specialists on, you're going to play better in different

219
00:13:29,492 --> 00:13:32,960
fields. That's true. That's a good point.

220
00:13:33,890 --> 00:13:37,060
I love that you took the metaphor to the next level. That's awesome.

221
00:13:39,750 --> 00:13:42,994
I think you mentioned whether it was in the virtual green room or here something

222
00:13:43,032 --> 00:13:46,454
called habanero. And I know you're not talking about just

223
00:13:46,652 --> 00:13:50,018
cooking. Right. Spicy habana. Yes, habana. I'm

224
00:13:50,034 --> 00:13:53,574
sorry. I had food on my mind, as is

225
00:13:53,612 --> 00:13:57,298
often. What is habana? Because I've

226
00:13:57,314 --> 00:14:00,818
heard whispers of it. I know we're recording this middle of

227
00:14:00,844 --> 00:14:04,534
May. There's going to be some announcements at the Red Hat Summit. Well, they'll

228
00:14:04,582 --> 00:14:07,946
probably already happen by the time this goes live, but what is

229
00:14:07,968 --> 00:14:11,266
it? So Havana is an architecture, an AI

230
00:14:11,318 --> 00:14:14,922
accelerator, and it's a specialty chips specifically

231
00:14:14,986 --> 00:14:18,414
designed for accelerating AI. And it's actually two

232
00:14:18,452 --> 00:14:22,286
chips. And the reason it's two chips is that you want, again, going back

233
00:14:22,308 --> 00:14:25,470
to what we were talking about, you want the right hardware for the AI workload.

234
00:14:25,630 --> 00:14:28,754
So you want to be able to have the right hardware to opt optimized for

235
00:14:28,792 --> 00:14:32,222
training flows and a separate set of hardware

236
00:14:32,286 --> 00:14:35,902
for cloud scale and hyperscale inferencing

237
00:14:35,966 --> 00:14:39,794
workloads. And so that's actually what Habana is. It's a two

238
00:14:39,832 --> 00:14:43,574
chip strategy. So habana gowdy which is out available.

239
00:14:43,772 --> 00:14:47,606
V two is available. V One has been out for some time. If

240
00:14:47,628 --> 00:14:51,018
you go to the Amazon cloud, you can get it today. It's also available in

241
00:14:51,024 --> 00:14:54,406
data centers, and a lot of universities have them in their high performance computing

242
00:14:54,438 --> 00:14:58,102
environments. And it's geared to doing that sort of scale,

243
00:14:58,246 --> 00:15:01,500
large data set training that you would find

244
00:15:03,070 --> 00:15:06,846
whether it be in a cloud kind of environment, a chat GPT level

245
00:15:06,948 --> 00:15:10,682
of analytic, or in the case of high performance computing.

246
00:15:10,746 --> 00:15:14,462
Whether you're doing climate modeling or flow dynamics, those kind of big

247
00:15:14,516 --> 00:15:18,354
training model sets that you want to be able to do at scale. And

248
00:15:18,392 --> 00:15:22,174
what's nice about it is that like your cloud scale, it scales with your architecture.

249
00:15:22,222 --> 00:15:25,954
So it allows you to be able to scale up your training based on

250
00:15:25,992 --> 00:15:29,398
the compute needs with an AI accelerator specifically tuned to

251
00:15:29,404 --> 00:15:33,154
that. The other chip, the Goya chip, is an inferencing

252
00:15:33,202 --> 00:15:37,014
chip. So it's again tuned for that inference. But the reason,

253
00:15:37,132 --> 00:15:40,794
again, this is for high end cloud scale hyperscale or things like high

254
00:15:40,832 --> 00:15:44,506
speed training, where you want to be able to do large amount of inference in

255
00:15:44,528 --> 00:15:47,754
as near or close to real time as possible against really

256
00:15:47,792 --> 00:15:51,546
complex kind of data flows that you're trying to do

257
00:15:51,568 --> 00:15:55,014
the analysis of. And again, looking at the right

258
00:15:55,072 --> 00:15:58,446
hardware, we wanted to make sure to not just meet what we call the sort

259
00:15:58,468 --> 00:16:02,286
of the normal scale. So the kind of things you would interact with when

260
00:16:02,308 --> 00:16:05,514
you're going to do fraud detection, but you also want to be able to handle

261
00:16:05,562 --> 00:16:09,202
really large scale inferencing because you're dealing with ingestion of multi data

262
00:16:09,256 --> 00:16:12,674
sets across multiple different domains and having to be able to do that

263
00:16:12,712 --> 00:16:16,178
inferencing in a streaming kind of mode. And that's really where the Goya chip

264
00:16:16,274 --> 00:16:20,114
shines, is an inferencing platform that can scale

265
00:16:20,162 --> 00:16:23,766
with the cloud. And that's really the Habana strategy is about giving you the

266
00:16:23,788 --> 00:16:27,606
hyperscalers and high performance computing, the equivalent of

267
00:16:27,628 --> 00:16:31,470
an AI custom chips. And that's really where Habana

268
00:16:31,490 --> 00:16:35,114
sits. And then when you look at sort of the majority of what most

269
00:16:35,152 --> 00:16:38,938
people will leverage in a cloud or on prem, what we've been

270
00:16:38,944 --> 00:16:42,718
doing there is adding new instructions to the CPU. So

271
00:16:42,804 --> 00:16:46,462
VNNI was the first really big one in AVX 512,

272
00:16:46,596 --> 00:16:49,534
which really accelerates the math that you're doing behind

273
00:16:49,652 --> 00:16:53,022
inferencing and training and give you those

274
00:16:53,076 --> 00:16:56,446
instructions. That software, whether it be Intel's OpenVINO software

275
00:16:56,558 --> 00:17:00,226
or TensorFlow or other frameworks can take advantage of

276
00:17:00,248 --> 00:17:03,998
that math to use hardware offload to accelerate the math that you're

277
00:17:04,014 --> 00:17:07,726
doing in your training and your inferencing workloads for most of your normal

278
00:17:07,758 --> 00:17:11,074
kind of AI. A lot of the AI we deal with, not the high performance

279
00:17:11,122 --> 00:17:14,326
computing style. And so you get the balance. And again, it goes back to what

280
00:17:14,348 --> 00:17:17,974
we talked about in the beginning, the right compute for the right AI. We've also

281
00:17:18,012 --> 00:17:21,610
introduced data center graphics because again, there are workloads that absolutely

282
00:17:21,680 --> 00:17:25,500
make sense for a GPU besides fun gaming. And

283
00:17:25,870 --> 00:17:29,254
that's really where you'll see GPU shine on, those kind of specialty

284
00:17:29,302 --> 00:17:32,858
workloads that take full advantage. And a lot of the deep learning object

285
00:17:32,944 --> 00:17:36,734
recognition ones work well on GPUs. They actually work well on other

286
00:17:36,772 --> 00:17:40,426
kind of platforms as well. And one of the things we're seeing in the Edge

287
00:17:40,538 --> 00:17:44,334
is a shift towards more customized approaches, whether that be using

288
00:17:44,372 --> 00:17:48,114
an FPGA as sort of a hardware platform that you can code

289
00:17:48,152 --> 00:17:51,694
in your algorithms to do inline inferencing, do feedback loop

290
00:17:51,742 --> 00:17:55,250
training. And you see this a lot of times in the image processing, video

291
00:17:55,320 --> 00:17:59,094
processing side, also in the signals processing. So whether it's five

292
00:17:59,132 --> 00:18:02,914
G and being able to do signal quality testing or signal acquisition

293
00:18:02,962 --> 00:18:06,594
and being able to do RF signal analysis, FPGAs

294
00:18:06,642 --> 00:18:10,258
actually really shine for that kind of workload. Where you want to put in your

295
00:18:10,284 --> 00:18:14,074
custom algorithm that you're going to actually test against or

296
00:18:14,112 --> 00:18:17,850
use as part of your conditioning. And then we get to the idea

297
00:18:17,920 --> 00:18:21,758
of what we call an ASIC. And that's where you know your workload, you

298
00:18:21,764 --> 00:18:24,734
know you're going to be doing this kind of inference. You can actually code that

299
00:18:24,772 --> 00:18:28,510
into a custom chip that will do just

300
00:18:28,660 --> 00:18:32,206
audio AI inferencing or

301
00:18:32,228 --> 00:18:36,062
do certain aspects of video coded. And this way you get the most

302
00:18:36,116 --> 00:18:39,826
performance in a low swap. And that's the idea here

303
00:18:39,848 --> 00:18:42,466
is you want to be able to handle everything from the pointy end of the

304
00:18:42,488 --> 00:18:46,306
spear, the Edge sensor and give it the ability to do AI as

305
00:18:46,328 --> 00:18:49,286
opposed to waiting for it to send the data to the cloud and get a

306
00:18:49,308 --> 00:18:52,294
decision. You want to be able to give it something, but it also has to

307
00:18:52,332 --> 00:18:56,166
operate at the size, weight and power that

308
00:18:56,188 --> 00:18:59,802
you'd expect from an Edge sensor. You obviously don't have a data center power

309
00:18:59,856 --> 00:19:03,706
system for your car, for your drone, or for

310
00:19:03,728 --> 00:19:07,498
your camera on the streetlight. Right. That would be a very heavy to

311
00:19:07,504 --> 00:19:11,146
fly that drone. That's okay.

312
00:19:11,248 --> 00:19:14,846
I'm curious how you kind of manage what

313
00:19:14,948 --> 00:19:18,602
I'm just going to make up words here, but like an innovation chain,

314
00:19:18,666 --> 00:19:21,840
I'm thinking about like supply chain management. And I know

315
00:19:22,450 --> 00:19:26,098
I've got experience in electronics engineering, and I

316
00:19:26,104 --> 00:19:29,842
know some of how much it takes to go into mind you my

317
00:19:29,896 --> 00:19:33,474
work was decades old, but this whole idea of getting

318
00:19:33,512 --> 00:19:37,206
ahead of the curve or at least being able to predict where the

319
00:19:37,228 --> 00:19:40,950
curve is going and how steep and when. That

320
00:19:41,020 --> 00:19:44,726
sounds like a huge challenge for figuring out what

321
00:19:44,748 --> 00:19:48,566
will be needed next. So what you're talking

322
00:19:48,588 --> 00:19:52,342
about is how does a company that's building out both the hardware and the infrastructure,

323
00:19:52,406 --> 00:19:56,246
stay ahead of, like you said, the week to week turnaround

324
00:19:56,278 --> 00:19:59,754
in the AI world. Part of that is having a diverse team

325
00:19:59,792 --> 00:20:03,514
of specialists. So the Intel Labs,

326
00:20:03,562 --> 00:20:07,390
which is our team that looks five to ten years out, is over 1000

327
00:20:07,460 --> 00:20:10,686
people who full time looking at process node technology,

328
00:20:10,868 --> 00:20:14,666
security, AI data science. They're across multiple domains

329
00:20:14,698 --> 00:20:17,790
and within each domain we have specialists in different areas.

330
00:20:18,370 --> 00:20:22,114
One of the really I'll give you a great example. Before Chat GPT blew up,

331
00:20:22,232 --> 00:20:25,938
I had two different of my AI specialists, one on the

332
00:20:25,944 --> 00:20:29,346
government side and one on the performance side. Start talking to me about this thing

333
00:20:29,368 --> 00:20:32,726
called Transformer. Like, oh, there's this really cool thing that we're seeing here, it's called

334
00:20:32,748 --> 00:20:36,358
a Transformer. And I'm like, okay, that's interesting, and tell me more. And they explain

335
00:20:36,444 --> 00:20:40,086
sort of how it worked. And then fast forward, six months later,

336
00:20:40,188 --> 00:20:43,386
Chat chips BT shows up and I'm like, I know what that is because that

337
00:20:43,408 --> 00:20:47,242
has the word Transformer. I've seen this. And again, it's about giving

338
00:20:47,296 --> 00:20:50,778
your people the ability to go out and look. I think one of the

339
00:20:50,784 --> 00:20:54,382
advantages of being at intel, and it's really why I've been here so long,

340
00:20:54,516 --> 00:20:58,094
is everyone knows intel inside.

341
00:20:58,212 --> 00:21:01,886
But there's something to that. Our chips are inside the

342
00:21:01,908 --> 00:21:05,742
edge. Clients are inside the financial services, healthcare,

343
00:21:05,886 --> 00:21:09,714
manufacturing, oil and gas. They're in the government system, they're in the cloud,

344
00:21:09,832 --> 00:21:13,650
we're in the network. Which means we see workloads both current

345
00:21:13,720 --> 00:21:17,314
and coming from all those different domains. So in some

346
00:21:17,352 --> 00:21:20,546
respects we're on the cutting edge because we're seeing what people do because they come

347
00:21:20,568 --> 00:21:23,458
to us, say, hey, I've got this software, I want to optimize on your hardware.

348
00:21:23,554 --> 00:21:26,546
What does it do? Well, it does blah, blah blah blah. I'm like, okay, let's

349
00:21:26,578 --> 00:21:30,130
help you. And then eventually that becomes open AI.

350
00:21:30,210 --> 00:21:34,042
That's the kind of thing because ultimately every startup, every big company

351
00:21:34,096 --> 00:21:37,418
wants to get the most out of their software and our teams. And one of

352
00:21:37,424 --> 00:21:41,190
the things people don't realize is intel has over 19,000 software engineers

353
00:21:41,350 --> 00:21:44,526
and a large majority of those do you know, they really divide up into three

354
00:21:44,548 --> 00:21:47,754
areas sort of research and pathfinding, ecosystem

355
00:21:47,802 --> 00:21:51,454
enabling, and then software development for

356
00:21:51,572 --> 00:21:55,406
compilers, software services, software tools. That ecosystem enabling team

357
00:21:55,428 --> 00:21:58,562
is a very robust team, it's been around for a very long time. Whose job

358
00:21:58,616 --> 00:22:02,222
is to make Microsoft Windows rock on intel, make Oracle

359
00:22:02,286 --> 00:22:05,954
rock on intel, make red hat rock on intel, make open source. We have

360
00:22:05,992 --> 00:22:09,714
over 1000 open source software developers whose full time job is committing

361
00:22:09,762 --> 00:22:13,474
to open source. We're actually one of the largest committers to open source

362
00:22:13,522 --> 00:22:17,154
community and a lot of what they do is build the optimized

363
00:22:17,202 --> 00:22:20,694
version of those Linux kernel libraries or to

364
00:22:20,892 --> 00:22:24,726
that AI model running on intel and give it away and open source

365
00:22:24,758 --> 00:22:28,454
it. We've created whole model zoos optimized for the variety of intel

366
00:22:28,502 --> 00:22:31,866
architecture because we know if you can run it best on intel, you will run

367
00:22:31,888 --> 00:22:35,530
it, and that consumes resources. We like that. But ultimately

368
00:22:35,610 --> 00:22:39,214
it gives us they call them bell cows, if you will.

369
00:22:39,332 --> 00:22:42,766
We're seeing those bell cows of what's coming next because they come to us and

370
00:22:42,788 --> 00:22:46,618
they say, hey, help us. And very few see us as competition because

371
00:22:46,644 --> 00:22:50,018
we're not going to go build the Chat GPT. We're not going to build a

372
00:22:50,024 --> 00:22:53,262
new operating system or a new sort of predictive maintenance

373
00:22:53,326 --> 00:22:57,026
solution. We're going to give you the architecture for you to run it

374
00:22:57,048 --> 00:23:00,486
best. And even our OEM, whether you buy from Dell or

375
00:23:00,508 --> 00:23:04,082
HP or from Lenovo, we don't care. You're buying intel hardware

376
00:23:04,226 --> 00:23:07,778
inside. And so let's help you take the best advantage of those platforms. And that's

377
00:23:07,794 --> 00:23:11,634
really been the approach from intel, is we want everyone's software

378
00:23:11,682 --> 00:23:15,494
to work. And even with the GPU vendors, they still run on a CPU

379
00:23:15,542 --> 00:23:19,226
platform. And so we want to make sure that that code runs best. So that,

380
00:23:19,328 --> 00:23:23,114
again, you're driving the overall consumption. We raise the bar for everybody. We

381
00:23:23,152 --> 00:23:26,798
raise the bar for everybody. Nice. Yeah. I

382
00:23:26,804 --> 00:23:30,286
think there's a lot to unpack there. Right. And I think one of the things

383
00:23:30,308 --> 00:23:33,698
you brought out, which is something that people don't, I don't think people have

384
00:23:33,784 --> 00:23:37,380
widely realized yet that Edge is probably going to be the next

385
00:23:37,750 --> 00:23:40,978
frontier in just

386
00:23:41,064 --> 00:23:44,802
computing. Right. Obviously the last ten years have all been about cloud. Right.

387
00:23:44,856 --> 00:23:48,290
But I think we're swifting as companies kind of take a look at the bills

388
00:23:48,370 --> 00:23:51,734
and realize that lift and shift was not a

389
00:23:51,772 --> 00:23:55,622
financially great decision. Right. Whether or not cloud is a good

390
00:23:55,676 --> 00:23:59,466
thing or not, I think it always goes back to those two

391
00:23:59,488 --> 00:24:03,050
words that every consultant and every It person always says it depends.

392
00:24:04,430 --> 00:24:08,074
Whereas previously it was last ten years was

393
00:24:08,112 --> 00:24:11,894
oh, definitely was the two words. But I think now we're realizing it depends.

394
00:24:11,942 --> 00:24:15,690
And I think one of the drivers for this are things like autonomous systems

395
00:24:15,770 --> 00:24:19,454
or drones or self driving cars, right. No matter how good

396
00:24:19,492 --> 00:24:22,830
5G is, and I can tell you I know all the dead spots

397
00:24:23,570 --> 00:24:26,560
in the DC area, but

398
00:24:27,750 --> 00:24:31,274
if you're driving along at 60 miles an hour, 100

399
00:24:31,332 --> 00:24:34,370
miles, 100 km/hour for our friends overseas,

400
00:24:34,950 --> 00:24:38,726
and like you said, is that a tree? Is that a shadow? Is that

401
00:24:38,748 --> 00:24:42,582
a person? Is that a grandma? Right. You don't want to wait on

402
00:24:42,636 --> 00:24:46,486
the latency to come back. You want the inference or the decision to

403
00:24:46,508 --> 00:24:50,186
be made on device. So you're really bumping up against the

404
00:24:50,208 --> 00:24:53,466
speed of light and you're talking nanoseconds, not

405
00:24:53,488 --> 00:24:54,780
milliseconds. Right.

406
00:24:57,470 --> 00:25:00,926
What do you see? Because you mentioned you want there to be

407
00:25:00,948 --> 00:25:04,718
sensors, but obviously these things have to be relatively low power. I guess in

408
00:25:04,724 --> 00:25:08,382
a car it doesn't matter as much, but certainly on a drone that

409
00:25:08,436 --> 00:25:09,230
matters.

410
00:25:12,050 --> 00:25:15,762
What sorts of challenges does intel see in that regard in terms

411
00:25:15,816 --> 00:25:19,330
of you want the most performance, but you want the most

412
00:25:19,400 --> 00:25:22,242
energy efficiency. That seems like two

413
00:25:22,376 --> 00:25:26,146
opposing forces. You would think that way, but if you

414
00:25:26,168 --> 00:25:29,334
look at Moore's Law and you look at what's really behind that, it's about

415
00:25:29,372 --> 00:25:32,966
reducing the size. And really that means the

416
00:25:32,988 --> 00:25:36,118
power and increasing the performance, increasing the amount of

417
00:25:36,204 --> 00:25:39,514
transistors. And that's really been what's driving compute all along, is how do we get

418
00:25:39,552 --> 00:25:43,226
to lower power per density. Now, where it

419
00:25:43,248 --> 00:25:47,002
becomes interesting is in the cloud. It's a cost measure. It's about getting

420
00:25:47,056 --> 00:25:50,794
more for your dollar in a car or in a

421
00:25:50,832 --> 00:25:54,286
drone or even in a factory floor. It's about being able to

422
00:25:54,308 --> 00:25:58,078
operate closer to where the decision needs to be made

423
00:25:58,244 --> 00:26:01,866
without having to, again, to have to power it and have that immense

424
00:26:01,898 --> 00:26:05,646
cost. Or in the case of a drone, the weight of the battery pack and

425
00:26:05,668 --> 00:26:09,314
so forth. So lower swap actually enables those edge use

426
00:26:09,352 --> 00:26:13,026
cases. And again, one of the things that people realize is that Edge can mean

427
00:26:13,048 --> 00:26:16,354
different things to different people. You talk to the cloud providers and Edge is just

428
00:26:16,472 --> 00:26:20,086
a couple of racks closer out of the cloud. On

429
00:26:20,108 --> 00:26:23,606
Prem, you look at Azure Stack or Snowball or these kind of

430
00:26:23,628 --> 00:26:27,446
approaches. It's really about pushing pieces of the cloud closer to the edge through like

431
00:26:27,468 --> 00:26:31,290
the core or they called it the

432
00:26:31,360 --> 00:26:34,906
fog back in the day. You look at the edge and

433
00:26:34,928 --> 00:26:38,458
you take a look at a Tesla, it's like a driving data center.

434
00:26:38,624 --> 00:26:42,122
There's compute capabilities in there. A plane is a flying data

435
00:26:42,176 --> 00:26:44,800
center. Your drones are getting to be more

436
00:26:45,330 --> 00:26:49,054
computing. And when you move from a

437
00:26:49,092 --> 00:26:52,078
discrete mode to a logical mode, and I've seen these already, where you have a

438
00:26:52,084 --> 00:26:55,842
drone who actually has one processor but multiple containers, so actually running

439
00:26:55,896 --> 00:26:59,442
multiple functions that could be thought of as different

440
00:26:59,496 --> 00:27:03,022
applications on different nodes, but now they've all been collapsed with either virtualization

441
00:27:03,166 --> 00:27:06,878
or container. So you can have navigation being one, you can be

442
00:27:06,904 --> 00:27:10,726
doing object detection and mapping with another, and then be able to do sort

443
00:27:10,748 --> 00:27:14,338
of other kinds of sensing like temperature

444
00:27:14,434 --> 00:27:18,086
or barometer and things like that and doing analysis in

445
00:27:18,108 --> 00:27:21,830
real time. One of the best examples that we demonstrated

446
00:27:21,990 --> 00:27:25,786
at our last year's Fed summit was a set of drones out

447
00:27:25,808 --> 00:27:29,386
mapping a region. They were going about their business, but they had a policy that

448
00:27:29,408 --> 00:27:33,066
if somebody walked into a specific area of interest, let's say in front of an

449
00:27:33,088 --> 00:27:36,926
embassy or in front of Lloyd or too long, that one of the drones would

450
00:27:36,948 --> 00:27:40,698
be retasked and go over and investigate and do facial

451
00:27:40,714 --> 00:27:43,646
recognition. All the things you want to do to make sure, hey, is this person

452
00:27:43,828 --> 00:27:47,534
up to no good? And it didn't require a reprogramming

453
00:27:47,582 --> 00:27:51,426
of a drone. It didn't require a special drone that was just the investigator. It

454
00:27:51,448 --> 00:27:55,282
would basically retask itself with a new. Mission in real time

455
00:27:55,416 --> 00:27:58,694
and go investigate. And when the person left that zone, it go back to its

456
00:27:58,732 --> 00:28:02,198
day job of mapping the environment. That's just sort of the tip of

457
00:28:02,204 --> 00:28:05,894
that simple prototype to show that even a very

458
00:28:05,932 --> 00:28:09,358
small autonomous system and these were like sort of my mini drones

459
00:28:09,394 --> 00:28:13,226
here, is capable of the compute necessary to

460
00:28:13,248 --> 00:28:16,906
do multimission kind of use cases. So the edge absolutely is

461
00:28:16,928 --> 00:28:20,570
that new frontier. And it's again similar to the cloud. When you say cloud,

462
00:28:20,640 --> 00:28:24,342
everyone thinks, oh, public cloud, really? Cloud is all those architectures

463
00:28:24,406 --> 00:28:27,562
all the way down to the edge. It's the way we develop those cloud native

464
00:28:27,626 --> 00:28:31,374
apps that can flow back and forth. So from a cloud provider, it's moving

465
00:28:31,412 --> 00:28:35,106
more of their cloud infrastructure closer to the edge. And what the

466
00:28:35,128 --> 00:28:38,718
edge, folks, whether it be the actual device or sensor manufacturers

467
00:28:38,814 --> 00:28:41,538
are looking at, is bringing some of those cloud

468
00:28:41,704 --> 00:28:44,622
capabilities to their device to operate

469
00:28:44,686 --> 00:28:48,406
independently. And there's a reason for that is that, number one, latency, like you

470
00:28:48,428 --> 00:28:52,054
mentioned, Frank, but also the cost of shipping all that

471
00:28:52,092 --> 00:28:55,878
data. No one wants to ship Raw 4K video feeds to the

472
00:28:55,884 --> 00:28:59,714
cloud just to be able to tell me, is that a tree?

473
00:28:59,842 --> 00:29:03,526
You want to be able to send the results that I saw a tree

474
00:29:03,558 --> 00:29:06,922
here with the longitudinal latitude, which is a small data

475
00:29:06,976 --> 00:29:10,614
packet, and let the sensor do the AI, do the inference

476
00:29:10,662 --> 00:29:14,494
at the edge. Right. And then you have the case

477
00:29:14,532 --> 00:29:18,174
where you're talking about planes or vehicles, right?

478
00:29:18,212 --> 00:29:21,566
Like the whole time it's tracking, did the wheel fall off? Did the wheel fall

479
00:29:21,588 --> 00:29:25,186
off? Did the wheel fall off? Right, but at one point when you get to

480
00:29:25,208 --> 00:29:28,834
your destination, the wheel either fell off or it didn't. Right.

481
00:29:28,872 --> 00:29:32,100
So you collapse that entire thing

482
00:29:33,030 --> 00:29:36,546
to one integer level or really not even an

483
00:29:36,568 --> 00:29:40,294
integer. Like a bit. Right, a bit. And then if the wheel does

484
00:29:40,332 --> 00:29:44,150
fall off, I'm sure there's plenty of other stuff you can pick up too,

485
00:29:44,220 --> 00:29:47,946
but hopefully nobody gets hurt. But I mean,

486
00:29:47,968 --> 00:29:51,674
ultimately you're right. The problem with data is so much

487
00:29:51,712 --> 00:29:55,562
that there's value, but there's a certain

488
00:29:55,616 --> 00:29:58,954
amount of we've gotten to the point where

489
00:29:59,152 --> 00:30:02,686
just because we can, we've done it. Right. Yeah, sure. Bring up that

490
00:30:02,708 --> 00:30:06,366
4K. If I'm a salesperson for one of those cloud

491
00:30:06,388 --> 00:30:09,600
providers. Yeah, man, bring in all that 4K data you want,

492
00:30:11,090 --> 00:30:14,786
we'll take it all. We'll be happy to charge you for it too. Right,

493
00:30:14,968 --> 00:30:18,020
but I think as we get to the point where

494
00:30:20,150 --> 00:30:23,634
there might just be too much data, I think people organizations are going to start

495
00:30:23,672 --> 00:30:27,414
thinking like, where can we scale back on the storage? Because

496
00:30:27,452 --> 00:30:31,254
we don't really need it unless there's some kind of regulatory reason for

497
00:30:31,292 --> 00:30:34,646
it. Now, one thing I want to double click on,

498
00:30:34,668 --> 00:30:38,486
because this is a fascinating conversation, we'd love to have you back

499
00:30:38,508 --> 00:30:42,106
on the show at some point. What's the

500
00:30:42,128 --> 00:30:45,482
deal with FPGA because you mentioned

501
00:30:45,536 --> 00:30:49,066
that and this was a huge deal. So a couple of things that are

502
00:30:49,088 --> 00:30:52,782
interesting is that I first heard about Transformers at

503
00:30:52,836 --> 00:30:55,786
the Microsoft has this internal data science conference

504
00:30:55,898 --> 00:30:59,662
MLADS, and they first talked about Transformers. I went into

505
00:30:59,796 --> 00:31:03,230
the talk and ten minutes, my head went

506
00:31:03,300 --> 00:31:07,006
boom, right? I didn't quite follow it. Somebody later on in the

507
00:31:07,028 --> 00:31:10,386
day in the reception area was kind enough to explain it, how it

508
00:31:10,408 --> 00:31:13,906
works. And one of the other things that came out of that conference was talking

509
00:31:13,928 --> 00:31:17,414
about the importance of FPGAs and what they're going to be like in the future.

510
00:31:17,532 --> 00:31:21,014
Now, again, I'm a data scientist. I really don't focus on

511
00:31:21,052 --> 00:31:24,294
hardware so much until when I need to buy new

512
00:31:24,332 --> 00:31:27,350
hardware, like a new desktop or laptop.

513
00:31:29,050 --> 00:31:32,806
What are FPGAs? And I remember hearing a lot about them and then

514
00:31:32,828 --> 00:31:35,674
they kind of went dark for a while and then now they're kind of coming

515
00:31:35,712 --> 00:31:39,114
back into vogue. Can you talk to us about, one, what they are and then

516
00:31:39,152 --> 00:31:42,474
two where you see they're going? Sure. So Ed and FPGA are a field

517
00:31:42,512 --> 00:31:46,030
programmable gate array. They've been around for forever. I mean, computer

518
00:31:46,100 --> 00:31:49,406
science engineers going back, electrical engineers going back to the

519
00:31:49,428 --> 00:31:53,246
80s played with FPGA. They were very early FPGA, but

520
00:31:53,268 --> 00:31:56,914
basically they're programmable hardware. That's really the way to think about it.

521
00:31:57,032 --> 00:32:00,766
You think about a CPU or an Ace or any chip it's

522
00:32:00,798 --> 00:32:04,226
laid down with its transistors, and the flow of those transit is

523
00:32:04,248 --> 00:32:07,614
fixed. CPU can do multiple software

524
00:32:07,662 --> 00:32:11,506
flows, but the instruction flow is the instruction

525
00:32:11,538 --> 00:32:14,966
flow. What makes FPGAs interesting is that you

526
00:32:14,988 --> 00:32:18,758
can create new RTL, new layouts of flows, what

527
00:32:18,764 --> 00:32:22,598
they call netlist of those instructions going across those transistors

528
00:32:22,774 --> 00:32:26,426
each time. You can go in and customize it after. So the

529
00:32:26,448 --> 00:32:29,466
manufacturing builds you a clean slate of a bunch of think about a bunch of

530
00:32:29,488 --> 00:32:33,014
rows, and then you program them to your specific need

531
00:32:33,152 --> 00:32:36,720
at a hardware style abstraction layer. So it gives you a much

532
00:32:37,170 --> 00:32:40,926
faster capability because you're now really writing in hardware. It's a lot more

533
00:32:40,948 --> 00:32:44,282
complex of a coding. It's not like doing Python,

534
00:32:44,426 --> 00:32:47,986
but what you get is a very optimized piece of

535
00:32:48,008 --> 00:32:51,426
hardware for your specific use case. And what's nice about that

536
00:32:51,448 --> 00:32:55,266
is one of the great examples is in signals conditioning. When

537
00:32:55,288 --> 00:32:59,138
you're doing like 5G research or testing signal amplitudes and

538
00:32:59,144 --> 00:33:02,678
things like that, as you put in your algorithm actually into hardware, you go out

539
00:33:02,684 --> 00:33:05,974
and test it. It works sort of here. I need to tweak it well, instead

540
00:33:06,012 --> 00:33:09,846
of going and spinning a new piece of hardware, you just upload new code and

541
00:33:09,868 --> 00:33:13,562
you go right in. So it's a much faster time of development for doing

542
00:33:13,616 --> 00:33:17,306
those custom things. What people have found when we start looking at sort of

543
00:33:17,328 --> 00:33:20,898
AI use cases and machine learning and pattern matching

544
00:33:21,014 --> 00:33:24,800
is that FPGA really lend themselves well

545
00:33:25,170 --> 00:33:28,926
to be able to create different kinds of architectural approaches to how

546
00:33:28,948 --> 00:33:32,714
you process that data flow. If you think about a GPU

547
00:33:32,762 --> 00:33:36,238
or CPU or even an ASIC, it's a fixed data flow. It's good for the

548
00:33:36,244 --> 00:33:39,934
things it was designed for. What FPGA allows you to do is to customize

549
00:33:39,982 --> 00:33:43,426
your flows based on what the data is or based on what your algorithm are.

550
00:33:43,528 --> 00:33:46,482
And so a lot of the FPGA work they were seeing in AI is people

551
00:33:46,536 --> 00:33:50,102
coding their AI algorithms or the machine learning algorithms right into

552
00:33:50,156 --> 00:33:53,746
hardware and then deploying it. And so it allows you to be able to deploy

553
00:33:53,778 --> 00:33:57,446
your thing quicker and you get pretty good performance. It's not as

554
00:33:57,468 --> 00:34:00,874
good as say, as a custom ASIC for your algorithm. And it's not as

555
00:34:00,912 --> 00:34:04,634
scalable really as like a software abstraction on running on a

556
00:34:04,672 --> 00:34:08,186
cloud set of CPUs. But for a lot of these training and

557
00:34:08,208 --> 00:34:11,854
inferencing use cases, one of the areas where it shines is in the whole

558
00:34:11,892 --> 00:34:15,630
area of neuromorphic processing. So a whole part of the AI machine learning

559
00:34:15,700 --> 00:34:19,326
space is modeling after brain activity or how our

560
00:34:19,348 --> 00:34:23,022
brains process. It's a whole field. FPGAs are actually

561
00:34:23,076 --> 00:34:26,754
well designed for those kind of algorithms that X 86 and

562
00:34:26,792 --> 00:34:30,338
other CPU style Arctic just aren't yet.

563
00:34:30,504 --> 00:34:34,274
And that's why FPGAs really shine in those environments, because you can create

564
00:34:34,312 --> 00:34:37,906
these linear sort of permutation flows that you find in neuromorphic

565
00:34:37,938 --> 00:34:41,574
algorithms. You just code those into the path for the

566
00:34:41,612 --> 00:34:45,190
FPGA. They're really good. You'll see, FPGAs are very often used

567
00:34:45,260 --> 00:34:49,014
in cellular and RF communications that are really good at those sort of

568
00:34:49,052 --> 00:34:52,826
channelizer and signal optimization and

569
00:34:52,848 --> 00:34:56,090
be able to do those kind of algorithms that you do on RF and

570
00:34:56,160 --> 00:34:59,978
Comps, again, really good for those kind of workflows. And so why we

571
00:34:59,984 --> 00:35:03,694
see the resurgence of FPGAs, although they've never gone away, you find them

572
00:35:03,732 --> 00:35:07,358
everywhere. Open up your big screen flat screen TV, you'll find a couple of

573
00:35:07,364 --> 00:35:11,086
FPGA in there. Where they're shining is because it

574
00:35:11,108 --> 00:35:14,894
allows you to do some rapid prototyping on AI. And because we're seeing

575
00:35:14,932 --> 00:35:18,574
now FPGAs come to the cloud. So you go to Azure has an FPGA

576
00:35:18,622 --> 00:35:22,446
cloud. You can now deploy those algorithms at cloud scale,

577
00:35:22,558 --> 00:35:26,386
or you can deploy an FPGA into your edge sensor and be able

578
00:35:26,408 --> 00:35:30,178
to do that real time, sort of. Let's go try this inferencing model. Oh, we're

579
00:35:30,194 --> 00:35:33,426
going to change the inferencing model. Let's go do that one. And where this becomes

580
00:35:33,458 --> 00:35:37,206
really interesting in those low slop environments is a modern FPGA is

581
00:35:37,228 --> 00:35:40,506
reprogrammable in milliseconds, which means you can go from one

582
00:35:40,528 --> 00:35:44,314
program to another by just pushing a firmware, if you will,

583
00:35:44,352 --> 00:35:47,590
update. And now you go from a 5G communications

584
00:35:47,670 --> 00:35:51,502
system to LTE or to a six G

585
00:35:51,556 --> 00:35:55,070
without actually going and swapping out the hardware. That's wild.

586
00:35:56,130 --> 00:35:59,678
That's wild. Yeah, it's exciting times. So

587
00:35:59,764 --> 00:36:02,800
with that, the updatable part of it,

588
00:36:05,350 --> 00:36:09,166
how do you secure that? Because I can easily see that being like particularly

589
00:36:09,198 --> 00:36:12,580
you work in the in the federal space, right? Like security

590
00:36:13,110 --> 00:36:16,658
is top of mind in that work. It should be top of mind everywhere,

591
00:36:16,754 --> 00:36:20,118
but in the near term it's top of mind, at

592
00:36:20,124 --> 00:36:23,970
least in the federal spaces. FPGA

593
00:36:24,050 --> 00:36:27,686
sounds like awesome, but it also sounds like that just seems

594
00:36:27,788 --> 00:36:31,590
dangerous in a lot of ways. You can reprogram it in milliseconds.

595
00:36:31,750 --> 00:36:35,434
There's got to be some kind of security story there. Oh absolutely. And

596
00:36:35,472 --> 00:36:39,162
Fpjs have actually in many cases led as far as the kind of security

597
00:36:39,216 --> 00:36:41,920
mechanisms built into the hardware for that very reason.

598
00:36:43,890 --> 00:36:47,134
At its core, at the core level, it's the same kind of approach you do

599
00:36:47,172 --> 00:36:50,778
for verifying your firmware on your system. It's signed

600
00:36:50,874 --> 00:36:54,666
by hardware so that basically you're verifying

601
00:36:54,698 --> 00:36:57,458
your load and if you're going to do an update, you're going to verify a

602
00:36:57,464 --> 00:37:01,314
signature against a hardware rooted key so that you make sure that only

603
00:37:01,352 --> 00:37:04,834
legitimate folks can do the update and that it's only be able to be done

604
00:37:04,872 --> 00:37:08,354
by someone who's got the permission. From a cryptographic

605
00:37:08,402 --> 00:37:12,150
perspective, what we find in the current FPGA that are out in the market

606
00:37:12,300 --> 00:37:15,430
is that they've built in a whole suite of security

607
00:37:15,500 --> 00:37:19,266
capabilities. Things like Puff Provably, unclonable

608
00:37:19,298 --> 00:37:22,794
functions, which is basically a hardware root key that is

609
00:37:22,912 --> 00:37:26,474
really secure as that hardware route of trust, signing in

610
00:37:26,512 --> 00:37:30,234
cryptography functions, anti tamper functions to make sure someone can't go

611
00:37:30,272 --> 00:37:33,834
pop open the lid or put in a jumper and try to try to change

612
00:37:33,872 --> 00:37:36,734
the code. So those kind of mechanisms have been in place for a long time

613
00:37:36,772 --> 00:37:40,606
because FPGAs have been used in such critical places. We find them in

614
00:37:40,628 --> 00:37:44,302
radar stations, we find them in systems and so they've been building security

615
00:37:44,356 --> 00:37:48,146
in for a very long time. And it's part of the workflow that when you

616
00:37:48,168 --> 00:37:52,014
build your code you're going to take advantage of these implicit, let's call them IP

617
00:37:52,062 --> 00:37:55,794
blocks that do security for your RTL, for your code that you're putting

618
00:37:55,832 --> 00:37:59,574
in place. The other important thing is that the way that the code works

619
00:37:59,612 --> 00:38:03,014
is once you lay it out, once you translate your software into that

620
00:38:03,052 --> 00:38:06,754
layout, the layout is you can't just sort of go and reverse engineer

621
00:38:06,802 --> 00:38:10,246
back. And so it's really a very powerful

622
00:38:10,438 --> 00:38:14,150
mechanism as opposed to say firmware. When you're it's software.

623
00:38:14,230 --> 00:38:18,026
If you think about the BIOS update, it's software that you're loading just deeper in

624
00:38:18,048 --> 00:38:21,866
your platform and if anyone wants to go inspect, you'll find

625
00:38:21,888 --> 00:38:25,262
there's a lot of software in the hardware that you don't realize is actually

626
00:38:25,316 --> 00:38:28,958
software. The same kind of security mechanism we did there. You verify it against a

627
00:38:28,964 --> 00:38:32,718
hardware of trust, you make sure it's signed before you run it

628
00:38:32,804 --> 00:38:36,046
and then you apply cryptography to make sure that it can't be changed or it's

629
00:38:36,078 --> 00:38:39,778
integrity protected. You find those same capabilities built into the

630
00:38:39,784 --> 00:38:43,394
hardware of an FPGA and the software development tools, the

631
00:38:43,432 --> 00:38:47,278
dialogue, the cordis and so forth have the mechanisms to take advantage.

632
00:38:47,374 --> 00:38:51,078
So again, programmers don't have to be security gurus. They basically say,

633
00:38:51,164 --> 00:38:54,418
I'm going to push this, and it's auto going to take advantage of those features.

634
00:38:54,514 --> 00:38:58,006
It's good because programmers historically are very bad security

635
00:38:58,108 --> 00:39:01,490
people. I say that. It says, yeah,

636
00:39:01,660 --> 00:39:05,386
it's its own specialty. And yeah, you can't be

637
00:39:05,408 --> 00:39:09,178
good at everything these days. There's too much. So I'm going

638
00:39:09,184 --> 00:39:12,734
to echo what Frank said earlier. Steve, we got to have you back.

639
00:39:12,852 --> 00:39:16,414
I really appreciate you being here. We could talk and geek out on

640
00:39:16,452 --> 00:39:19,982
hardware stuff forever, but we want to

641
00:39:20,036 --> 00:39:23,726
pivot and go to our questions and if that's

642
00:39:23,758 --> 00:39:26,946
okay, we want to start with unless Frank, unless you had anything else you wanted

643
00:39:26,968 --> 00:39:30,146
to do before. Let me

644
00:39:30,168 --> 00:39:33,380
rephrase. No.

645
00:39:34,710 --> 00:39:38,150
In the virtual green room, you talked about some things that are going on and

646
00:39:38,220 --> 00:39:42,006
kind of operationally and

647
00:39:42,188 --> 00:39:45,958
wow, we didn't even get there. I mean, I

648
00:39:45,964 --> 00:39:49,338
think the important thing I took from this conversation is

649
00:39:49,504 --> 00:39:53,258
that one, GPUs, they are important, but

650
00:39:53,264 --> 00:39:55,900
they're not the whole story. And two,

651
00:39:57,470 --> 00:40:00,922
at the end of the day, chat

652
00:40:00,986 --> 00:40:04,714
GPT, any of these magical looking AI

653
00:40:04,762 --> 00:40:08,174
models, magical seeming, right. They're all mass,

654
00:40:08,292 --> 00:40:12,074
right? Yeah. And being beneath the math are electrons

655
00:40:12,122 --> 00:40:15,822
bouncing around inside these microscopic chips. And

656
00:40:15,876 --> 00:40:19,426
there's all sorts of things you could do to tweak and improve that, even if

657
00:40:19,448 --> 00:40:23,282
it's like a billionth of a second, right? A billionth of a second times

658
00:40:23,336 --> 00:40:25,300
a billion adds up.

659
00:40:26,970 --> 00:40:30,278
And that adds up in terms of whether you're driving a car

660
00:40:30,364 --> 00:40:32,840
or you're flying a plane or

661
00:40:34,410 --> 00:40:37,922
you're a company like AWS or Microsoft,

662
00:40:37,986 --> 00:40:41,578
where, hey, if I save one compute second per

663
00:40:41,664 --> 00:40:45,066
transaction, I do trillions of those a day. And that's real

664
00:40:45,088 --> 00:40:48,826
money. Exactly. And that's the thing that blew my mind. But yeah,

665
00:40:48,928 --> 00:40:52,206
let's switch because we could geek out for hours. Because this is very

666
00:40:52,228 --> 00:40:55,934
true. Yeah. Amazing.

667
00:40:56,132 --> 00:40:59,806
It really is. So how did you find your

668
00:40:59,828 --> 00:41:03,294
way into not so much data, but it how did you find your way into

669
00:41:03,332 --> 00:41:06,994
data? Did you find it or did

670
00:41:07,032 --> 00:41:10,514
it find you or hardware specifically? So ring. It's a really good

671
00:41:10,552 --> 00:41:14,162
question and going back to the very beginning, actually, I started

672
00:41:14,216 --> 00:41:16,850
out in the molecular biology

673
00:41:17,270 --> 00:41:20,342
bioresearch side of the camp, going all the way back. I was going to be

674
00:41:20,396 --> 00:41:24,006
a research biologist and probably still be there today,

675
00:41:24,108 --> 00:41:27,878
except for a couple of key life events early in

676
00:41:27,884 --> 00:41:31,562
the early ninety s, I was a hacker as a kid.

677
00:41:31,616 --> 00:41:35,434
I loved seeing how things fell apart and how to code and break code

678
00:41:35,472 --> 00:41:39,066
and things like that. But in the late 80s, there really wasn't a

679
00:41:39,088 --> 00:41:42,586
career other than a COBOL programmer, which

680
00:41:42,688 --> 00:41:46,426
wasn't an exciting career at the time. So I went the bio route,

681
00:41:46,458 --> 00:41:49,934
which was my, the love. And right after I graduated and was going to start

682
00:41:49,972 --> 00:41:52,880
med school, I had a year off and

683
00:41:53,490 --> 00:41:56,766
someone had some money, wanted to do a startupy thing and they knew I was

684
00:41:56,788 --> 00:42:00,114
a hacker and say, hey, why don't you help me get this thing running? And

685
00:42:00,312 --> 00:42:03,314
I'm thinking, well, med school is expensive. This would be a good way to help

686
00:42:03,352 --> 00:42:06,834
pay for it. And so I started my first company in

687
00:42:06,872 --> 00:42:10,566
95 and after three months just fell in love with everything that was

688
00:42:10,588 --> 00:42:14,214
going on. It was the exciting time to be in the internet. Got to apply

689
00:42:14,252 --> 00:42:17,942
some of my security hacker background in an interesting way

690
00:42:17,996 --> 00:42:21,290
and had some really good mentors. People like Bruce Schneier,

691
00:42:22,110 --> 00:42:25,606
the writer of Applied Cryptography sort of took Zebru Schneider.

692
00:42:25,718 --> 00:42:29,526
Zebrus Schneider was one of my mentors and took me under his wing.

693
00:42:29,558 --> 00:42:33,366
And like I say, I sucked his brain dry as best as I could. But

694
00:42:33,408 --> 00:42:37,070
really it just sort of got the opportunity to get on the ground floor

695
00:42:37,410 --> 00:42:41,214
right before Netscape went public. So really early days on

696
00:42:41,252 --> 00:42:44,766
a startup in the email encryption space and then one thing led to another and

697
00:42:44,788 --> 00:42:48,626
I just felt this was what I was going to do. And for the next

698
00:42:48,728 --> 00:42:52,366
sort of several years, I did multiple security startups throughout

699
00:42:52,398 --> 00:42:56,174
the then in 2005 got acquired

700
00:42:56,222 --> 00:42:59,766
by intel. I like to joke, I'm still trying to figure out

701
00:42:59,868 --> 00:43:03,634
how I ended up here for 18 years. But I think what intel

702
00:43:03,682 --> 00:43:07,286
has provided me and provides a lot of our folks is the ability to sort

703
00:43:07,308 --> 00:43:11,050
of innovate in an environment where a, you've got a big company

704
00:43:11,120 --> 00:43:14,634
behind you helping you do that. But one of the best

705
00:43:14,672 --> 00:43:18,234
reasons why I think intel has been fun for me, my most

706
00:43:18,272 --> 00:43:21,902
successful startup, we had 500 of Fortune Thousand companies using

707
00:43:21,956 --> 00:43:25,742
our product. The first project I worked on in intel went to 40 million

708
00:43:25,796 --> 00:43:28,862
PCs. So the impact is just

709
00:43:28,996 --> 00:43:32,494
unbelievable. Now from the data

710
00:43:32,532 --> 00:43:36,290
side again, at the end of the day, like you mentioned earlier, underneath the data,

711
00:43:36,360 --> 00:43:40,046
underneath the machine learning, underneath the AI, and even before we were talking about AI

712
00:43:40,078 --> 00:43:43,806
was machine learning and advanced pattern matching. There's electrons

713
00:43:43,838 --> 00:43:47,506
moving around it's running on hardware. And so a lot of what my

714
00:43:47,528 --> 00:43:50,934
job has been before I came to the federal team was looking for ways to

715
00:43:50,972 --> 00:43:54,726
innovate or take advantage of new use cases in software, to

716
00:43:54,748 --> 00:43:57,766
take advantage of hardware in interesting ways. And so we call that

717
00:43:57,788 --> 00:44:01,394
pathfinding. So you think about our labs or thinking about the next generation

718
00:44:01,442 --> 00:44:04,842
hardware five to ten years out, I ran the team, the security

719
00:44:04,896 --> 00:44:08,634
pathfinding team that was looking at the two to five year horizon. I

720
00:44:08,672 --> 00:44:11,786
knew this was the hardware platform that was going to be there next year. What

721
00:44:11,808 --> 00:44:15,054
would be some interesting things I could do with it to either advance security or

722
00:44:15,092 --> 00:44:18,558
increase security, that was my area domain. And so things like

723
00:44:18,644 --> 00:44:22,234
antimalware technologies, cloud security, before they knew how to spell

724
00:44:22,282 --> 00:44:26,050
cloud. We called it virtualization security first and things like that.

725
00:44:26,200 --> 00:44:30,034
Web security, that was the fluffy stuff. That was Steve's world while

726
00:44:30,072 --> 00:44:33,854
the hardware engineers are figuring out low level cryptography and hardware

727
00:44:33,902 --> 00:44:37,250
roots of trust. And we sort of worked in tandem to innovate.

728
00:44:38,150 --> 00:44:41,926
And so as things like data science started to take off, it was like,

729
00:44:41,948 --> 00:44:45,746
this is a key area both from a security and perspective. How do I secure

730
00:44:45,778 --> 00:44:49,478
that data? How do I secure the algorithms? How do I use that? I mean,

731
00:44:49,484 --> 00:44:52,646
one of the really cool things is being able to use machine learning and AI

732
00:44:52,678 --> 00:44:54,380
and apply it to the cyber problem.

733
00:44:56,270 --> 00:44:59,878
And when you start doing things like that, you immediately run to, well, we've

734
00:44:59,894 --> 00:45:03,674
got too much data flowing in. I mean, the classic example is streaming

735
00:45:03,722 --> 00:45:07,294
analytics on network at network speed. Well, how do you do

736
00:45:07,412 --> 00:45:10,910
deep packet inspection at gigabit or higher

737
00:45:10,980 --> 00:45:14,786
speeds without losing data? That's a big problem. That's where hardware can

738
00:45:14,808 --> 00:45:17,810
help save you, that you just can't do in software.

739
00:45:18,630 --> 00:45:22,402
And then when I transitioned to the federal team and took over and

740
00:45:22,456 --> 00:45:26,226
drove our federal technology practice, you really opened the door to

741
00:45:26,248 --> 00:45:28,866
all the different use cases. And one of the things I like about the federal

742
00:45:28,898 --> 00:45:32,678
government is that it's a macrocosm of all verticals. You want to

743
00:45:32,684 --> 00:45:36,230
talk finance, you've got IRS and CMS, some of the largest

744
00:45:37,130 --> 00:45:40,770
processing of financial data. You want to talk healthcare, the VA is the

745
00:45:40,780 --> 00:45:44,474
largest provider of healthcare, the largest insurer in the world. You want to talk

746
00:45:44,512 --> 00:45:48,058
logistics, DoD logistics is huge. So

747
00:45:48,224 --> 00:45:51,722
you sort of look at it, every kind of use case you'll find in government.

748
00:45:51,776 --> 00:45:55,214
So it's really a good way of looking at all the different verticals. And they

749
00:45:55,252 --> 00:45:58,382
all have unique or interesting data problems. There's some

750
00:45:58,436 --> 00:46:02,158
commonality. And one of the things I really like about the federal government is that

751
00:46:02,164 --> 00:46:05,874
you get that commonality across the divisions. They all are having trouble doing data

752
00:46:05,912 --> 00:46:09,074
ingestion. That is just fundamental. It doesn't matter if you're the federal government or

753
00:46:09,112 --> 00:46:12,834
Citibank or startup in Silicon Valley. Data ingestion is hard

754
00:46:12,952 --> 00:46:16,546
and doing it at scale and being able to then do something

755
00:46:16,648 --> 00:46:20,418
once you've got the data. And I like to use the analogy

756
00:46:20,434 --> 00:46:24,086
of an iceberg. So AI, Chat, GPU, all these are the tip of the

757
00:46:24,108 --> 00:46:27,750
iceberg. That's the cool, sexy stuff you can do, the hard work,

758
00:46:27,820 --> 00:46:31,434
the data curation, data wrangling is all the work that has to be done before

759
00:46:31,472 --> 00:46:35,174
you ever get there. And that's data ingestion, it's labeling, it's curation,

760
00:46:35,222 --> 00:46:39,002
it's data set management, it's all that stuff. And then layer in things like

761
00:46:39,056 --> 00:46:42,906
removing bias or dealing with bias and securing and integrity, protecting your

762
00:46:42,928 --> 00:46:46,686
data. Like all those things have to happen before you ever start having

763
00:46:46,708 --> 00:46:50,110
the fun math that happens towards the end of that curve.

764
00:46:51,010 --> 00:46:54,558
That's where you find that coming out. Everyone is challenged with those things, and I

765
00:46:54,564 --> 00:46:58,146
think that's where the excitement is today. No, you definitely hear in your

766
00:46:58,168 --> 00:47:01,666
voice, sorry, Andy. Yeah, definitely. No, it's okay. We refer to

767
00:47:01,688 --> 00:47:05,506
that as kind of a joke that's been going on

768
00:47:05,528 --> 00:47:08,840
for seven years now. We say, first you get the data,

769
00:47:09,210 --> 00:47:12,694
and that's 90% of the work. We know

770
00:47:12,732 --> 00:47:16,546
that and your iceberg analogy fits that, Frank.

771
00:47:16,578 --> 00:47:20,266
We need a shirt that has a picture of an iceberg against us. First you

772
00:47:20,288 --> 00:47:24,090
get the data under the I like that. I'm definitely going to do that.

773
00:47:24,240 --> 00:47:27,900
We launched a magazine, actually, yesterday as we record this, and

774
00:47:28,430 --> 00:47:32,126
the cartoon segment is called First You Get the Data. And it

775
00:47:32,228 --> 00:47:36,046
kind of like cringy things that you'll hear about data, and one

776
00:47:36,068 --> 00:47:39,774
of them was like, yeah, first we get the data. My

777
00:47:39,812 --> 00:47:43,518
favorite was how

778
00:47:43,524 --> 00:47:47,218
to prep and clean the data. And they were like, oh, no, our data is

779
00:47:47,224 --> 00:47:50,046
already in the normalized database. We don't need to clean it or prep it. It's

780
00:47:50,078 --> 00:47:52,660
already ready. Like, oh, boy.

781
00:47:54,790 --> 00:47:58,550
You need you need a picture of someone throwing data into a washing machine.

782
00:48:00,330 --> 00:48:03,400
That's a good shirt. We could do that. Yeah,

783
00:48:04,250 --> 00:48:07,560
no, that's cool. And I think you bring up something that I think,

784
00:48:08,990 --> 00:48:12,646
folks, we don't know our exact age demographic. We have a rough

785
00:48:12,678 --> 00:48:16,522
idea, but if there's anyone, let's say, under the age of 30,

786
00:48:16,576 --> 00:48:20,326
right in the car with the parents

787
00:48:20,358 --> 00:48:24,094
or they're listening, it's hard to imagine the time because we're about the same age.

788
00:48:24,132 --> 00:48:25,470
I think you're a little older.

789
00:48:28,530 --> 00:48:32,286
If this was not seen as a good career path, like, coding was not the

790
00:48:32,308 --> 00:48:35,374
whole learn to code movement is a modern

791
00:48:35,422 --> 00:48:39,138
phenomenon. I started my college career to be a

792
00:48:39,144 --> 00:48:41,780
chemical engineer because

793
00:48:43,110 --> 00:48:46,646
I had to convince my parents that software engineering was a

794
00:48:46,668 --> 00:48:50,214
viable career path. And my mom, God rest her

795
00:48:50,252 --> 00:48:53,986
souls, was like, I don't want my baby to be one of those weird

796
00:48:54,018 --> 00:48:56,600
people in the basement. Right?

797
00:48:58,010 --> 00:49:01,786
And then my dad, God rest his soul, was like because when

798
00:49:01,808 --> 00:49:05,306
they came to visit me, I had a Sunday print out of the New York

799
00:49:05,328 --> 00:49:09,018
Times, which of course had the job section, which was

800
00:49:09,184 --> 00:49:13,034
at one point like a book. Right. And look at all these

801
00:49:13,072 --> 00:49:16,846
jobs for computer programming. This is a thing. And my

802
00:49:16,868 --> 00:49:19,614
dad looked through it, and he saw all the starting salaries, and it was like

803
00:49:19,652 --> 00:49:22,990
seven or eight pages of near six figure

804
00:49:23,060 --> 00:49:26,562
salaries in the early 90s, which was a lot of money back then, right?

805
00:49:26,616 --> 00:49:30,146
Yeah. Like, looking through, like, on Wall Street stuff. And

806
00:49:30,248 --> 00:49:32,500
he's like, I'm sold. And it's like

807
00:49:34,470 --> 00:49:36,180
and my mom was like, no.

808
00:49:38,470 --> 00:49:41,798
That is literally, like, my experience as well. When I told my parents that I

809
00:49:41,804 --> 00:49:45,266
was going to not go to the research biology route and do the MD

810
00:49:45,298 --> 00:49:48,246
PhD, I was going to go into the security thing. They wanted to do an

811
00:49:48,268 --> 00:49:50,200
intervention. They thought something was wrong.

812
00:49:52,010 --> 00:49:55,786
About two years. In 96, after I'd done the start, for about

813
00:49:55,808 --> 00:49:59,286
a year and a half, there was an article in the New York Times, Paul

814
00:49:59,318 --> 00:50:03,018
Cotcher, had done the timing attacks against RSA, and it

815
00:50:03,024 --> 00:50:06,846
was front page news. And when you read down the first blurb, it says, 22

816
00:50:06,868 --> 00:50:10,526
year old bio student from Stanford cracks RSA encryption. So

817
00:50:10,548 --> 00:50:13,434
I cut that out and faxed it to my parents because they have an email

818
00:50:13,492 --> 00:50:17,186
yet and said, look, another bio student doing security. It can

819
00:50:17,208 --> 00:50:21,058
happen. Right? That's funny. One of

820
00:50:21,064 --> 00:50:24,798
the best web developers I ever worked with, his degree was in biology

821
00:50:24,974 --> 00:50:28,514
as well. And I think there's something to be said about understanding natural

822
00:50:28,562 --> 00:50:32,114
systems, and I think there's some pattern matching gifts

823
00:50:32,242 --> 00:50:35,974
that go along with that. I know my friend was that way as well. And

824
00:50:36,012 --> 00:50:39,546
Frank, when your mom said she didn't want you to be one of those

825
00:50:39,568 --> 00:50:43,130
weirdos in the basement that flew through my head, but I

826
00:50:43,200 --> 00:50:45,450
maintained discipline was too late.

827
00:50:47,630 --> 00:50:51,150
And I could say the same for me as well. Too late.

828
00:50:52,130 --> 00:50:55,886
In her defense, my mom stayed with us in a house that my

829
00:50:55,908 --> 00:50:58,000
wife also works in technology too.

830
00:50:59,890 --> 00:51:03,474
She had an entire suite in our basement of our

831
00:51:03,512 --> 00:51:06,658
house, which was not

832
00:51:06,824 --> 00:51:09,780
windows, walk out yard, everything.

833
00:51:10,790 --> 00:51:14,626
It worked out well. Sometimes

834
00:51:14,728 --> 00:51:18,546
your parents my mother encouraged it without realizing. She allowed me to buy

835
00:51:18,568 --> 00:51:22,054
the haze modem and connect it to our phone. And I did get

836
00:51:22,092 --> 00:51:25,794
disciplined when I had that $1,000 phone bill from dialing into BBS's overnight.

837
00:51:25,842 --> 00:51:29,480
But they should have seen it coming. Yeah,

838
00:51:30,410 --> 00:51:33,610
my mom freaked out when I wanted a modem. She's like, no, absolutely

839
00:51:33,680 --> 00:51:37,366
not. And my dad was like, yeah, you probably should stay out of trouble.

840
00:51:37,398 --> 00:51:41,046
It's easy to stay out of trouble. Then. I think I was lucky

841
00:51:41,078 --> 00:51:44,926
that my parents didn't know what a modem was, so I didn't know what

842
00:51:44,948 --> 00:51:48,686
they were getting me. Right. This

843
00:51:48,708 --> 00:51:52,462
is awesome. But I want to jump to question too sure. And ask, what's your

844
00:51:52,516 --> 00:51:56,226
favorite part of your current gig? Favorite part of my good

845
00:51:56,248 --> 00:52:00,062
gig? I think honestly, I thrive on being challenged,

846
00:52:00,126 --> 00:52:03,954
on trying to solve big hairy problems. I think that's what has always

847
00:52:03,992 --> 00:52:07,426
excited me is present to me with something that isn't being done well today and

848
00:52:07,448 --> 00:52:09,606
trying to figure out how to do it. And I think one of the things

849
00:52:09,628 --> 00:52:13,094
that I love about my job is meeting with government customers who

850
00:52:13,132 --> 00:52:16,914
have big hairy problems and looking at a variety

851
00:52:16,962 --> 00:52:20,266
of technologies. And I think what makes my role somewhat unique at intel, so we

852
00:52:20,288 --> 00:52:23,494
have like a CTO for memory and a CTO for various

853
00:52:23,542 --> 00:52:27,386
architectures is my role is pan intel so I can look

854
00:52:27,408 --> 00:52:30,534
across FPGAs server parts,

855
00:52:30,662 --> 00:52:34,462
networking, and sort of see that collective of where do the bits can

856
00:52:34,516 --> 00:52:38,350
come together to solve big hairy problems. And that's really, I find

857
00:52:38,420 --> 00:52:41,806
keeps me very excited is that every day I could be talking about an

858
00:52:41,828 --> 00:52:45,566
IoT problem today with an edge sensor, and they're

859
00:52:45,598 --> 00:52:49,118
talking about petabytes of data being processed in the cloud tomorrow.

860
00:52:49,294 --> 00:52:53,106
It's looking across the technology domains and again, coming

861
00:52:53,128 --> 00:52:56,822
from a background of cybersecurity, which again looking at various different domains from a security

862
00:52:56,876 --> 00:53:00,482
perspective, but then adding to that AI, high performance computing,

863
00:53:00,546 --> 00:53:04,386
it's a technology playground, right? And the federal

864
00:53:04,418 --> 00:53:07,430
government, when I first joined Microsoft,

865
00:53:08,170 --> 00:53:11,338
I was in the public sector, part of doing basically

866
00:53:11,424 --> 00:53:15,146
technology developer evangelism for the federal government. And a lot

867
00:53:15,168 --> 00:53:18,934
of my commercial sector colleagues were like, wow, it must be really boring

868
00:53:18,982 --> 00:53:21,360
there. I might be like, you know,

869
00:53:23,730 --> 00:53:26,880
we see things that you don't see

870
00:53:28,530 --> 00:53:31,694
and what it is, is like there's interesting work going on, but the folks doing

871
00:53:31,732 --> 00:53:35,442
interesting work for many reasons do not want

872
00:53:35,496 --> 00:53:39,234
a lot of attention. Indeed. So you see

873
00:53:39,272 --> 00:53:43,106
some things that like, wow, see, I hadn't really

874
00:53:43,128 --> 00:53:46,894
thought of that type moments. Well, decades

875
00:53:46,942 --> 00:53:50,594
ago I spent just a little bit of time in a really odd shaped

876
00:53:50,642 --> 00:53:54,214
building up that way. Just a touch of

877
00:53:54,252 --> 00:53:58,026
time. So I can have five it did. So

878
00:53:58,048 --> 00:54:01,818
I can go yes and amen everything

879
00:54:01,984 --> 00:54:05,690
you both have shared about. So now we have three. Complete

880
00:54:05,760 --> 00:54:08,970
the sentences. When I'm not working, I enjoy blank.

881
00:54:09,470 --> 00:54:13,198
Spending time with my kids. I have two small children and they keep me young

882
00:54:13,284 --> 00:54:16,846
and full of fun and keep

883
00:54:16,868 --> 00:54:19,760
me trying to stay in shape to keep up with them.

884
00:54:20,930 --> 00:54:24,580
Very cool. Both Frank and I have

885
00:54:25,030 --> 00:54:28,606
children as well. Frank has the younger kids. I'm

886
00:54:28,638 --> 00:54:32,098
probably the old guy in this conversation now that I think about it.

887
00:54:32,184 --> 00:54:35,794
But number two, complete this sentences. I think the

888
00:54:35,832 --> 00:54:38,550
coolest thing in technology today is blank.

889
00:54:39,850 --> 00:54:42,040
One thing that is a tough question,

890
00:54:43,690 --> 00:54:47,062
I would have to say. So the two things that I think are really cool.

891
00:54:47,196 --> 00:54:50,630
Number one, again, not the chat GPT, but

892
00:54:50,780 --> 00:54:54,074
what the future will do with that capability is one

893
00:54:54,112 --> 00:54:57,926
area. And then again, because I'm a security geek at heart, post quantum

894
00:54:57,958 --> 00:55:01,254
crypto is going to be fun. Figuring out the next generation of algorithms

895
00:55:01,302 --> 00:55:05,118
and how robust they'll be once quantum computing comes online.

896
00:55:05,284 --> 00:55:08,958
I think that's an exciting area of math that is going to

897
00:55:08,964 --> 00:55:12,746
spurn a lot of mathematic. Academia is

898
00:55:12,788 --> 00:55:15,780
excited because it's a renewed interest in that space

899
00:55:16,150 --> 00:55:19,934
and the algorithms are really interesting. The lattice

900
00:55:19,982 --> 00:55:23,700
space structures are fun area of math to look at. Nice.

901
00:55:24,810 --> 00:55:28,534
Interesting. The third and

902
00:55:28,572 --> 00:55:32,086
final, complete the sentence. I look forward

903
00:55:32,188 --> 00:55:35,302
to the day when I can use technology to

904
00:55:35,436 --> 00:55:39,254
blank. So I'm going to give you two answers. I look

905
00:55:39,292 --> 00:55:42,586
forward to the day when I can draw something on a

906
00:55:42,608 --> 00:55:46,234
whiteboard and it turns into code. That's one thing I'm looking forward

907
00:55:46,272 --> 00:55:49,914
to. Oh, nice. I can totally and that's not that

908
00:55:49,952 --> 00:55:53,274
far off. It's not, I think a little bit of sort of the

909
00:55:53,392 --> 00:55:56,540
image to text, image to code. I think

910
00:55:56,910 --> 00:55:59,779
building box, you have to be able to read my horrible handwriting. That's going to

911
00:55:59,779 --> 00:56:03,038
take an AI in its own right. But I would love a day. When I

912
00:56:03,044 --> 00:56:06,466
can start draw my design like I like to do I'm a whiteboard kind of

913
00:56:06,488 --> 00:56:10,306
guy, and then have it create a prototype. I think that's one thing

914
00:56:10,328 --> 00:56:14,018
I'm looking forward to. And then I think

915
00:56:14,104 --> 00:56:17,766
the other thing is I'm looking forward to the day when

916
00:56:17,948 --> 00:56:21,702
augmented reality becomes reality, where it's not just

917
00:56:21,756 --> 00:56:24,630
a cool toy, but where we actually see it integrated

918
00:56:25,450 --> 00:56:28,854
into our daily lives. And I'm not talking to glasses and all that. I'm talking

919
00:56:28,892 --> 00:56:32,326
about having the digital world and our physical world actually start to make

920
00:56:32,348 --> 00:56:36,042
sense instead of it being a throwaway toy and I think we're seeing

921
00:56:36,096 --> 00:56:39,274
pockets of it, but I think that the future is going to hold a lot

922
00:56:39,312 --> 00:56:43,146
more of that immersive experience that we only see in movies today. I think

923
00:56:43,168 --> 00:56:47,006
those are the two things from a technology perspective, I'm looking forward to.

924
00:56:47,108 --> 00:56:50,394
Although I have to say, if I can get that, the code from the whiteboard

925
00:56:50,442 --> 00:56:54,178
is going to make me a lot more efficient. No, that's true. And

926
00:56:54,264 --> 00:56:57,250
it's funny because things that once seemed impossible

927
00:56:59,030 --> 00:57:02,834
are now possible and even mundane. So I remember

928
00:57:02,872 --> 00:57:05,298
when I was a kid, there was a story, there was like a story we

929
00:57:05,304 --> 00:57:08,706
read about a kid who wrote a built a homework machine, right? And this was

930
00:57:08,728 --> 00:57:12,006
like first or second grade and a bunch of us kids were like, yeah, how

931
00:57:12,028 --> 00:57:14,886
do we do this? We got to make one of those. Now you look at

932
00:57:14,908 --> 00:57:18,754
Chat GPT, obviously we abandoned the effort

933
00:57:18,802 --> 00:57:22,506
because it just wasn't possible at the time. But you look at how kids

934
00:57:22,528 --> 00:57:26,090
are using Chat GPU today, that machine exists

935
00:57:26,510 --> 00:57:30,106
not in the way or the shape or form we could have imagined, but

936
00:57:30,208 --> 00:57:33,582
it's definitely here. So to have that whiteboard to code

937
00:57:33,636 --> 00:57:36,830
thing, it's totally

938
00:57:37,970 --> 00:57:41,806
within sight. Whether it'll be within reach, only time will

939
00:57:41,828 --> 00:57:45,426
tell. Probably a few weeks. If there are VCs out there listening, this is an

940
00:57:45,448 --> 00:57:48,850
idea to invest in, for sure. I would love to see

941
00:57:48,920 --> 00:57:52,542
especially for you, Steve. I'd love to see whiteboard

942
00:57:52,606 --> 00:57:56,194
two FPGA code. That'd be even

943
00:57:56,232 --> 00:57:59,734
better. We're just combining ideas. There you go.

944
00:57:59,852 --> 00:58:03,526
I know that would make some of my engineers happy. There you go. Really

945
00:58:03,628 --> 00:58:07,414
cool stuff. So we ask all of our guests to

946
00:58:07,452 --> 00:58:11,078
share something different about yourself. But we caution

947
00:58:11,174 --> 00:58:14,874
everyone to be fair that remember, we're trying to keep

948
00:58:14,912 --> 00:58:18,746
our clean rating at itunes, so please keep that in

949
00:58:18,768 --> 00:58:21,360
mind. So something different about me.

950
00:58:22,050 --> 00:58:25,546
Well, I guess one thing we've already talked about that I have a bio

951
00:58:25,578 --> 00:58:29,134
background, but the other thing I like to do is I play

952
00:58:29,172 --> 00:58:32,714
tournament poker. I am an avid

953
00:58:32,762 --> 00:58:36,466
poker player when not in COVID Lockdowns and things like

954
00:58:36,488 --> 00:58:39,982
that. I played in the World Series back in 2013.

955
00:58:40,126 --> 00:58:43,778
Really? That's something I like to do as a

956
00:58:43,784 --> 00:58:47,282
past. It's a different use of my skills, of sort of social

957
00:58:47,336 --> 00:58:51,046
engineering, if you will. And I like the tournament play

958
00:58:51,068 --> 00:58:54,678
because it's sort of a long game. Right? Well, I have a

959
00:58:54,684 --> 00:58:58,266
stack of money and I'd love to learn more about

960
00:58:58,368 --> 00:59:02,122
is that the joke? All you need is you're always

961
00:59:02,176 --> 00:59:05,866
welcome to my table. I'm lying about the

962
00:59:05,888 --> 00:59:09,514
money. My wife is

963
00:59:09,552 --> 00:59:13,180
actually a pretty good poker player, and when she was pregnant with our second,

964
00:59:15,070 --> 00:59:18,846
she's short and she would carry a stool with her because she would have

965
00:59:18,868 --> 00:59:22,046
to set up and her feet didn't reach the floor. And I think I gave

966
00:59:22,068 --> 00:59:25,730
her like $100 in seed money and said, go knock yourself out.

967
00:59:25,800 --> 00:59:29,122
And she came back like she was spending money. I think she turned that into

968
00:59:29,176 --> 00:59:32,434
something like two grand before she had to quit and go have

969
00:59:32,472 --> 00:59:36,226
Emma. I

970
00:59:36,248 --> 00:59:40,066
would love to see you, because I don't think she's

971
00:59:40,098 --> 00:59:43,558
your level by any stretch, but she did okay. We should have

972
00:59:43,564 --> 00:59:46,806
a data driven poker tournament. We should. There we

973
00:59:46,828 --> 00:59:50,506
go. That's an idea, Frank. The other time we had an

974
00:59:50,528 --> 00:59:53,930
idea of somebody on the live stream said we should do like an ATV

975
00:59:54,910 --> 00:59:58,730
race or something because we always go off track. That's kind of the joke.

976
00:59:59,870 --> 01:00:03,706
Very true. But no, that's cool. Audible is a sponsor

977
01:00:03,738 --> 01:00:07,466
of data driven can you recommend a good book? Ideally

978
01:00:07,498 --> 01:00:11,246
audiobook if you do, audiobooks if not. Sure. Absolutely. Actually, I just

979
01:00:11,268 --> 01:00:14,206
finished one that I think would be perfect sort of summation of this. So

980
01:00:14,228 --> 01:00:17,460
Chips is an excellent book.

981
01:00:19,270 --> 01:00:22,866
You think it's talking about today, but it gives you the history of how we

982
01:00:22,888 --> 01:00:26,706
got here. And even one of the things I thought was really interesting is

983
01:00:26,728 --> 01:00:30,022
some of the decisions that were made early on from the

984
01:00:30,076 --> 01:00:33,526
policy, the government policies that we've seen and how it

985
01:00:33,548 --> 01:00:37,222
affects where we are today. Fascinating reading. So, yes, absolutely.

986
01:00:37,276 --> 01:00:41,074
Chips wars, it's available on Audible because I literally just finished reading

987
01:00:41,122 --> 01:00:44,426
listening to it on Audible. So that would definitely be a book I would

988
01:00:44,448 --> 01:00:48,054
recommend. Cool. I watched a show called Halt and Catch

989
01:00:48,102 --> 01:00:51,486
Fire a few years ago when it was at, and it was similar. It was

990
01:00:51,508 --> 01:00:55,118
in that vein of when things were developing and trying basically

991
01:00:55,204 --> 01:00:58,974
the laptop development story. And of course it was

992
01:00:59,012 --> 01:01:02,766
fiction, but I know enough about it to

993
01:01:02,788 --> 01:01:06,606
know there were some true parallels in there. So this

994
01:01:06,628 --> 01:01:08,818
would be very appealing to me. I'm going to get it. I hadn't heard of

995
01:01:08,824 --> 01:01:12,114
it. Thank you for recommending and our listeners can go to

996
01:01:12,152 --> 01:01:15,918
thedatadedrivenbook.com I didn't test it today, Frank.

997
01:01:16,014 --> 01:01:19,606
Some days it's moody, but if you go there, it should

998
01:01:19,628 --> 01:01:23,446
redirect you to Audible. And if you decide you get a free book on us.

999
01:01:23,548 --> 01:01:27,394
And if you decide later to sign up, then it buys

1000
01:01:27,442 --> 01:01:31,286
Frank a cup of coffee. So when

1001
01:01:31,308 --> 01:01:33,338
you do that, we get a little bit out of it. It's a great way

1002
01:01:33,344 --> 01:01:36,540
to support the show and we really appreciate it.

1003
01:01:36,990 --> 01:01:40,220
Awesome. And where can people find out more about you and

1004
01:01:40,830 --> 01:01:44,634
what the federal team at intel is doing. So find out more about

1005
01:01:44,672 --> 01:01:48,334
me, go to my LinkedIn page. That's S-O-R-R-I-N on

1006
01:01:48,372 --> 01:01:52,106
LinkedIn. And then to find out more of what intel is doing in public sector,

1007
01:01:52,138 --> 01:01:55,746
just go to Intel.com public sector and it will redirect you to our

1008
01:01:55,768 --> 01:01:59,246
Government Solutions page. It covers everything from AI

1009
01:01:59,358 --> 01:02:02,834
data science to Cybersecurity to Edge, with lots of white

1010
01:02:02,872 --> 01:02:06,706
papers. Use cases podcasts with folks like myself and

1011
01:02:06,728 --> 01:02:09,974
others that are recording content on how intel is helping our

1012
01:02:10,012 --> 01:02:13,798
ecosystem. So definitely come check us out. Awesome.

1013
01:02:13,964 --> 01:02:17,462
And with that, I'll let Bailey finish the show. Now that was some

1014
01:02:17,516 --> 01:02:21,254
show. Is it me or are the shows getting better? It could be my

1015
01:02:21,292 --> 01:02:24,774
bias that leads me to say that, but I figured I would ask to get

1016
01:02:24,812 --> 01:02:28,614
more input. After all, what's an AI without good

1017
01:02:28,652 --> 01:02:32,286
input and a feedback loop? Speaking of feedback, have you

1018
01:02:32,308 --> 01:02:36,106
checked out Data Driven magazine yet? We are looking for writers

1019
01:02:36,138 --> 01:02:38,400
for the Autumn 2023 issue.