Welcome to show 350 of data driven.
In this episode, Frank and Andy interview Chris McDermott,
VP of engineering at Wallaroo. Wallaroo helps
customers operationalize machine learning to ROI in the cloud,
in decentralized networks, and at the edge. It's
a fun conversation on MLOps and the future of intelligence systems
and model management. Now on to the show.
Hello, and welcome to data driven, the podcast where we explore the emergent
fields of artificial intelligence, Data science and of course, data
engineering, the fundamental thing that kind of underpins it all. And with
me on this epic road trip down the information superhighway
Is my favorite data engineer of all time, Andy Leonard. How's it going
Andy? Good Frank, how are you? I'm doing alright. I'm doing
alright. We just, we're chatting
in the, virtual green room about some of the logistical challenges we had,
with Microsoft Bookings and how Kind of like you can only have, you
know, like, remember that the pick any 2 triangle, right? Good, fast, and
cheap? Yep. Yep. Like, we can only have 2 things, 2 features of what we
needed to do. Right. Alright.
Despite logistical challenges, we are excited here to have,
Chris McDermott who is the VP of engineering at Wallaroo,
and, he is a passionate, and intellectually
curious professional With excellent communication skills, he
loves hard problems, then he must have definitely loved the
process to get on the show, And,
have yet to meet 1 he couldn't solve somehow. Maybe we should get you,
Chris, to help us with our scheduling stuff. Really? You visit
later? Yeah. So welcome to the
show, Chris. Thank you. Thank you. It's great to be on. It's nice to meet
you both. Well, likewise. Likewise. So you're coming to us from the,
Mile High City. That's right. Awesome place. It's, I was
there once, for internal Microsoft
conference actually. Oh, nice. And beautiful town, like, it was
just really cool. I think it was the 2nd biggest
event that in Denver history was the Microsoft thing. Wow.
And they they literally ran out of hotel rooms like it was.
Oh, wow. It was pretty wild. Yeah. I think it was, just
before one of the big parties had a convention there. And,
they Oh, yeah. Yeah. Yeah. Yeah. I was so I'm actually
slated to head back there next year for a Red Hat
conference, so we'll see Let's see if the hotel situation has
improved. I think it's improved a little bit. The city's been growing a lot. So,
Yeah. Lots of government. Isn't Denver the place that has, like,
the large bear up against the conference center that
Yeah. Yeah. Yeah. Yeah. Yeah. That's exactly right. A giant blue bear appearing in the
window of the conference center. Yeah. I was there. And,
and and I remembered that. That was the That was the first thing I
remember. It was, I was there in
2007 for a Kind of a Microsoft conference. It was
a, Professional Association for SQL
Server. That's what it was called back then. And, I was
actually the first one I spoke at. I've spoken at a bunch since then,
but 2007 in Denver was the first. And,
yeah. Like, I echo what Frank said, beautiful city
and, just very picture picturesque. Yeah.
Yeah. The weather in the mountains are beautiful. Mhmm. Yeah. And
it's funny, like, you know, on the East Coast, we talk about mountains,
but It's nothing like that. Like Right. Yeah. We quite the
same. You would laugh at what we call mountains. Yeah. Right. But
I remember a Robin Williams bit Where he said something like that
people he admired the people in Denver because they got to
Denver and they looked at the mountains and went, Well, I can't say what
he said, but he had a kind of an Elon
moment.
There's so many of those. There's so many No more. We're stopping right here. We're
not going over those mountains. So,
You're VP of engineering at Wallaroo. So tell us a little bit about Wallaroo.
Mhmm. Plus you're also ex data robot too. That that's interesting.
Yes. Yep. Exadata robot. Yeah. So I've been working in the machine
learning and AI space for, about 7 years now, I guess, or 6
years. And, it's been really fun. You know, it's, it's a
good time to be in the business. There's a lot of development
happening, very fast pace of change, which I appreciate.
And, you know, Wallaroo has been really great. Like,
the team is fantastic, and the people are wonderful. And it it's a lot of
fun working, with people that you enjoy hanging out with and and you respect
and everything, that's that's very important to me. That's awesome. But I also just
I think the product is awesome. It's really, I think,
playing well in the market. Like, we are focusing on making it as easy as
possible to deploy And manage machine learning models.
And the focus is really on any model using any framework and being
able to deploy onto sort of any architecture, any hardware,
and being able to leverage GPUs if you need them or different kinds of CPUs,
different acceleration libraries that people have tailored to the different
architectures. And, honestly, there are not a lot
of other solutions that tackle those 2 problems for people. Right.
A lot of the other companies that we're competing with, they are trying to be
like an end to end solution or, like, really force people into, you know, their
platform. So you train on their platform, you deploy on their platform, you manage on
their platform. But it's very limiting in terms of what you can bring on to
the platform and and being able to, deploy on the different types of
architectures and, platforms and things like that. So it's really
exciting. It's fun. I think that's really important that you bring up
the CPU solutions. As I've been tinkering,
you know, the past couple of years with, you know, with the
different, different platforms that are out there, it's
That's definitely a smaller market, but maybe it's emerging now. I'm
just not sure. Mhmm. Yeah. I wonder yeah. Sorry. Go ahead.
Well, I was gonna say, you know, a lot of the time people conflate training
and inferring, which is, you know, sort of the 2 different stages. Like,
1st, you have to train a model, but then you use the model to make
inferences, which, you know, it's really like asking the model to make a prediction
or you give it some input and it gives you some output. And,
they're they're very, very different tasks. And just because, you know,
like, you may wanna use some hardware GPUs for training Doesn't
necessarily mean mean that you need the GPUs when you are in production and
you're asking it for predictions. A lot of the time, you know,
The model is small enough that you really don't need to, but there's
so much hype. It it's hard sometimes to separate the hype from the, you
know, The real stuff and Yeah. Yeah. The hype the hype
machine is real. I mean, like, it's and and and I I wanna get your
thoughts on, you know, I mean, I love generative AI. I'm not
knocking generative AI, but it feels like it's taken all the oxygen out of the
room for All the other kinds of AI.
Yeah. Yeah. Yeah. Because there are a lot of, you
know, great models. I like XGBoost is a very standard one. It's
been around for, you know, a long time, meaning at least for, you know, 5
or 10 years now or something. But, that really honestly solves so
many problems, and it's such a Small, easy model to deploy.
I I wish people would focus more on on that kind of thing rather than
hype. Right. No. That's a good point. And I think you
bring up an interesting point because not all not
all AI workloads are created equal. Right? Obviously, there's,
I heard this term the other day and I had to spit my coffee out
because it was just so funny. Legacy AI. Yeah.
Yeah. There's generative AI now. There's legacy AI. That's
crazy talk. You know? And I was just like,
wow. But,
you know, because, you know, legacy AI, basically,
you're not using deep learning, you're not using neural networks,
Generally, you don't get a good boost from GPU's.
Correct. Right. And that's something that when when you tell that to
Even tactical decision makers, they they they
kinda look at you like, you know, what sorcery is that? Like, you know, because
they'll they'll They'll say, like, oh, we don't have enough GPUs. There's no budget for
GPUs. Like, what what what types of workloads are you running? And I
tell them, it's like, well, it's not really a concern for you. Like, you don't
need them. Yeah. And you see, you know,
the the the the people who are doing the actual data science, they're like, yeah,
duh, that's what we're trying to tell you. Yeah. But you see, like, the leaders
of these teams are like, like, you know,
it's, now Just for my own education,
there wasn't there something called RAPIDS, and it was an
acronym that let you use GPU's For
certain types of like XGBoost, I think was one of them. Random
forest there. I don't know. Oh. You See, it's funny because
it was an it's an NVIDIA thing and obviously it only optimizes on. But,
like, it was I remember Hearing about
it in 2019, and I'm thinking, wow, this is gonna change
everything, and you haven't heard of it. Only,
like, one ever per other person I met in the wild has ever heard of
it, and he was at the same conference I was at where we heard about
it. So I'm like, That's kind of unusual,
but, you know, we gotta watch so
fast, you know, and it's really hard to tell sometimes What
what, which new developments are gonna end up being the future and which ones
are gonna end up as dead ends? Right. You know, and even all
the transformer stuff that that is powering GPT and and those similar types
of models, I think that was originally written up in a
white paper in, like, 2017 something. Mhmm. And it just kinda sat around for a
while, and nobody paid a whole lot of attention to it until OpenAI really
ran with it. So yeah. Pension is all you
need. I think that's was that the paper? Sounds right.
Yeah. And then we're gonna go. Oh, sorry. Go ahead. Sorry,
Andy. I cut you off your point. No. I I don't wanna go too far
downstream before I say cred boost for using the phrase I don't
know. Oh, nice. Somebody with your
credentials, you know, saying I don't know. That's that's super
cool. So cred Honestly, there's way too much to know. There's no
way anyone person could know that. I I like to joke. I
haven't checked my phone or, like, news Feed in like half an hour
and I'm like woefully behind now. Yeah.
But it feels that way like in the whole Oh, no. It does. Yeah. Especially
it was especially interesting when the whole drama on OpenAI, and I
don't wanna go down that rabbit hole too far. But when all of that soap
opera kinda unfolded Yeah. Yep. It was kind of like,
what's the latest? Like, is he back? Is he gone? Is he working at Microsoft?
Like, he did work at Microsoft for like 10 minutes, and now he doesn't.
Like, Yeah. You know, at
at some some point down the middle of it, it's like, call me when this
is over, and I'll deal with the, things yeah. I'll
check-in again. But that's just the human
side of it, let alone the let alone technology side of it.
So Operationalization. I think that's gonna
be the buzzword. Obviously, chatty b t and JennyIs, taking
all the air out of there. And I think the next buzzword It's gonna be
operationalization. 1, because it's kinda
hard to say, and I'm not gonna lie, I've had to practice.
But, it's something that I think companies and organizations
that adopt AI, whether it's legacy AI
Or generative AI. They're gonna have to realize, like, it's one thing to build
the model, and then it becomes a, okay, now
what? Yeah. Yeah. Well and models
really are just like any other software. It's not something that you just
write once and you, you know, Throw it out there, and it runs forever
without being touched. Right? All of it requires care and feeding, and
and machine learning models are no different. So, I think
part of it is, you know, how do you deploy it? And then, you know,
how do you keep that that deployment up to date, you know, getting critical
patches and vulnerability fixes and things like that. But also
how do you monitor the model and how it's performing and how it's performing
relative to the real world, Because the world doesn't stand
still right. So even if the model was trained on some data and it was
98% accurate when it was trained, as the world shifts and
and the situation around it shifts, that accuracy will
almost certainly start to degrade over time. So You need to monitor that. You need
to know when to retrain the model. And you have to be kind of
keeping track of, new training data. Right? So the
the the new environment that the model is operating in, you need to be recording
all of the the inputs and also paying attention to the ground truth of, You
know, what was the outcome of that prediction that the model made? Was it accurate
or not, after the fact? And and correlating that back into your
training data So you can retrain the models and, you know, keep them going
over time. And that's just, you know, assuming you're gonna be using the same model
forever. But as we just finished talking about new models coming out all the
time, new approaches, new techniques. So, yeah, it really is
is something you've gotta pay attention to. It's an
extremely Yeah. It's an extremely dynamic space.
Mhmm. I've heard this called
MLOps for the longest time. Mhmm. Mhmm. But I've also heard a new term
kinda pop up on the radar called AI ops Mhmm.
For this. What do you call it? I
generally call it MLOps. You know, one, I
I sort of per like, AI and ML, there's an
interesting, you know, difference there in in terms of who uses the different terms and
when they use them. For me, AI is
more of a general term that I use conversationally. And most of the time if
I'm trying to be fairly technical and specific, I'll usually revert to ML,
Because in fact, most of these things are machine learning. AI is a much more
nebulous concept, and I I don't even think everybody agrees on on what AI is
or What the threshold would be, you know, if you're
doing statistical analysis, I think most people probably would not call that
AI. But there are a lot of machine learning models that do work that way.
And and that's definitely, like, part of the gradient. You know? I've
noticed that too. Like, there it is a gradient too. Like, there's not a, like,
a hard, like, You know, typically it depends on the audience. Right? If they're
if they're BDMs, business decision makers, they're gonna use
AI. Yeah. They're technically focused people. They tend to prefer the term
ML. Yeah. That's also been my experience Interesting.
Quite often. So I like MLOps because, one,
it sort of grounds you a little bit more in that technical perspective. Mhmm.
And, and it's sort of a like, To me, I think I came up
through DevOps a lot of my, you know, first half of my career was DevOps
and infrastructure and things like that. And, I
think part of the appeal of the term MLOps is it taps into a
lot of the DevOps, associations. Right? And
Right. The concepts and the themes of DevOps, which is really about,
merging different skill sets and breaking down silos and getting different teams to
communicate with each other and And to collaborate more,
being more dynamic. Not just, you
know, putting software out there and and letting it run forever, but Keeping it up
to date, monitoring it, recording the logs, you know, all of that kind of
stuff, and and getting into a flow of continuous
deployment, you You know, continuous integration, continuous testing, continuous
deployment. And I think on the ML side, that's also where
MLOps really shines and and is bringing those themes
to the party, rather than a data scientist training a
model, deploying it, and, you know, Throwing it over the
wall to to, like, an operations team or something. It's
getting all these different teams and skill sets to work together. It's
building a continuous, you know, pipeline, with
monitoring and and feedback loops and so on. So that's that's why
I like MLOps. No. I like that too. So in order to prevent
any hate mail come in or or but actually comments, AI
ops is also used, I've heard, in,
the telcos and network operators tend to have a term
called AI ops, where they use AI to help operate their network. So that is
Got it. It's it's a it's a namespace collision,
which I've free further which I prefer MLOps for to avoid the namespace
collision, plus all the reasons you said. You
know, what's interesting is and I came from a software engineering
background and, you know, and I'll be honest, I was not
initially a big, believer in in DevOps, but
kind of as time went on, I became a convert. But I think
that, you know, when you look at how AI models, ML
models, whatever, how they get operationalized.
You look at it And I I often I often can tell
who the fans of the new Star Wars movies are by using this analogy,
because I'll say it's The 2015 Star
Wars movie and the 1977 movie, DevOps.
DevOps being kinda like the original, episode 4 And then the
new, the the first new one, right? It's the same
plot. I mean, the characters have changed, some things are
different, But very effectively, it's the same plot. And, you
know, some people will laugh like you did, and some people I can
see will, Their their faces turn red. But,
but I mean it's like it's it's the same plat plot. The names, the places
have some have changed. But you're right. I mean, I think and there's a lot
of lessons we can learn Yeah. In the ML community
from the DevOps world. Right? Because, You know prior
to DevOps, you know, the developers and operations had a
very antagonistic relationship for the most part. I'm sure there's always
exceptions. You know, I was I was joking that they would only meet,
they only have to interact 3 times a year, and one of those was the
holiday Christmas party. You know what I mean? And
Yeah. But if you wanna deploy something in a far more agile
way where they have to, you know, you put it In some extreme cases, every
few hours, some new bit of code gets gets pushed up. That's obviously on
on one fore end of the spectrum. But for the most part, you know, a
couple times a month is not unreasonable. You have to automate that. You have to
have processes in place. Yep. And I I see a day, and if that day
has not already come, I would be surprised, That AI is gonna be the
same thing or ML. Right? You're you're gonna have to get but to your point,
right, this is a continuous process, You know? Yep. Yeah.
We can't get away with, you know, you have this isolated team of data scientists.
They they they kinda go off to their little area 51 type labs
in secret. Right. I then come back with some model,
and and I'm guilty of this too. I've done this. Right? Where I'm like, I
built the model. I'm done. I did the math. I did the hard
part. How do you get the play it? Not my problem. Not my
problem. And it's funny, like Yeah. You know, I caught
myself. Right. I caught myself doing that as I, you know, you
know, doing that. Like recently, I had to I had to do a demo
and I had to work on a kind of a It's basically a predictive
maintenance type thing, and I took all this data, had the model, and I
just said, here's the here's the link to the model, Have at it,
pal. Mhmm. And then as I sent that, I was like, you know, I should
probably be more involved in getting this on a race for it.
Right. Yeah. Yeah. Yeah. Yeah. No. I think that's a big part of it.
Another big part of it though is, scale, you know. And I think scale
scaling of compute and, how How people were using compute and how
much compute was required was a big part of what drove DevOps.
You know, if you were a sysadmin responsible for a 100
servers, That's, you know, challenging, but it's feasible. Like, you can do
that. You can keep them all up to date. You can keep them all in
sync with each other. Make sure they they all have the same patch levels and
and so on. But you scale that up to a 1,000 servers?
That gets a lot trickier. You try to go to a 100,000 or, you know,
if you're doing Internet scale things like Google or Facebook or somebody, We're talking
millions, tens of millions. And Right. That level of scale
requires you know, everything has to be automated. Everything has
to Has to work that way and it has to be resilient and it has
to, you know, have automatic fail over and stuff. You know, there's the,
x k CD where they're, You know, they get to a certain point. They're just
roping off entire data centers and being like, alright. We're throwing that one away and
moving on to the next one. And for AI, I
think a lot of the same stuff is happening. When, you know, 10 years ago
or so when when people were just getting started on this journey And as an
entity, as a business entity, if you're talking about 1 or 2 use
cases, you know, you can have humans curate that stuff and hand
craft it, hand roll it, hand deploy it, and hand manage it. But
if you're a a big enterprise company and you you wanna have hundreds of use
cases in production or thousands or tens of thousands, there's just no way.
You have to automate it. No. That that that's a that's
an excellent point. Like, one way I've heard to describe is that if you're
baking a loaf of bread for your family and friends Or loads of
bread. You can do it in your kitchen. Right? You don't have to do anything
special, but if you're the Wonder Bread Corporation or
Mhmm. And you wanna deliver at that scale, that's no longer an
option. Mhmm. And I think that we're at that point where and
correct me if I'm wrong, where I think AI and ML adoption or AI
adoption is still new enough where there's enough of naivete out
there of, oh, we don't need to scale to that degree. Like, we don't need
the production line. I think I think that's ending. I think we're getting close to
the the end of that era, but that's kind of been my yeah. I think
so too. Yeah. Because they're they're more and more, ML
tools in everybody's toolbox. Right? So you were talking about telcos
routing network traffic using ML models. That's not
gonna be 1 model. Right? Like, with latency and and
everything else, you're gonna need, you know, Very small. Lots
and lots of very small models deployed on every router, every top of rack
switch, every, you know, whatever 5 gs cell phone tower,
whatever you're talking about. There are a lot of cell phone towers. So you're
not managing 1 model. You're managing a fleet of models, right, across
different geos and all kinds of things. No. That's that's an
excellent point. Sorry, Andy. That's okay. It does seem to scale like that,
though. Right? It's almost it's almost tectonic. There's
a whole new layer going down. You know? That's that's the new surface.
I noticed on the website, I I popped over to wallaroo dot,
aiwallar00.ai.
And I noticed a familiar looking, blurb just
below the top of page. And it's familiar to me because, I
started off in business intelligence. I'm still working in BI.
And there's a note, 90% of AI
initiatives Failed to produce ROI. And I saw this
in, you know, it's very similar number, 85% in,
in BI back in the day. It's probably still true. So where
does that number come from? Well, I think it reflects a lot of
things. You know? Some of them we were just talking about and
and where MLOps is coming from is is, a lot of the failure
modes were teams not really working with each other. Right?
Somebody decided we should be doing AI, so they hired the data scientist.
And the data scientist works in the corner for a while and,
You know, 1, they don't have access to all the data. They don't know what
the data is, where to find it, how to access it, how to clean it,
what it means to the business. There there are a whole set of challenges there.
And then, you know, they may train some models and and get something, you
know, to a point where they think that it's gonna solve a problem. But Then
you've got to work with an IT organization to stand up infrastructure. You've got to
work with somebody to package the model and build, you know, an API around
it or a UI of some sort And figure out how to deploy
it, train people on how to use it, and and actually, like,
somehow integrate it into your business process So that it's
it's driving business outcomes. And all of those are really tough
challenges. And all of them require breaking down those
silos and getting a bunch of different People within an organization to
talk to each other and communicate and to work together to solve something.
I don't think ML or AI is is a magic wand that you just
wave and magically provide value to a business. You've got to really
think about What is your business doing? And, you
know, machine learning at at heart, it it's
really just like a a more efficient way of
Making decisions, you know, faster and more accurately,
and with less human input. And so you've got to look for places where your
business can either save a lot of money or make a lot of money by
being able to answer a a simple question repeatedly very,
very efficient. And that sounds easy, but in practice,
defining a business problem is often one of the hardest parts. So now I'm
seeing even more parallels. Uh-huh. Yeah.
You know, that was the problem we were trying to solve, with
business intelligence as well. So didn't mean to cut you off. Sorry about
that. No worries. So I yeah. I think I agree with you. It it there
are tons of parallels there. I think there are a lot of similar lessons to
be learned, and I think we are applying them in this In this space in
ways that we've applied them to other spaces in the past. I also
think there are technical challenges. You know, part of it is the field is moving
so fast. So there's just this constant stream
of of new frameworks, new models, new techniques, and you
have to kinda stay on top of that. You have to be careful with your
tool selection, to make sure you're not, you know,
going whole hog into some tool. That sounds
great today, but it's just not flexible, and it's not gonna be able to support,
like, all these new things that are coming out. Yeah.
Or that company could have internal internal political
strife, which was crazy talk. Right? Cast Absolutely. Right. Cast
doubt on their future. Alright. That would never happen. That would never
happen. Sorry. Yeah. You were talking about privacy, which I think is
another key thing. Yeah. Data residency, data privacy, see data
security. You know, all of those things matter tremendously.
And for for a business trying to, get
value out of AI and ML. You know, a lot of it, depends on
having good data and, Cleaning it and curating it
and getting it ready for things. But then it it forces the
the organization to really kind of do an inventory. What do we have? What's useful?
What's not useful? Well, how much do we store? How much do we not store?
How do we comply with various regulatory
environments? Right? GDPR is is the big one everybody, you know,
loves to throw out there. It's it's big and it's complicated, but, you know,
things like that matter a lot. And And there's 300 +1000000 people behind
that. They're covered or whatever. I think that, you know, that that
is not only do they have a big stick, but they have a big arm
that they can wave that stick wet. Yes. You
know, if if a small country with, like, you know, 50 people in it, and
that could something like GDPR, people would just walk around it. But I think
that, a block with I've heard different numbers, but
it's for, you know, pushing 4 to 500,000,000
people. That's a huge that's a big enough market nobody can really ignore.
Yeah. What's interesting is on the LinkedIn page
for Wallaroo I love the website, by the way. I checked that out too. Thank
you. It talks about decentralized
networks Mhmm. And at the edge. Yes. What how would
you define decentralized network? Yeah. This is a big new push for us that we've
been focused on for, I mean, we've been focused on it kinda for the
last year, but it was a lot of, development on on the back end. And
we just released kind of our 1st edge features and product,
in October, so it's kind of a new thing for us. But,
As you think about ML and edge or ML and AI,
and the the fleets of models that we talked about and all these use cases
And, you know, telcos and and five g cell phone towers and all of those
types of things, intersecting with data and data
residency and privacy and security, It it really seems to
indicate to me and and to us at Wallaroo in general that the
future is lots and lots of models being deployed in lots of
locations. And I think that one
big sort of industry wide theme that I'm seeing is if the
last 20 years, let's say, was the story of Everybody
picking up from their colos and moving to the cloud and centralizing
all of their IT, I think that the next 20 years are gonna be
Not like deconstructing the cloud. I think the clouds are here to stay and they're
gonna continue to grow, right, year over year. But there will be more
of a push out to more edge computing environments. Cell phones
are getting more and more powerful. Cars are getting more and more powerful. Like, there's
more computer stuff happening, all over the place, and the compute
available, the memory and the storage available is all through the roof compared to
what it was 20 years ago. And, I think we're
gonna see more push for smaller, more specific machine learning models, And
they're gonna be pushed out to all these edge locations so that they can run
close to where the data is. So you're not schlepping this sensitive data all over
the Internet and other people's networks. Yeah.
But, you know, you're taking advantage of of compute resources that you
have local to the data and making very fast decisions,
you know, very efficiently. So I I have to jump in
because, you you just made me feel really good.
About a year ago, I built a large server here
at home, which I hadn't done in a decade. Actually, my my
20 year old son built it. But he and he helped me with,
with picking out the new shiny fast parts, on it because I was
so out of practice with this such confessing.
But, and it's really cool to see, you know, all of his All of
his skills. He does edge. We just picked up the
Raspberry Pis are back in stock, finally. Yep. And I just picked up,
like, 3 for $35, You know, the 1 gig force.
Yep. Anyway, super excited about that. One of the things I built
at the time I built a box About a year ago, you
couldn't do a local GPT or anything close
to that. And I said, Eventually, we're
gonna be able to do this. I I made that guess, and it was a
guess. Yeah. But about 6 months later, about 6 months
ago, All of a sudden, I started seeing these 7,000,000,000
token machines showing up and it started clicking.
It was like, holy smokes, you can do this. I did make one stupid mistake
and he didn't catch me on it. I bought a 12 gig GPU
because that's super crazy huge From 10 years
ago. And that wasn't super crazy huge at all. No. No.
No. But it's interesting. Now they're back now. They can run on, You know, on
the 12 gigs. And like you said, you mentioned the CPU models. So I just
learned a ton as I've been going through this. And, That
it's it's very encouraging to hear that. I had not heard anybody
say edge and running small ML models on the edge.
That's, I mean, that's what we've been trying to do here. And I I love
the redundant you know, the idea of a redundant array of whatevers,
you know, MLs. It's almost like a swarm of MLs. I've heard,
yeah. Yeah. Yeah. That's true. Right? And, you know, I think there's a lot
of interesting stuff happening on the battlefields in Ukraine right now drones.
And Right. That Yeah. Was also a fascinating space and
very much, I think, heading in the direction of lots of ML running at the
edge. It's it's funny you mentioned that. So I live in a DC area,
and, I was at a government tech
symposium about 2, 3 weeks ago now. And
they were talking about that that, you know, edge is gonna be much more important
in the future of warfare. And he said presumably
elsewhere too. Right? He was permanent primarily a government in defense. It was definitely a
military industrial complex, type of type of event. But he was
explaining like, you know, in the past, you know, 20 years,
we've not dealt with adversaries. We've
only dealt with adversaries in in battle space conditions
where it was, you know, we controlled the airwaves.
Mhmm. And he, I think he used an interesting term. We
had airspace and electromagnetic electromagnetic
dominance. I was also like, Wow. Yeah. That was yeah. Yeah. I was, like, oh,
that's interesting. So, like, the whole idea of these disconnected
decentralized networks, I mean, I think
you're I think you're spot on. It's the future for
geopolitical reasons, but also just for, you know,
Privacy and just kind of flexibility reasons. Yeah. The
question I have though is, like,
Organizations can barely manage the infrastructure they have now and barely manage
the software they have now. What are they gonna do when the software starts Not
thinking for itself, but, like, this becomes another workload Yeah. On
top of that. Like, what Well, for one thing, that's why Wallaroo It
is focused where we are, and we're trying to build this platform to help people,
you know, with this capability of being able to deploy models and manage a fleet
of them at the edge. Because, yeah, there aren't a lot of good
solutions for that today. Yeah. Interesting. I I think the
general answer to your question is probably some combination of cloud and edge.
You know, like, it does make sense to centralize a lot of things, and it
makes the the maintenance easier and, more efficient. And
You can get some economies of scale and, you know, all that kind of stuff.
But, we are gonna have to get good at managing a bunch of,
disparate types of things in desperate locations. I think all of
us. Interesting.
So this is the part of the show where we'll switch over to
The premade questions, and for your convenience,
I will, paste that in here.
Hopefully, paste it. And there we go.
So You had an interesting career looking at LinkedIn. You were at
SendGrid. You were then you were at DataRobot, and you said you made a switch
into the the data world, which begs the question, How did you
find your way into data? Did data find you or did you
find your way to data? I I
guess that is a good question. I think that, it was probably a
little bit of both.
Finding my way to data, I think that the beginning of the story is probably
at SendGrid. And I joined SendGrid as a DevOps engineer.
And to be honest, I had not really heard of SendGrid at the time. I
knew a little bit about it, but it, you know, I didn't really understand what
it was, too much with the scale. SendGrid, by the way, is now owned by
Twilio. But they have an API for sending email, and
they make it just really easy to integrate with, websites and applications
and and software so you don't have to worry about SMTP and, you know,
DKIM signing and all the other, like, gnarly bits of of
email. Turns out that Sengrid had a
ton of data. They're handling billions of emails a day,
and, you know, there's a lot of metadata there. The the actual data of the
email and and so on, the recipients and who to send it to and all
that stuff. And so working in that space,
I was dealing with tons and tons and tons of data. I mean, we
had, we were using mostly MySQL, and we had these
massive massive clusters. I think we had,
like, 30 or 40, you know, schemas under management. Each
schema was a cluster of anywhere from, Like, 6
to 40 plus servers, Wow.
You know, with lots of compute and everything else. So that was probably my
1st foray into, like, really thinking about data as a first class
citizen. And, and even to the extent of, like,
You know, building an architecture around the data. Right? So
that you can optimize the flow of the data, and being able to store it
and process it and transmit it fast enough to keep up with, with the
flow. And so, yeah, from there,
you know, had a lot of fun, learned a lot of things about, startups
about industry, about, DevOps and and all kinds of
things. Management as well and leadership because that's where I first,
started managing teams And then moved to data robot and,
into the ML space. And then it was a whole another learning journey
about, you know, data,
engineering, feature engineering, transformation tools. How do you
curate your data? And how do you really, like, know what you
have and inventory it and, make it available
to people within the business so that they can get value out of it.
Interesting. Very much. So our next question
is what's your favorite part of your current gig?
I think it's actually, I'm gonna cheat and I'm gonna say I have 2 favorite
things. And I I kind of always have I I
Figured out this formula a while back, in terms of what
motivates me. And it's one part the people that I work with
and another part, the problems that I had yet to solve.
So I wanna work with smart people. I I really don't like being like, feeling
like the smartest person in the room. I much prefer to surround myself
with people that are smarter than me and I respect and I can learn
from. But that also, you know, I enjoy. Right?
We spend a lot of time at work, so it helps to to enjoy the
people that you're working with. True. So that's a big part of it. And
then, finding tough problems, hard challenges. You know, if I
don't have hard challenges to keep me, to keep my mind
engaged and occupied, I start to get bored and, that's no fun. I
prefer to to always have something new to to to, you know, be chewing
at. So, yeah, good people, smart people,
and hard challenges. That is that is really awesome. I feel the
same way about about both of those things. The, for me though, I
I, Trying to find people that are smarter than me is
really easy. So I I enjoy that part a
lot. Like Frank. Frank is smarter than me.
Well, thank you. So
we have a couple of, complete this sentence, questions.
The first one is, when I'm not working, I enjoy
blank. When I'm not working, I enjoy
reading. I Enjoy movies. I go biking sometimes.
That's part enjoyment, part exercise. You know, it's good for me, but,
There's a lot of good, road biking in particular around Denver and a lot of
beautiful scenery. So you can, you know, just ride for a while and find yourself
up in the mountains or something, which is great. Yeah.
Traveling, cooking, all these things are good.
Our next fill in the blank is I think the coolest thing about
technology today is blank.
I I don't think it's necessarily something about today, but I think the coolest thing
about technology is how it builds on itself. I remember
Years years ago, I was studying for the CCNA exam, and
that was such a formative moment for me to
suddenly understand How networks worked all the way
from the physical, you know, sending
electricity down a copper wire, and it can be on or it can be off.
And that's it. Right? And you can do that really, really fast. Switch from on
to off, on to off, on to off, all the way up to,
web 2.0 and and Ajax and, you know, Asynchronous
JavaScript stuff happening in Google Maps. Right? And I can just drag my map
around. It's just mind blowing. And, honestly,
like, that That journey from the zeros and the
ones up to Google Maps, that was, you
know, what, 50, 60 years of,
technology building on itself of people solving very small simple
problems, but you add up all those small simple solutions and you get
something incredibly complex And absolutely mind blowing.
Excellent. Very interesting. The last, the 3rd and
final, Complete the sentence. I look forward to the
day when I can use technology to do blank.
I I would love, a Personal assistant, you know, like
Jarvis from from Marvel Comics or something or, I don't know,
from I I'm big into sci fi and and things like that when I read.
So, there are plenty of examples, but some kind of a smart personal
assistant that, you know, I can chat with and it keeps track of my calendar
and reminds me of appointments and, you know, when to call
my dad and whatever else, stuff like that. I just think that's
so cool. And I don't you know, with Especially with all the new LLMs
and and GPT stuff that's happening, I don't think we're super far from that. So
it's kind of exciting to me. No. You're right. Like, I
you know, if you watched, you know, when I was a kid, Star Trek next
generation was on, And the way that they were able to interact with the
computer just through their voice. Yep. And I mean, the 1st Star
Trek show had that too, but, like, the way the conversations I thought were more
richer and more kinda interactive. Mhmm. Mhmm. We
have a lot of that now. Yeah. I think some of the fundamental pieces are
in place now. Yeah. It'll probably take a little while to put
them all together and make it work right. But yeah. Agreed.
So our next one is, share something different about yourself.
But we, always remind guests that we're trying to keep our clean
rating. Yeah. On Itunes. So
I don't know. I think one of the more interesting things about my
Journey is that I don't have a background, like a a degree
in anything technical. I went to college and I got
my undergrad Studying Greek and Latin and classics. And
so it was mostly history, archaeology, languages, and things like
that. And Computers have always been a hobby of mine and and I
definitely did some computer science stuff in high school. I took 1 or 2 classes
in college, but I didn't really make my way into that
Professionally until a few years after college.
And, you know, honestly, I I don't think it's hurt me at
all. And in many ways, I think it's helped me partly
because, you know, it it helps a lot with management and leadership, just
to To kind of have a broad background and and understand, you know,
different people and perspectives and and where they might be coming from.
And I'm sure that some of the languages, you know, studying languages helped me
picking up computer languages as well. I think there are a lot of similarities in
In, human languages and and computer, you know, programming languages. But
What? Yeah. But, yeah, it is somewhat unique, and I don't run
into too many other classics majors, At, you know, tech startups.
I could definitely see the convergence, especially now when we're talking about
LLMs and the like. Right. You know, the the
nearest neighbor algorithms and all of that that are that are being applied
because my understanding is that's that's, You know, that's how that
works as it picks the next best word Right. You know, in a in a
sentence. And so syntax and grammar and all of the things you
studied in-depth, That should be very helpful.
Yeah. No. That that's awesome. There
is that good value in,
like a classics education. I I went to Jesuit
High School and Jesuit College, you know. Mhmm. I was forced into studying Latin
and things like that, like, didn't do it voluntarily. I'm not gonna
admit that, not do that. But but like as I get older, like, it's
definitely like, Oh, I get this. Like, you
know, especially when dealing with a lot of lawyers, there's a lot of Latin in
that. And so I'll hear them, like, you know, Excuse some
words. I'm like, I think I know what that means. Yeah.
Audible sponsors data driven.
And you mentioned you read a lot. Do you do audiobooks and
sci fi? Do you have any recommendations? Yeah.
There was a really good book that I read recently. Like, this is maybe
a year ago or something, but, best book I've read recently.
It's, The title of the book is called Seeing Like a State,
and it's by, James c Scott. The the longer
subtitle is, something like how Some
schemes to improve the human condition have failed or something like that. But,
it talks about this concept of legibility and how a lot of
The developments over the course of the enlightenment, the industrial revolution,
and, in the last few 100 years in in
Our society have been primarily
driven by the centralization of power in states
And the state needing to administer all of these people,
taxes, lands, land ownership, and all these different things.
And, you know, as part of, like, the the enlightenment, the
scientific revolution, we all got very enamored with, like,
rational thought and Logic and and all of this stuff. And
we thought, we're understanding the principles of the universe. We can predict
the motions of the planets and all these things. Well, we can solve all these
problems about, you know, around human civilization and humans as well.
And in a lot of cases, it failed. Right? And we didn't know as much
as we thought we did. And one of the sort of basic,
like, premises of the book, I guess, or arguments that it's trying to make is
that we routinely Underestimate, the
complexity of the natural world and how necessary it is.
And we think we can Simplify things and strip out all these
variables and go, you know, monocultures in our in our agriculture,
for example, and do industrial scale agriculture. You need
timber for building ships. Great. We'll just plant Norwegian pines in straight
rows. This is gonna be great. It's so predictable. We know exactly what,
You know, an acre of that will yield after 10 years.
But it turns out you can't strip out all the variables because the whole thing
falls apart. You need the complexity of the ecosystem to keep all those trees
healthy. And so all that predictability you thought you had
disappears within a couple of generations because, it can't
sustain itself. Wow. So, anyway, it it's a very, like,
complicated book. I'm not really doing it justice,
but I definitely recommend it. Interesting. It's on Audible.
Yeah, yeah, so definitely check it out.
The show. So if you go to the date is ribbon book.com,
you'll be routed to an Audible page. And if you choose to get a subscription,
to Audible. You will give us
you'll get a free book, and then we'll get like a little bit of a
bump on the head, and pat on the back, and Probably enough to
buy a cup of coffee. It started Which will share. Which will
share. Yes. Yes. And the final question,
Where can people learn more about you and Wallaroo? And they even
made that rhyme. Yeah. Great. I
think the best place to go is the Wallaroo website, which, as
Andy mentioned earlier is wallaroo.ai. So wallar00.ai.
And we've got a ton of great stuff on there. Lots of, you know,
documentation and and white papers and, tutorials and things about the
product and what we're doing there. And for myself,
I'm on LinkedIn. That's probably the easiest place to find me, Chris McDermott.
And, I think I even have that as my, like, LinkedIn
Profile name or whatever sits in the, you know, in the URL.
Cool. It is, actually c s m
c s McDermott. Okay. Well, thank you. Close. I was just looking at
it, and I was also looking at the website. It's a very nice website. Thank
you. Great design. And, although I can't design
great websites, when I look at one, I can tell whether it's great or
not. Me too. Me too. Same boat. I can't do it myself, but I definitely
appreciate it. I I can't cook, but I appreciate a good meal. There
we go. Yeah. That's it. And with that, we'll let
Bailey finish the show. Thanks, Frank and
Andy. And thank you, Chris, for putting up with our broken
calendaring system. Satya should really look into that
now that the drama around open a I is over.
Well, over for now at least. Maybe g p t
five can fix it.