Machine learning is when you take a bunch of data and you have a clear
goal and you're training the model to fulfill that goal. And
I'm aware AI is also built on machine learning, but the distinction I make is
that I say, well, AI as we use the term now, is basically you
take a bunch of data, you don't really have a goal,
but you're training it anyways. And I mean, that's roughly how most
of these language models are trained, right? A new season deserves a proper
host. As Data Driven kicks off season 10,
Bailey, your favorite semi sentient British AI co host,
has returned to the microphone after a 15 episode absence.
While our temporary substitutes did their best, and several deserve
honorable mention for bravery under difficult conditions. It's time to bring
back the class, the sass, and the occasional dose of dry British
skepticism. To open the season. Frank Lavinia
sits down with Sebastian Wernicke, author of Data Inspired, for
a fascinating discussion about why organizations struggle to turn data
into action, how leaders create data driven cultures, and why the
biggest obstacle to better decisions may not be technology at all. It may
be the wonderfully irrational humans making them. So put the
kettle on, adjust your dashboards, and join us as we begin season
10 of Data Driven.
Well, hello and welcome back to Data Driven, the podcast where we explore the emerging
industry that is data science, AI,
and all of it is underpinned by data engineering. However, my
favorite data engineer in the world will not be here. But
again, I think as the world focuses on AI, I think we really need to
step back and think about data. So with that, today we
have Sebastian Wernicke, who is a leading expert in data and
AI strategy. And he believes that the key to unlocking
data's power lies not in technology, but in leaders
fostering a culture of evidence and
inquiry, which I think is very, very true.
You could throw all the AI, all the agents you want, but if you don't
have the data right, you don't have it raw. You don't have anything.
And he has three acclaimed TED talks, which is cool. He's reached over 5 million
viewers and. Well, welcome to the show, Sebastian.
Well, thank you. Thank you for having me. Great to be here. Yeah,
I'm excited to have you. Tell me about this
one. I mean, it's pretty cool. You've had TED Talks, right?
That's pretty epic. So you've been doing this for 20
years, right? Yeah. Data was not really taken
serious. Arguably it's not taken seriously enough today,
but I would say 20 years ago, it Certainly was not taken seriously.
Yeah, well, it was sometimes taken seriously, but
I think that's sort of a resonating theme. So the
interesting thing is, I think if you've been in this field for so long,
is that you really notice history repeating again and
again and again. Right. So we had the era of big data,
if anybody even remembers the term. Then came analytics,
then came digitalization, and now it's AI.
And in the end it always comes back to data and whether you
manage to take all the great insights that the data
is giving you and actually implement them in the organization and
create some change ultimately. Right. That's all what data is
for. If you don't create change with data, you don't need all that
expense and investment into it. Yeah, it's a good
way to put it. And even before that, even before there was big data, there
were OLAP cubes. I remember I actually worked for,
in the virtual green room we were talking about. You're based in Germany. I used
to live in Frankfurt and prior to joining
Deutsche Bank, I had worked at basf, or the big
chemical company Americans would know as basf.
And I remember sitting in the cubicle of
one of the SAP gurus that we had talking about OLAP
cubes and all this crazy stuff. And I just was
like, you ever have a conversation that is very, you look
back on it now and it was very prescient, you know what I mean? It
was very future facing. And then at the time you're
sitting there and you're like, you partly understand what was happening and
you partly are very. I'm not sure if this
person's crazy, you know, And I
remember she was one of the DBAs that we had in doing advanced analytics.
And you know, she said that, you know, my goal is to figure out, you
know, will rainfall in Western Australia impact
prices here, which will ultimately impact, you know, how we
go to market with, you know, at the time, a chemical
company. And I just remember sitting there thinking like, I can't
tell if that's brilliant or crazy. It was
data science. Because before the term was invented in a way. Exactly.
I mean, that's really what it was. It was. And you know, now when you
say that, it's not so crazy. Right? Because, you know, we had a previous
call, we were talking about, you know, finance people actually
would. It'll be the episode prior to this. So hopefully you've listened to that. Not,
not you, but you, the audience you're welcome to listen to,
but where they talked about finding Alpha, like finding the
signal that matters before anyone else does. And that's really
what she was doing. It just, you know, in
the late 90s that sounded a little crazy,
but I think it's also one of these things that's still true today. I mean,
everybody's trying to find the advantage in the data. And I think
it's also of course interesting to then see with
everybody just accumulating more and more data analytics becoming
much, much faster. We don't even fully know, I think what
AI is going to do to the speed of the
generation of insights, not the adoption, but just the speed of generation.
And everybody's still trying to find that edge. And
I think that as people increasingly look at data
for a while you could have an edge by
looking at the data to optimize and to go in these incremental
ways where it's like the 1% optimizations over time add up.
And I think that's become the baseline. That's just the expectation
right now you have to do that already. And so
sort of a transition happening, I think where you now need to think about, well,
I'm already using data for that. I'm optimizing my processes,
I'm adding the automation. So where's the next
edge going to come from? And I strongly believe that it's
now again a shift to the anomalies and the
shift to the outliers and not trying to look
to data to give you those straight answers where you go for, I mean that's
not going away just to be clear, that's. But that's baseline expectation
now you need to add to that to look into the data and say, okay,
what's the next interesting question? Where am I going to find that
new product idea? Where am I going to find that new idea
for revamping a process that
maybe nobody dares to rethink today? And I think that's
the exciting part of what everybody who's in data
can be working on today. That you sort of transcend that,
look for just the correlation and say, okay, what's on the
fringes here? That's a good way to put it. I
think you touched on something that is important.
Leaders really have to foster
a culture here of looking for evidence and data.
It has to be pervasive, I think, because finding anomalies and data,
you're right, that's kind of like baseline now.
Right. But in order to do this you
really have to have teach people respect for data and an
understanding of basically evidence based approaches to things.
Right. Escalate when needed, but don't escalate. Don't use
data for data's sake. Right. And how do
leaders do that? Right. How do you, you have to educate the leaders, I assume,
right? Yeah. Well, I mean, I'm. Oh, and I just, I think I
have to preface that a little bit because I'm always very careful with that. I
think leaders get told 20
times at least a day what should be their priority and what
leaders should do. Leaders must do this, leaders should do that, and
so on. I am going to try and make the case
that here. I think it is really a leader's job. And
the reason, I think is quite simple.
Within an organization, leadership usually
thrives on being right. I mean, that's what we look for in leaders.
We look for the confidence, we look to them to say,
this is the way we're going. This is what I believe in.
And if you take this change with data
seriously, it basically flips the whole thing on its head. Because
your baseline assumption, I think almost has to be, well,
we're kind of wrong today. It may be right, what we're doing for the business
right now, but every single day we have to start
looking where are we wrong? And then correct that.
And everybody in an organization, of course,
is smart, so they will look to leadership to see
what's the culture here. And I mean, we all know that culture, it's
a bit of a fuzzy word, but I think you can easily unpack it. I
mean, it's not what's on the PowerPoint slides, it's not what's on the posters
in the hallway. And yet culture is who gets hired, who gets
promoted and who gets fired. And that's
the baseline of it. And people will look at that. And
so leaders are the ones, I think, that get to set the tone in the
room, that get to shape these things. And if you are in
a culture where, for example, the executive will say, well, we're a
data driven company, and then turns around and promotes the head of marketing who
has publicly proclaimed that he really has a good gut feeling and never
trusts the data. Everybody has learned, well, you're not really a data
driven organization. And so that's, I think, where
leadership really comes in. It's only leaders who can
create these safe spaces and signal to everybody
it's okay to come with data that disagrees with the status
quo. It's okay to come with an analysis that disagrees
with what I have said for the past year, for the past two years.
I'm willing to challenge and change my
Convictions. And only when that is done on a very regular
basis and the organization can observe that, I
think. Are you creating the culture that is that fertile ground for
these very small insights?
Maybe at first, you know, but they need the space to be explored, they need
the space to grow and maybe to even create some
experiments to further validate that. That's
true. And I think the idea of promoting people with
gut feelings, and I'm not that gut feelings are necessarily
bad, but I think you're right, it sends the wrong
message. You coined a term called data frustrated,
which I think is pretty accurate. So how did you get to that
term? Data frustrated, by the way, is just the pre stage to becoming
data cynical at some point. So it's kind of
like the stages of grief, right? There are stages to it.
Yeah, well, I think it was just a feeling
that I perceived whenever I was working with
my clients on data projects that there was always
that point where you would sit together maybe within the project setting, maybe when you
go to dinner afterwards, where people would tell you, oh my God,
we've invested so much time, effort and money into
data, you know, hundreds of millions going to spend another few
millions, hundreds of millions in the future. But we're not
really satisfied with the results we're getting. What's happening here?
And I think that question came up again and again and again. And I think
that's the very definition of frustration. You sort of
notice something isn't going as you want and at the same time
you're not finding an answer or maybe you think you have the answer
and then you try that and it doesn't move forward. Now
an interesting thing is the data on that. It confirms
that. So I found a couple of surveys
from various years. It starts out in 2010, and there's another one,
2019, 2024, where some
consultancies asked executives what
are the top 10 reasons why you're not happy
with the results that you're getting out of your Data projects?
And 2010, it was three reasons that came out on top.
It was, well, we don't think we have enough management attention on it.
We don't really understand the business case as much as we'd like to, and
we don't think we have the skills. Now the interesting thing is in
2019 they did a similar survey and the same
three reasons come out on top. And then in
2024 the same three reasons come out on top again.
They change orders. Sometimes when AI comes, everybody says, ah, we probably
don't have the skills. But I mean how frustra as that you
think you know the top three reasons standing in your way.
And they also don't sound that difficult. I mean, you know, if there's not enough
management attention, pay attention. If you don't have the skills, do some
training, calculate some business cases. But apparently that's not the solution.
And I think that's where data frustration ultimately comes from.
So over these just decade and a half now, the problems have
been the same. So like, how do you,
what is really the problem? Are those the problem? Because if it's something, if you
know something is a problem for 15 years, you don't address it. There's an
underlying problem that maybe you're misidentifying the problem.
You're putting the blame on the wrong things. Like what do you think it is?
I think it's that missing cultural component that if
you think about some of the things we just discussed. Right. The
space that leaders need to create for having controversial
discussions, which is also sometimes known as psychological safety.
And on the other hand, you have all this technology that's creating
measurements that needs to process the data. And whether it's
a cube, it's a data lake, or a data mesh, doesn't matter.
These two discussions never happen in the same room. You
have one part of the organization thinking about the next technology cycle,
constructing architectures, discussing
what's the best way to organize data. And then you have other parts of
the organization that are thinking about culture, that are thinking about
transformation, that are thinking about leadership education.
And these two things are never brought together. And I think
that's the issue. So it's not a sort of technology versus culture.
And you don't need technology. All you need is culture. And it's also not
data or gut feeling. Of course, you need both because data is never
going to give you all the answers. So a good intuition is quite helpful in
many cases. But it's about really
bringing these two elements together and integrating them.
And for me, that's the missing component where, you
know, the what, what, what the surveys express. I think
that, that management attention, the, the skills, the
business case, I think these are symptoms and so you
can't really treat them as the causes of the data frustration. It's just
what resurfaces when you don't pay enough attention to the cultural element.
So how do you get. So it sounds like you brought up something very, very
real, like the people working on the actual technology. 10
and I'm guilty of this. I'm a technologist, right?
Yeah. I mean, no, all Our listeners
are right. So like I often will catch myself like when I'm like deep down
a technical rabbit hole, like, wait a minute, what am I actually doing here?
That's a skill that was not easy to develop.
But how do you, I mean, is it people working
on kind of the, the cultural side of things, like, don't they need to talk
to the technology people and like, because, because
historically, and this goes back to when I was sitting in the
cube at BASF where, you know, the, that cube,
that cubicle was in the basement and behind, you know, regular. It was in the
basement. And then the, the, the, the DBAs were like in a sealed off
portion of the basement. Right. And like, you know,
I don't know that that's kind. And if you ever seen the TV show, the
IT crowd, it's a British show. Yeah, yeah. You know, they were kept
in the basement too. Like I think historically it was not
seen as crucial to the business.
Right. It was kind of a back office job and people, it was kind of
pushed to the side. And I think it's been years since that's
really been true. But
I think what we're seeing is a bit of the lingering effects of that. Is
that a fair thing to say?
Well, I mean, I think there's a bit of a
problematic history here where I think for a very long time
it was mostly perceived as a cost factor.
And so the incentive was to optimize for cost
and optimize for efficiency. And suddenly we're expecting
technology to drive transformation
and innovation. And of course that completely
changes the incentivization. And I think it also creates
sometimes these gaps within technology organizations
where there's one part that says, okay, for years we've been trained on
ensuring efficiency, security,
reliability, and suddenly we're supposed to open up this huge
experimentation stage. Other things continue,
you know, as we, as we want them. So that
is definitely something to recognize and acknowledge. But on the other hand,
I'm not a big fan of always saying, okay, you know, the
other people should do something. Clearly everybody needs to talk
with each other. But I think there's also something that we can do on the
technology side or more specifically. So I've been running data science teams for
many years now. So for example, what I do in my teams
is whenever they ask me for training, what I will
propose to them are trainings that I think you might call
soft skills. But I think they're just essential skills. So I will
send them to communication trainings,
stakeholder management trainings. We will
together Talk about decision making, how that works, how you
influence people in a room, how decisions are really made. Because
I think many data scientists come in with the impression the decision is
made at that meeting where they come in with a PowerPoint slide. And that
insight, you know, that everybody will say, oh, brilliant,
finally we have that insight. We're now going to change our ways.
Which I cannot blame them because that's, as a,
you would intuitively think that. And it's I think also
a good belief in humanity if you would think I could simply
influence people like that. But I think it's adding
that part of understanding to the technical
profession. That's something that as a technical team you can do. That's
also something that as a technical leader you can do to just
add these additional skills. I was two weeks ago at a
data science conference and I was speaking in the AI and
MLOps track with a very strange
topic. I just talked about decision making for half an hour
and I was really worried that the resonance would be okay. We came here to
look at a Python notebook and suddenly this guy is talking about psychology. So what's
going on here? But the resonance and the number of questions
I got, I mean, they just showed me there's a real openness, almost like a
craving to understand that because everybody has
been in that room. I know I have many, many times where
we had all the analysis, right? The data was good,
we had scrubbed it, we had understood it, the area under the curve or
whatever quality measure, it was good. And we showed them. I don't
know, when you drive your trucks around Southeast Asia, you can save 15%
of fuel. And then a year later, half a year later, trucks are still
driving around like they used to. I mean, that's such a frustrating experience
on the data and analytics side that I think people are really,
really eager to learn how do I overcome this? And
how can I be more effective in
actually changing something and actually
being effective
in my job and being seen with what I do.
And maybe we can use that as a platform and
basis to expand it from the technology side.
And of course it's not a one sided thing,
right? At the same time, when I speak, let's say with
HR leaders, I always emphasize
you need to make sure that there is more technical understanding in the
organization. You cannot treat data
as this simple API where you basically say request dashboard and then
suddenly the dashboard is built a couple of weeks after. That's not how it works.
You need to understand how this works. You need to also understand how
data works. And what, what it can and cannot give you, because
otherwise you're just coming at this with the wrong expectations and
you're almost bound to be disappointed in the end. That's
a good way to put it because I think one of the naive things I
thought in my youth too, is that we would make better decisions if only
we had the data. And then that didn't work out. Well,
maybe, maybe it was about access and discoverability in the data,
but I think ultimately the problem is a human problem. And
it's funny you mentioned that. Right. You know, people go to a tech conference, they
expect to see jupyter notebooks, et cetera, et cetera. Right. They expect to see code.
I just got back actually really late last night from DevOps days,
Austin. And a number of the talks
were not technical. They were about influence and how decisions are
made and Campbell's Law. And now
I was working the business. I wasn't able to see the whole conference, but the
way that those talks resonated with the crowd I thought was very
interesting because that was, you know, soft skills. And the whole
soft versus hard skills goes back to apparently US military
training, right? Where hard skills. Okay, yeah, yeah, yeah. So. So apparently the
origin is not that soft. It means it's easy or it's not important or it's
fluffy. It really means like, you know, you know, actual
kinetic things that are hard. Tanks, guns,
bullets, that sort of thing, Missiles and, you know, dealing with people. Things that are
soft, like, you know, living things. Right. So that's
apparently the origin of it. So. Which is. It's an unfortunate term because I think
when people hear soft skills, they're like, yeah, right. Especially engineers.
And. But I just found that interesting because I think, I think a lot of
IT professionals have gotten to the point where they get very frustrated because
the data says one thing. Right. But the process
hasn't changed. Right. Or they're still making the same bad decisions or
not data driven situations. And, you
know, and I think you have a lot of people in leadership
roles, not in it, but outside of it, that are data
frustrated. Yeah, yeah, go ahead. No,
no, we, we call them delivery skills in, in my team, that's ultimately
what. Well, I, I know that some people like to refer to soft skills also
as, as human skills, but I'm also not happy with that because, you know, if
you don't have the skills, you're not human. No, that, that doesn't work. But, but
we call them delivery skills because that I think brought home that aspect of. Well,
if you want to deliver. If you want to deliver impact, here's the skills
you need and there's the core skills, your technical component, of course,
you got to have that clear. But you have to know how
to deliver these insights into an organization that isn't
just waiting for the next statistical analysis to
be delivered. But it's unintuitive. And
that was one of the fascinating things I found when researching for the book. So
the book contains an entire chapter on psychology because I just
found it so fascinating to understand that
interplay of data and human brains and
what happens. And turns out there's actually even a study from the year I
was born. So it's been around for a while where
they took a couple of students in Stanford and asked
them, what's your opinion on capital punishment? So they took strong, emotional,
controversial topic, and then they showed them a fake study.
And that study was made up of data that was
constructed in a way so that you could imagine. Half the data
gave you arguments for capital punishment and half the data would give you arguments
against. And the researchers just wanted to find out, okay, can data change
people's minds? And the fascinating thing is they found out,
yes it can, but in exactly the opposite direction that you
want. So the people going in that are pro capital punishment, they come out
and say, oh yeah, I finally found, found the data to confirm my beliefs. There
was also a bit of sketchy data in there that said I'm not right, but
that's sketchy. And the sources, I don't believe them. And the people that were
against capital punishment, they came out with exactly the same feeling. They said,
oh, there was so much good data in there against capital
punishment, I must be right. A bit of sketchy data that was pro, but
that can't be right, I don't believe it. And that's, that's
an insight that's been around for so long and of course it's been replicated
enormous amount of time. And
just go on social media you will see that effect applied at scale,
very much so, the echo bubbles and everything. But we still tend to operate
on this, what I like to call the data deficit theory, that it's just like,
oh, all we're missing is, you know, the saying, right, the right
data to the right people at the right time and suddenly
things will improve for the better. And psychology and
research for decades has shown us this is not the
case. We don't like to be proven wrong and our brain
will do a lot of tricks and a lot of self convincing
to just make us very Very reassured that
we're right. Yeah, I mean that's
really, is that, is that the,
that's really a human problem. How does,
obviously it's also been shown in monkeys actually. Oh really? Okay, so
it might be a biological problem. Right. And I wonder,
I wonder, you know, will you, will we see as model LLMs
get better at reasoning and kind of holding opinions, will they do the same
thing? Is it maybe just part of, it's just a function of the system?
I don't know, it's, I mean the, the,
the thing is, so that's what I find is one of the bigger
dangers of AI actually because so I
differentiate in the book pretty clearly between machine learning and AI.
So I'm not sure it's canonical, but I think it's useful. So basically I
say machine learning is when you take a bunch of data and you have a
clear goal and you're training the model to fulfill that goal.
And I'm aware AI is also built on machine learning, but the distinction I
make is that I say, well, AI as we use the term now is
basically you take a bunch of data, you don't really have a goal,
but you're training it anyways. And I mean, that's roughly how most of
these language models are trained, right? You feed them all of these texts and
then you say my goal is to make the user happy and
for many people to be happy with the answers. So what
does that lead to? Well, first of all, you're training a model that
by definition is going to be extremely convincing
because that model has been trained on how do I circumvent all of
these psychological traps, how do I make people feel good all the time?
And that's all the sycophancy discussion of course we're having. But I think it's also
more subtle. There's some studies that actually
looked at, for example, models generating propaganda and
found out that language models are much more effective than humans
at generating propaganda. Because I think just the way they're trained,
there's some implicit mechanisms that these models have learned
that they can exploit. And so the ironic thing is that
actually you should be trusting machine learning because machine
learning, lots of data, clear goal, there's some
statistical proof that you can make after a while and say,
okay, of course it's better than a human. The classic examples, the cancer
detection, now the self driving cars. But it turns out
in studies that people just don't trust machine learning.
There's a really weird study I found from Wharton, for example, where they
showed people that a machine learning model was superior
to their own decision making. And then they asked them,
as they do in these studies, you can earn some money here and you can
either trust your gut or you can trust the machine learning model.
Everybody on average, of course, went with their
own feelings. Even though they had seen that the model performs
better, they just needed to catch it doing something
wrong and that they would immediately say, I don't trust this thing.
Then there's a weird mechanism that they added where people could start to influence the
results and suddenly they trusted the model more, even though the model didn't
change at all. So very, very weird effects. But
what I'm getting at is machine learning is something that's extremely
trustworthy and yet our brains are just wired to
distrust it. It all seems so mechanical, so
mathematical, so unhuman. I think that's what many people
say. Right. They don't feel comfortable with that. And then here along
come these language models where you should definitely not trust
them. They have not been trained on a specific goal. We have no idea what's
going on under the hood. They're pretty good, to be fair, but
still weird effects happening.
And yet implicitly, we completely trust them just because of the way that they
interact and because they have this amazing, amazing human,
like, user interface to interact with us. And
that's something I think we're going to have to grapple with that suddenly we found
a machine that can be very convincing but actually shouldn't be very
trustworthy. That is a very good way to
put it. I actually just saw a video last night where they were talking about
how an AI will map human emotions
and not so much map the emotions. I'll send you a link to the videos
from the infographics show. And, you
know, it was not meant for. It was meant for, like, the general public, I
think, to watch. But, like, there were things in there where. And it makes sense,
right, because they, they basically categorize or
store words in. We'll call them spaces. Right. So they,
they. Certain words that you will use will indicate that you're in a certain
emotional state. So. And it was very
fascinating. I was like, this sounds.
It sounds dangerous. Right. And so
it'll know, like, if it can. It can know, like, if it's. Depending on what
the prompt is and how it was trained, it can use different words to kind
of guide you back to whatever emotional state it wants you to go in,
which is, you know, a very
dangerous weapon. I dare say it's manipulation at
its very best. Yeah. And on the other manipulation at scale.
Yeah. And on the other hand, not surprising that the models would be able to
do that. I mean, you know, they know 20 TV shows you watched and they
can recommend you a movie that you never thought of. But still like, so
why wouldn't it work with emotional states? I think, of course our brain likes
to suggest to us that we're so complex to figure out where
perhaps in fact we are. Not always at least.
Yeah. If you ever watch a Star Trek Next Generation and you watch it with
like today's vision. Right. You
know, like they, there are lines in there that I find kind of funny with,
you know, 20, 26 kind of
vision. Right. Is, yeah, I'm an AI. I, I, I can be
completely impartial. That was what Data said in a couple
of times. And I like, oh, we were so innocent
then. And there were a few other things where, you know, a big
subplot. Picard would say, well, you know, you really can't calculate
the human condition or something like that. It was very, I think it was very
much a product of its time. Yeah, yeah. That's the one thing
they really didn't quite get right. It's so interesting.
Yeah, I hadn't thought about that. Yeah. Like, you know, if you watch
them like now, it's kind of like, you'll see
particularly when they talk about AI and kind of how AI people interact
with AI, some of it is very, very, very much on point. Right. You'll, you
know, there's one episode, any episode, where they interact with the computer through
voice. Very much how we interact with voice assistants today. Right.
Yeah. They didn't anticipate the seductiveness and the human likeness.
Right? Yeah, no, they know, they, I mean it was just, and honestly, who
really could? Right? No, yeah. Unless you're like
extremely paranoid like Philip K. Dick. Right. Like, you know,
but sometimes, sometimes paranoia is another way to say you're
further ahead of the curve than other people.
But that's what science fiction is all about.
Exactly. Right. Like sometimes the crazier it comes out
when it's mentioned, the more it'll has long lasting effect.
So we mentioned kind of like data driven and data frustrated. What is
data inspiration then? Is that kind
of the end, the ideal end state is data inspiration?
Like what I think the goal is to add data
inspiration to the data driven organization. So
I don't think it's any way plausible to
argue, oh, let's swing the pendulum around. Of course we're
going to have dashboards. Of course we're going to have optimization, of course we're going
to have automation. I mean, that's all what a data driven organization
really is about. But I think the important part about being
data inspired, I think there's two components that I deeply care
about. One is that I think the notion of data driven
can also have this notion of here's a preset path to
average and everything's just going to be optimized.
And in the end, that's going to kill innovation, it's going to
kill creativity, and it's also not going to be very fun.
And I think that's not true when it comes to data, or at least that's
also not how I perceive data. I know that when many people hear the term,
they think about these tables and numbers and things that are not very
exciting. But in the end, if you think about it,
there's a lot of innovation potential in data. And
I myself, I find myself being creative with data.
You connect data sets that probably weren't meant to be connected, but you find
this new, exciting insight. We have art created with
data that is amazing. We have things like data journalism, which I
find some of the most interesting and most fascinating forms
of journalism. So I think we need to recognize that data
is more than these cold hard facts that just tell us go
left, go right, but that there is a lot of
additional potential in here. And ultimately for a
organization or a company, I think that's also the potential for
transformation. So not in the way that you have, you know, the
data driven part, that's opt, that optimizes what's
already there. And then you have a few creative people with these
brilliant strategic ideas that are going to take the organization to the next
level. I think that's wrong thinking. You know, that's again, having
two conversations in different rooms that should be in a single room. You are going
to find your next breakthrough strategy and your next
breakthrough product, most likely in the data. If you
connect everything that you have, if you allow for experiments,
if you, as we said, look at the outliers, look at the
anomalies. And I think that's the important aspect that I
like to emphasize here when I say data inspired, that you
just don't forget about these aspects and
don't just build organizations that are happy when the dashboard
is green. Yeah, yeah.
You know, there's a lot of organizations and I've worked for them in the
past, that it basically it's a scorecard world.
Yeah. And as long as you hit those certain numbers and
the mental gymnastics and I
wouldn't say shady, but I would say awkward ethical
situations people would throw themselves into. I mean, it can be
shady. That's the. So. So, so there's an interesting thing
I found that most of the
inventors of data driven methods, so one of the
famous one W. Edward Deming, for example, one of the inventors of
statistical process control, all of them
come to the realization that steering a company by
numbers is a really, really bad idea.
So they all have that insight. And with Deming it's
quite extreme and also unfortunate. So there's this quote,
and I'm sure you've heard it right. What gets
measured gets managed or what gets measured gets done.
And oftentimes when people put that on a slide, they will quote Peter Drucker. Now
there's a whole webpage on the Peter Drucker Institute where they say
Drucker never said that. And the
actual quote comes from Deming. But the
really, really annoying thing is he actually said the exact opposite. So
his full quote is, it's wrong to assume that if you
can't measure it, you can't manage it.
Yeah, and he only gets cited with that second part. He must be turning around
in his grave because he was so against management by
numbers that he put it on his list of seven deadly
sins of management. It was so important to him that people
realized that you need to honor what can't be measured. And that
exactly what you're saying, if you just steer by numbers, you're going to end up
in a shady place. And I think that's not just theory. I mean there's lots
of public examples. So for example, ge, you know, for year
after year after year delivered extremely stable
profits, unusually stable profits, because that's what they measured
themselves by until somebody figured out, well, the
accounting that led to these stable profits might not
entirely have been kosher to say it very carefully
or the. I mean while we're talking from Germany, the
Volkswagen diesel scandal, right, Dieselgate,
there were a couple of engineers who were told, I want you to build an
efficient engine and if you don't, you're fired. Well, and that's the
only number I care about. So the engineers made very,
very sure that the engines would hit that number.
But of course that led them to other
troubles down the line. So I think that's also one of the real
dangers. If we're just talking about data driven organizations that ultimately
or very easily actually it can lead to this notion of
we steer by numbers and we Invent a few
KPIs, maybe complex ones. And as long as we hit the KPIs,
the business will be fine. And of course, what you start doing is you start
managing the numbers and not the business. And both can diverge
quite a bit. Yeah, no, that's.
Incentives drive behavior. So if you
change the incentive, the incentives should be running the business, not hitting
a KPI. Right. KPI's are a means to an. No. Like,
you know, and it's funny because, like, I've seen people do really dumb things
just because that was in their KPIs. And I, I just,
I, I did not know that about Deming, like, because he's often
cited as like, you know, the management by numbers guy. But the fact that
I hear that, I'm like, oh, my God, like, he really must be rolling over
in his grave. I know, I know. And I mean,
also, just to defend the people that want to steer by numbers,
I have these discussions in my team all the time. For example,
performance reviews, where I'm very much against
doing performance reviews based on metrics because
I really believe that these won't do people justice. But then
maybe a performance review isn't going the way you thought it would be. And
what's the immediate reaction that you get? Well, people will come to you and say,
okay, next time I want numbers, because that will give me reassurance.
And when I hit my numbers, I know I've done a good job. And I
think in management that may just be the same, right, that you say, well, how
do we know we're doing a good job if we don't have objective measurement, if
we don't have numbers to hold ourselves to
as a standard? And of course, as always, I think
there's a bit of a balancing issue. The interesting part is
you may know this from project management and project management, you have this triangle
where a project can be cheap, it can be good, it
can be fast. Unfortunately, you only get two out of three.
And with data, I think there's a similar triangle,
which is data in my mind. It can be simple, it
can be accurate, and it can be universal, which means it
tells you something about the entire business. And I also think you only get two
out of three. So if, for example, you have something that's
accurate and simple, it won't be very universal. It will be
measuring a tiny process somewhere in manufacturing hall 4D,
but not tell you how the whole business is doing.
And so what does that mean? Everybody wants accurate. And I
think when it comes to steering a business or steering a business unit, you want
universal. So what's the one you need to forego? That
is simplicity. And you really need to engage with the complexity. And
that's what some organizations do. I mean, I know everybody's a bit tired about
hearing about Amazon, but if you look at what Amazon does in a business meeting
and there's various books about that, they will look at 400, 500
different KPIs in a single meeting
just because they want to make sure that they're not optimizing for one number
at the cost of the rest of the business or managing one number up and
the other one goes down. And so I think they have deeply
understood this principle that, okay, we want accurate, we want
universal, so we need to go complex. Sorry.
Yeah, I mean, you're right because you know, you can only
model, particularly the specificity part.
Right. Like, yeah, that, that, that I think is very, and it's very
tempting to assume that you can have one model to rule them all,
but that's just not. I mean, I think finally when it comes to
LLMs, I think people are finally figuring that one out, right? Where you'll have different,
smaller, fine tuned models versus models with
trillions of parameters that'll cost ridiculous amount of money to run.
Yeah, well also I think the whole agent discussion of course
is going this way, right? You're realizing we won't have the universal
agent that will do everything, but people are creating these really, really
specialized tools that can then interact with each
other and play to their goals. So we're sort of doing the
opposite. Right. At first we said, well, we don't have a special or specified
goal for creating these AI models. And now we're sort of trimming
it down again and saying, well actually we'd like a bit of a goal in
there. Right, Right, yeah. No, that's
interesting. What
I'm looking at some of the notes here and this is what is a
data resistant mind. Because I think that's interesting, I think it's an
interesting concept because I think I know what you mean. But I'm like, I never
had a label for it before. I think the data resistant
mind, I mean it starts with things like the Stanford study, right?
So our mind is just confronted with data and
just takes it the wrong way. But there's other effects
that have been shown. So I mentioned the monkey
study. So again, apparently they like to study brains. In Stanford
there's a study that's much younger where they had a monkey looking at dot
patterns on a screen and the monkey essentially had to decide which way
are the Dots moving, and then press a button, depending on what it thought.
The researchers, what they did is they wired up the monkey's brain,
which led them to some fascinating and scary result,
which is a couple of seconds before the monkey actually was pressing a
button, they already knew which way it was going to decide
and they could predict that. But then of course, they
did the following thing. They said, okay, if we can predict how the monkey's going
to decide, we're going to show it opposing signals. So we're
going to have some of the dots suddenly move in a different direction
and see whether the monkey changes its mind. And
what they found was the closer the monkey was to a
decision, the more it simply ignored that new information.
So as the brain was sort of forming the signal from the
noise, it was stronger and stronger and stronger in its
belief and it filtered out anything that was contradicting that
belief. And now, of course always very dangerous
to transfer monkey brain studies to human brains,
but I think ultimately that's the same way that we function as
well. The closer we are to having made up our mind, the more certain we
are of something, the more likely we are to reject contradicting
data. Which of course is a very, very painful irony, because
when would data be the most valuable? When we're certain
of something and it actually contradicts us. And so
we need to recognize the data resistant mind
on an individual level. But then of course, many companies,
organizations, it's not just a single brain. So suddenly you have
lots and lots of data resistant minds interacting with
each other and interacting with each other on different
timescales. And so that again shows you
how ridiculous actually that notion of the data deficit theory is, right?
If you just think that in this jumble of noise you can throw in a
few numbers and they're going to influence a decision, it's futile. You
really need to think about the entire mechanism from beginning to end. And I
mean, we all know this or have been in a situation,
there is no single decision point or decision meeting. Just like in
the monkey brain, where they found there's at first a bit of chaos and it
starts to form a decision signal. I think that resonates very much with how
I perceive organizations making decisions. There's a notion here,
bit of politics over there, and suddenly at some point you feel,
well, things are probably moving in this direction and suddenly everybody is moving in that
direction. And that that is a very
data resistant mechanism that we need to,
where we need to put some engineering into that decision process and the decision
framework, the decision Architecture to make it work. It's not going to happen by
default. Yeah, I often wonder, because if that happens in
monkeys, it happens. Assuming it happens in people, it happens in
organizations. There's got to have been some evolutionary
advantage to it, right? Things don't. Things
don't exist in a vacuum is one of my beliefs. Now, again, maybe I'll see
data and I'll fight it. You tell me that I'm wrong, I'll fight it.
But I really think that things, particularly natural
things like behavior in animals or in groups of people,
or there has to be some kind of evolutionary
advantage. I wonder how we got here. Right? That's the engineering me,
like, how did we get here? That's a fascinating
question. And there's actually some people that have thought about that. So I
only very briefly mentioned that in the book. There's a book by two
researchers called Mercier and Sperber who make a
very fascinating argument. So they start with the
following premise. They say, if rational
thinking is so good, why don't we see it all over
the place? Why are we humans the only creatures that have
evolved rational thoughts if it's so good? And
the conclusion they come to which you can
subscribe to or controversial, but I find it very interesting is that they say, well,
maybe rational thought did not evolve for
actually thinking through a decision before we make it.
Maybe rational thought evolved to rationalize and to
justify decisions after they have been made,
which again, I'm not entirely sure whether I subscribe to that
myself fully, but I think it would make a lot of sense
in that context, if you think about
rational thought not being the decision making process that's in
control of everything, but something that comes in after
and helps us justify it to ourselves and others,
how we came to a certain conclusion.
An interesting piece of research that adds to that. So there's
various studies that have been done with humans that have
had a particular kind of brain damage where
they're not able to process emotions, so they have a damage
to a certain part of the prefrontal cortex.
And you would think that these people are the perfect rational
decision makers. They have lost the capability to process emotions.
And it turns out also otherwise they behave normally, they, they talk
normally, they have high IQ scores and everything. And what the
researchers are finding is that these people are
completely incapable of making any decision at
all. They can't even decide. Yes, they cannot even
decide what to have for lunch because they will find
an additional rational reason and yet another additional rational
reason for why this sandwich might be better. Than the salad or why the
salad might be better than the sandwich or the burger. So these people become
incapable of decision making and these
studies. So there's a brain researcher, Antonio Damasio, who
started that. Their conclusion is
that most decisions just
cannot be made on a purely rational level. So you need
emotion to tip the scale at some point to move you
in that direction. So that mechanism that stands in the way
where ultimately, you know, the emotion is telling us, I'm not going to listen to
this data and I'm going to subscribe to that data
has a flip side to it where it might actually be
the part of the brain that allows us to make
decisions in the first place, because otherwise we would just be overwhelmed
by data that doesn't give us a clear direction.
And that's true, Link. And also too, thinking requires effort,
FDA requires calories, and calories were not at a surplus until
very recently in human history. Yes.
So I think maybe there's something. It's an optimization trick, Right.
Maybe that's what it is. It's a fascinating thing.
And we can go down that rabbit hole. But one of the things I thought
was interesting is that you said that there were four ways to
derail your data driven journey.
Yeah, yeah. So what, what are those four things to avoid?
Yeah, so, so the first one, and this comes really out of the project experience,
I think with, with many data projects, they start out by
saying, let's do a quick win or let's do a pilot or
let's do a lighthouse project. So let's take the lighthouse project for
example. And I think the assumption there is everybody thinks data
is hard and you know, people might not like to engage with it.
So what if we build a very impressive lighthouse project,
then surely everybody must look at this and say, oh, this was a
great idea, we're going to come along. It
doesn't pan out that way for various reasons. I think one of the main reasons
is that the organization will look at this lighthouse project and say, well,
okay, you constructed this in a place where it was very
easy to construct a lighthouse. You know, usually these lighthouse projects,
they get all the management attention, they get the budget.
If for some reason they don't work, they get more attention, they get more budget
because everybody's already committed to them. And so they don't get the
organization moving. So I think this notion of let's not engage with
the difficult parts of this, just build a lighthouse, I think is one of the
main traps that I see now.
Another trap is that I think
let's say out of a good intention, which is the intention to
avoid complexity. Oftentimes
use cases, various data use cases will be
analyzed in a very isolated fashion from each other. So
I think many people working in the data space will have seen this. You take
the individual use case and then it gets a score, maybe the
complexity and the business value gets put just this two
by two matrix. And then you sort of try to find, oh, we're going to
do the ones that are sort of medium difficulty and we want to
get a lot of business value out of it. And
what that does is I think it neglects the fact
that many of these use cases will be related and all of them will be
tied to building a solid data foundation. And
so when there's oftentimes I think a complaint of saying,
oh, you know, we're doing the same data transformation in 10 different
places. And oftentimes where that comes from is because the use cases
are planned and implemented in isolation, because nobody wants to deal with a
complex topic of touching the entire data
foundation. And so I offer some workshop
formats and some methodologies that I found useful to
actually make sure that you put all of these use cases on a
common ground basis and are able to connect them
to each other. Then there's
another interesting one, which is that
oftentimes we believe that the data
and the technology is going to be the hard part
about a data project. So the
example that I previously mentioned, we were actually
optimizing trucks driving through Southeast Asia and
we were launching in that thinking that the hard part would be
designing the algorithm, because as you and probably a lot of
listeners know, finding optimal routes, it is a hard problem, and especially
with different parameters coming in and so on. But we
actually developed an algorithm that was doing pretty pretty well and pretty, pretty
fast. What we had completely
overlooked and underestimated were two things. One is the
complexity of restructuring a warehouse, because in order to
drive these optimal routes, you would have to have completely
restructured the way the warehouse is working. And that was a
complexity that everybody was just afraid of because you would essentially
be, it's almost like made to order, right? You would say, okay, put this package
here and then put this package there. And that's not how it works. They get
their pallets and the pallets get into the truck entirely.
And the other part was simply politics because the hard
part there was, well, if you tell people you can save 20% of fuel,
you're also telling them you have been wasting 20% of fuel in
the past. And so we should have been a
bit more careful about that, which I can completely understand. And it
was also a very political organization in many places.
So the data project was
hard, but it wasn't the hard part. And I think oftentimes when we
say, let's implement a use case, we think about, where are we going to get
the data, who's going to write the algorithm, how are we going to get that
into production? It and completely neglecting all of these other
factors, which ties into the fourth trap, which is
ignoring the human factor. And I think this is something that
comes very much out of the data deficit theory, where we think,
oh, once we show this amazing technology to
everybody, everybody will be happy. And I use this example in the book of saying,
let's say you are using a language model to make an automated
marketing newsletter. As a technology person, you would think
this is amazing. I can categorize the customers
into clusters, then I can address each of these clusters
specifically. I can measure the response rates, I can measure the open
rates. All of these really fantastic things. Not very
expensive. The technology is there. Let's go for
it. What you don't recognize is that there's a whole bunch of people
in the organization that will think very differently about it. The
marketing department feels like it's losing control. You might have, if it's a
retailer, let's say you will have a category manager, maybe that says, well, I'm
incentivized on the sales here, so if your automated
newsletter is suddenly disrupting what gets sold and what gets
left behind, my bonus is gone. The warehouse manager
says, oh, your newsletter is suddenly going to change our
orders. You know, can we even have that stock ready in time? I don't know.
And so that's, I think, the big, big human factor that is often
underestimated. And maybe as a corollary of that, I think also
sometimes, and I'm guilty of that myself, you know,
sometimes as data people, we just miss out on some of the operational
realities. I know one project we weren't involved in, but it was
involving a steel manufacturer and they had an
agency that had designed an app for the workers
to sort of see how the machines were doing. The only problem was
that app didn't really work with the mandatory safety gloves that everybody
had to wear. Had these big, big, big thick
mittens trying to control an iPad. Doing that.
Not really the way it works. That's come up quite a bit,
actually. The notion that they get these apps and for whatever reason,
the end users were never consulted or brought into
the process. Amazing. Isn't it? Yeah. Yeah.
The book is called
Data Inspired and the author is
Sebastian Wernicke who's been here speaking with us.
Thank you. And where can folks get the book on Amazon, on
Audible, everywhere where books are sold. And
if you get enough on Amazon, they will surely be available on Audible or as
an audiobook soon. So we're still working on that. I
do love myself a good audiobook. Yeah, me too. Me too. So
hopefully that's going to be in the making soon. And I mean, if you want
to find out more, you can go to datainspired.org that's my website
and I'm always happy to connect with folks on LinkedIn. So that's where I
am. That's where I regularly post. So get in touch and reach out and let's
have some data conversations. Excellent. Thank you. I'll let the
outro music play.
Sam.