The rising star of MLOps.
Is that how you feel about it?
What do people get wrong about MLOps the most, do you think?
Many people feel that they need to have all these shiny tools to do MLOps.
And I don't think it's true.
If you could go back the 10 years that you mentioned and you were to
choose whether to go into the same career path, would you do it again?
Yeah, I would definitely do it again.
you've got a very nice backdrop there with Iron Man
It's all Lego.
It's also flowers.
Also Lego.
Because I have two cats and if you have any normal flowers or even dried flowers,
they will try to eat them and they're not interested in Lego flowers, luckily.
So that's why only Lego flowers at our house.
And, I'm fan of Marvel and that's why we also have company
called Marvelous Mellops.
Because Marvel and the Mellops, these are two things I really like.
That all makes sense.
Yes.
So I have a lot of Marvel stuff here.
you can just see two, but I know there are a lot in the room.
Okay, Maria, so you've been described to me by multiple people at this
stage by the rising star of MLOps.
Is that how you feel about it?
I don't know in terms of following, maybe on LinkedIn, it is definitely growing.
a bit more than a year ago, I had, A bit more than 1000 followers.
And now I believe 46,000.
And when this episode comes out, I hope 50,000.
That is my goal.
and higher, obviously.
I just felt that there is not so much people writing about MLOps, talking about
MLOps, and I've been doing it for many years, so I felt like that's the right
moment for me to start talking about it.
I think I saw somewhere that it's over a decade now.
And You describe yourself as a dinosaur of AI, which I don't think
is entirely accurate, but why MLOps?
Why was it interesting enough for you to really double down on that?
So my career started more than 10 years ago as a data analyst, because there was
not so much in data, available back then.
data scientists didn't really exist in the Netherlands at least.
back then, and I started economics, econometrics, and I never really liked
economics per se, but I liked the data part in it and we were using R.
to do some modeling.
And that's the part that I liked and data analytics felt like the closest to it.
So that's how I got started.
And, even though I formerly was called the data analyst, we were already
building some churn model acquisition models for telco in R and then I
transitioned more into data science.
So I started learning Python and actually building first applications.
And there was no one who could put those in production, actually
integrated with the real applications that the customer facing.
And, that's how it all got started, actually.
I teamed up with some software engineers in the company that
were helping me to implement that.
and then I felt like that's a good moment to actually start introducing it to a
broader data science community to teach them, some best DevOps practices, like
using version control, having CI/CD pipelines, registration in place, and
that's what I started promoting back then.
I felt that it was needed because it took so much time for data scientists
to have something that could actually have impact, and not just some analysis.
yeah, actually introducing CICD pipelines, as I mentioned, and version
control was already a big step, because no one was using it back then.
There were some shared drives where people would put some shared code maybe
and Jupyter notebooks and that those were popular still of course very popular
but I feel like now the community is really becoming more professional and It
will be even more needed in the future.
So that's how it all started for me and Since then, I've built multiple MLOps
frameworks with different set of tools.
It started all with some proprietary software, then we moved to all
open source with Kubernetes, and then to AWS native infrastructure,
and now I use Databricks a lot.
So for the last three years, I'm building it on Databricks.
So would you say that this is what you would describe as a hockey stick
moment for, ML, this marriage of, DevOps practices added to the previous stack of R
and Jupyter notebooks and stuff like that?
Is that really what enabled this growth of MLOps and improvement in scale of how
people deploy this machine learning stuff?
Yeah, I think that was indeed such a moment.
And what I personally saw in my career that around 2015 or so
that a lot of companies started investing into data science.
Also, the company where I worked back then.
And after this investment was done.
There was very low return on that investment.
And that's because of this, separation between IT and, data
science departments, which were originally part of business teams.
So to actually implement something, you would have to create requests
to IT teams, and it can take some time before it gets implemented.
So actually giving freedom and, permissions to data science
teams to implementing themselves.
That's what makes a huge difference.
And some companies are getting there, but.
I feel like there is still a really large amount of companies that
don't use these best practices yet.
I think the question that a lot of people have is, and I'm asking that because I
went on to marvelous MLOps.substack.com, a little plug here, and, I found an
interesting article, the ultimate must haves and nice haves for MLOps and LLMOps.
And maybe we talk a little bit more about what MLOps versus LLMOps is, but,
when I was looking at the illustration at the top of that looks very similar
to any modern deployment stack minus, obviously the things that are a little
bit more, specific, like the vector databases, although, you could use
that for other purposes as well.
So why don't we just call it DevOps or, why do we need a specific term for MLOps?
I think MLOps is, an extension of DevOps.
It's a bit more complicated than that.
For DevOps, you don't have one thing that is you have in MLOps.
it's a model that can change when the data changes, so even if your
model hasn't changed the artifact itself But the data distribution has
changed and you get bad predictions.
Code didn't change nothing changed, but you get bad predictions that would
never ever happen in standard DevOps, unless maybe data schema changes, but
if nothing changes, but just some data distribution changes, things can fail.
And that's a huge difference and that's a lot of complexity to
how you need to manage things.
and that's, I think what a lot of people that ask these questions
usually don't realize that you need to have a lot of things in place to
make sure you can track it, properly.
And you need to know, for any deployment that you have, what code
that deployment, came from, what data that deployment came from, and
also what model artifact was created.
Need to have the track of all these aspects in one place and be able to roll
back if needed or modify things if needed.
that's a much different process than what you have for DevOps.
I also noticed that you've got on that list LLM monitoring.
I wonder how LLM monitoring works for anybody.
And a lot of people listening to this will be software engineers
who are trying to peek into this entire AI thing and see whether it's
worth spending more time on that.
How does LLM monitoring work?
I think a LLM monitoring has a lot, in common with NLP, model monitoring.
When you basically have anything text-based and you want to
monitor the quality of it that's something that you would have.
Things like human in the loop that is required usually for this kind of
systems is part of this monitoring.
You need to have some model evaluation metrics in place as well.
But those are usually not as good as having some actual person
checking and giving feedback.
so that aspect, I think is what makes it a little different, but there are
a lot of other aspects that come to LLM deployment comparing to, standard
machine learning, basically everything goes back to prompt engineering.
And, you add a lot of different things to the prompt and
you need to optimize prompt.
And there are different optimizers that are available now.
And one of the best ones I think is adult flow.
that's a product by Lee Yin.
I really recommend you to talk to her.
I think she's amazing.
so she's building one of the best optimizers for prompt, engineering
out there and basically knowing what you send to the LLMs, the whole prompt
structure and what are the parts of the prompt and making it, really clear.
I think that's also a very important part of this traceability for LLMs.
So what do people get wrong about MLOps the most do you think?
I think that many people feel that they need to have all
these shiny tools, to do MLOps.
And I don't think it's true.
As you mentioned earlier, a lot of components are similar to a
standard software engineering, the differences in principles.
But you could still use the same tools.
You don't need other tools for version control or CI/CD or orchestration.
you need some extra tools like, model registries, p vector, databases
and the whole LLM, stack, but overall For the standard machine
learning, you don't need other tools.
I think the best would be just to go and look, what do you already have within
your organization, what tools are there software engineers use and just use the
same thing because the field evolves very fast and the tools change very fast.
And you would have to go through migrations maybe every two, three years
when there is a big change in the tool.
And you want to limit that as much as possible by limiting amount of
tools and only use them if you really need it and maybe build some features
yourself if you really have to.
but I believe you need to stay pragmatic about it.
And that's what a lot of people get from.
So what would be your advice to overcome that?
take a look at what you already have and just use what you have.
And if you really miss a feature, build it maybe first and
otherwise look for alternatives.
It depends also on what type of organization you're part of.
If you're part of large corporate organization, getting any new vendor
on board is a lot of work, and it's often not really worth it in my opinion.
it's a lot of bureaucracy involved, so yeah, building your own thing
maybe can be a better approach just because of this complexity
And what do you think is the next thing in MLOps that needs solving?
What would be the next breakthrough moment, what would have to happen
in your prediction to make the field better, bigger, or easier, or, superior
in any metric of your choosing?
I think what needs to happen is the professionalizing the data
scientists, but there are different types of data scientists, right?
There are people that are doing research.
I don't think they need to learn mlops, but people that do data science for
any organization where machine learning models are professionally deployed.
I think it makes sense to invest into following the best practices and that
there is no other team that takes your code, rewrites it and deploys it.
I don't believe in that.
I really believe in data scientists taking this responsibility and it's
mainly people choice, not really tooling choice that needs to happen, for
things to become faster in the future.
Yeah.
All this humans.
Oh, man.
All right.
Let's switch gears a little bit to talk about Marvelous Ops.
you mentioned at the beginning that the name was inspired by Marvel and that you
seem to enjoy that universe quite a lot.
Tell us more about, how it started, with whom you're doing it and what
you're trying to achieve with it.
it started quite some time ago, actually, my colleague Basak, we worked together at
Ahold Delhaize, where I currently work.
She now works at booking.
we worked together in the same team and we were building MLOps
framework with Databricks.
And we felt what we are doing, you can't really find anywhere written down,
that a lot of different instructions are lacking, certain principles
and we decided to write about it.
we actually wanted to have that more as a part of corporate
blog, but it wasn't allowed.
So we just started our own blog and, we've been writing there Since then,
and we also have, LinkedIn page and I'm also very active on LinkedIn to write
about MLOps, to actually educate people on Databricks specifically a lot of
tutorials are notebook based and that's something we are not advocating for.
We advocate for tools that promote better software engineering practices.
we basically want to educate people and now we go even further
with that by providing a course.
So we have an end to end MLOps with Databricks course that
actually takes the real use cases.
And we show how to implement it end to end, how to implement, best software
engineering practices, how to go about some tricky things, how to track your
data, how to track your code, how to track your models and how to, rollback
as well, if needed, and what tools, what features in Databricks you should
use the best for certain scenarios.
That sounds like you're actually going pretty deep into the details, right?
Are you going to teach people how to use Git and, CI/CD?
Is that all included or do you send them back to see the basics somewhere else?
And you focus more on the AI related part of it?
we expect that people actually know what Git is and how to do basic CI/CD.
so basically our target audience is not, beginners, not People that just start
with a machine learning that the people that are already in the field for a
bit doing data science, maybe machine learning, engineering, and want to know
how to do it better with Databricks.
So we are going more deep into details.
Let's say if you have a model, in MLflow, how do you track things with it?
How do you add certain tags to it?
How you search for experiments, how you need to register your model in Unity
catalog, what is required for that.
And when model is registered, how you deploy your model
using model serving endpoints.
And what if you want to roll back, how you would do that?
A lot of different things.
And we cover also Databricks asset bundles as the way to deploy things and
a lot of tricky things, like how to deal with private packages, which I believe
is a very annoying topic to deal with.
a lot of details you can't find easily.
When is the course coming out?
It is ready for enrollment and, we start the cohort on 14th of October.
It's a seven week cohort where we give feedback to people when
they do assignments and they are in touch with us constantly.
to get information from us if they have any questions.
So that's actually an opportunity to work with us, on a daily basis.
And how do people find your course?
Where do you go to sign up?
It's on Maven.
So I can give you a link, where you go and enroll, but it's basically on
Maven, if you just look for end to end, MLOps with Databricks, you will find it.
It is under engineering and data and engineering, I believe, and it's also
a top selling course at the moment.
congratulations.
I think a lot of people will be interested in that and I hope it goes well.
Is that the first cohort you're running?
thank you.
Yes, it is, it is our first cohort.
It is actually coming from our three year experience of working
with Databricks and even broader experience with MLOps in general.
That's something that we did internally and our struggles to do things with
Databricks properly for all these years are basically condensed in this course.
Okay.
So everybody listening to this, please go and check out the course.
And for you, Maria, I guess the one last question that I really have is
if you could go back the 10 years that you mentioned, And you were to
choose whether to go into the same career path or a different one.
Would you do it again?
Yeah, I would definitely do it again.
I think everything that happens with you in life in general happens
to you with the best possible way.
it brought me to the person I am now, so I wouldn't change
anything in my life in general.
and also not in my career path.
And if you are not happy with your career path, you can always change things, right?
for anybody who is at the beginning of their career and listening to this,
what would be your word of advice?
So I think in the beginning, it's good to go broad.
Try a lot of different things.
if you like, data science, go to try data science.
If you like analytics, some people do like analytics, engineering, building
dashboard, go try do that work together with software engineers, that are
working on some features for the website, for example, and learn what they do.
Just understand as much as possible, as broad as possible, and then find
things that triggers you the most, what you like the most, and just
go deeper in that specific thing.
I believe that's a good recipe for achieving what you like.
And if you end up not liking it, you can always change, right?
Absolutely.
That sounds like very solid advice.
Maria, thank you very much for your time.
For anybody who wants to go and follow Maria, Maria on LinkedIn,
marvelous ML ops dot sub stack dot com is where you find her writing.
And for the course, you just heard how to find it and to end MLOps with Databricks.
Thank you so much for your time and I'll see you next time.
Thank you.
Have a great day.
Bye bye.