What are service level agreements and why are they absolutely

essential to managing complex, multi service

modern applications? Today I continue my discussion on

modern ops with Beth Long. Are you ready? Let's

go. This is the Modern

Digital Business podcast, the technical leader's guide to

modernizing your applications and digital business. Whether you're a

business technology leader or a small business innovator, keeping

up with the digital business revolution is a must. Here to help make

it easier with actionable insights and recommendations, as well as

thoughtful interviews with industry experts. Lee Atchison

in this episode of Modern Digital Business, I continue my conversation on

Modern operations with my good friend SRE

engineer and Operations manager Beth Long. So, Beth,

great to see you again today. And today we wanted to

talk about SRE terminology and

measurements, and it's fantastic that we have a

SRE in our miss in order to do that. So I'm glad you're

here. Great. Let's get started on

this. So SRE cyber liability Engineer

is tied very closely to the concept of DevOps,

but they're really not the same thing. Can you start out

by telling us what's the difference between DevOps and SREs?

I love this question. I've talked about this a number of times. And I'm going

to get back at you for asking me this by flipping it around and asking

you the same thing in a minute, but I'll take a stab at it. So

SRE site reliability engineering originated out of

Google, gee, almost 20 years ago now, I

guess. Yeah. And it was

really a discipline that was a

response to the

pressures of managing technology

at Google scale. And so

a lot of the practices that are associated with site reliability

engineering are the things that Google

developed internally to help them manage their scale as

they grew and then began to evangelize out to the wider

community. And so now a lot of those practices have been adopted more

widely and have been iterated upon. But

that's the origins of site reliability engineering

and the origins of DevOps are

a little bit more cross

cutting, a little bit more democratic, I

guess, and came out of

people around the same time

realizing that the

siloing of development and operations

was leading to unhealthy

patterns in the software engineering.

So people like John ALSPAW, who we'll talk about a

little bit later probably, if we touch on incidents at all, were

prominent in kind of saying, let's rethink how we're doing

the software engineering practice. So DevOps really focused

on integrating development and operations so

that those functions were

shared more as opposed to completely siloed. And

site reliability engineering was a set of practices

around maintaining stability and reliability

of large scale web operations. And so there

are some foundational topics that I'd like to ask you about actually,

around things like service level indicators, objectives and

agreements and a wide number of other

practices. So this is a very wandering answer to say

that the major difference between the two I think is really kind

of one of ancestry and how they started and

SRE sort of being a set of

practices and DevOps being more of a

philosophy and an approach to the development

environment. Yeah, it's almost like the

SRE is a practice that occurs within a DevOps model, but

it exists independently as well too. But it's a role

within DevOps. But not the only role within DevOps. Right?

Yeah. Now what's interesting is you

hear both DevOps and SRE talked about as

practices but also you'll hear about SREs talked

about as a profession, but yet

you don't talk about DevOps as a profession.

And in fact people do, but usually it's considered a

negative. I'm a DevOps engineer. No, there's no such thing as a

DevOps engineer. So is that

also part come from the historical

nature of where it came from or is there really is a difference there that

matters? This is a great question and something I hoped we

would touch on because I still kind of cringe a little bit when I see

DevOps engineer, but I've come to understand why

that job title has meaning. Because

there are organizations that for a number of reasons,

including the size of the organization, the history

of it, its composition, sometimes it does make

sense to have people focus

on the kinds of things that happen at the

boundary of development and operations.

And so you'll get DevOps engineers who focus on internal

tooling build and deploy pipelines

that of

activity. Yeah, I always hate the word DevOps engineer applying

to that as opposed to like infrastructure engineer or tooling

engineering. But you're right, you do hear that, you hear that

apply there. What it almost seems like though is you

hear a large organization say DevOps is good,

we need to go to DevOps. Okay, you and you are now DevOps

engineers. Exactly. And that's not the way it's

done, of course. And often they become the ones then that

focus on the tooling and kind of become those tooling engineers and keep

the DevOps title. And it's not always a good

history that brings you to that situation.

Yeah. And to answer your original question, I

think

there's a little bit of a CRISPR definition around

what a site reliability engineer does,

but there's still a lot of fuzz in the definition and there's a lot

of range in if someone says they're an SRE, what they actually do. It's

still going to be quite a wide range of options. But

the origins of site reliability engineering go back to

bringing the software engineering discipline

into the operations realm. And so again you see this sort

of both SRE and DevOps are really about

crossing that boundary, but it's

almost. In the opposite direction of what exactly. Yeah, exactly.

DevOps is more about bringing Ops into dev and

SRE is more about bringing the processes

of development into operations. Right. And so you are much more

likely to end up with an SRE group

that is sort of helping the whole organization level up with those

things. Whereas a DevOps organization,

at least in the way that I tend to use DevOps and I think you

and I are similar in this a DevOps organization is going to

be you're on call for your own services rather. Than having

an operations center and some of those things that are more at the

organizational scale as opposed to SRE tending

to be more likely that you're going to have a group of

people that are bringing those things to the organization. Yeah,

in that manner, SRE group or SRE

engineers is more akin to like an architecture group and

architects, they're assigned to individual parts of the

project, but they also have some global responsibilities as

well and shared knowledge. And whether they're in

one group or distributed is a

much more fluid question. That depends on the

organization versus a clear cut who should be in which group.

Sort of a model that is more akin to what

happens in DevOps. I like that distinction. So now

we know SRE is not the same as DevOps and we understand the difference

between them. That's great. So you bring me, get me back on something. Now

you had said, I'm not really looking

forward to that, whatever that is.

If there's one thing that's iconically associated with

SRE, I think it's fair to say that it's service level indicators

and service level objectives and service level agreements.

Slis, SLOs, SLAs. The acronyms

confuse everybody, even those who have been using them for years.

And I know that you have a very pragmatic approach

to kind of tackling some of these questions. So I'd love first for folks that

aren't deeply aware of those, maybe kind of set the scene and then I'd

love to kind of hear your take on how you can implement those.

Well, sure. I even confuse Slis and

SLOs. And so I'm going to need help with the definition if we're going

to define what the three are. But I'd almost prefer to avoid the

definitions and talk about what the problem is that's going on there. What the

problem is, is what all of them are trying to indicate

is the health of something, the health of a code base,

the health of a service, the health of an application.

Now historically the word SLA service

level.

Agreement. Agreement? Yeah.

SLA service level agreement comes

from inter customer connections.

So you have a provider of a service, of an

application that has a customer, and that customer says, we'll buy

your service, but I need a SLA service level

agreement that specifies how well or

what your service is going to do for me. And often those

agreements are around things like uptime

latency, how fast the application will work,

how many users can be connected to it. There could be a

thousand different dimensions on how it's measured, but it's usually some form of

measurement of a guarantee to the customer

of what the service or application that

the provider of that will guarantee in

exchange for usually money in the case of a customer

relationship. So an SLA has a very long history.

It's been around for a long time. The word SLA probably goes back

long before either one of us were born, because it applies

to contract work in general, not just software or

computer work. And so it's been around for a long time. But

what's happened in I believe it was Google

is the one who started the Slo or the SLAI model. I believe they're

the ones that did part of the SRE revolution. It included with

them. But what was decided was

we need some way to at a smaller scale as we

take this large application and now internally

divide it into services and into microservices and into

its various components. And especially in DevOps

models, we needed a way to say this part of the

service has requirements that it must

perform to. It has obligations it

needs to be able to handle in order to serve the needs

of the other services around it.

And so Google created new terms called Slis and SLOs

in order to distinguish them from

SLAs for how you

measure those parts of the application. And the

idea is Slis and SLOs are internal

measurements for internal customers, and SLAs were

external measures for external customers. That's where I have

my problem, because in my mind, in a service oriented

architecture, in a service oriented

team model, if you own a service

and other services depend on you, those other teams

are your customers. The fact that they sit down the hall from

you or right next to you, or on another floor, but in the same

company is irrelevant. They're still your customers.

Whether they're an internal customer or an external customer doesn't matter.

They're your customers. You need to keep them happy for

your application to perform as expected. So when

you provide a service that someone else is depending

on, and you specify what the requirements are for running that

service, those are service level agreements. Those are the

agreements that you have with the other service owners

of how your service will behave. There's no

difference between those SLAs as the external ones. So

don't call them something different, because that implies there's something less.

Right? An Slo implies an

internal agreement, which of course, internal agreements aren't

official agreements. Well, they're not as important, right?

SLA implies an external agreement, which is important because we're

talking about customers here. They're all customers. They're all

external. They're all SLAs. When you make an

agreement that your application performs a certain way, there's

no difference in whether or not that agreement is made with another team within your

organization or to an external customer. They're just as important

because guess what? If you break your agreement

for how your service performs with another team. That's not going to

just affect the other team. That's going to affect all the teams that they depend

on, and ultimately it's going to affect the customer. So it all

matters. They're all just as important. So let's not invent

new terms to describe them. In my mind, they're all

SLAs. So if you have 100 service teams

within your organization and they have their

criteria for how they are expected to perform

to support the other service owners, those

are SLAs. Those expectations are

service level agreements. They need to be treated at the same level of

importance as the customer level service level

agreements. I find that really interesting because you're getting

at the fact that words matter and what we

call things matter, because I think there are a lot of

really interesting organizational challenges with implementing

SLOs and SLAs effectively. And one of them sort of on the

flip side, is that when teams

talk about service level objectives,

they often sort of set them arbitrarily based

on, okay, these are the things that I can measure and these are

my objectives as the owner. And

what you're getting at is the fact that these really need to be

agreements. They need to be hashed out with product

owners and technical leads and people who are

deeply familiar with the customer, whether that customer is internal or

external. Stay tuned for our next Modern Ops segment

when Beth and I continue our discussion on modern application

operations by talking about ownership in a modern

operations world. Thank you for

tuning in to Modern Digital Business. This podcast exists

because of the support of you, my listeners. If you enjoy what you

hear, will you please leave a review on Apple podcasts or

directly on our website at MDB FM. Slash

Reviews if you'd like to suggest a topic for an episode or

you're interested in becoming a guest, please contact me directly by

sending me a message at MDB FM contact.

And if you'd like to record a quick question or comment, click the

microphone icon in the lower right hand corner of our website.

Your recording might be featured on a future episode. To

make sure you get every new episode when they become available, click

subscribe in your favorite podcast Player, or check out our website at

MDB FM. If you want to learn more from me,

then check out one of my books, courses or articles by going to Lee

Atchison.com, and all of these links are included in the show.

Notes. Thank you for listening and welcome to the world of the

Modern Digital Business.