Siddharth Pareek, Vice President of Consulting and Global Practice Lead for DevOps CoE at NatWest Group, shares his insight and practical guidance for starting and achieving domain-oriented observability.

Enjoy the Humans of DevOps Podcast? We’re incredibly grateful to be voted one of the Best 25 DevOps Podcasts by Feedspot.

Want access to more DevOps-focused content and learning? When you join SKILup IT Learning you gain the tools, resources and knowledge to help your organization adapt and respond to the challenges of today.

Have questions, feedback or just want to chat about the podcast? Send us an email at podcast@devopsinstitute.com

Transcript

Narrator 0:01
You’re listening to the Humans of DevOps Podcast, a podcast focused on advancing the humans of DevOps through skills, knowledge, ideas and learning, or the SKIL framework. Here’s your host DevOps Institute CEO Jayne Groll.

Jayne Groll 0:15
Hi, everyone, it’s Jayne Groll, CEO of the DevOps Institute and welcome to another episode of the Humans of DevOps Podcast. I’m particularly excited because, as you know, the episodes this month are focusing on a trending pattern on a practice known as observability. And today, I’m very excited because we have Siddharth curry, who is not only going to talk to us about domain oriented observability, but also bring an enterprise perspective to the discussion. Hi, Sid, welcome. Why don’t you tell the audience a little bit about yourself?

Siddharth Pareek 0:58
Hi, Jane. Good team. Hi. Hello, everyone. My name is Siddharth Pareek. I’m based in New Delhi, India. So I’m working for natters group. And our group is Bank of branch if you’ve heard of natus bank nathless, International fraud Bank of Scotland, also bang could spank are all under the bigger umbrella of the Napa scrub. So I work as the DevOps Center of Excellence global leader for Napa escrow. Fantastic. So you know, as you know, the concept or the practice known as observability, the interest in observability has suddenly become a buzzword in DevOps, particularly as it relates to operations SRE, it’s been a pattern in the software world for for quite a while. And now we’re talking about domain oriented observability. Tell us a little bit about the correlation. Tell us a little bit about your opinion on why observability has certainly become of enterprise interest, why observability has become as observability has been since ages in different form and shape in the organizations in the research in the software design patterns, why it has gained importance is because till now what we were talking about was we were talking about monitoring and alerting things. But that was specific to the technical aspect of the applications and systems. But those monitorings and toolings, were not bringing out the business aspect of the monitoring and alerting. What I mean by saying this is not I business does not wants to hear across the CPU utilization jaw, the server utilization is what they really want to hear across is how much failure has happened when the customer payment section, you know, for example, or for how long duration was the customer there in the session, while buying the products on the e commerce website. And hence, we want to move from the technical aspect to the business aspect. And hence observability plays a role. So in short, if I’m observable, then I can monitor you.

Jayne Groll 3:27
Well, and that’s interesting, because you’re right monitoring, I’m an old IT director, monitoring was very much first of all very reactive. And second of all very component are very discreet in terms of what was being monitored and observability has a big external piece. And so talk a little bit about domain oriented observability. Can you give us an idea of what that what that specifically means?

Siddharth Pareek 3:54
Okay, explain that. Let me give you a brief about the history before I pondered upon the domain observability was the observer aspect or the pattern has been since ages in the software design, per se. And for example, the observer pattern has been one of the well known gang of four design patterns, in which an object called the subject maintains a list of its dependents, called the observers. And then it notifies them automatically of any state change, usually by calling one of the methods. The problem that this pattern was trying to solve was that in large, monolithic design, were not able to scale well as new graphing or monitoring requirements were delivered. And Thurs this pattern was identified and introduced. Now, as per Google the observability is tooling or a technical solution that allows the team to actively debug their systems. So observability is based on exploring the properties and patterns not in define it once. Now, if you Let’s go back to history observability isn’t a new term, but it was first introduced by an Hungarian engineer, known as Rudolph Kalman for linear dynamic systems and mid 90s. And then 5060s, for which he actually received the Medal of Honor, two from the Institute of Electrical and Electronic Engineers, which is popularly known as the IEEE. Now, coming back to the question of the domain object oriented observability. So, in 2019, one of the, you know, observations, which was made on Martin Fowler’s webpage, by a gentleman named as Pete Hodgson, talked about domain oriented observability. He said that when it comes to observability, the smees get too technical, you know, something which I was talking about earlier between technical and business, and they mess up their codes and design in order to introduce them. So, there needs to be a pattern, which is cleaner, and give us the leverage to add business relevant observability. So, some called the domain oriented observability as the practice of observing one system, using the ubiquitous language of your to pain.

Jayne Groll 6:40
And so that’s very interesting, because, again, we’re in the business into this. And and I think in it, you know, we’re pretty young, as an industry goes, or as a business unit goes, I mean, if you think about it, we’re, what, 4050 years old, maybe right? And so up until this point, I think a lot of the observations, no pun, right has been about we’ve been very it centric, we’ve been very pleased oriented. And we’re not really looking enough at the business drivers that are delivering value. And you know, and I know certainly when we look at at where we are today, this becomes pretty important. So are these patterns in concrete being solution eyes for programmers at the moment? Is it only about ops, or it doesn’t have a strong developer relationship too?

Siddharth Pareek 7:38
The domain oriented observability talks about the dev aspect more? Because that where the journey of the application server service starts? So to answer your question, whether they are have been any concrete socializing, dance, yes. So for example, in terms of language, like JavaScript, we have domain oriented observability. for it. If you talk about language like C sharp, we have the implementations done by the, you know, the researchers and the community for it. So yes, it’s just not the pattern of framework, there has been solutions or implementations in place for it to.

Jayne Groll 8:21
And so does it replace monitoring? I don’t think it does. But but there’s a piece of it right? So what is the relationship to monitoring?

Siddharth Pareek 8:30
See, as I said earlier, also, I’ll make it short, not. If I’m observable, then I, then I can monitor you. So what happens is, at one end, we are improving our monitoring tools from nag us to now we have, you know, Angel tools like Ansible, Chef puppet and other things. But the important thing is, you are bringing new monitoring tools, but are you improving the observability of your application of your service? If that is not the case, you know, how rich tools you bring in for monitoring, the end goal will not be achieved. So, again, in short, yes. The you know, the more it becomes observable, the better the monitoring becomes.

Jayne Groll 9:19
I am certainly agree with that. And I think that there’s a shift, right, there’s a paradigm shift. That’s as much cultural as it is technical. Can you talk a little bit about the culture of observability, particularly, you know, inside your own environments, you know, what is what is the cultural change from an enterprise perspective needed for observability

Siddharth Pareek 9:50
One of the things that is needed is, you should stop talking on lunches and your meals or dinners, not your server or yours. CPU or your, you know, high capacity utilization of natural bang bandwidth to your business people, you should talk to the business folks in the language in which they understand them that, you know, coming back to the previous example, which I took talk about that this month, you had zero person payment failures that happened on the portal, you had customers have spending more than, you know, 100% of the duration on this website, then they did previous year, you had an increase in, you know, for example, the access of the catalog, or on your portal, more than what it was earlier. And that, to bring that language in picture, you need to change your thought process, you need to change your technologies, you need to change your process, and you need to share your observability

Jayne Groll 11:00
What’s interesting, too, because, you know, we’ve been trying to achieve that for a very long time. And it were we would have business speak, as even internally, right, not even just speaking to the business, but speaking to each other in business be more than about CPU, or, you know, looking at networks or looking at cloud, right looking, looking at the technical aspects of it. But it doesn’t come naturally to a lot of IT organizations because it is part of our culture. So I guess the question is, what can organizations do? And maybe it’s something that you guys have done already? What can we do to help it think in those terms, speak in those terms. And, and then, of course, if you think in speech, and hopefully you’ll act, thinking in the same way that the business people do, I mean, think about this, if you were marketing, and you were talking to the business leaders, you would likely speak a lot of the same language. If you weren’t, right, you would likely speak a lot of the same language. You know, when I was in it, one of the reasons it didn’t have a seat at the executive table for a very long time, is because we didn’t understand so what are some? So I’m gonna ask you two questions, in simpler words, can want to achieve domain oriented observability? And how can we help this? This isn’t even just behavioral? So it’s more of a mental or mentality? How do we get it thinking and speaking and therefore acting the way that we want to? So it’s really two parts of the same question, I guess, today.

Siddharth Pareek 12:47
So to answer the first question, how can one achieve domain oriented observability? Now, when you talk about observability, it is very broad in scope. You know, it talks about from low level technical Matrice, to high level business key performance indicators. On the technical end of the spectrum, we can track things like memory network, thread counts, garbage collections, and blah, blah, blah. On the other hand, of the spectrum is our business or the domain metrics that we’re talking about, which, you know, mind track things like you know, which I’ve talked about the cart, abandonment rate or the session durations or the payment failure rates. Because these higher level metrics are specific to each system, they usually require handrolled instrumentation logic. Now, this is in contrast to your low level technical instrumentation, which is more generic and often achieved without much modification to a systems code base, beyond, let’s say, perhaps injecting some sort of monitoring agents at boot time. Now, it is also important to note that your higher level, product oriented metrics are more valuable always, by definition, because they are more closely reflect that the system is performing towards its intended business goal. So by adding the instrumentations that track these valuable metrics, we achieve domain oriented observability.

Jayne Groll 14:44
And could the instrumentation then, when it’s producing its logs or when it’s producing whatever output? Could it be framed in business terms, as opposed to technical terms? Is that a way that we can train it to use the same language. And again, I’ve seen enough monitoring logs over the course of my career to know not always so right. It’s not always so that the the logs or the evidence that comes out is, is not it centric. But have you ever seen that? I mean, is and again, I’m kind of aiming for how do we get there?

Siddharth Pareek 15:25
The answer is a bit technical in nature. So, I would say that, you know, generally what happens is, when you are designing your systems, what generally you do is, you try to instrument the code, for example, involving a vendor’s library to your dependencies, and then calling those libraries directly into your code. Now, in that case, what happens is your code becomes less reusable. Right. Now. Now for this, there have been a lot of solutions, sizing that has been done. There was a gentleman on styler treat, we said, you know, let’s have a solution involving your centralized inter process collections. Then there was other researchers, Pete Hodgson, who, you know, brought this term domain oriented observability, we say, let’s have abstracting collections in the process. So they found out that there is a simpler within process approach that is easier for the devs, to understand, to test to configure to maintain and to operate. And it is the fusion of that describes by having your event based observability. And collecting the instrumentation context. In business terms, what it happens is that when, you know, for example, if I pick up an example of Netflix, so what Netflix done in their learning phase of observability was that they said they understood that storing your raw applications logs won’t scale up, right. So to to address the scalability, what Netflix did was they switched from, you know, streaming the logs, and filtering them on some selected criterias. So transforming them in more memory and persisting them as needed. The point 1.2, what Netflix did was, they said that our observability tools needs to access various persistent data types. Now, choosing which kind of database to store a given data type depends on how each particular data type is returned and retrieve. The third thing, what Netflix did was that, besides analyzing their logs and request traces, they included mattresses for observability as well. So they didn’t wanted mattresses for your CPUs and all they said, Let’s have metrics for observability as well. So, for example, by exploring the mattress, anomaly detections and metrics correlation, they learn how to define actionable alerting beyond just the threshold solo teams, where it came from a learning path, right and gave them again, you know, we talked about machine learning now, we talked about human learning, right, but it gives me insight. I’m sorry, I interrupted you, we’re gonna add something to that. No, no, absolutely. So too, you know, when you said, How could business be involved in observability? So they said that we would, how would be involved, let’s have metric systems or have the ability and agree with business. That was one thing that they said that we do not want to burn on Tumblr are logs and traces into the conventional modes of, you know, the text files and other files. But we would filter those logs at the runtime. So only the runtime based on the runtime queries, the logs will be shared or saved. So that agreement also did happen with the business. Thank you.

Jayne Groll 19:22
Yeah, thank you. Because, again, this is the type of tangible information. I think that particularly those that are new, whether you’re a developer, because your right developers today need to know how to write their code, they need to know how to build their code, they need to know how to test their code, secure their coat code deployed and operate it. And so having that ability to observe and you know, it’s funny, I looked up the dictionary definition right? Observer, but you know, to observe is to see to notice, right? So it means to be aware, as opposed to again and our traditional approach to event management or monitoring were really very, very similar. And so we were only aware of the dashboard, we’re only aware of the green and the amber and the red. And there’s nothing wrong with that. But it’s only one piece of a much larger paradigm. Right? Right, right. Absolutely. Yeah. So so I do appreciate that. And I think that our listeners will love hearing this. From a very, very practical perspective, this is still a fairly new topic. And as you know, in August 20, is DevOps Institute skillup. De is going to be entirely about observability. Because I think that the topic is trending very quickly, it delivers, you know, when done, right, it delivers, I think, a different my and a better view in terms of, you know, what the environments are looking like. But I think more than that, it is going to be part of this new evolution, you know, we don’t know what the new normal is going to look like. And so equipping the IT professional, particularly the developers, with the kind of tools, and also a different way of thinking, I think is important. So before we wrap up, any last tips on getting started with observability?

Siddharth Pareek 21:26
You said great things one thing, which I want to call out is thing from the business perspective. Point, one. Second point is, how do you can think from the business perspective is stop thinking from the dev perspective, ask questions, which matters to the business rather than to the developers is stop. You know, as I said earlier, also stop dumping your logs in the large monolithic log files and saving it for no use actually, only have those logs only have those records, which means not to you, but to business. And that is one way of moving towards the domain service. observability. And the last but not the thing says you should notice how to separate observability not from your code, but from your domain logic. Thank you.

Jayne Groll 22:23
Awesome. Thank you. This has really been fantastic Siddharth and thank you so much for sharing your knowledge and your advice because I think it’s very, very valuable. Again, I so appreciate a set of party joining us today sharing about domain oriented observability. It is observability month. So August is all about observability. So if you’re listening, Siddharth is one of our ambassadors are very much appreciate that August 28 is DevOps Institute’s monthly skillup day and this month, it’s going to be entirely about observability. Please go go up to www DevOps. institute.com become a free member register for observability or any of our future skillup days. Again, Sarah, thank you so much. For those of you that are listening, we are continuing to do our humans a DevOps podcast, and I hope you stay safe and I hope you stay well. Thanks very much.

Narrator 23:26
Thanks for listening to this episode of the Humans of DevOps podcast. Don’t forget to join our global community to get access to even more great resources like this. Until next time, remember, part of something bigger than yourself. You belong