On this episode of the Humans of DevOps, Jamal Walsh returns! Jamal was previously on episode 36.
Jason Baum and Jamal discuss:
- Should security be more of a consideration for all DevOps teams? How does this relate to DevSecOps?
- Why is security becoming more of a priority for businesses?
- How will this impact the role of security specialists?
- What are the biggest security issues facing DevOps teams?
Thanks to our episode sponsor Kolide! Want access to more content like this? Gain the tools, resources and knowledge to help your organization adapt and respond to challenges by becoming a member of DevOps Institute. Get started for free: https://www.devopsinstitute.com/membership/
Have questions, feedback or just want to chat? Send us an email at [email protected]
You’re listening to the Humans of DevOps podcast, a podcast focused on advancing the humans of DevOps through skills, knowledge, ideas and learning, or the SKIL framework.
Jamal Walsh 00:16
I think what’s hard in when it comes to software development and security is thinking like someone who wants to attack kill your system, your services. And I think working out how you embed that into your engineering teams is kind of the big challenge around dev SEC ops.
Jason Baum 00:34
Hey, everyone, it’s Jason Baum, Director of Member experience at DevOps Institute, and this is the humans of DevOps podcast. Welcome back. Hope you had a great week. Last Tuesday, February 8, we actually celebrated safer internet day. Never heard of it. I actually heard hadn’t heard of it either. But safer internet Day was celebrated for the first time in 2005. And the goal is to spread awareness about online privacy and security. This year, its theme was together for a better internet and called upon stakeholders to join together to make the internet a safer and better place for all sounds pretty great to me. And so with that in mind, we found our next guest and begged him to come back to the show. Jamal Walsh is here with me to discuss online security and more specifically, def SEC ops. Jamal is a passionate agile and DevOps practitioner with a keen interest in the human side of agile and DevOps practices. He also happens to be a DevOps Institute ambassador and was a guest of this podcast back on episode 36. So if you’d like to learn more about Jamal, I definitely encourage you to have a listen to that episode. We discussed everything from applying DevOps practices to legacy platforms to airline hangars to mountain biking, and more. So definitely have a listen to that you’ll learn a whole lot about Jamal. It’s a lot of fun. And that description sounds about right for this podcast. So Jamal, welcome back to the podcast. Thanks so much for coming.
Jamal Walsh 02:12
Thanks, Jason. Great to be here. Again, always a fun conversation. I think this one today is probably front and center of everything I’m doing at the moment. So I’m really looking forward to having a chat with you.
Jason Baum 02:24
Awesome. Well, we’re excited to have you, and are you ready to get human again? Oh, yes.
Jamal Walsh 02:29
Jason Baum 02:32
Excellent. So DevSecOps. So there’s DevOps, I know of all the ops, there’s a ton of ops. Yeah. So DevSecOps, I know, applying DevOps principles to security practices, I’m assuming, and then what?
Jamal Walsh 02:48
Yeah, I guess I guess it. So it’s always an interesting one, right. I think I think when people see dev SEC ops and wonder what it means, for me personally, I think it’s about ensuring you apply security thinking, to you know, every step of your kind of software lifecycle. You know, and that can be from really early on when you’re actually discussing and designing things that you’re going to implement to the actual kind of operation and support and monitoring of the stuff that you build. So it’s kind of all-encompassing, in my view, you know, it’s about thinking about security every step of the way. And obviously, embedding that within your development teams, and ensuring you know, the things that you build a secure, and you do it in a way that doesn’t slow you down, because I think a lot of the old security practices can sometimes get in the way of being, you know, agile and delivering things quickly to customers. So yeah, that’s that’s my personal view on it anyway. Yeah.
Jason Baum 03:50
When you say the security practices, I mean, I think of I mean, on the front end side of using something like two factor are taken. Ah, gosh, I can’t say the word. So hard. Yeah. Thank you. It’s so hard to even say the word, let alone then have to do it and get the text message or the email. And so yeah, I mean, it becomes cumbersome. When you’re applying it to software development, is it kind of like when you’re testing something you like build it to break it? And just so you’re building something to kind of get around it? Is that what you mean by that? Or
Jamal Walsh 04:25
I think, I think, I think what’s hard in when it comes to software development, and security is thinking like a malicious actor, someone who wants to, he wants to attack your, your system, your services. And I think, I think, you know, a lot of software engineers kind of find that hard because essentially, they want to build, you know, they want to build great products, great features. And I guess it’s hard to actually think about how could someone take advantage of what I’m building? What can they do, how could they access it and what could they do if they actually access it? What kind of information can they get and what kind of problems causes as an organization. So yeah, I think it’s the mindset and the way of thinking is very different to kind of your standard engineering practices, and kind of working out how you embed that into your engineering teams is kind of the big challenge around DevStack. Ops. Yeah, it’s
Jason Baum 05:16
interesting, because it’s not really a bug, right? It’s working, it’s working fine. But how would someone circumvent yesterday, the rules that exist, I guess, the logic that exists?
Jamal Walsh 05:29
Yeah, yeah. And you have to, you know, you really have to think about it every step of the way. Because, you know, even from the point, you start thinking about what you’re going to build and how you’re going to build it, it’s a we do something called threat modeling, a very group. And that means we will take a design, before we even code anything, we’ll design the system, and then we’ll do something called Threat Modeling before we even write a line of code. And that will allow us to think, like a malicious actor. And think about the, you know, the attack vectors of the application, where someone where there might be some exploits and things like that. And then, as we design and implement the system, we’ll take those risks. And we’ll make sure we put security controls in place to mitigate anything we’ve identified really early on. Because the last point you want to find a problem with security is in production. So you know, there’s lots of different steps you can take, right? Through your software development lifecycle workflow, you know, it’s this whole, this whole old concept of shifting stuff left, security is the same, right? The, the sooner the, if you move it more towards the left, then you can find these problems sooner, fix them sooner, and the impact of those things is much less. So this
Jason Baum 06:49
is the humans of DevOps. And so I find it interesting to get in the head of a malicious actor, what does that look like? How does it how do how does one channel that that malicious actor rather than just, you know, looking at the code, and where one might, I think, I’m assuming you have to get in their brain a little bit, right?
Jamal Walsh 07:10
Absolutely. I mean, we have, we have security experts working in the business, and we also have partners who will speak to as well. It’s, I think, it’s, it’s really hard to put yourself in that mindset, because it’s not a natural mindset. You know, you know, it’s not a day to day you don’t think on it, you know, a good engineer is not thinking about how, how they can take advantage of a software system. So I think, yeah, I think, I think helping people get in that mindset, you need to help people with experience of that. And, you know, I’ve worked with some great security consultants, pen testers, and, you know, there’s these concepts of red and blue teams and purple teams in the security space. And it’s purely their job to think in that way. But I think it’s really important that they don’t just do that alone, that they sit with other engineers, and, you know, your QA engineers, your software engineers, and impart that knowledge in that way of thinking on to the engineer, so they can think about those things when developing software.
Jason Baum 08:20
So it’s pretty safe to say that security should be more of a consideration for all dev SEC ops teams.
Jamal Walsh 08:26
Yes, absolutely. 100%. I mean, you know, you just have to look at the, you know, I think there was the lock for J incident recently, where, you know, the logging package had been activated, there was an extra zero-day vulnerability in log for J. And, you know, instantly people are scrambling around trying to patch that. And, you know, some companies were, you know, fell foul of that, because they, they weren’t able to patch their systems quick enough, or they were unlucky enough to have someone take advantage of that, that that vulnerability. And then there are so many ransomware attacks happening these days, I mean, that, that, that for me at the moment that the kind of plural proliferation of the ransomware attacks at the moment, is staggering. And that that for me and for the company I’m working with at the moment and probably most companies is probably the scariest thing right now from a security point of view. You know, if, if one of these bad actors gets in and can get this ransomware on your machines, you know, they can completely encrypt your entire data. I mean, there’s a company in the UK called KP snacks. They actually make my favorite brand of crisps be fooled hoops. And they were attacked with ransomware just last month, and that’s completely affected their supply chain massively. So none of their crisps are in the shops at the moment. And while you’re pretty angry, I’m an unhappy customer. Yeah, that
Jason Baum 09:50
would drive me crazy.
Jamal Walsh 09:53
You know this is this is you know, this is the kind of the fallout from you know, not, you know, behaving those exploits. How In your systems, they can cause real problems. You know, from a, from a, from a brand point of view and just from a trainer trying to fix these kinds of things is real.
Jason Baum 10:09
Yeah, I mean, let’s talk about that. So it’s it’s safer internet day and just talking about security and or last week was safer internet day. And we’re talking about security. And we’re not talking about just safer internet for our children to go on to I mean, obviously, that’s very important. And, and all that. But we’re also talking about safer internet as far as practices to keep your data safe and companies safe and their data safe. More importantly, because your data’s their data. So where it How does that all fit in? We talked about the ransomware attack, what are they after? What are they looking to exploit? And why should that become more of a priority for businesses? Or why is that a priority for businesses? Right?
Jamal Walsh 10:54
So I think ransomware is, I think, one of the things as well as I don’t think, I don’t think we actually know how rife is because I think a lot of companies just pay the ransom and get the concept. Yeah, and so. So you know, a lot of these things we’re not aware of even happen, because the companies don’t want to publicize the fact that this has happened in a lot of cases, because it affects their brand. And in some in some scenarios they do, and they have to because they’re just impacted so badly. But from a, you know, a mitigation point of view, there are so many things you need to think about, you know, you know, encrypting your data at rest, making sure you have regular backups, and your backups are stored completely separate from everything else that you’re doing. You know, and just being able to practicing the disaster recovery side of things to ensure that if, you know, if something like that does happen, how quickly can you restore? How much is it going to impact you? And, you know, sometimes, with these attacks, you just don’t know, you know, if you’re, if you’re a large enterprise, and they’ve managed to infiltrate a large portion of your network, then you know, recovering from that can be a hell of a, you know, a hell of a job. And, you know, sometimes the cost of trying to recover from that, versus the ransom demands, you know, this is why a lot of companies kind of, you know, weighing those things up, and obviously, just some of them are settling, without even, you know, telling anyone or letting anyone know that something’s happened. But you know, there are, there are lots of things you can do to mitigate it. But again, I think it’s more about planning, planning for it to happen, I think is the most important thing, and making sure you’ve got the right processes and tools and steps in place to and you practice, you know, what you would do in that kind of scenario, I think is really important.
Jason Baum 12:51
Well, it kind of goes back to what you said, catching it in production, right? I mean, those earlier phases, so that you don’t get to that point where you’re working too far.
Jamal Walsh 13:01
Yeah, so there’s the kind of the processes you take between during your development lifecycle. So you’ve got things like Threat Modeling really early on to identify any potential gaps in your security and the design of your system and its architecture. And then next, you’ve got kind of checking your code to ensure that there are no vulnerabilities being developed within the source code itself. And then the big one at the moment is kind of the dependencies that a lot of you know, organizations pull into the software that they’re developing. So you know, you’re pulling in packages from external sources, and you want to be checking, you know, scanning those dependencies and ensuring there are no vulnerabilities in the software that you’re pulling in from other people. You know, and then there’s obviously, load lots of other stuff you can do in the development part, if you’re using containers, you can have scanning, you can scan the images of your operating systems in those containers, and then all the way down to kind of securely monitoring your, your, your website and applications from, you know, putting web application firewalls in place. And there’s tons of, you know, kind of bought detection software that will detect if people are trying to do credential stuffing on your website and things like that. So there’s, there’s a vast amount of things you can do as an organization to kind of protect yourself.
Jason Baum 14:25
We’re investing a lot in security, and I would assume that businesses, you know, obviously they see this threat. We’ve we talked about that, but how has that kind of changed or the landscape of the role of US security specialists, you know, the type of people that companies are looking to, to hire bring in to solve this?
Jamal Walsh 14:48
Yeah, I think I think there are different types of security specialists. So there, you get your consultant types, you’ll come in and consult around a large, you know, security in you, especially in an enterprise organized organization, is vast. I mean, you’ve got the, you know, from, you know, uses laptops, and the networks and all of that, that they use to the software they’re developing. I mean, security as a subject matter in a large organization is enormous. And generally, that’s where you’ll have security consultants, and, and C ISO type people who kind of manage the whole scope of that. But from a software development point of view, I think it’s always good to have someone who deals with security having a software development background because I think that allows you to have a conversation about security and software development at a level that really helps your engineers understand, you know, the implications and the cost of not doing security properly.
Jason Baum 15:54
Today’s episode of the Humans of DevOps podcast is sponsored by Kolide, Kolide is an endpoint security solution that sends your employees important and timely security recommendations for their Linux, Mac and Windows devices, right inside Slack collide is perfect for organizations that care deeply about compliance and security, but don’t want to get there by locking down devices to the point where they become unusable, instead of frustrating your employees collide educates them about security, and device management while directing them to fix important problems. You can try Kolide with all its features on an unlimited number of devices, free for 14 days, no credit card required. Visit Kolide.com/HODP to sign up today. That’s kolide.com/HODP enter your email when prompted to receive your free Kolide gift bundle after trial activation.
Yeah, and I’d be remiss to mention that our sponsor Kolide, we had Jason Miller, the CEO of Kolide on and talked about, he’s the author of honor security. And we talked about how, you know, all these threats are becoming more sophisticated, the steps to prevent them are becoming more sophisticated. But then you have a whole line of all the people who are employed by you, and you need to take steps to make sure that they are being safe, and that they are not compromising the organization. And but you need to do it in an honest way. And one that’s not too cumbersome. And I’m assuming that’s a big piece of what plays into this, how do you make the employees feel like, the big brother isn’t like just watching every move that they take?
Jamal Walsh 17:46
Yeah, it’s really interesting. And you know, I work we work, we work in a financially regulated business as well. So, you know, it’s even more stringent in financially regulated businesses, when it comes to security and things like that. There are certain changes happening in the PCI compliance space where you have to start, you know, talking about how you’re securing your software development. And they never, never seen that in any kind of PCI audits before, and now they’re really starting, you know, if you’re financially regulated, and you’re doing payments and things like that, then yeah, they’re starting to really delve deeper into your architecture and ensuring your engineers are kind of up to date with everything that’s going on. There’s, there’s obviously a lot of training. And I think the other thing we seen in one of the latest PCI audits that we had to do is we had to prove that our engineers were taking regular security training. So you know, these, these are the things that are starting to happen now with, especially in the regulated space is that you’re now having to show that the engineers that are developing your software that is financially regulated, for example, have taken some relevant training to ensure that they practice secure coding and things like that.
Jason Baum 19:05
It’s so funny, I remember at one of my very first jobs, we during orientation, you all go into the room and they have the chief, you know of information technology, and they sit you know, they have everybody sit down, I do the presentation on if you get a suspicious email, make sure you forward up to us and don’t open it. I feel like we have come such a long way from that, you know, that that orientation meeting, but in many ways, it still holds true, but yeah, it’s like, this is a completely different world.
Jamal Walsh 19:37
Yeah, so we have a we have some software in the company that actually sends out malicious emails. Purposely. Oh, really to see if there yeah, if you click on them, it will tell you if you forward it on like you’re supposed to you get a little pat on the back. So
Jason Baum 19:53
I was gonna say what happens if you click on it? Do they like send yell?
Jamal Walsh 19:57
No, no, no, it’s much more friendly than that. I think it’s just This day is to help people recognize when, as you know, I think it’s really helpful for everyone, right? It’s not just for, for work, it’s people in their, in their personal circumstances and things like that. Being able to, you know, recognize those kinds of malicious requests and emails and things like that is a really, you know, important thing, not just inside workbook personally as well.
Jason Baum 20:20
Yeah, I mean, you see it all the time, my Twitter was hacked, my Facebook was hacked, please don’t respond to this email for me. It’s not me. Yeah. It’s, it’s, it’s very prevalent. So what are the biggest security issues facing DevOps teams?
Jamal Walsh 20:34
Um, I think, I think, I think training is one of the biggest things, kind of, you know, from, from a, from a DevOps team point of view, it’s about having multidisciplinary teams, where you all work together to deliver, you know, fast, secure software. And I think, I think from a security point of view, it’s, it’s, it’s bringing everyone up to speed, to be able to understand, you know, what it means to be secure what, what tools and processes, can you and there’s so much to think about? Engineers have a lot to think about anyway when it comes to software development. And then this just adds another layer of complexity on top of that. So yeah, it’s, it’s, it’s, for me, the biggest challenge, and the biggest issue is, is understanding how you can improve that awareness. And, you know, add those security skills and mindsets to the engineers, not just software engineers, QA engineers, you know, all the roles within your, your kind of your DevOps team.
Jason Baum 21:43
And you’re doing all that, and you can’t slow things down. I think that’s right. I mean, that’s a huge piece.
Jamal Walsh 21:50
So that is probably the most important part of the whole thing for me is the fact that if you get it wrong, security can absolutely cripple you. Because, you know, security can turn around and go, you can’t release that into production. Yeah. And then you stopped. And it’s for me, it’s finding the right balance of security versus getting features and products out to your customers. And that part is the balance is the most heart the hardest thing to kind of find wins in that scenario. Well, it’s, it’s a negotiation, right? Security wants you to be 100%, secure, and you want to get a product out to the customer. And in the end, it’s a lot of risk management. So it’s about, you know, understanding, you know, why you’re not going to do something and saying, you know, if you’re not going to, if you’re not going to implement a specific thing, because it may take a long time and the risk is quite low, then it’s something you can discuss and negotiate and say, right, well, maybe, you know, we’ll do this, this and this, which will give us, you know, this level of security, and then we will start to develop the other parts and add that over time. It’s about for me, it’s about continuous improvement if you try and if you try and go for 100%, security upfront, you’re never going to deliver anything to your customers. So yeah, it’s about continuous improvement and working with security to gradually get more and more and more secure.
Jason Baum 23:20
What are the little things we can do because, you know, celebrating the Safer Internet day and with that general theme of together for a better internet, and things that we can all do personally to help secure ourselves or family make the internet just in general safer? What are the little things that we can do?
Jamal Walsh 23:41
So my number absolute number one tip is to get a password manager. So I use LastPass other password managers are available. But yeah, I think I think, you know, we spoke before about credential stuffing. So for those who don’t know what credential stuffing is, it’s where malicious actors go on the dark web, get a list of credentials that have been taken from a hacked website. So if your credentials are in that list, your email and your password, they use credential stuffing, then and what they’ll do is they’ll take that email address and that password, and they will go off to hundreds and hundreds and 1000s of websites and they will try and access. Those are the sites Facebook, you know all the social sites with those credentials that they’ve taken from another system. And if they get access to that, that then got access to you know, and if you’re using the same email and password on all those different sites, you’re going to be in real trouble pretty quickly. So having Password Manager and making sure all your passwords are unique in each different system and password managers make that really easy. Some of them will even go in and automatically change your passwords every month for you. So yeah, that would be my number one thing, And then the second one is, anywhere you’re doing any kind of financial transactions or anything to do with money because this is where, you know, bad actors are really focusing on that’s where they want to get your hard-earned cash. I would definitely have MFA, so multi-factor authentication, any kind of banking or anything like that anyway, anywhere where you’re spending if MFA is available, and switch it on, get the app on your phone, and use the apps don’t use text MFA, because text MFA sends a coding clear text. So always try and use an authenticator app on your phone.
Jason Baum 25:36
I mean, are we at the point where biometrics needs to be part of this? And eventually, you know, some of the most secure sites that I’ve been on at least, the government sites use biometrics, but at this point?
Jamal Walsh 25:49
Yeah, I think I think there’s a big push to I think, I read a statistic that MFA, I mean, MFA can use different types of biometrics is a way of having multi-factor authentication, there are different, there are different levels of multi-factor authentication, biometrics being one of them. But you know, just the fact that, you know, when you look at the stats around MFA, and how many, you know, the percentage of malicious access to a person’s account is stopped. I think it’s somewhat like 99% of, you know, accounts that have MFA or not or not. You it’s very difficult to get an account that has MFA on basically,
Jason Baum 26:35
yeah, yeah. That’s, that’s great. So back in 2009, I was part of an organization association that had frank a big nail speak. Frank Abagnale is the Catch Me If You Can the main character, he now works for the FBI. But he’s known for forging checks and money and, but he’s also I mean, he’s also he was a kind of a hacker as well. And, and obviously, the FBI has, like, recruited him, but we also had Colin Powell. And, and the two of them were speaking so frank McNeil, during his presentation, did one of the most amazing things I’ve ever seen anybody do during a presentation, and he’s like, I’m gonna hack Colin Powell. He’s like, I’m gonna steal his identity is a Colin Powell is pretty, pretty important, dude. Right? I mean, he’s, he’s pretty high up there in the government, you think he’s super-secure everything secure, right. And he’s like, I’m going to hack him. And not only I’m going to hack him, I’m going to steal basically his identity in 15 minutes. And he’s like, all I need is his address, his birthday, and, and his password. And he got everything. And he got it in 15 minutes. And so he stole compounds identity now, I would hope that things are a little more secure now, in 2022, than they were in 2009.
Jamal Walsh 27:57
Wow. So I think what you’re talking about, there’s something called social engineering. And I think it’s, it’s a really big thing at the moment. So it comes down to privacy, and people not realizing what privacy means. And when you go on to the social media sites, the more information you publish about yourself, the easier it is for people to then take advantage of that. So it’s really important that you think about what you’re posting online, what you’re sharing online, who you’re accepting as friends online, like just accepting random people, and then not realizing that by accepting that request, you are then exposing a ton of data that can be used against you, right? So you always have to think about, you know, how much am I sharing here? Do I know this person? You know, it’s kind of like, it’s kind of like, you know, going out in the street, would you hand a complete stranger your well, it was all your ID cards,
Jason Baum 28:57
right? I was just gonna use the example of like, when you post on the internet, like in a social media setting, and if it’s a public post, and you’re basically saying you’re on vacation, it’s like, would you just stand out with a megaphone and announce to the whole neighborhood? I am going on vacation now. At my house is empty. No one’s home? No. But we don’t think of it that way.
Jamal Walsh 29:18
No, we should shoot. Absolutely. Yeah. Yeah.
Jason Baum 29:22
Thank you so much, Jamal. I mean, we could talk about this is a really interesting topic. It’s a very timely topic. And I hope you know, now through this podcast and all the different means that you have out there that we can all learn to be safer together for a better internet, not just for our data, but for our families and ourselves. Yeah, absolutely. Thanks again for coming on. I’m going to ask our last questions. So you were on Last time we asked. We asked a question very much focused on you And what was something that you would like to share that no one else knows. So if you’re curious about what Jamal’s answer was to that question, you have to go back and listen to episode 36. I’m not gonna, I’m not gonna repeat it here. So there you go. But our question today for you is, what’s something everybody in your industry should stop or start doing? Immediately?
Jamal Walsh 30:21
It’s a really good question. So I’ve ever seen that kind of, say pretty much anywhere and everywhere I’ve worked, which is to stop starting and start finishing. And that basically means stop trying to do too many things at once, break your work down into small, achievable pieces, and start delivering stuff. I think, I think sometimes people have a concept that by doing many things at once, they’re delivering a lot, when actually you’re not. If you just deliver one thing at a time, when you deliver that one thing, move on to the next, you’re actually getting value delivered much quicker. And it’s a concept that you know, should be practicing in DevOps and all DevOps teams is the fact that, you know, if you try and work on 10 things at the same time, there are only so many things you can you know, it takes longer for those 10 things to finish, right. Whereas if you just work on the one thing, finish that, and then start in the next thing. Easier said than done. I’ve got to admit, I do have trouble with it myself sometimes. But yeah, I just keep reminding myself every day.
Jason Baum 31:28
I think getting to the finish line. Don’t we all need that feeling of like you did it? You know, it’s it keeps you going and motivated to do the next thing. Exactly, exactly. Well, thanks so much, Jamal. I really appreciate it. You’re always welcome to come back.
Jamal Walsh 31:43
Anytime I love. I love having a chat with you. It’s great. Awesome.
Jason Baum 31:47
And thank you for listening to this episode of the humans of DevOps Podcast. I’m going to end this episode the same way I always do encourage you to become a member of DevOps Institute to get access to even more great resources just like this one. Until next time, stay safe, stay healthy, and most of all, stay human, live long and prosper.
Thanks for listening to this episode of the humans of DevOps podcast. Don’t forget to join our global community to get access to even more great resources like this. Until next time, remember, you are part of something bigger than yourself. You belong