Join Eveline Oehrlich and Max Howell, CEO of tea.xyz and creator of Homebrew, to discuss open source including “the Nebraska problem,” challenges, and more.

Max Howell is the CEO of tea.xyz and creator of Homebrew. Brew was one of the largest open source projects of all time, with tens of millions of users to this day. He has set out to make tea: a new, better package manager than brew that utilizes web3 technology to remunerate value back to open source devs.

The Humans of DevOps Podcast is incredibly grateful to be voted one of the Best 25 DevOps Podcasts by Feedspot.

Learn more about getting DevOps certified at devopsinstitute.com

Transcript

Narrator 00:02
You’re listening to the humans of DevOps podcast, a podcast focused on advancing the humans of DevOps through skills, knowledge, ideas, and learning, or the skil framework.

Max Howell 00:17
We see this all over the place with open source. I think a lot of the time the people who make successful open source, making a hobby project, some kind, something they need, and then it becomes an essential piece of internet infrastructure.

Eveline Oehrlich 00:33
In December of 2004, a group of IIT students from DAP University in Copenhagen has launched a open source beer, the one that a version was described by those students as a great tasting, energetic beer, and its world first open source beer. It’s based on a classic ale brewing tradition. But with added Guarana for a natural energy boost. The aim was to see what happens when an open source structure is applied to a universally known product like beer. Hello there. This is evolutionarily ish. We are not here to talk about beer. But we’re here to talk about open source. And I brought this introduction because I think it fits in multiple ways you will find out yourself. But I’m a beer Brewer by Hobby so that caught my attention. Our podcast title today is open source, Abreu and t. Now a few more details, which is very different 97% of commercial code contains open source and I actually learned this from our guest. When looking at some of the tech trends such as containerization, like Docker and Kubernetes, originally developed at Google and released as open source in 2014. Big data like Apache Spark with Hadoop or Kafka, a lot of open source is out there. Again, as I said, Welcome to humans of DevOps podcast, I’m Avalon early Chief Research Officer at DevOps Institute. Today, we have a very special guest, and I’m excited to have him with us for multiple reasons. Max Howell, CEO of T dot x, y, z. Hello, Max. Welcome to our podcast.

Max Howell 02:33
Thank you. I’m pleased to be here.

Eveline Oehrlich 02:35
Yes, excited for you to be with us. Tell us the first thing really I’m curious about is your journey into an across open source.

Max Howell 02:48
Yeah, well, it’s a fairly good story. So I did a chemistry degree because I thought I wanted to be a scientist. And in the process of doing that degree, and then doing a year in the industry, I discovered that actually, it was really not for me, I was working with this device that measured surface tension for like various chemical solutions. And I realized that if I kept working there, like in 10 years, I would still be using that single machine and making very minor contributions to the world. So I fell into this sort of depressing funk. And I discovered open source. Programming was always something that I had considered a hobby. My dad taught me when I was six, which, you know, very young, but I only used it to like make video games essentially, like most kids do when they learned to program. And I never really considered it for a career until my career was on the rocks. And I didn’t know what I was doing myself. So propelled into open source by installing Linux. And I loved the community, I loved how everyone was super passionate about what they were doing, and just doing it because they wanted to do something that changed the world or improved their workflow or just just making cool things. And I ended up working on a few apps with people from all over the world and it was just like nothing else. For me. Honestly, it’s, it’s difficult to find it as good as it was when I first got into it. And we were all on IRC together. And doing that led me to getting a job in the industry, which you know, it’s it’s kind of luck in many ways. But there was a company in London that used one of the apps I was working on and invited me to go and interview and it wasn’t really as much of an interview as just having a chat. So I actually got myself into the industry. And a few years later after that, you know, I kept doing open source and so I created homebrew, which is one of the biggest open source projects of all time. At this point, and I’m sure you’ll have some more questions about that.

Eveline Oehrlich 05:05
Yep. Yep, keep going, because I will come, we’ll come back to that.

Max Howell 05:10
So I created homebrew because at the time, we were making all these different apps. I was at last firm in London, and we made six different apps. One was for Linux and Mac and Windows, we have an iPhone app, and an Android app, and even this Blackberry app. And we we build them all on Mac, because Mac was like the unified platform, time, and what it really has become the platform for development now. But this was 2008. And at that time, like, developers hadn’t really decided what platform they’re using, it was all over the place, it was still still a lot of people using Windows for development, which is not as common nowadays. And in the our office was an awful lot of people on Linux, and I was one of the few who converted to Mac. Because Apple had a bad rep. With developers. It’s hard to believe now. But well, it’s easier to believe over the last few years. But there was certainly a period where everyone in development used to back. But that was new. And we were using it because it was the platform that you could do Android Dev, you could do Linux Dev, you could have a Windows VM running. So we could do all six platforms quite easily. And like the package management solutions were the acceptable, they weren’t great. And they certainly weren’t designed for developers, their impact managers turned out with Linux. And in many ways, they were the thing which defined the Linux flavor. And it’s still still the case. But I kind of felt that they were designed for sis ops and DevOps, but not development. So I wanted to build something that was more for that. And so I got on with it. And after a few months, I realized that it was kind of neat. So I open sourced it. And then it took a few months before anyone noticed it. But then when he got noticed, it took off, like amazingly quickly, huge amounts of contribution, but a lot of excitement, because I’ve managed to tap into something which people wanted, needed, especially with this burgeoning developer platform, which was what the Mac was becoming. And yeah, so here we are, like, almost 14 years later. And yeah, it’s it’s an enormous project. At this point. It’s very hard to meet people that haven’t heard of it or used it.

Eveline Oehrlich 07:40
You know, I have watched your speak at Web Summit in 2022. And you talked about something there called the Nebraska problem. And And honestly, I’ve been in infrastructure operations for a long time, but I’m not a developer. So maybe that’s why I’ve never really heard. I’ve understood, I’ve heard of it, but I really don’t understand or didn’t understand what it actually meant. So can you give us a quick explanation of what is meant by the Nebraska problem? And how it relates to the open source? And if possible, any examples you have seen recently? Maybe?

Max Howell 08:25
Yeah, sure. So what we call the Nebraska problem is the open source funding problem. And we call it that because there’s this famous XKCD comic, you’ll see it trotted out whenever there’s any discussion about open source funding issues, and represent it shows like this tower of blocks, and it represents all modern digital infrastructure. And then near the bottom of the towers is precarious, little pillar, and it’s holding the whole thing up. And there’s an arrow pointing to it, which says, this open source package maintained by some someone from Nebraska, since 2003, thanklessly. Are a suddenly experienced this with homebrew, like, it was kind of the first time I’d really experienced it, because even though I’d done some relatively successful open source before that, well, you know, I did work full time on a few projects before that, but I was I didn’t have a job and didn’t know what I was going to do with myself. I was still living with my parents. So it was to do it. With homebrew, I had to have two full time jobs. One was paid and one which didn’t. And I took some time off here and there so I could work on homebrew full time. But I never had enough money saved up to work on it for a long period of time, but I couldn’t abandon it because it was well it was my easily at the time like my greatest creation. I was extremely proud of it but also people depended on it. And I Couldn’t let them down, I just didn’t see how I could. So I work at, you know, my office job, and often was doing some homebrew stuff to be honest. And then in the evenings and the weekends, yeah, I didn’t really have a social life and just worked on homebrew. And that was like, wanted or needed, but it wasn’t sustainable. And in the end, I did burn out on it. And I haven’t really worked on homebrew since 2016. I passed it to the community, which, you know, I was, I’m lucky that I could, that suddenly other people turned up. And this is how open source is and how it should work. Of course, community is essential to open source. And so it was good that I could hand it over, but I can personally keep working on it. And we see this all over the place with open source, I think a lot of the time the people who make successful open source are just making a hobby project for some kind, something they need. And then it becomes an essential piece of internet infrastructure. And they’re stuck in a situation where they either like, abandon it. And then that’s not what they want to do. They can’t afford to work on it as much as they need to. While they try to find funding, and let’s face it most most funding doesn’t doesn’t work very well for open source. A lot of the time then Nebraska projects is what we call them, like famous one couple of years ago was locked for J which is a logging piece of logging software for Java applications. And there was a massive exploit found in it where you could root servers just by typing stuff in if the stuff you talked to him would go through the logging software in some capacity. And no one had really heard of this package, because it was deep in the stack. And the sounds themselves often works, right. It’s like these essential pieces of software have been built. But over time, they’ve just been buried on the things that are built on top of them. And you got really mature pieces of software, which still need to be maintained and everyone’s forgotten exist. A lot of for Jay was a great example of that. And they fixed the bug. And they asked that maybe they could get some funding in future so that they could afford to spend more time actually maintaining the software and maybe making sure it doesn’t have these security holes. And I don’t think they ever received anything, because once the bug was fixed, everyone forgot about them again. And then more recently, there was core J S, nine D in downloads in is since it was released, the core of the base of every node app that exists, every you know, everything was built on Node or uses Node, and an essential piece of infrastructure really, but the person who maintains it is giving up because they’re just fed up with the fact that they don’t get any funding. Everyone uses this thing. Everyone is extremely entitled about how they treat open source. I’m afraid to say it’s often true. And he’s just like, well, I can’t afford it I’m giving up. And what’s going to happen, I don’t know, it’s not a good thing to use at the infrastructure of and, you know, I, I’ve done loads of open source. And the truth is, I really should just been doing open source all these years like I created one homebrew with the small amount of free time I had. People I myself shouldn’t have to make a choice between working on software that improves the world and is beneficial to like all layers of the software stack. Or some of the some of the jobs I did were not particularly beneficial to the world, really. So that’s the Nebraska problem.

Narrator 13:48
Skill up it learning is the perfect online destination to learn about DevOps and digital transformation, anytime, anywhere. Our digital learning platform provides you with all the resources you need to upskill and learn about these important topics, including expert led course videos, access to certification, prep courses, and a community of supportive peers. Our subscription based model makes it easy for you to gain the skills needed for success in the modern workplace. Visit DevOps institute.com now and explore our different learning plans.

Eveline Oehrlich 14:21
Wow, wow, quite quite involved. quite complicated. And when you started out just said, you know, this noble cause of contributing to the world really with what you have done with homebrew. We’ll get to T XYZ in a minute. But I wanted to ask a couple other questions. Because of the topic of open source, there’s still some people seem to not understand that perspective. So that’s why I was very, very interested in your thoughts and your journey there. Let’s switch a little bit to the benefits in terms of of leveraging open source. You know, take it from either the benefits into from a developer perspective collaborating and into the open source community, or for the benefits of the community over in its large meaning the world right. Talk a little bit to us about what you see the benefits are of open for open, not open force, but open source.

Max Howell 15:26
Yeah, open source is really interesting. I don’t think that human history is really got any other equivalent examples. Or markets usually up here, because there’s an economic incentive there. Like I don’t know if anything else like this. But I’ve been thinking about this quite a lot recently. And I remember how in the 90s, the entire software stack was basically owned by Microsoft. And, you know, there was numerous issues with this. But certainly, as the Internet became a thing, it became more and more clear that Microsoft was holding the Internet back, because Internet Explorer was the only browser that anyone used, and they didn’t want to advance it. They didn’t care, too, there are no real incentive. And it was this kind of attitude, which led to open source taking more and more of a hold. In the end, like now, it’s difficult to imagine a world where open source isn’t the base of every stack, like we say, 97% of all commercial software has open source involved in it in some capacity, and like, this is a really free percent that doesn’t, I’d like to see what those source bases and understand how they get away with not having any open source at all. It Over time we, we built the open source app, and we swapped off the Microsoft stack. And it became like the point where developers realized that they not only gained like all this functionality for free, but also they were getting these step ups in productivity. And proficiency is open source is usually designed to be like these tiny little self contained pieces that you slop together, and you build on top of each other. And then you’ve got all this slight advanced functionality. But in the process of replacing Microsoft with open source, we never transferred all that money that makes over making while they were trillions of dollars in. And instead, we just sort of ended up with Freebase without real any maintenance on it, which, you know, we’re trying to solve AT T realized the the value of open sources. People throw out ideas and the good ones stick, there’s no marketing behind it. There’s no like someone’s trying to sell you something that actually they know isn’t a good fit for you. You put some open source out there. And if it works for the developers of this world, they adopt it? Well, this is sort of people’s world, they adopt it. And then over time, if it continues to fit, it builds until it becomes like a mature block in the stack.

Eveline Oehrlich 18:17
So with that there are challenges as well. What are some challenges you would say? Again, from multiple perspectives, take it from the developer, take it from the community or take it from the software world of vendors out there?

Max Howell 18:39
Well, I think vendors have the highest risk right now, right? But if you’re building an app, and it has 6000 dependencies, either, there’s no way you can vet that. And be sure that all those dependencies are secure. First off, also without any malicious insertions of any kind. It’s one thing we’re going to try and fix with what we’re doing at T. But I do wonder how, you know, if I had to report to someone and say that our app is secure, we we can guarantee that I don’t know how I’d say that with a straight face 6000 depths in there. So there’s a lot of risk there for vendors. I think developers, most of them just enjoy open source existing because it really does make computing more enjoyable in general, especially now. Like 14 years ago when I made homebrew, like the amount of people doing open sources so much smaller. And now you can’t go a day without dozens of new things being released continuously. And some of those are great. But in most developers feel bad that they just consume this open source and don’t give back but I don’t see how they can effectively give back right because you can try and spot through a few open source steps here and there, but we’re all using 1000s and 1000s of open source projects. And it’s just infeasible without some layer of automation to contribute to those properly. But open source developers are the ones that I feel for but you know, most of them are just do it, they start off by doing side projects, it really is, it’s like, it’s one of the beautiful things about software engineering is that it’s so cheap and easy. All you need is time, that if you’re gonna lay there, and you got some time, you can, you can create some open source. But what happens with all the open source devs, I’ve talked to who are, have got successful projects is that after a while, it’s a huge amount of work. And you just can’t abandon that. You have to somehow find the time to do it. People on myself, we’ve, the way I approached, it was always to try and make it so that as much as possible, it was robust. And then but if you did break, it would tell the user what they can do to try and fix it themselves. And they’ll make sure the documentation was as good as I can get it. And that would reduce the burden on me as much as I could, as much as possible. And then, you know, if you if you’re lucky, a community turns up, mostly open source, I’ve done communities that turn up, homebrew was

Eveline Oehrlich 21:24
really quite an exception. Hmm, fascinating. Absolutely fascinating. So again, at the Web Summit, you said, I love this vision you painted, you said, you want to support those who are passionate on open source development, but make sure that they get rewarded. And you actually call it I love this to digital revolution for developers. What do you mean by that? And then we’ll get to T XYZ, because I think there is a relationship. So tell us about the digital revolution for the developers?

Max Howell 21:58
Well, let’s face it, in the sense of web two, as we call it, which was, you know, mid 2000s, till, kind of recently, I’d say, a huge amount of money was made by people building on top of open source. And it was gradual, you know, like the way open source became so essential to what we’re building and software was gradual. And I think a lot of these essential pieces, if they were built by hackers who were building them for other hackers, and then when web two turned up and figured out how to monetize all that, there wasn’t any transition of wealth. And so that’s why it’s a digital revolution. These huge, mega billion dollar companies that have never really given any substantial amount to the open source that they depend on. It’s not right, like we were okay with it as it happened, because it was gradual. But now we like hanging on. You know, why am I working two full time jobs when Microsoft has billions and billions of dollars coming in every month? Doesn’t make sense. So what we’re trying to build a T is a way to successfully remunerate how open source is funded. There’s been lots of attempts, sponsorship, bounties, things like this, but we see that as being reward favorites. And some people make a lot of money with sponsorship. But the truth is, there’s millions of open source packages. And most of them are Nebraska projects that nobody knows about. React doesn’t need to be funded, because Facebook, pay for it essentially, which is another thing we want to fix instantly. I don’t believe that these huge corporations should have so much agenda over how these open source projects that the community needs to figure out what makes sense for open source. Facebook could just like close down react tomorrow. I’d want with what we’re building a tee is for these developers at Facebook and Microsoft who do work on open source or who are just extremely talented people to be able to quit and work on open source full time because the pay is exactly the same or better. Even, frankly, we’re building an economy on code. And if you make a bunch of successful open source packages, providing you’re using the T protocol, you could be much wealthier than you are currently as a software developer.

Eveline Oehrlich 24:35
Interesting. So how can developers get involved if I am sure we have a few of those who are going ooh, I love this. I love hearing this. How can they get involved?

Max Howell 24:49
Well, four months ago I released the tea package manager so tea is the successor to brew you know I’ve stopped working on brew in mid to 2010s. And I didn’t stop thinking about what could be better about broom. But I didn’t really have a good incentive I feel to make another brutal I’d already done it. And I remember the tireless hours that I spent working on it robust, defying it, filling out the package graph, answering people’s support questions, and just like trying to make sure it was a successful and useful piece of software. But I didn’t really want to go and do that again. I think I like doing things once usually, but But 18 months ago, while I was looking into web three, and crypto, which basically was the first time I’ve looked into this stuff, because I’ve never been particularly interested in it. I thought Bitcoin was pretty impressive. As a wonderful mystery story to it also pretty ingenious, how it works, and how it has successfully become so large, even though it’s not controlled. But I never really thought that there was much more that was interesting there for me. But as I was diving into web three stuff, I started to see the different contracts were actually pretty interesting, I had this moment of inspiration while messing around an open sea with NF Ts. Because NF T’s allowed you to well view the open sea digital contract anyway forced any repurchases to put 10% back to the original creator. I was like, Oh, so you can write a digital contract that funnels money automatically to different entities. And I realized suddenly that the package graph of all open source was that kind of, if you put a digital contract on it anyway, you could funnel money to all the dependencies, all the packages. So all these Nebraska projects could get like little bits of token from any insertion of token near the top. And that’s how it works, right? Like people sponsor these big projects of near the top. And it’s all the ones underneath that that are failing, have are maintained by people without any things. And they’re essential pieces of the digital infrastructure. Without them, everything would collapse. I like everything being that fragile is scary, like people remember that for a long time? How it could all collapse any moment, but somehow it doesn’t. And we’re just waiting for a bigger claps really. But also, like, think about what would happen if it was all funded correctly. People myself, like make, like occasionally big and important projects. But we have to do it in our spare time. What if I was working on that full time? And then, you know, we find security holes and open SSL every two months? Well, if it was funded correctly, so that well, either of us so could really afford to hire people who could, you know, like kind of like a corporation is what we’re thinking things light, each of these big projects will have their own dowels on the T protocol. And they can operate them. Similar to a kind of corporation, they can hire people using the token, they can split the rewards that they’re getting from the tee protocol across everyone is contributing to open SSL. So if you open a pull request, you could end up getting paid for that pull request. Because the DAO is a set of smart contracts that figure out how to channel the token based on the governance structure that those projects have chosen. CL I came up with the idea, like build a package manager and build it on top of blockchain. So that open source can be remunerated, then we managed to raise 80 million, and we’ve been building it since. So I released the package manager itself four months ago, got six sales and styles now got some really passionate users. I think it’s special. And if anyone is interested in getting involved with what we’re doing, that’s the that’s the place to start. Currently, what we got is github.com/t. XYZ and you can check it out, install it, and I think it’s a better package manager the brew it’s, it’s more than a package manager, really, the ideas I had over the years for how the package manager could be improved. I realized that it’s a unique piece of software, right? It sits underneath all other tooling. There’s so many things that you can do, which nobody’s really tried before. I don’t really get it I think perhaps package managers just aren’t very sexy. Is the truth and don’t many people that are interested in working on them but for some reason I’ve always enjoyed it. I love getting into that sort of area of it.

Eveline Oehrlich 29:50
It is a brilliant idea. You are brilliant. I am honored to talk to you. If I come up with a name for this that I will share it with You but it is. It’s certainly a super cool and absolutely honorable, really honorable thing you’re doing. I love the idea. I’m, I’m very grateful. Now I have one closing question. Hmm, maybe it involves beer drinking, but what do you do for fun? What do you do for fun if you don’t do these brilliant things and manage these fantastic things? You are You were you were talking about?

Max Howell 30:30
Yeah, well, so like, programming was always a hobbies. Switch when I switch to doing it for career did become kind of what I did one as well. But certainly, you know, I used to live in the UK. I live in the States now. And I used to love going to pubs friends. So there was some beer drinking in my palace for sure. But I got a son now he’s 10 months. And so there’s certainly less beer in my life at the moment. Otherwise, for fun, so Blimey, I do like video games here and there on Sunday not playing them as much as I used to. And I like hiking or going out or camping isn’t some of the things I really enjoy.

Eveline Oehrlich 31:15
Excellent. Well, if you make it to my part of the world, I’m happy to take you out for beer. We got some pretty good beer in Germany. So yes, yes. Well, this has been fantastic. Thank you so much Max. We have been talking to max Howell CEO of T dot XYZ and brilliant thought leader in the topic of open source Mexican, thank you so much for joining me today on humans of DevOps podcast. Thank you very much. Humans of DevOps podcast is produced by DevOps Institute. Our audio production team includes Julia pape, Daniel Newman, Schultz and Brendan Lee, shout out to my colleagues who do a fantastic job. I’m humans of DevOps podcast, Executive Producer, evolutionarily, if you would like to join us on a podcast, please contact us at humans of DevOps podcast at DevOps institute.com. I’m evolutionarily talk to you soon.

Narrator 32:19
Thanks for listening to this episode of the humans of DevOps podcast. Don’t forget to join our global community to get access to even more great resources like this. Until next time, remember, you are part of something bigger than yourself. You belong