How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer

Gergely.

Allright, I'm going to assume most of you—uh, show of hands—who subscribes to The Pragmatic Engineer. Oh my god.

Wow.

Uh, he is, uh, he needs no introduction then. Let's getright into it. Um, what is token maxing, and should everyone here be doing it?

So I, I heard about token maxing a week ago, or like a week and a half ago first, and you know, some people have been doing it for longer. And I tweeted about it, I think, three days ago, saying, "Oh, there's this token maxing," and again, you see it on social media.

And my DMs were blowing up from, from people at large companies. I don't want to name names, but like, you know, Meta, Microsoft, uh, some, some other ones as well, like the likes of IBM, and, and so, so many more.

And the story is a little bit different every—at every company on why people are doing it and whether they like it or whether they think it's good. But there's a few, few common themes. One is token output at these larger companies is measured in, in some way.

There's like either a leaderboard, or there's a way to look up your, your peers. Salesforce, for example, you can check the spend, the, the money spend that every, every person at the company did. You can like search in a tool that someone built, and it shows how many dollars they spent on, on AI-related tokens.

And, you know, first there's this number, then there's this uncertainty on, in the tech industry,right? We're kind of hearing layoffs, like massive cuts at the likes of Block. And I mean, there, like, no matter how much tokens people spend, they were let go independent of this.

But people start to think, like, does—is it part of performance evaluations or promotions or all that? And the answer is, hmm, kind of. So inside of Meta, I talk with managers, and in the performance evaluation, they have this data point, which is one of many data points,right?

The same way as, as like diffs or impact or, or code reviews of how helpful this person is. But they do, just like with any data point, they sometimes pull it and use it. So typically, just like any data point, it can be weaponized.

So like a low performer with low impact and a low token count, clearly not even trying, so. And a high performer with high impact and high token count, clearly that's innovating, and this must be doing good. So inside of these companies specifically, I talk with a lot of people at, at Meta.

And again, this is not representative of 100% of Meta, but they had this leaderboard where people showed up and they had like massive amounts of tokens, and a lot of engineers got just scared, worried. So they started to token max to try to generate tokens.

Stories that I've heard first, or secondhand from these people who, who, who told me firsthand is, for example, instead of reading the documentation, I will ask the agent to summarize it for me and ask questions, even though it doesn't do a good job answering it, but my token count goes up.

People just want to not be in the bottom 25% or bottom 50% for token count where these things are measured. Inside of Microsoft, again, there's a leaderboard, and I'm talking with people, they're like, it's ridiculous, like how some people are just running autonomous agents to build junk, honestly, for the sake of having that number go up.

And, and sometimes it gets ridiculous because like inside of Meta, they had this leaderboard, they got rid of it after an article came out, and it looked amazed stupid, honestly. So whoever built it, like, just, just like closed it down.

People are still token maxing, by the way, because there's this, this thinking that it might have gone, but, you know, we're engineers. And don't forget, these are high-paying jobs,right? Like, you don't really want to lose a job over something stupid as like you didn't have high enough token count, and that's how it feels.

But inside Salesforce, there's a target of minimum spend per month. Like, I think it's like $175 between things. So like, people are like, again, you kind of like, you know, beginning of the month, like just token max to get there.

So it's, it's, it's weird, and it started as a joke earlier. Like a few months ago, token maxing was really just people like going crazy and enjoying this thing and building cool stuff, but it's kind of turned into, in a lot of companies, I, I think it's just a culturally weird thing.

So it's a weird time to be in because I remember lines of code used to be when, when early, uh, developer productivity tools came out, like Velocity and Pluralsight Flow, they kind of measured lines of code and, and number of QPRs.

And we know that was stupid, and people kind of optimized for that at companies that did it. But it's, it's almost like now it's a top-running companies, like Meta, Microsoft, who are incentivizing people to do just stupid stuff, honestly.

Yeah, those are wild stories. And one of the things—you're clapping for that. That deserves another full conversation. Uh, one of the things I liked about talking with you and subscribing to your newsletter is that you basically kind of anonymize all these stories from, from real, uh, incidents and real examples.

Um, why is it that, uh, is, is it still worth it,right? With all the flaws, uh, you know, when you have good hearts law, like whatever, whatever gets measured gets, uh, sort of abused. With all the flaws, is it still worth it?

Uh, you know, is, is, is AI basically still making us faster overall? Like the cost of token maxing is still, with all these like really ridiculous examples, is it still net worth it?

Yeah, so don't forget, like the reason token maxing is probably a thing is, like, let's just go back to six months ago where I, I, I was at a, I was at a CTO like dinner, conference, whatever, like a bunch of CTOs gathered, CTO level people.

This, this was in Amsterdam, and we had like, like a bunch of people. And they were talking, and, and one of the CTOs, like the, the, the Amazon of the Netherlands, uh, there, there's a, an e-commerce company was saying like, "Hey, like, everyone, like, I have a problem.

Like engineers on my team are really skeptical of AI, and they're not really using it, the AI tools." Don't forget, this was before Opus 4.5, and those models were, were out. They were not as, as productive. We had, uh, we already had a cursor and, and the like, and they subscribed.

They're like, "There's not using it that much on existing code bases,"right? And, and next to them, uh, the head of the Dutch national bank said like, "Oh, we don't have that problem. Our engineers are using it because our, our mission is to regulate this thing, so we need to understand it," and they're kind of motivated.

And there was this time where experienced engineers were kind of holding off because if you had an existing code base and used AI, cursor, whatever on it, it was mildly useful, if that even. And these engineers were like, "Why should I use a tool if it doesn't help me refactor, it doesn't find the bug, it doesn't do what I need to do?"

And leadership saw they're not really using it. And they kept hearing, you know, the likes of Anthropic, for example, was already saying how they're writing a lot of their code with, with Claude code, uh, and it just keeps increasing.

And Anthropic's, you know, like revenue is going up like this. So those leaders are kind of, they might be confusing correlation and, and, and, you know, like which one comes first, but they're like, "Well, we should be using it more because probably good things will happen and thus bad things will happen if we don't use it."

So the whole targeting and measuring things, it actually came from leadership wanting, "We want our engineers to use friggin' AI. I don't care what it is." And it, it was a bit of a push, like, "We know this is bad, but it's, it's better than not using it."

The best example is Coinbase, where, uh, Brian Armstrong, the CEO, just like fired an engineer. He sent an email saying, "Everyone like needs to get on board and use AI tools, and whoever doesn't use it in a week, I'll have a conversation with them."

And then I think a week later, Saturday, he fired an engineer. And, you know, like this, again, high-paying job, like we're talking base salary of like 3,400K, thousand dollars per, per year, uh, and then both equity and everything on, on top of it.

Like they got the message, everyone just started to just, you know, like use it. And, you know, back to your question. So on, on one end, there, there's a push. And look, I feel it's a little bit like this is going to be controversial, but have you ever worried, wondered why big tech love to do lead coat style interviews, algorithmical interviews, which have nothing to do with the job?

And, and we know it's the case. And there's a lot of criticism for this, and they've been doing this since, since like 20 years. But here's the thing, it selects for a specific type of person. It selects for the person who's smart and willing to put up with absolute bullshit to get the job.

And this person, you know, they will study two months pre-AI, two months or three months of lead code, which again, makes no sense on the job, but you do it, you get in there, and this person will be putting to, to put up with bullshit that makes absolute no sense to keep the job.

So token maxing happens at large companies, and people are putting up with this BS. And look, a lot of them are smart, and they will make the most of it. Some of them will build cool stuff. Uh, it's, it's the reality, I think, of big tech.

So we're in this weird place where big tech is a bit weirder than startups where, you know, no one cares about token maxing. They care about like just building stuff and, you know, use whatever makes sense. Uh, don't, people will care about the cost.

Yeah.

But, but going back to your question, like, like, you know, like is, is it making us productive as, as a whole? Like individually, it certainly is. And as teams, we're kind of like a bit question mark as we should be moving faster.

And there are a few companies that do. Anthropic is a good example, but a bunch of companies are like not. It's, it's, it seems it's hard to retrofit all this AI into like the way we have been working.

Yeah. Uh, one of my favorite studies from last year was the METER study where they, uh, did a blind test of, uh, people and their expectations of productivity,right? And basically the, the end result was they felt 20% more productive, but their demonstrated results was actually they were 20% less productive on average.

Yes, but that, that study was very interesting because they.

It was very small sample size.

It was 30 people, and there were one outlier, uh, who actually was way more productive.

Quentin Anthony, we, we interviewed him on the pod. Yeah.

Yeah. Yeah. So he was the one productive AI engineer.

But anyway, so, uh, actually my theory is that, uh, something that I've seen on my team is that I've been enabling coding agents for the rest of my team who are non-technical,right? And, uh, you as the engineer may not be more, much, that much more productive because, and, and you can be more productive if you, uh, attend AIE.

But, uh, if you actually enable your non-coding, uh, your, your non-coding co-collaborators to code, actually they are more productive because they don't have to wait for you,right? And, and that's that unlock of like, oh, suddenly you have serverless developers, basically.

Uh, and I think, I think that's, that organizational coding thing is different than studying pull request level productivity for the individual developer.

Yeah. And, and the thing that still, I still remember to this date, I, I talked with Simon Willison, I think in 2024, so two years after ChatGPT came out, and he was Simon Willison, top commenter on Hacker News, or he's, he's.

That's his, that's not his title, man. Top commenter on Hacker News. What the fuck?

No, he's, he's.

Credo Django, top blogger. Yeah. Uh, prompt injections. Uh, yeah.

Yeah. He's actually not top commenter. He's the most submitted blogger because he blogs so much.

Yeah.

Like, like, and he's, but he told me back then, he said like this thing, AI is, is just so hard to, to get good at. He's like, there's no manual. And he's like, I've been doing it back then for two years, and I'm still, I'm still figuring out what works and what doesn't.

I keep changing my workflows. And I think that's something that is a bit hard for us. Two things about AI that for any of us engineers is hard to understand. One is it just takes a long time to get good at it, and you need to keep doing it.

And the second thing is understanding the theory will not make you better at using the tools, which is an absolute mindfuck, honestly, because we're so used to, you know, you understand how the compiler works, how assembly works. Okay, you will now be more efficient if you want to write low-level code because you know how it works.

But with, with these things, I mean, you, you can, of course, it's, it's helpful to understand how, how the, the architecture underlying works, attention, uh, the, the different, the, the different probability sets, et cetera, et cetera, but it will not help you get a sense for how you can use it.

And then once you figure out how you can be more productive, if you're, if you're inside of a team, again, it kind of breaks and you have to relearn again. But, but the more effort you put into it, it, like, it's clearer that it's, it's working, it's helpful.

And I think it, it's, the teams I'm seeing getting more value out of it, low ego, open to learning, open to leaving your priors behind. The word priors, I have not used forever, and I feel we're in the stage where like, just, just leave your priors behind, just have an open mind.

Like, don't leave your experience behind, but, you know, be open to it.

Yeah. Zooming out a little bit, how is the role of the software engineer changing?

I think it's always, this was always coming, but AI is just, just speeding it up. Uh, even before AI, a few, it's interesting how you see like startups in, in many ways, venture funded startups are kind of front running what the industry will be cashing up because venture funded startups are about fast growth, uh, doing things, moving fast with smaller teams because smaller teams mean, mean smaller comps, even pre-AI.

So a lot, a lot of these venture funded startups start to expect a lot wider range of roles from engineers. For example, DevOps as a whole, uh, inside VC funded companies from the mid-2010s, every engineer was kind of like responsible for the code they deployed, but like more traditional companies, they had more money, more, sorry, more, less pressure.

They kind of have dedicated DevOps teams and some of those things. So in, in the industry, like the software engineer is now becoming like the kind of the tester role has collapsed into software engineer. We, most companies don't have dedicated testers.

Very, very few do. DevOps collapsed into here. Uh, and now we're starting to have the product role also starting to come. So a lot of companies, even like in 2022, before AI started to hire for product engineers, that's happening faster.

And I think the, the last push that AI is doing is even for early career engineers, there's a lot more seniority expected or, or senior like things, planning about things, knowing about the business. So I, I, I, I think the role is expectations are, are higher.

Teams are also getting smaller everywhere. I talked with someone at John Deere, 200-person, uh, 200-year-old company, sorry. Uh, you know, like they do tractors and, and all, all, all that stuff. And, and inside of that company, one of their, their VP of engineering was telling me how they're actually seeing that their two pizza teams are now just one pizza team inside of that company.

It's the reality partially because of these tools.

My, my joke used to be, I am a one pizza team because I eat a lot of pizza, but, uh, it depends how much pizza you eat. Uh, there's, uh, so I'm sorry to interrupt. I don't know if I cut you off in some, uh, critical point.

Uh, there's a comment saying, I've heard it twice, even among this audience, where a lot of people are saying that, oh, uh, you're no longer an engineer. Everyone's an engineering manager now, and you've been an engineering manager. And I wonder if you agree with that or if you have a different take, you know, because basically you're, the, the, the common analogy is that you're no longer a software engineer.

You're just managing engineering agents,right?

Yeah. If you've been a manager before, that is an absolute bullshit. So, so here, here's the thing. The, like, yes, you are a manager without all the things that no one wants to become a manager for. The, the, when you become an engineering manager, hands up if you are or have been an engineering manager,right?

Hands up if you actually, if you've not been and you want to be one.

That's about 15, 20%.

Allright. You, you come and talk to me afterwards. I'll, I'll, I'll, I'll tell you there's a hand up there. I'll talk you out of it. So,

so what you think you become an engineering manager to like help people's career, maybe have higher salary, higher impact, all, you know, there can be a lot of dynamics, but the reality is, is, is you, you become more removed from the product and you have to deal with people problems.

And the thing with, with agents is you don't have to deal with people drama, people problems, conflict between your team. I mean, uh, unless the next generation of agents starts to fight with each other, I think that'll be something.

But you, you actually, you, you do have to orchestrate, but it's more like a tech lead role or, or, or, or experienced engineer where, where you're like mentoring, uh, mentoring engineers, but you don't have the people management. You don't need to worry about the personal problems.

So it's actually a lot more kind of empowering. And I was talking with, uh, the podcast was, was just out, uh, yesterday with, with DHH, uh, creator of Ruby on Rails, who said, you know, people tell, told him like, okay, it's, it's like managing things and he's not excited about managing agents, but he feels it's more like a mech suit where you have like, you can do seven things at once, you can do a lot faster and you're in control.

And that's more what it feels like. So there's orchestration, yes, but it's very different to management. And also the, the really, really bad thing or honestly shitty thing about management, if you, if you make it into management, which makes it hard, also rewarding later when you, you tell yourself at least this thing is you start a project with all these people under you, you know, congratulations, you've got 10 people, wonderful.

And you start a project and in six months you will see some results of the decision that you made. With agents, it's just so much faster. So the, the feedback loop is faster. So I, I think it's, it's not much of it except for the orchestration.

And, and, and for that, everyone's going to have their own flavor. Some people will, will have the tendency to like run multiple agents and they're good at this or weak good at it. Some people just do like two agents.

Michelle Hashimoto, I interviewed him, he has two agents. He always has one agent running the, the, no, he has one background agent that he doesn't, that's it. He's like, two is enough for me. Great.

Yeah. Yeah. Uh, we're figuring out the patterns. Um, uh, I want to hit you on large tech infra. Uh, this is something that I think both of us are very excited by, by, uh, good infra, which is a very niche, uh, interest.

What are you seeing?

It's wild to see how much of the, so I, I said that from externally, a lot of companies, a lot of big tech companies, especially the ones that are spending a bunch on AI and have platforms and all that, you're not seeing too much like more come out.

Like Uber is a good example. I'm not seeing too many more features come out of Uber or new products launch. And they're like, but what's going on? They are really investing in AI. But when you look inside, there's a whole lot of buzz.

They are rebuilding their complete infra, you know, they're, and I'm not talking about they're buying Cursor or, or Claude Code or all that. They're doing that as well, but they're completely, they're building their own custom background coding agents that is integrated into their monorepo.

They are, are having, uh, their own MCP gateway that is, is now integrated into service discovery. Their on-call tooling is being retooled. Their internal code review system is like, like categorizing based on risk. They are like, and Uber is one example, but everyone else, Airbnb, Intercom, Meta, Microsoft, even mid-sized companies are just building so much internal infra.

And I was asking myself like, why? It on one end, it feels like such a waste. But when I worked at Uber for four years, I realized they spent so much on, on internal platform. There's two reasons. One is honestly, it's a, it's a low risk way to get good with AI, uh, to be hands-on.

And these companies want to be hands-on, but maybe you shouldn't start with shipping AI features no one wants into your code base. Second of all, because these, these companies have such, so much code that never fit in a context window, by building custom solutions and just basic, basic raggens, that kind of stuff, they will have better results than off-the-shelf vendors.

So they already have a win. And number three, honestly, is anything that has AI in it gets funded. So there's this joke of if you're in the developer platform team and you're asking for more headcount, like, good luck with that.

Oh, developer platform. Oh, but say that you want to get two extra headcount for agent experience. Done. Uh, so, so there's that part as well. But, but all.

And agent experience is just a CLI.

Yeah. Pretty much. But all these companies inside, there's so much buzz and so much work. Everyone's building their own custom system. So I'm kind of wondering how long this will take, but I think for next year, this is going to happen.

So if you either have friends or if you're work, if you're working at a company, you'll see, but talk with, with friends at other large companies and you will probably see you are all building the same thing. If you're at a large company and you're not already building an MCP gateway, what are you even doing?

Yeah. Um, actually a lot of these topics are exactly the things I curated for tomorrow. Uh, it's just, uh, fantastic to have you as the closing keynote for today because, uh, it's, it's like an appetizer for tomorrow. We have talks about MCP gateway and, uh, all these sort of AI architecture and infra things.

And I do think like, uh, infra, like taking AI infra seriously as a company is, uh, very misunderstood, not, not that well understood. Andright now you just kind of learn by example from people because there's not really like a textbook or anything like about it.

So the way I think about this, because again, from if you just kind of step out and we love to criticize big tech of how they're wasting money here and there. And by the way, we love to criticize Google.

And I'm kind of thinking to myself like, hang on, what if Google actually executed well? Like, like do we want that? And you know, they would kill all the startups, but, but what they're doing makes, makes sense. And Shopify is an example where I'm like, huh, I'm starting to get why it makes sense to do all this stuff.

So Shopify in 2021, they were the first company to have access to, uh, GitHub Copilot. What happened is the, the head of engineering, Farhan Tawar, heard about GitHub Copilot being developed internally inside of GitHub. And he pinged Thomas Dymke, the CEO of GitHub at the time, and said, "Hey, Thomas, I heard you guys are doing Copilot."

And he's like, "Yeah, we are. It's internal." He's like, "I, I'd like to get access to it." He's like, "Yeah, but it's not for sale." He's like, "No, no, no, you don't understand. I, I didn't ask if it's for sale.

We would like to roll it out to all of Shopify and in return, we will give you feedback for 3,000 people for, you know, as honest feedback all the time." And so they got it a year before it was out anywhere.

And they incurred a lot of churn. It wasn't that great initially. And, and they went through all of this stuff. And then Shopify was the first company to onboard to like a bunch of other tools and they gave unlimited budget.

And they're spending so much time ironing out bugs. But the reason they're doing it, this is what like made me click, is they are trading off churn and expense and spending a lot more money to be at the forefront of this.

They are a few months ahead or six months ahead of their competition. And for them, it's worth it. It's not worth it for anyone else,right? If you're, if you're at a company where your business is like something, something physical and you don't care, like, yeah, just, just wait out.

It, it, it'll come. But for a lot of us in the tech industry, this churn is worth it. Plus what Farhan told me is like, 'cause he actually told me he's kind of worried about the cost now, but he was like, look, like it's still worth it because if I, it would look silly if I said you cannot have these tools, how would I hire the best?

So it's, it's innovation, recruitment, and it kind of makes sense when you think about it. And the weird thing, everyone's doing it at the same time. So it looks silly, but it, it's rational.

Uh, my next podcast is with Mikael Parekk and the CTO of Shopify and, uh, the sheer amount of machine learning that they do and infra that they set up for their customers makes me want to be a customer.

You know, that's, that's, that's like the best, uh, endorsement I can give. Um, I'm going to get meta a little bit and talk about Pragmatic Engineer. Uh, you and I kind of started ish in COVID. Uh, you just left Uber.

Uh, how has it been growing? What, what are the main stats that you're proud of that, uh, you'd like to share with the world?

Yeah. So I, I started Pragmatic Engineer. I, I, I'm a joke that if it wasn't for COVID, I, I would probably never have started the, this thing because what happened with COVID is, uh, Uber had layoffs and most of the tech industry was doing great, but Uber was not.

And my team, uh, was hit by layoffs and then we, we had to disperse the remaining people at other teams because our mission no longer made sense. And it was just like a, the morale was low, my morale was low.

So I was like, let me take a break. I wanted to write some books. Swix was writing his book, The Coding Career.

Yeah. Some of you have read it. I've met some of you.

Yeah. And, and that, that's how we met there. And then, uh, my plan was to write a book and then start, start up some startup, something, something platform engineer, control C, control V from what Uber was doing inside.

And that's actually almost all Uber success, Uber startups. It's amazing. Temporal is, is, is from there. Atmosphere.

If I, by the way, if I did not start AI engineer, I would have started platform engineer. That, that would have been the industry conference. Yeah.

Love it. Uh, and then I started, I started the Pragmatic Engineer, uh, a year after I left Uber. Uh, it was just an experiment. Um, I figured no one, Substack was taking off, no one was writing about software engineering in depth.

And I just acted all confident, saying, pretended that I knew what I was doing. The first article was about Uber's platform and program split that no one had written about publicly before. And it's a, it's a free article.

You can, you can now check it out. Uh, and it was like when you feel product-market fit, that's what I felt almost immediately. The first week before I published anything, just a confident Twitter post, I had a hundred people pay upfront a hundred dollars for the whole year.

I was like, whoa, I haven't published anything. In six weeks, I was at a thousand people paying for this thing that didn't exist before, which was my old Uber base salary back, back in Amsterdam. And it just kept going up.

So like I, I figured like when you find product-market fit, this is like outside of, like there's this rule, like if you find product-market fit, just keep doing what you're doing. So for me, I just kept writing that one article.

I got all these interview requests, collaborations, podcasts. I just said no to all of them 'cause I knew the most important thing was to do what makes it successful, which is that one article. And later it turned into two articles.

And for two years, this is all I did, just two articles. And after two years, I looked up and I was like, huh, like this is actually working. People like doing it. I like doing it. There's a future in that.

And that's when I decided I actually want to turn this into a business that I don't burn out because for two years, every vacation I went to, I was working 50, 60 hours. I was always thinking, I was writing, I, I couldn't really let go.

So I started to grow the team a little bit. Uh, uh, I, I, I Alan Bird, the first tech industry researcher, Ellen, she's a ex, ex, uh.

She's here,right?

Uh, Ellen's not here. Um, Jessica is, who, who just joined, uh, later.

Yeah.

And then, uh, so now it was two of us. Uh, and I started a podcast a year and a half ago because I talked with so many people, I figured it was a bit of a shame to, to, to not have it.

So the Pragmatic Engineer became the number one paid technology newsletter about four months after starting. It stayed there for three years. Now semi-analysis has.

Dylan versus, uh, you guys. Um, yeah, no, congrats on your success. Uh, I think you're also a leading tech voice in Europe, which I think you're sort of proudly sort of, uh, upholding that over here, which I would really wanted to feature.

Thank you for your support for AIE and, uh, everyone, thank you to.

Awesome.

Thanks, man.