Copilots Everywhere: Thomas Dohmke and Eugene Yan

I'm delighted and honored to welcome a very special guest, Thomas Dohmke. Thomas has been fascinated by software development since his childhood in Germany, and he's built a career building tools devs love and accelerating innovations that are changing software development.

Currently, Thomas is CEO at GitHub, where he has overseen the launch of the first at-scale AI developer tool, GitHub Copilot. So please join me in welcoming to the stage Thomas Dohmke.

Thank you, Thomas.

Yeah.

Oof, it's bright out here.

Yeah. Well, thank you, everyone. Thank you, Thomas. Let's start with Copilot. Many people have shared their own takes on the Copilot origin story. But what was your personal experience seeing it in GitHub? I don't know, you had a sneak preview.

Take us back to the start in 2020.

So imagine it's 2020. It's lockdown here in San Francisco and Seattle, everywhere where GitHub engineers are sitting. So like all of you, probably we're on a Zoom call. One of us had early access to a new model that OpenAI had just released in preview, a version of GPT-3 called Codex.

And one had the Ugur, I think, had the keyboard, the leader of GitHub Next at the time. And we were dictating prompts and asked the model to write some code. And I think the first aha moment that I had is that you could ask it to write JavaScript code and put the curly braces in theright places and whatnot.

And you could ask it to write Python code. And the model, in a way, it doesn't work like a compiler. It doesn't have a syntax tree. It doesn't know these things. Or you could also argue it knows them exactly like we know it.

So that was probably the first moment. We kept building, we kept exploring the model, and then decided we built this auto-completion copilot that was the first copilot. And we built it all while being remote, while being on lockdown.

Wow.

So if your investors tell you today you need to be in a room and on the front of a whiteboard, you can innovate if you want to while being in your home offices around the world. I think the next moment was that we shipped a preview to our internal engineers.

We call it a staff ship at GitHub. And the NPS survey with those engineers was through the roof, I think 72, 73, something like that.

Wow.

And typical of our early-stage products, especially with a large language model and all the hallucinations and the UI wasn't really figured out yet, is much lower. So that was kind of like a holy shit moment that we had.

And as the product then shipped in mid-2021, and COVID was still going, we started looking at telemetry. And the team came and says it writes about 25% of the code in those files where it was enabled. And I remember saying, don't believe this.

Like, your telemetry is wrong. Please go back and validate that. And it turned out that was actuallyright. And by now, it's about half the code that's written. Some languages like Java even have a higher acceptance rate and more lines written.

And so I think this journey that we went through over the first two years really was like one moment after another where we saw the future of AI long before ChatGPT actually opened everybody else's mind.

Amazing. And now it's available to everyone here as well. So I think Copilot started as an auto-completing IDE, and now it's all over GitHub. I know I have PR bots, et cetera. What do you do to make Copilot integrated across all of GitHub?

What are some experiments? What worked? What didn't work?

I think the first thing is to think about, what do I do as a leader, as the CEO of a company? And it's really about constantly reconfiguring our approach. So much of the AI world is changing almost daily.

There's some news on the information elsewhere every morning. And so there is no more AI, I have a long-term strategy, I have my features all laid out, and I just work through the backlog. It's really like operating as agile as possible, even as we are a 3,000-person company as part of one of the largest companies on the planet.

The second is that we really try to meet the developer where they are. We say, we're not trying to build an AI engineer. We are trying to build AI for engineers, a human-centric approach. That's what the name Copilot ultimately visualizes.

But also, we're trying to make the developers' lives better because we are developers ourselves. And every productivity improvement we can find ultimately helps us at GitHub to build our AI product. So that really is the approach, like looking at what can we do next to make our work a little bit easier of building more features for Copilot.

You mentioned a great point. You're trying to meet the developer where they are. So for now, we've been bringing the AI to the IDE.

Yeah.

Are we going to try to bring the developer, the IDE, closer to the AI? How are you thinking about that?

You know, the idea of bringing AI into the IDE, really into ghost text, auto-completions, was a way of getting around hallucinations. It was a way of saying, OK, the model is not always going to be perfect, but neither are auto-completions,right?

Like whether you have auto-completions in your Google Docs or in your email or in your editor in the old intellisense way, as you're typing, it cannot know what you want it to type. And so you're used to adjusting your typing.

And then you find this moment when you press the Tab key. And even without auto-completions, if you think about what developers do in the editor, while they write code, and the best developers write a lot of code before they get stuck.

And the newbies and those that rarely write code like I get stuck more often. And then you Control-Tab or Command-Tab into your browser, and you open Google or Stack Overflow, GitHub,right? And what you do there is you find code, and you argue with other developers, and then you copy and paste that code into your editor.

And you modify that as well. So it's kind of like, in a way, Stack Overflow has as many hallucinations as the model might have. And not because the answers are bad, but because the world is changing so much.

I quote a little bit on iPhone projects and Swift. And there's always a new Swift version after WWDC or a new Xcode version. And so things have changed of how you use the APIs. And so it keeps the developer in the flow.

That really the crucial thing here was we didn't in the world 10 years ago, we probably wouldn't even call this AI. We would just call it smarter auto-completion. And the AI piece is not the core piece. The core feature of Copilot is that it helps developers to stay in the flow to get the job done and not be in this constant distraction between the editor and the browser.

That's a great point. And I think a few months ago, you wrote this post about Workspace. What was the journey to creating Workspace? And maybe for folks who are unfamiliar with it, what is Workspace?

Yeah, so you already mentioned auto-completion. That's how we started. In November 2022, ChatGPT happened. So early 2023, we added Chat and GPT-4 to Copilot in the IDE as a separate sidebar window. So we have that available. And it has RAG and all the information, the context available in the IDE.

But ever since, we have been thinking, how can we make the developer flow even easier? And Workspace does exactly that. It takes a GitHub issue or just the task and idea that you write down on GitHub.com. And it helps you then, as part of your code base repositories, to figure out how to implement that change.

It bridges from the issue, from the task description, into the pull request, into the code. And the magic behind this is that, A, the human is still in the center. So every step of that way, writing a specification, analyzing the current repo, the current behavior, and then using your description to figure out how do you modify this.

Then writing the plan, which shows you how to change all the files, to the implementation, which is the diff view, if you will. The human can interact, can change those bullet points, can change the code. And what that really does is it gives you a pair programmer that helps you to explore the code base,right?

Because the challenge we all have as engineers is that as soon as you get moved on to a new project, or you want to modify an open-source project, or you're just coming back from vacation, you're trying to remember what is implemented where in your 1,000-plus files.

Navigating the code base is the first challenge you have, figuring out what's the current behavior and what's the new behavior. So having an AI native, a Copilot native developer environment that helps you along that journey that you're naturally also doing in your IDE.

And that really is the key here. It's not about building an autonomous agent. I'm sure you have heard a lot about that in the last three days. It's about building agents that helps us as humans to solve a task and learn along the way as we figure out, oh, there's this test file that I also have to modify if I want to implement this feature.

I love the point that you mentioned, which is not building autonomous agents and also helping the developers. So how should non-developers use Workspace?

They can. And in fact, once we announced this last year at GitHub Universe in November, I think the first email we got with feedback was from a program manager, our product manager, saying, this is awesome because now I can not only write a user story or a work item, I can also see what that would mean to implement in the code base.

In many ways, the biggest challenge we have today is can we be as specific as possible when we write down a task as product managers or as engineers ourselves. Often, everything is obvious until it is not. And then

you kind of need to size the task,right? Like, how long will it take? And the mythical man month, I think the pragmatic engineer had that a couple of weeks ago, is still true. Most estimates are half the time that the job actually takes.

And so we're really bad at estimating how much time it takes to get work done, whether it's encoding or whether it's building houses or roads or infrastructure. And so Workspace helps you with that as it helps you to figure out what I just described.

Is it actually specific enough to write the code for that or to even figure out what the plan would look like?

Can you share a bit about your vision on how you think we will build and code in natural language and how it will help us collaborate better, devs and PMs, coders and non-coders, across languages and across the world?

For me, the very first thing that you say in natural language, as I have it on my t-shirt here, Copilot speaks your language, is because Chat, these large language models that we're using today in GitHub Copilot, in many other AI applications, are the same models that are also helping us in Chat agents.

They speak almost any language, any major human language. And so whether you want to explore coding in English and you don't understand the concepts of true/false, Boolean logic yet, or whether you want to learn that in German, in Hindi, in Brazilian Portuguese, in Spanish, in Chinese, you can do that now.

And if I look at kids today in school, most of them are growing up with mobile phones. When you go into a restaurant here in San Francisco or in Seattle or elsewhere in the world at night, you probably see a family with little kids where the kids have their phone because the parents want to enjoy five minutes on their own.

And then as then kids grow up, they see Super Mario or Minecraft, and they get into gaming. And that naturally, that means, how can I create my own game? How can I create my own web page? Copilot enables that.

And it enables that in the language that the kids grow up with, which for the majority of the humans on this planet is not English. And so that's number one. It democratizes access to technology. It also democratizes access for those that don't have parents at home that have a technical background or that don't have parents at home that have infinite patience, but most parents do not.

I have two kids. I speak from my own experience. At some point, you're just done with explaining the world to your kids, and you just want to switch on the TV and watch this Netflix show. But that keeps going.

If you look into the professional context, one of the biggest challenges we have is if you would join my company or I join your company tomorrow, the biggest challenge we have is what's all the institutional knowledge? How are things being done?

And what we don't like as humans is asking 1,000 questions, especially if you're a new employee in a big company. You're like having this anxiety in your head that everybody else thinks you're dumb. Why did you get hired in the first place?

So Copilot also democratizes access to all the information in companies. And I think that is going to be changing how we work, not only for developers in the workforce, but for really every human.

Thank you. Thank you, Thomas, for sharing your vision. I guess the next thing I want to ask is maybe a little bit more unhinged. Speaking of agents, in your opinion, what makes an agent or a Copilot? What's your definition of an agent?

I think an agent is like an AI dishwasher. You fill it with the dishes, and you let it do its thing. And then at the end, you take the output, and you put it back into the shelves,right? And today, we have we used to call it bots

or CI/CD. In many ways, that's an autonomous agent,right? You push a pull request, and you run your CI/CD, GitHub Actions, or a similar product. Many compute primitives that we have today are agents as they get a job done on their own.

My monitoring to figure out if GitHub up or down is somewhat autonomous. Hopefully, it pages somebody without us hearing from you that you cannot access your repository. So I think in many ways, what we are building is still tools that help us to get the job done.

And there's many jobs that developers have to get done, many jobs that now AI engineers need to get done. You saw it on the slide earlier, all the things that are also still true, even though you can automate things with large language models.

And a lot of work in software engineering is bogging us down. A lot of boilerplate, a lot of security compliance. That Friday evening, when you want to enjoy the barbecue because the sun is out, and instead, you have to update all your log4j dependencies,right?

Like security tooling, in fact, is creating more work. It's not a dishwasher. It's actually a tool that shows you that it tells you that the dishes are dirty, and then you have to do the dishes yourself today. And so that's security tooling,right?

It just adds stuff to our backlog while we actually want to work on the creative side. And we want to build new features. We want to build innovative products that creative things. I think many software developers do not understand themselves as a production worker.

They understand themselves as artists, as creators.

Yes.

And but our companies, our governments, the world is requiring us to do a lot of other work. And we need AI tools, auto fix, things that scan not only for security issues, but then fixes those security issues. We need those pieces supported by AI.

So we have more time for the things we do want to do, and AI takes over the things we don't want to do. And that's where the agents will go.

Fantastic. What's an agent you want to have, and how far are we from it?

I mean, I want to have these agents that burn down all my security backlog. It's as in any company, the challenge is that I have way too many of these items. And there isn't really a book you can buy that tells you, as an engineering manager, how to balance those two things.

You cannot do all your work into security compliance, accessibility, and whatnot. And you cannot put all your work into innovation because your customers will lose all your trust the moment you have a security issue that threatens their data.

And as such, you have to balance those two things. Or you find AI agents that bring the work down. And I think as any leader of a software development company, I always want to go faster. I always want to get that feature done faster.

And I'm sure it's the same for you folks at Amazon. When I have an idea and I ask my folks, how long will it take to implement that, the estimate I'm getting is like I'm scratching my head. I'm thinking, I could do that myself faster than waiting for my team to do it.

But of course, that's not the truth. The truth is that there's so many other things in the process these days that we need to find new abstraction layers that help us to get control over our development lifecycle again.

That's a great point. So last question. Do you have any advice for devs, both new and experienced, on how they should navigate this new world of tools, this new world of abstractions in what some say is the biggest technology innovation since the internet?

I think the most exciting thing about this new technology, as you saw it hopefully over the last three days at this conference, is that we are moving into a new world of software development. And there have been multiple step functions over my life.

I was bornright before the PC was invented. I remember getting my Commodore 64 on a PC in the '90s. I remember the open source and the internet. And internet open source before the internet was buying CDs and DVDs in bookstores.

The internet came, SourceForge, and then GitHub came. All of a sudden, developers started collaborating. The mobile wave came. And every time we had those step functions, software development got more exciting. And I think we are again at that step function.

It means we can embrace our nerdy tools. We can build new shit. And I think it's really like for me, as the CEO of GitHub, I don't get to touch code often. And so when I get to touch code on a Sunday afternoon, I don't want to spend all my time updating all my.

OK, that's all we had. Thank you, Thomas. Please join me.

Thank you so much.

And around us, and then we want to bring the fun back. And that's the AI brings the fun back into software development. And so I want to encourage you all to go back home and do some good stuff and get a great experience of AI.