GitHub Next Explorations: Rahul Pandita

My name is Rahul Pandita, and I am a researcher at Github Next, uh, and today we're going to talk about some of the GitHub Next explorations. Uh, now, before we begin, who among you have heard of GitHub Next?

Oh, cool, quite a few of you. That will make it go much easier and much faster. Allright, for those of you who don't know us, we are, uh, about 20 bunch of researchers, senior-ish level, uh, developers, and mostly code builders, uh, tool builders, uh, who work outside of the regular product, uh, and report directly to our CEO, uh, and that's by design.

And, and our goal is to explore the future of software engineering like you all are doing in, in your day-to-day jobs. And the, and, and the reason for exploring that is that, like, once we do our explorations, we toss it on, and we pass it on our learnings to the product and development teams so that they can build really compelling products like the Copilot that you all have used, hopefully, at some point of time.

As an aside, uh, for people who are following us on Twitter, uh, I don't look anything like my picture over here. I'm the one in the green background. But we do have Devon in our team. He's not an automated AI.

He's a very real person, and he looks exactly like the person on the topright corner on that slide. Allright, since we have gotten that out of the way, let's talk about, let's get back to the future of software engineering with regards to GenAI.

So here's what Andrew Ng, uh, who single-handedly trained a whole generation of machine learning engineers, uh, has to say about, uh, AI: that it's just as electricity. It's the new electricity. It's going to transform the software development and almost every other field, just like electricity did 100 years ago.

So what does that mean? Here's a picture of what a manufacturing facility looked like before electrification. There used to be a giant, uh, mostly coal-powered steam turbine or steam engine located centrally, which used to turn these giant, uh, giant shafts, which will turn these auxiliary shafts, so forth and so on, and individual workers would connect to these shafts using the belt and pulley system,right?

And, and these engines were, like, really, really huge. So, so it was the workers, the whole architecture of the factory was designed around this steam engine, and, and the whole workflow was around this steam engine. And, and it was the workers who were working around the technology rather than the technology working for people,right?

And along in 19, uh, 1880 came these electric motors, uh, and, and they had the potential to revolutionize, uh, the, the manufacturing sector. Why? Because unlike steam engines or steam, uh, motors, they retained their efficiency when they were smaller,right?

So, so you could basically redesign the entire factory floor plan. So you would think that, "Wow, this is great," and everyone would jump on this. But it was not until the 1920s where these became the mainstream. So early 1880s to late 1920s, what was happening for about these 40 years?

What was happening was exploration and experimentation. People were trying to figure out, uh, how to use this technology, how to make it better, how to de-risk it to a point that, that the use of this technology becomes the norm rather than the exception.

And that's what we do at GitHub Next,right? Our charter is to explore the future of software engineering, and with the emphasis on the word "explore,"right? Because if we knew what the future of software engineering in context of AI looks like, we would just build it.

That's more efficient. But unfortunately, we do not. So what we have to resort to is exploration. We just try out different things, rapidly prototype, experiment, and figure out whether something works or not. And if it works, then we put it out in front of our customers, uh, and, and users, and we learn from them, and then we finally transform into a product.

Oftentimes, an idea begins as inside our Next, uh, as a functional prototype, which goes through heavy dogfooding inside the Next team. If it survives that, then we move on to the next level of dogfooding that is inside the company.

If it survives that, then we move on to the next level, which is releasing it as a tech preview, uh, to the early adopters. We learn from that. If it survives that, then it may have a chance to become a product like that, a product in the future.

And we can kill or we can shelve any of these exploration at any point of time if we are not getting theright signal so that we can explore other areas. We did that with the Copilot. So yes, Copilot started off as a Next experiment.

And since that, we have created many other experiments like Copilot for CLI, Copilot Voice, GitHub Blogs, SpecLang, so forth and so on. Uh, a lot of these have transformed into a product of their own. So you can see some of them as, uh, GitHub product offerings.

A lot of them have been absorbed into existing products, uh, and, and you will see them as a part of the existing products. And a significant number of them have been shelved. We've learned what we learned from those experiments and figured out that this is not theright time for that kind of exploration or the exploration itself was flawed.

So, but we learned from them, and we will keep that learning and use that in our next, uh, Next explorations. So that was an overview of GitHub Next. And today I'm going to talk about two sp-specific explorations. Uh, one is the Next edit suggestions in the Copilot Workspace that are currently active, uh, from, from GitHub Next perspective.

And, uh, specifically, I'm talk I'm going to talk about what their motivations was and, and how they came to be and what are the future plans for that. So first off, uh, Copilot Next edit suggestions,right? So what if it started off with this question, "What if GhostX could be more intelligent?"right?

So we all know what Copilot does. Uh, it provides you the code completions in your current context,right? While it's, like, really, really good at creating new code, but that's not what we all do,right? We, we, we almost always edit existing code, which involves, uh, editing, adding, deleting lines at multiple locations in a program,right?

What if GhostX was good at that as well? And that's what this exploration is. We call it Next edit suggestion, which provides you suggestions not only at the current cursor level, but provides you suggestions what else needs to change in a program.

But enough talking. Let's jump onto a demo.

Allright, here, I am going to add this parameter in this Python program, and the Next edit suggestion automatically picks it up and says that, "Hey, you need to update your method definition." Once we update the method definition, it says that, "Hey, you need to add these, uh, these, these arguments."

And once that has been updated, then it will go back and say, "Hey, uh, now the code document, uh, is not, is not in line with what the code is actually doing." And it goes ahead and edits that and updates that as well.

And the same thing repeats when I add one more, uh, parameter.

Allright, so that was Copilot Next, uh, edit suggestions experiment. Uh, we are, we're still not ready yet. We are still, uh, experimenting with a bunch of other stuff like, you know, uh, is the GhostX completion theright, uh, modality for it, or do we need to figure out a different, different way of presenting those suggestions?

What if the location of the Next edit is not visible in the current viewport, or what if the location is in a file that is not even open in an editor? Uh, most importantly, we are also working on fine-tuning the models specifically for this use case.

The idea being that, like, if we want the Next edit suggestions to be, uh, accurate and we want it to be very useful, then the suggestions need to be on point. And once we are done with these further sub-explorations and we feel that it has gotten through our internal dogfooding standard, Next edit suggestions would be coming out either as a standalone, uh, tech preview from Next or as a part of an existing Next product, uh, sometime in your IDE, uh, in the next few months.

Allright, so there was code completions, but let's move from the code completions to the task completions land. Uh, why do you ask? Why, why move from the task completions? It just turns out, uh, that while code is, like, an important artifact, uh, that comes out of software development, but it's not the only artifact.

Software development involves this inner loop where you begin with a task. The idea is like, "What am I supposed to do? Uh, how am I, uh, what, what is the specific thing that I'm trying to do?" And followed by, uh, "How do I go about doing that thing?

What are the frameworks that are at my disposal? What are the programming languages that are, that are at my disposal? What are the kind of, uh, what is the existing code that's there? What, how do I write a new code that is consistent with those codes?"

So that's becomes a sort of a specification. And once you understand where you are, then you sort of try to decide, like, "Where am I going with it? Like, how does the final product look like?" Once you have zeroed in on that, then you go about, "What specific file changes do I need to make to, to, to get to that final product?"

And it sort of becomes a plan. And once you get to the plan, then you go to the implementation part. And that forms this loop of software development, and we call it inner loop. And we would like the AI to be helpful in all those aspects of that inner loop.

And that's why we built Copilot Workspace. In mind you, like all Next explorations, it did not start as Copilot Workspace. It started as individual explorations. For instance, we started to figure out, "Can we use natural language to as a functional specification of program?"

So there is a SpecLang exploration. We in parallel, we were trying to figure out if we can improve the code completions by providing, by prompting the model with the runtime information. And all of those things combined and with the user feedback combined into this one bigger exploration called Copilot Workspace.

And we were also talking to our users. Like, we, we, we wanted to talk to developers, and we wanted to ask that, "Hey, we are building this thing. How would you like AI to support you? What are your major pain points?"

And one, and a few things became very, very clear while talking to our users,right? So first thing is that the most, uh, difficulty that people faced was getting started on a task. Like, "How do I, I know that an issue is assigned to me.

How do I get started on it?" Followed by, "How do I trust the output of the AI? I don't trust it." And more importantly, they figured out that problem solving is what software development is about, and I, they would like to retain that problem solving, uh, aspects of it.

And they would like the help of AI in the form of a thought partner or a sparring partner or a second brain, which they can collaborate with to solve a problem. And lastly, and most importantly, they would like to retain control.

Developers are in control, not the other way around. And with this feedback, we built Copilot Workspace. So what is it? It allows you to, it simplifies getting started. So one-click proposal on, on your tasks. It has a built-in runtime that allows you to quickly verify what the, the code that has been provided by the AI.

It has an environment which is built for iteration. So if you feel that AI is going in the wrong direction, you can just go and quickly correct it. And most importantly, it is designed for collaboration. So you can just share, uh, your code or your work as a part of the GitHub pull request, or you can share your work or share your workspace with your colleagues if you're not comfortable with it.

But let's, enough talking, let's just get into a demo about it,right? So this is MonoSpace, which is another GitHub exploration. So if we are to write code, let's write code in style. And these are the four, uh, is a family of MonoSpace fonts that has been released by GitHub.

And, and this is a website that outlines a bunch of, uh, features of these fonts. And over here, somewhere over here is this playground which says that, uh, that here are how the syntax highlighting looks across different languages.

Notice that it is missing Rust. And Rust appears to be the next cool thing that all the cool kids are doing. So we would like to update this MonoSpace website with a Rust example as well. So how do I get started?

So I've created this issue, or somebody has created this issue. It just happens to be me for the purpose of this demo that I would like to create. I would like to add a Rust example to the font playground.

And I can just click this button over here, and it will open the Copilot Workspace for me. And through the magic of caching, you can see that it quickly generates the specification and propo uh, current specification and the proposed specification.

Uh, why caching? Uh, because I had to finish this demo in time. But trust me, it's not a matter of hours. It does happen in a matter of minutes,right? And, and for the, those of you who are interested, I would like to do a live demo for you in the Microsoft booth after this task.

Allright, so what is the current specification? It just goes and figures out, does the website have this playground that contains a, uh, Rust package? And it says that it doesn't. And it goes to the target state. Would, where would the target, what does the target state look like?

And it would say that, yes, the website will have the specific package for syntax highlighting. The website will have, uh, this package in, in, in package.json. And then I will update a bunch of other files. It looks nice, and I'll go and generate a plan for it.

Again, through the magic of caching, a plan has been generated, and it will tell you that these three files, these three files need to be updated. And I will, it, it appears that this seems to be at theright level of modality.

Then I will go ahead and implement it. And yes, magic of caching again, what we see is the files that are over here. Uh, now this seems nice. And, but what about the iterator part? What you can do is at any given point of time, if you feel that something is notright, you can just go ahead and say that, "Okay, add Rust to the language mappings," and say, "Add code documentation."

And you can edit at any given point of time. And what you can also do is that you can edit via chat over here, and you can say that, "Hey, I want to edit this one specific location. How do I go about in doing this?"

I'm not going to do this because it's going to go through the whole iteration loop, and then the illusion of the caching will break, and it will take a lot of time. But I would like to do show that in live demos afterwards.

But how do I trust whether this is, in fact, theright thing? So I will open up this integrated terminal, and I will say, uh, "Install and run this repo." Allright, so what's going to happen is, uh, that a suggestion is going to load and apparently not theright thing.

But I can quickly go and edit it and say that, "Allright, this is the command that I'm specifically looking for," and I can go and run. Now this will run this command in an actual terminal, and we'll see the output in some, some point of time.

Uh, and, and you can see that actually this, this code does compile. What we also have is a preview. What we can do is open the live preview. I don't trust it. It's, uh, it will say that it's just going to be a second, but it takes longer than that while that loads.

What are the other things? Uh, one of the things that you would say is that, "Hey, you wrote a very simple command in the terminal. You said npm. You could actually type that thing in the terminal." And yes, you'reright.

I can type that thing. But think about that in a mobile setting when you can open Copilot Workspace in a mobile plat uh, in, in, in on your phone. It becomes very tedious to type those symbols,right? And if you have used the, the mobile keyboard, it's not very useful for that.

So what I'm going to so, so that's why we use this natural language way of, uh, writing these commands in the terminal, uh, so that it can help you when you're on the go. It can synthesize those commands.

And hopefully the website has loaded, and there is a Rust example,right? Cool.

That was a demo, and thank you.

We are working we are not stopping there. We are working on a bunch of these improvements, and I can talk about these improvements, uh, on one-on-one basis with you. And, uh, and, and you already saw some of the improvements like the runtime support to synthesize the terminal commands and, and faster file completions using, uh, to, to make the Copilot Workspace better.

But there are other Next explorations that are also active, like how do we rethink the developer learning with AI, and how does the code review change if the majority of the code that, that is now being written is by AI?

So what does that mean? And some of these explorations will, will work out, and some of these explorations you will see as tech reviews, and some of these explorations will kill because we don't know where they're going. So in summary, I'm saying that we do not know what the future of AI is, but what we know is explorations is the way to get it.

And with all your help, we'll jointly explore the space so that we don't have to wait like electricity. We don't have to wait for 40 years to get to a place where to, to get to a place with software development where we enjoy the benefits of AI.

You have been a lovely audience. That is my time. I really appreciate you. And if you have more questions, if you want to have live demos, I'm available in the Microsoft booth, uh, in like two salons over that side.

Thank you so much.