# How Coding Agents change Software Development Forever - Hailong Zhang AI Engineer · 2025-02-22 Hailong Zhang presents how coding agents, particularly the unit test agent Gru, transform software development by automating routine tasks while humans focus on creative work. In the future workflow, synchronous agents like GitHub Copilot and Cursor assist in real-time, while asynchronous agents like Gru autonomously handle tasks such as writing tests, fixing bugs, and submitting pull requests. Gru, built to boost unit tests, detects code changes from pull requests, generates and runs tests, and submits a new PR with a summary and coverage improvement. In production, over 50% of Gru's PRs are accepted by humans, and it handles 80% of unit tests in its own repo, making it the top contributor. To build such agents, Zhang emphasizes defining a clear problem (e.g., unit tests, not generic software engineering), creating datasets and evaluation harnesses, selecting and fine-tuning LLMs per stage, building task-specific context from environment data, and developing an agent operating system to reuse common infrastructure across different tasks. ## Questions this episode answers ### What is Gru and what are its results in automating unit tests? Gru is an AI coding agent that automatically writes unit tests for code changes in pull requests. It detects changes, generates tests, runs them, and submits a PR with a summary and coverage improvement for human review. In production, over 50% of Gru’s pull requests are merged, and it handles around 80% of unit tests in its own repository, making it the top contributor. [3:49](https://aie.addtry.com/35a72397-3020-47a8-9bb2-1c2264d71bce?t=229000) ### What is the difference between synchronous and asynchronous AI coding agents? Synchronous agents like GitHub Copilot work inside the IDE simultaneously with the developer, providing suggestions as you type. Asynchronous agents are bots that run autonomously in the development workflow, triggered manually or automatically, and complete tasks like writing tests without human attention, then submit a deliverable. Hailong Zhang says both types are important for future software development. [1:11](https://aie.addtry.com/35a72397-3020-47a8-9bb2-1c2264d71bce?t=71000) ### What approach did Hailong Zhang use to build the Gru unit test agent? Hailong Zhang first defines a clear, concrete problem like unit testing. Then, they build evaluation datasets and harnesses, evaluate frontier models from OpenAI, Anthropic, etc., to pick the best per stage, fine-tune models such as GPT-4o with human-labeled tests, build context for each language and framework, and use an agent operating system to orchestrate shared tools and runtime. [5:59](https://aie.addtry.com/35a72397-3020-47a8-9bb2-1c2264d71bce?t=359000) ## Key moments - **[0:00] Agenda** - **[0:32] AI Shift** - **[1:11] Sync & Async** - [1:11] Hailong Zhang distinguishes between synchronous coding agents (like Copilot) and asynchronous agents (like GitHub bots) that work autonomously. - **[2:26] Agent Workflow** - **[2:55] Test Gru** - [2:55] Unit testing is now more critical because AI-generated code is produced so fast it's prone to bugs, says Hailong Zhang. - [3:59] Gru, an AI agent, triggers on pull requests, writes unit tests, runs them, and submits a pull request for human review. - [4:49] Gru's generated unit test pull requests are merged over 50% of the time in production, a meaningful starting point. - [5:19] Gru already handles 80% of unit tests in its own repo and is the team's top contributor; Zhang predicts more agents will become contributors in 2025. - **[5:51] Building Agents** - [5:59] "Unit test is a problem, but software engineering is not" — Hailong Zhang on agent scoping. - [7:57] Hailong Zhang introduces an agent operating system (agent OS) that abstracts common infrastructure to rapidly build agents for different tasks. ## Speakers - **Hailong Zhang** (guest) ## Topics Agent Skill Design ## Mentioned Anthropic (company), DeepSeek (company), Google (company), OpenAI (company), Cursor (product), GPT-4o (product), GitHub (product), GitHub Copilot (product), Gru (product) ## Transcript ### Agenda **Hailong Zhang** [0:02] Hi, everyone. In this presentation I'm going to talk about coding agents and how to work with coding agents. This is my agenda. First, I'm going to talk about the future of software development workflow, and then I'm going to show you an example of a coding agent, and this agent will boost the unit test. And finally, I'm going to dive deeper into how we solved real-life problems with agents, how we build the agents. ### AI Shift **Hailong Zhang** [0:32] Generative AI has shifted a lot of things, and people believe AI coding is going to be the future. And we believe a lot of the routine works will be handled by AI, including entry-level coding tasks, debugging, testing, and operating. But still, I think the creative works, including product design, architecture design, and difficult issues, still need to be handled by humans. So it must be a human and AI agents collaborate together to solve problems in the same workflow in the future. ### Sync & Async **Hailong Zhang** [1:11] The collaboration has two types. First is synchronous, and another one is asynchronous. I think people may be pretty familiar with the synchronous one, GitHub Copilot or Cursor. This is AI live inside your IDE, and this AI works simultaneously with human. When you're typing, it's working,right? So we have already had this kind of product for a long time, since 2020, and it's getting mature since 2023 and rapidly grows in 2024, and it's widely adopted today. But the asynchronous one is pretty new, and it just started in 2024. The asynchronous one is more like a bot inside your workflow, for example, GitHub Bot. And this bot can be triggered either manually or automatically. And this bot will complete the task without humans' attention, fully autonomous, and will submit a deliverable once it's done. So it's totally different experience, and these two types of agents are all important. And I think people need all these kinds of agents to solve the real-life problems. ### Agent Workflow **Hailong Zhang** [2:26] Imagine this is your workflow in the future, and the workflow will have a lot of AI agents, small agents living inside your repos, workflows, in different stages, to solve different problems, including unit tests, including fixing bugs, writing documents, submitting code reviews, and releasing from these kinds of tasks. And so human can focus on the more creative part. ### Test Gru **Hailong Zhang** [2:55] Next, I'm going to show you a detailed coding agent, which will boost unit test. I think unit test is more important today than before, because in the AI era, a lot of code is generated by AI. As we just talked, if you're using Cursor, you must be familiar with tab. For each of the tab, multiple lines of code in multiple parts of the file will be generated. And it's very hard for you to focus on every generation, and you may overlook something. The coding speed is really, really fast, but it's also easy to generate bugs. So one of the ways to solve the issue, or prevent the issue, is to write a unit test. People talk about unit test. People think unit test is important. But when writing unit test, people hate it. Yeah, developers, every developer hates to write unit test. So we built a Gru, an AI agent, to help developers to write a unit test and managing unit test. This is a screenshot that pull request of unit test generated by Gru. And the process is triggered either manually or automatically by a pull request. When humans submit a pull request, Gru will detect the change of the code and determine if this change needs more unit test or change existing unit test. And Gru will do the coding work, writing the test, and to run the test to make sure everything works, and then prepare and submit a pull request, including all the information showing in this screenshot, the summary of the test, and the coverage improvement of the test. Now, human needs to review this pull request and determine if this unit test is good or not to merge into the repo. Let's look at the performance of Gru in production. This is also a screenshot from GitHub. It's a list of pull requests. And as you can see, over 50% of the pull requests are merged and accepted by humans. So 50% is not a very large figure, and there are a lot of rooms for improvement. But I think in real life, 50% is already meaningful in production and helps solve a lot of problems. And also, Gru handles around 80% of the unit test itself in its own repo. We are using Gru every day, and most of our tests are generated by Gru. From the commits you can see, Gru is already in the counts of commits. Gru is already the first contributor in our team. And we believe more and more agents will become contributors in people's repo this year, 2025. Next, I'm going to dive a little deeper into how we build Gru to solve real-life problems. ### Building Agents **Hailong Zhang** [5:59] To build an agent, I think the first and the most important thing is to define the problem itself. A clear, concrete, and doable problem is crucial. For example, unit test is a problem, but software engineering is not a problem. And once we have the problem, we need to build the data sets for the evaluation purpose and also the evaluation harness. And then we work on LLMs, building context. And last, we need agent OS or framework to orchestrate everything together. As we all know, agents work on top of LLMs, and Gru works on all frontier models, either from OpenAI, Anthropic, DeepSeek, Google, etc. And we evaluate the models on the different scenarios, try to find the best model for each of the scenarios. That means even within the same job, Gru may use different LLMs for different stages. And also, we fine-tune models to improve the performance. For example, we fine-tune GPT-4o with human-labeled unit test code to improve the generation of the test code. Building context in agents is really important. And we think it's necessary to build the context for specific tasks. For example, here in unit test, we build the context for each of the languages and the frameworks. And we also gather all the information from the environment. For example, the GitHub issues, code reviews, commits, pull requests, README, code itself. And all this information needs to be gathered together and to be filtered and to be selected to fit into the context. We talk about unit test a lot, but our vision is not only unit test. We want to build agents for different software engineering tasks. For example, the refactor task, the E2E test task. It is almost impossible for us to build every agent from scratch. So we build the framework, or we can call it agent operating system, agent OS. Different tasks may share similar runtime, similar tools, similar context. So we abstract the common infrastructure. And this operating system will enable us to build agents in this domain in a really fast pace. The agent era is coming. Let's embrace agents in workflow. Thanks for watching. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.