Enhancing Quality and Security in CI: Gunjan Patel

My name is Gunjan Patel. Um, I'm a Director of Engineering at Palo Alto Networks. Uh, like many of you, in the last year and a half or two years I've pivoted to working on GenerativeAI. Um, the talk today is about self-evolving code.

I'm not here to sell you anything; none of this is a product from our company. Uh, this is, uh, a side project that I've been exploring. Before we get started with this talk, um, I would like to dedicate this talk to my close friend, Nikhil, who passed away two weeks ago.

Um, it was unexpected, uh, but, um, he was very excited for me to come and do this talk. Uh, and he was very enthusiastic about this conference itself. Um, so I want to dedicate this talk to him.

Okay. Let's start with the coding flow. Uh, if you've read the book called The Flow, um, it talks about this zone where, uh, your skills—on one axis you have your skills and how challenging the problem is—and the next, uh, next axis.

As you're starting to work on a project, you should be somewhere within that green zone to stay in the flow state.

In reality, what happens is, in the beginning you have an idea, you start writing some code, initially crank out some code. Then you have a little bit of a difficult problem, so it cranks up the, uh, direction a little bit to the anxiety, may even go a little bit into anxiety.

And then you have to write unit tests, and then it goes straight to the boredom part. Uh, because it's, uh, your skill levels are quite higher, but nobody likes writing unit tests, uh, or code documentation.

So what's the problem there? As a developer, you're—when you're writing code, uh, when you're developing software, writing code is a small portion of that entire flow,right? Uh, you have to work with a lot of stuff like, you know, uh, building Kubernetes, uh, like manifests, building Dockerfile, etc., deploying stuff, writing unit tests.

So then there's this Copilot duct tape, yes, uh, having something like a Copilot that would help you get through a lot of technical hurdles on, uh, coding side. Whenever you see something challenging, you can overcome that. Um, but we are trying to outsource more and more of that to do offline,right?

So you have—you as a developer have more time to write code. So the key part in this talk is about outsourcing a boring task, uh, to CI.

Um, uh, I had to insert this meme here, um, that, like, on both ends of the spectrum, people use ChatGPT in the middle. The mid, uh, such as myself, is like juggling all these different tools, trying to figure out how to write the code.

Okay. Let's get into this. Uh, so what's the difference between Copilot and, uh, I just coined this term, Ghost Pilot? Uh, Copilots are made for quick, just-in-time decisions,right? Uh, Copilot examples are, uh, Source Graph Corey, like Gwen just talked about earlier, uh, and GitHub Copilot.

They work with developers live. So developers are impatient. You need response in seconds. Uh, uh, how good of a tool it is, partly measured by how quick the responses are,right? Um,

code review—uh, code reviewers are made to have deliberate thinking,right? When you write code and then, uh, code reviewers reviewing your code, it's deliberate thinking. It's not just, "Guess the next word," uh, at this point. Uh, you need to ref-reflect on the answers that, uh, that you're providing or, uh, like a human code reviewer.

Uh, being aware of the full context and, uh, to update the answer,right? So some of those things require—they need time. They require iteration. Uh, so one, one example is, uh, this book called Thinking Fast and Slow. Uh, so if you think about Copilot as thinking fast, uh, the slow system is this, uh, Ghost Pilot system.

So this is the high-level architecture, uh, I'm talking about here. Developer writes, um, code using, uh, a Copilot or IDE assistant, uh, tool. They check in the code in the—in CI, in the—during the pull request process. Uh, the first step is, uh, improving code varia—uh, code comments and variable names.

Second step is adding and running—running unit tests in a loop. Uh, and I'll go into each of these steps in more detail. Uh, adding unit tests here early on, uh, sorry. Let me talk about the first step. For doing the first step properly means adding unit tests makes more sense,right?

Like, you need AI to understand what is the intent of this piece of code. Uh, adding unit tests early on sets a baseline behavior for the code, that this is—if this is the input, this is the expected output.

So it's, uh, it's setting the baseline. Then third step is, based on the environmental context, identify security issues, security best practices that are not followed. Um, and fourth step is fixing them. And after fix—proposing a fix, not fixing them, proposing a fix for a human to review, uh, run the unit tests again.

So let's go into each of these steps. So first one, why improve variable names, uh, and code comments? Uh, yesterday there was a session here from Manuel. Uh, that he, he wrote this line: "LLMs are cultural technology,"right? LLMs are built based on all the knowledge in the world, not just coding knowledge.

Like, most of us here, we know a lot more about coding than art or how humans behave, uh, psychology, etc., where LLMs have all that knowledge stuffed into them. We should figure out how to utilize that completely,right? So an example is, if you're working at a finance company, for example, and use a variable named rev, it makes sense to you, like revenue, in that con—small context of your code.

Uh, but improving it to annual revenue. And then in the subsequent steps, if, uh, LLM is run, uh, using different, um, different LLM providers, it will pull in its existing knowledge from all the finance world. So there may be some corner cases that may not have been discovered if you just used the variable named rev versus annual revenue.

Um, the second part is, uh, improving good, uh, improving code comments,right? Uh, code comments can get out of date quickly. If someone is in a hurry, makes change to a piece of code without improving the code comment, uh, then it's out of date,right?

Like, it doesn't state the developer's intent. So improving that, um, to make sure that LLMs understand what is intended behavior of this code, even though—even if the code may not reflect it. And that's why we have bugs,right? There's the intent.

Nobody intends to have a bug. Uh, so making that intention of the code block clear. That's why we improve the code comments. So this is a small example of that. Um, you can see, uh, in CI, it's adding high-level flow of what is expected from this code, uh, code block.

Um, here, like, it's a server, so I just put port, and it improved to port number because it could be port name or port number. Uh, I don't think about those things, but, uh, when it comes to adding unit tests, AI-generated unit tests, it may—it becomes important.

Second one is adding unit tests. So adding unit tests, as I said earlier, it's to set the baseline behavior. Um, uh, then once we have the baseline behavior, we can add a second layer of unit tests, uh, for covering corner cases.

Third one is to get context from historical bugs. Uh, what a looking at Jira, fetching for this project, we have seen this kind of bugs in the—in the past. So adding additional precautions to make sure those cases are covered.

So what are the steps like? It's not asking an LLM to say, "Generate unit tests for this code." It doesn't do a very good job. Uh, and this is where having iterations comes into play,right? So first one we do is, um, because we have all the time in the world.

People are not rushing for CI jobs to complete. I mean, yes, a lot of people are impatient, uh, but it's not like when you're writing code, uh, with an IDE-based assistant, you need response quickly. So what we do in this one is set the baseline.

So, uh, LLM outputs the expected behavior of this code. N-not the person who wrote it. LLM will read it and say, "Okay, this is the expected behavior." Uh, it will take things into consideration like, uh, code comments, README file, and maybe, uh, PRD document.

It may not go into function-level details, but overall intent is there. Uh, then in the next step, uh, we added adversarial mindset. Uh, you can use the same model or a different model and say, uh, system prompt is, uh, you assume adversarial mindset, and what are the edge cases?

Think about weird characters, min, max, all the boundary conditions, and add list out the unit test cases. Don't write the unit tests yet. Just list it out. Then, uh, personalize the code. Personalizing happens from team priorities that, uh, are written in context.yaml file, which I'll cover in a second, um, and historic bugs again.

So personalizing which unit tests to add based on that. So it will add a few more unit tests at the end that are specifically applicable to this code and this application. And then the last step is, here's the unit test cases we need to cover.

Generate unit tests. And then it will do a much better job, and then it can go in a loop until they pass,right? Uh, because again, it's cheap labor, free labor.

Uh, this is what the context looks like. Uh, context is used in the, uh, step three, but things like what is the cloud provider, what is the service, where is it deployed, is it a company internal tool or a customer-facing tool, is there PII data in it, um, is it a front-end or back-end, uh, application, networking, PRD, where—where is the Jira project to pull all the bug reports and, uh, more information on this project, Slack channel.

Now, this is important. I haven't implemented this part yet, but where it could go and ask follow-up questions to the user, saying, "Hey, I'm not sure about this context. It's missing this context." Similar to, uh, if you use Perplexity, uh, and you use Perplexity Pro, you ask a question, it's not clear, we'll ask you a question back.

Uh, so that's what this is designed to do. Uh, and your company security policies, um, guidelines from your infosec team, if you have any.

Then second part of the context is built, uh, using AI. So AI infers this context. Uh, what language is this? Is this IaC code or is this back-end code, uh, shell code, etc.? Uh, because you will have different priorities based on that.

Uh, what are the focus areas? That comes from historical bugs, uh, SQL injection, timeout errors, concurrent users, uh, whatever issues and escalations you've had in the past. Um, here, uh, Gwen has done a great job on opencontext.org. Uh, that's a potential integration point here, uh, as well, for bringing in additional context.

Finding security bugs. Now, uh, security issues. So security best practice. This is not just looking. It's different from running static code analysis and, uh, diff running SAST-based tools. It's finding secu which security best practices are not followed that your company may have policies around.

Uh, logical flow-based issues. Uh, I'll show you an example in a bit. Uh, and prioritize which ones need fixing,right? Not just identify. Because when you ask AI, "Find issues," it's going to find issues. Uh, it's never going to say, "No, it's fine."

I mean, it's a small issue, but it's going to say something is wrong.

So how do we do that? This is—this is a little bit of a weird side now,right? Uh, what I'm doing here is simulating three AI employees,right? Um, one is a Red Team Engineer, one is a Python developer, assuming there's Python code, and an engineering manager.

Uh, engineering managers are useful for something, uh, other than, uh, talking in meetings. Uh, so what context am I providing the Red Team Engineer? Security policies and the code that's written, uh, or the code that's being reviewed. Python developer will assume the identity of the person who wrote this code, uh, and the PRD, uh, product requirements.

Engineering manager will have business side of things, uh, business context. So, uh, Zoom meeting transcripts, uh, Slack channel conversations, and PRD. Now, there is a prompt to have these three people take turn, multi-turn, and then debate among each other on each of the security issues on identifying what is the risks associated with it and which ones need to be fixed and what is the effort for it.

And at the end, after they're done with the debate, uh, self-reflect on the answer and say, "Okay, this is, you know." And that could be a separate model. It could look at the answer and say, "Okay, this is matching the original intent or not."

Uh, some fun things you could try. Um, I was listening to a podcast recently, and they talked about Odyssey Journey, which is like you think about, like, three potential paths you could take, uh, and then time travel back and then talk to your past self.

Now, that's not really possible in real life, but in here, we can do that, um, with AI. It's—it's all fun, um, but it, uh, there is some research paper that show that this is actually more effective than just using LLMs, uh, as a standard tool.

Uh, this is an example. Uh, this was a Kubernetes bug, uh, in the Go core. It was a critical severity bug. Uh, and I made sure to find this issue from after the training end. So GPT-3.5, uh, training date end.

Uh, this was—this wouldn't have been picked up in any of the SAST tools because it's a logical thing. This if state statement needs to go from here to here, and that caused the, uh, security vulnerability. Now, that's something that's when I run it through this, it—it finds that issue.

Last one is fixing the security issues. So this is outcome-based, uh, and it's again, suggestion, uh, of what to fix. And then human reviewer comes in at the end and, uh, decides which ones to actually fix. For each of the fix, explain the reasoning why it's prioritized, uh, that all the conversation that the three virtual employees had have had, uh, it's summarized in three bullet points.

Uh, so human reviewer has the context and security, uh, policy, uh, citation. And at the end, make sure before a human comes in, run the unit tests that were establishing the baseline early on, so a human doesn't have to waste their time, uh, with verifying the fix.

Um, this is not a great example, but a simple example of what it would look like in a, uh, code comment, uh, hard-coded API key. I mean, most people wouldn't have that. Uh, what is the risk or what is the effort score?

Uh, what is the recommendation? Which is the code snippet? And then fix it afterwards. Uh, that's it. This—these are—this is my information. This is the GitHub repo. Uh, I will upload the code, uh, the GitLab CI and GitHub Actions, uh, files there in the next few days.

Uh, so bear with me. Uh, but thank you for coming.