Does GenAI "belong" to data scientists? — Phil Hetzel, Braintrust

What we're going to do today is we're going to talk about whether agents or agent development really belong to data science or machine learning engineers. How many people here would describe themselves as either A, a data scientist, or 2, or B, a machine learning engineer?

OK. This is going to be awesome. I'm glad that no one has brought any rotten tomatoes, because the answer that I'm going to give is probably not going to be exactly to your liking, but give me a chance to justify why.

What we're going to do today is talk through these couple of things. I'll introduce myself, introduce the company that I work for, and then we'll get into the topic. I'll probably rip through the slides pretty quickly and hopefully give some time for Q&A.

But before I do that, I'll introduce myself. My name is Phil Hetzel. I lead the solutions engineering team at Braintrust. Braintrust, I'll go over that in a second, but the solutions engineering team is basically I think there's a seatright there.

Yeah. Solutions engineering team is basically the people that allow our customers to get the most value out of the platform as quickly as possible. Prior to Braintrust, I spent 12 years in consulting and systems implementation. My last role in consulting was leading the global Databricks business unit at a company called Slalom Consulting.

And I noticed that a lot of my customers were really prolific at creating generative AI proofs of concepts, but not nearly as good at bringing those proofs of concepts to production. So I started using Braintrust as a user first, and I liked the product so much that I applied for a job.

And I've been here for about a year since that happened. Outside of work, I like to play chess, but I'm not very good at it. And I like to spend time with my wife and my dachshund, Pistil Pete.

He's pictured over there. He's the one in brown and not the one in black. The one in black is me. The one in brown is him. What is Braintrust? Braintrust is an agent quality platform. The way that we perform agent quality is two different pillars: evals and observability.

Evals, those are the things that you're doing in experimentation as you're tweaking and building your agent to become confident in your agent's execution once you push it to production. Agent observability, to us, means that once it is in production, that you remain confident in its execution once it's confronted with real usage and real users.

There are some other ancillary things that the platform does, but in general, that's what we do. I'm really not going to talk about the product today. If you're interested about the product, you can find me at the booth downstairs.

But other than that, very happy to get into the content. Just some observations that I've seen through the last year of watching some of the top teams building agents across many different industries. I think there are two different types of organizations that we work with.

There's the traditional enterprise, and there's the AI natives. Traditional enterprise approach agentic development a little bit differently than AI natives. Traditional enterprise, a person in charge, a person of note, CEO or CIO, will read something in the CIO or CEO monthly magazine that says that they need to be building agents.

And then they'll tell their delegate that you need to be building agents, because that's the thing that's going to take us to the AI promise land. And then that will get further delegated to an existing ML or data science platform team who already have a lot of the tooling in place.

And since generative AI has AI in the name, it's a pretty natural fit to hand over that capability to an existing AI or data science team. Is anyone kind of in that bucket today where they just got they were machine learning platform engineers previously, and they just kind of got handed generative AI?

Because that's kind of seemed like the best fit. Yeah, got it. And there's no judgment or connotation here, just something that I've observed. There's a whole other set of companies that are more AI native that think less about what already exists, because nothing really existed before generative AI started to gain popularity for these companies.

In fact, they started building their entire offering around agents. So rather than having an AI ML platform team, they'll just have a small team of engineers that are agile enough to grow with the times. And rather than having very specific segments of things that they do, everyone is very much cross-functional across both product engineering and AI engineering.

The other thing that's interesting about these AI natives is that since these are typically smaller companies, each person has more proximity to the problem, i.e., they have a better understanding of what the end agent is actually meant to solve.

Two different differences between traditional ML and generative AI. The model's already built. So much of what data scientists and machine learning engineers is going through that data pipeline of training a model. What do we do when the model is already built?

And the other interesting thing, or other interesting nuance, I should say, is that if you want to add value to these models, then you can add values not necessarily with feature engineering, but with natural language, which could bring in a different skill set to the conversation.

So just to make this more clear, this is I know that it's so much more complicated than that. But abstracted to a certain level, this is kind of what data scientists and machine learning engineers do. It is a data pipeline of training and testing, making sure that you're not overfitting, eventually deploying that model where it can be used by some downstream product team.

That is kind of what data scientists and machine learning engineers are used to doing. This, though, has already been done. Anthropic and OpenAI and Mistral, they've already done the data process of grabbing that data, putting it through the pipeline, training the underlying LLM, and then deploying it through an endpoint so that their consumers can use it.

The one nuance here is that

while Anthropic and OpenAI and Mistral will be doing testing of their own, we still need to, as AI teams, we still need to perform evals after we've implemented those APIs into our product. That needs to be very important to us.

That's really the only nuance here between these two images. How do you change this is kind of going to these two differences before. The second difference, how do we change these predictive applications before and after? Traditional ML, you either add more data to retrain it, or you're performing feature engineering to adjust how the underlying model is performing.

And then you're performing a lot of A/B testing to understand how your model changes has provided lift or not provided lift. With generative AI, you can, since that model is already trained and irrespective of performing any fine-tuning on that model, which is pretty rare, the way that you can change that behavior is just by changing the inputs, the prompts, the context that you're giving that model.

So there's a lot of folks that will be performing context engineering on top of these models. And those folks could have a better understanding of how real users could be using the agent. They'll have closer proximity to the problem.

So I'm going to make the case for and against agents belonging to data scientists. Let's say I was debating the position of it really does belong to data scientists and traditional machine learning engineers. Agents use models. In our organization, models are governed by data scientists.

So data scientists will have a lot of underlying knowledge about how neural nets work and thus how LLMs work. Because of that, they're going to have a far better appreciation of the risks inherent with using this very complex technology.

Other thing here is that they'll have very rigorous processes to push models and model assets to production. They will understand some type of testing process that they can use to keep the company safe and make sure that end users are getting the experience that they need.

And then number three, very related, just very rigorous mindset around testing. The counterpart to that is, again, models are already built, so we don't necessarily need to do any training and testing. Entirely different pipeline. We're not doing the whole cross-validation dance.

And this is probably the biggest argument. Does an AI or, sorry, does an ML engineer or a data scientist really know what they're testing for? One of the things that I've noticed with some of these teams is they will really lock on to the traditional ML engineer metrics, like precision recall, F1.

And they'll obsess over those metrics, because that is what has gotten them there up to that point. But when you're analyzing agents, it is far broader of a surface area that you need to be evaluating. You need to be evaluating the functional performance of that agent rather than just technical performance across that toolbox that we're traditionally used to working with.

So argument here is, let's say that I was arguing that agents belong to non-data scientists, which could be both technical and non-technical experts. We could make the case that LLMs are just APIs. Product engineers are very used to using APIs as they build applications.

That is a massive part of what they do is reaching out, grabbing information from another system based upon some payload, and bringing that information back in a way that's useful to the end users. That is a thing that product engineers do.

The other thing about agents that is unique is that if you have a very complex agent, it could be running across many different types of compute. If it's some distributed agent, i.e., you have a supervisor agent up here, and then it's calling different child or sub-agents that might be running on different infrastructure.

And as they're running on different infrastructure, they might be calling different systems as a result. That can be a very complex systems problem that might not be up the alley of someone with more of a statistics or maths background.

And then finally, more on the non-technical side, it's really valuable to have subject matter experts or product managers be able to control the actual prompts that we're seeding the agent with. These people are the ones that have the most proximity to the problem that the agent is trying to solve.

So there's a lot of lift in having a non-technical person have a lot of say in how the agent performs. Not only that, there can be a very large human annotation workflow that goes into making great agents, where as you see these interactions, if you are a non-technical person but has a lot of domain expertise for how the agent is supposed to be performing, that non-technical person can look into an agent trace and describe whether or not the agent is performing well or not performing well, and most importantly, why that's the case.

So where I'm landing with all this is not that all of the people that raise their hand that says, yes, I'm proudly a data scientist or ML engineer, I am not going to stand here and say, well, guess what?

You need to completely refresh your skill set. That would be a very silly thing to say, because I kind of figure there would be a lot of data scientists in the room, and I'm at least smarter than that.

But it does make sense to have a very diverse team when you're building these platforms. It makes sense to bring both non-technical and different types of technical people into the fold. How can data scientists add value to building agents?

A couple of different ways. Also, all these ways are irrespective of actually helping to build the product itself. I think that's inherent. With the tools that we have available to us now, it's actually quite easy for us to be able to add value to a product, even if you are not coming from a product engineering background.

But what I think is really valuable is data scientists can add, to use an overloaded term, add the guardrails to this process. A lot of people are very aggressive in how they implement LLMs. They don't understand how the underlying technology works.

They don't come from a stats background. I think data scientists can be the adult in the room during those situations and say, the LLM, this is how it's trained. It's just predicting token after token. It doesn't actually know anything, really.

It's just a bunch of stats problems at the end of the day. I also think that LLM as judge is a huge part of the eval process when you're building agentic applications. Again, people are very tempted to just believe LLM as judges when they're performing evals.

They're just prompts and models at the end of the day. And it's very easy to be able to create some labeled data set and perform the traditional recall, precision, and F1 style metrics on those, which data scientists will have expertise to.

And the last one, this is the most technical one, of course. If you do need to fine-tune an open source model very specifically to your use case, that's probably going to be the most fun and technical thing where data scientists and machine learning engineers can add a ton of value.

The ideal mix here, in addition to that top section, we want both product, application, and systems engineers to be able to implement those requirements into the product itself that the non-technical experts are giving to them. We want to make sure that the systems that we're building around these agents, i.e., where the agents are executing, is such that it's going to lead to a great user experience.

And then finally, and this is probably something the data scientists can pitch into as well, implement actual eval and observability pipelines so you have that feedback loop of what's happening in production and what's happening in experimentation. For non-technical experts, we want them to be performing a ton of human annotation and a lot of prompt and context engineering.

They have the closest proximity to their problem. You need to bring them into the fold if you want to have a very relevant agent to your use case. So what's next? Answer is always in the middle. So I hope I didn't fully insult half the people in the room today.

If I have, then you can feel free to come to the Braintrust booth and give me an earful. That's completely fine. But the idea here is that it's not a value for data scientists. Just make sure that you're bringing more folks into the room as you're building agents.

Two minutes for questions. I know that we're keeping a very tight timeline. Yes, sir.

Yeah, really good.

I like the conclusion. But I do have a question on the framing. So I view agents as a tool. So anybody in the organization could build and own an agent in theory. And they may have more domain expertise in the data science, the engineering side.

Rather than thinking about who owns the tool, this is the machine learning team, or that's the computing team, who owns the laptop. How do you view it as thinking about it based on the problem that's being solved and seeing agents as a tool to solve the problem, rather than the agents as the.

I think you're thinking about it I think you're thinking about it the exact same way. It's a product that a diverse team builds. I think the mistake that I see a lot of typically traditional companies make is they say, oh, we're making another predictive model.

And they isolate it to the ML engineers or data scientists and say, go build these agent things. I think we're actually thinking about it very, very similarly. Yeah. Maybe time for one more question. Yes.

Yeah, I really echo the message that you're trying to say.

I think I'm just curious about actually the tooling that you are offering about closing the loop. And I think that, at least from my experience, the missing delta is the tooling to facilitate the traditional machine learning into intra-like variation and the labeling.

Is that something that Braintrust is thinking about?

Yeah.

Yeah, how to make it easy for a domain expert to update the system and update the systems?

Yeah, for sure. Yeah, there's a lot of things that we do to lean into that domain expert persona. We do have a human labeling component as a part of our platform. And we do have an agent and prompt playground where people can experiment with their own prompts and send them to the underlying agents themselves.

Yeah.

When you say how to, what is the system of way that you keep the evaluator up to date? And then also the system up to date and understand when you make, I guess, the error analysis, if the error is based on the evaluator or the systems.

The idea is that we gather data from production to continually add to that offline data set that we're evaluating upon. And then hopefully we're gaining grounded data along the way where we can kind of self-check ourselves to understand if our evals are starting to align more to human agreement or not, or if they're diverging.

Yeah. OK, everyone, that's my time. I really appreciate the attention today. If there are any more questions, find me downstairs.