How to build Enterprise Aware Agents - Chau Tran, Glean

Thanks, Alex, for the introduction. That was a very impressive LLM-generated summary of me. I've never heard it before, but nice. So today I'm going to talk to you about something that has been keeping me up at night, probably some of you too.

So how to build enterprise-aware agents, how to bring the brilliance of AI into the messy, complex realities of how your business is operated. So let's jump straight to the hottest question of the month for AI builders: should I build workflows, or should I build agents?

So what are workflows? Workflows are systems where LLMs and tools are orchestrated through predefined code paths. So there are two main ways you can represent workflows. The first way is through imperative code base. So these are the workflows where you write a program that calls LLMs, read the response, and then call tools.

And so do this in a traditional programming flow. And then here you have direct control of the execution of all the steps. The second way to represent workflow is through declarative graphs. So in this way, you sort of represent your workflow as a graph where nodes are sort of like steps where you can call tools or call LLMs.

And then there's sort of an edge between nodes. So you kind of define the structure, but not the execution. And the execution of this is usually handled by some workflow frameworks. So I'm not going to go into the details of pros and cons for these two approaches.

But the main point here is, for workflows, you get structure and predictability. So if you run a workflow today, it will mostly behave the same way if you run it tomorrow.

On the other hand, we have agents, which are systems where LLMs sort of dynamically direct their own processes of deciding how to achieve a task, like decide what tools to call, what step to take, depends on the task itself.

So the core agent loop is pretty simple. So it receives a task or a goal from a human. And then it sort of enters this iterative loop where it plans what to do and then executes that action, and then reads the results from the environment and sort of iterates until it gets all the results.

It's one and then responds to the user.

So what are the trade-offs between workflows and agents? Workflows are sort of like the Toyota of AI systems. It's very predictable. It's good for when you want to automate repetitive tasks or encode existing best practices or know-how in your business.

It's usually lower cost and lower latency because you don't have to spend time on all these LLM calls to decide what to do. And they're also easier to debug because you have this code or this graph that you can manually pinpoint at which step is going wrong in the execution.

And in building workflows, humans are sort of in control. You can control your destiny. Even given imperfect LLMs, you can sort of do tweaks and engineering so that your tasks workright now. On the other hand, agents are sort of like the Tesla of AI systems.

It's more open-ended. It's good for researching unsolved problems. It's also usually good at taking advantage of better and better LLM capabilities because here the AI is in control. Generally, it's higher cost and latency because you need LLMs to figure out what to do.

But the upside is there's less logic to maintain. The call loop is very simple. And sometimes you get these hints of brilliance that always feels like everything is going to be automated in a few months. The problem is your Tesla, it works very well most of the time, but sometimes it still takes the wrong exit on the highway.

And that's when you kind of miss your Toyota.

And the decision to build workflows or agents is a pretty tricky one because it depends highly on the state of the LLM. So some workflows that don't work in the agentic loop now might start to work later in a few months when a new model has come out.

So it's a really huge dilemma.

But recently, one thought that's sort of really changed how I think about it is, what if you don't really have to choose? So if you think of agents, what they do is when you give the agent a task, it will figure out the steps that need to be done to achieve that task.

So you give it a task, it figures out the one step, takes the action, figures out the next step. And then at the end, when the agent finishes the execution, and then you look at the trace of what happened, all those series of steps is a workflow.

So if I represent this in a

programming kind of way, then the agent takes a task, and then generates a workflow to achieve that task.

So if we think of it this way, the agent takes a task and generates a workflow, then you can sort of see there are really good synergies between workflows and agents. So the first thing is you can actually use workflows as evaluation for your agents.

So let's say in your company, you can collect a huge amount of golden workflows. Given a task, this is the steps that need to be done to solve that task. And you have a huge list of those sort of handbooks on how to do things in your company.

Then you can actually evaluate your agents by giving it a task, seeing what it did, and comparing it to the golden workflow. Did it actually figure out theright steps? So this is a little bit different from evaluating end to end.

You are not judging agents by the end response, but by whether it actually did theright step to get to that end response.

The second and even better way for workflows to help agents is, given that same golden workflows library, you can also use it to train your agents.

So here you truly get the best of both worlds, where with the data feeding, your agents will be able to execute the exact workflow that you have in your library for the known task. But then

it can also rely on its own internal reasoning capabilities to sort of compose different workflows together to achieve new tasks and even use its own reasoning to kind of extend what you teach it, but make it better.

And then agents can also help workflows as well. One way to do that is for workflow-building platforms, you can use an agent to generate the workflows. So this is sort of how Glean agents work under the hood, where the user can give the workflow builder a sort of natural language description of the task it is trying to achieve.

And then we run an agent implementation to figure out the steps that are needed to achieve that workflow. Then the user can sort of make edits or add, change the workflow that the agent was proposing.

And lastly, and I think is the most powerful

synergy, is you can use agents as a workflow discovery engine. So you ship an agent. Users try to accomplish new tasks with your agent. And then when they find that the agent did a good job, then you can sort of save that workflow as like, OK, this is how you do this task in my company.

And then over time, you can use this as training data to help agents get better.

Cool. So that was the main points of my talk. I guess maybe some of you are thinking, do we still need this kind of stuff in a world where we have AGI? So here's my thought experiment and why I think this may be still needed after AGI.

So AGI is going to be a super intelligent employee. But if AGI doesn't know about how your company works, it's sort of like a really good employee who just joined and doesn't know about all the business practices and still needs onboarding, needs to know who to talk to to get unblocked, and all the very nuanced ways of doing things in the enterprise.

So what is enterprise-aware AGI? So enterprise-aware AGI is fully onboarded, very intelligent, knows the ways your company do things. And

one key kind of insight that I think is, there are many acceptable ways to achieve a task. But there's a gap between an acceptable output versus a great output. One example is competitor analysis. Sure, it can do some basic Google search and read some notes outside to do some competitor analysis.

But does it actually follow the protocols or the processes that your company defines? And does it actually address all the key metrics that your executives really care about? So

given all this data, tasks, and golden workflows, how do you actually train your agents using those data? So this is the second part of my talk.

So there are two main ways we have experimented with. The first one is through fine-tuning. There are sort of two main flavors of fine-tuning here. One is supervised fine-tuning, where you give an input and an expected output, and you train your model to just mimic that behavior.

The second way is through RLHF, where you don't have a golden label, but you sort of have a rating or a reward when you know this task, this workflow, is it a good one or is it a bad one?

So then you can sort of run your favorite optimization algorithms to fine-tune the LLM. So the pros of this method is that it can learn really well when you have a lot of data. If you have a huge amount of tasks and workflows, it can really learn, sort of generalize across different tasks and combine workflows.

The problem here is, one, you kind of have to create a fog from the front TLLM. So you start with some LLM, you do some fine-tuning. And then by the time the fine-tuning finishes, maybe there's a new and better model already come out, and you have to redo this whole process again.

And the second is any change to your training data, you need to do retraining. So if you have a new tool, then maybe some of the existing workflow is outdated, and you have to retrain. If you change some business priorities or business processes, then you have to redo the training again.

And it's also not super flexible for personalization. So given the same task, maybe different teams or different employees might actually have different optimal workflows to do those tasks. And fine-tuning is not super well-suited for those use cases. Then comes the second option, which is dynamic prompting through search.

So given the same label data from task to a golden workflow, you build a really good search engine for tasks so that you can find similar tasks given a new task. So then at runtime, to accomplish a new task, we'll find the most similar tasks in the training data.

And then you feed the representation of those workflows to the LLM as the examples. So here you really have a spectrum of determinism and creativity. So when there's no workflow that sort of matches your input task, then the LLMs are in control.

It can use its creativity to generate a new workflow. But when there's a high confidence match of something that you have done before, then the LLM will sort of give you a workflow that's very similar to what was in the training data.

So one very concrete example, come back to the competitor analysis example before. So you collected this huge list of tasks to workflow. And then when a new task, say, what competitor have we been running into recently, then it will retrieve how to analyze each competitor.

And then you will find a workflow on how to find your recent customer calls. And then the LLM will take those examples and sort of generate a composed workflow where it reads customer calls, reads internal messages, extracts competitors, and then runs analysis for each of them.

OK, so comparison time. Fine-tuning RLHF is very strong when you have a lot of data that you want to generalize. Dynamic prompting with search is more flexible, also gives you better interpretability that you can sort of look into the exact examples that were affecting your outputs.

And fine-tuning is good for learning generalized behaviors, where the ground truth labels don't change over time or across different users. Dynamic prompting with search is better for learning customized behaviors or the last-mile quality gap, where requirements are changing quickly.

One sort of analogy I think about fine-tuning versus dynamic prompting is fine-tuning is very similar to building customized hardware. So when you have a sort of task that you really want to optimize for and the requirements don't change over time, you can really build custom hardware that does it very well.

But it's sort of costly when you change your requirements. Compared to dynamic prompting, it's more like writing software, not as optimized, but you can just change them very quickly.

Last point, so how do we actually build this workflow search? So how do you give it a task, find similar tasks? I would say it's very similar to building document search. And there are two main components to this.

The first one is what everyone usually thinks of when they think of search, which is a textual similarity. Given this task, what are some of the similar-sounding tasks that are in the training data? And here the sort of

golden recipe is hybrid search within lexical vector embeddings, reranking, late interaction, all that. But what I found is in the enterprise settings, pure textual similarity is not enough. When you give users the choice to create workflows and write documents, when you want to search for something, there will be hundreds or thousands of similar-looking documents or workflows.

And the problem becomes, how do you choose theright one? So which is what I call as authoritativeness here. And to solve this problem, then you kind of have to go into knowledge graph. So if this workflow is created by someone who I work closely with, it has a high success rate, and people post about it on Slack, then it's more likely to be theright one.

So all the tricks in the recommended system

world also apply here for workflow search. And these kind of authoritativeness signals are very hard to encode directly into an LLM, which is why we sort of have to have a separate system that does the search for workflows.

Cool. So key takeaways: workflows good for determinism, humans are in control. Agents more open-ended, AI is in control. And the synergy between agents and workflows is workflows can be used for agents' evaluation. Workflows are used for agents' training.

And agents are used for workflows discovery. Fine-tuning is good for generalized behaviors. Dynamic prompting with search is good for personalized behaviors. Allright, I still have one minute and 30 seconds. Maybe time for one question.

Is there actually one question?

Yeah.

I'm kind of curious. On the fine-tuning end, I think just now you mentioned that we made it a lot of data. How about with the new RLVR? Can we set a reward model so we don't have to provide as much data for the fine-tuning process?

So the question was I'll try to reinterpret it. Let me know if it's wrong. How much data do we need to do fine-tuning given the new.

RLVR?

RLVR. That's a very difficult question to answer because it really depends on how out of distribution your task is compared to the internal knowledge of the LLM. But I'll catch you after, and we can talk more.

Yes.

Thank you.

We're good for all of the questions.