# How to Build Planning Agents without losing control - Yogendra Miraje, Factset AI Engineer · 2025-07-23 Yogendra Miraje from FactSet explains how to build controllable planning agents using agentic workflows and planning by subgoal division, arguing they balance reliability and flexibility for enterprises. He distinguishes workflow agents (static) from agentic workflows (dynamic), and introduces a blueprint generator that pre-selects tools and provides high-level natural language plans to reduce planner cognitive load. He details an architecture using LangGraph nodes (blueprint generator, planner, executor, joiner) and emphasizes designing tools from an agent's perspective with clear purpose, description, and contracts. Miraje stresses evals using code-based and LLM-as-judge techniques, and advises against agentic workflows for fixed tasks, strict compliance, or low-latency needs. The talk includes a concrete example of preparing for NVIDIA's earnings call, showing how blueprints enable structured responses and interpretability. ## Questions this episode answers ### How does using a 'blueprint' help maintain control over planning agents in AI? Yogendra Miraje introduces the concept of a 'blueprint' — a high-level plan in natural language that outlines the steps a workflow should follow. This blueprint reduces cognitive load on the planner by limiting the tools it needs to consider and providing a guide. Miraje found that adding a blueprint as a separate component in the architecture (inspired by the LLM Compiler approach) allows for finer control, easier collaboration with non-technical teams, and better interpretation of agent behavior, while still retaining flexibility. [8:01](https://aie.addtry.com/c988c03b-0b2e-4a3f-b1c8-68f3697fdf8e?t=481000) ### What is the difference between a workflow agent and an agentic workflow? Yogendra Miraje distinguishes these often-confused terms: a workflow agent is a predefined, static workflow executed by an agent, where the workflow is in control. An agentic workflow, by contrast, is a dynamic workflow planned and run by the agent itself, with the agent always in control. This means agentic workflows have higher 'agenticness' and autonomy, enabling them to plan steps based on goals and feedback, while workflow agents offer more reliability and predictability for fixed tasks. [2:32](https://aie.addtry.com/c988c03b-0b2e-4a3f-b1c8-68f3697fdf8e?t=152000) ### How should I design AI agent tools to work with existing enterprise microservices? Miraje advises that the relationship between tools and microservices is not one-to-one; you need to design tools from the agent's perspective, providing clear purpose, description, and input/output contracts. He recommends using standards like MCP to build tool servers once and reuse them across AI applications. Spend time thinking about what the agent needs to know to select and invoke the right tool, and add validation checks to act as a brake, ensuring the agent behaves correctly within the enterprise context. [6:37](https://aie.addtry.com/c988c03b-0b2e-4a3f-b1c8-68f3697fdf8e?t=397000) ## Key moments - **[0:00] Intro** - **[0:39] Control Problem** - [0:39] Yogendra Miraje: "When we develop AI applications, it feels like driving a monster truck through a crowded mall with tiny joysticks." - **[1:25] Definitions** - [1:25] Yogendra Miraje defines an augmented LLM as an LLM combined with tools and memory, a workflow as a static path, and an agent as an autonomous system with a feedback loop. - **[1:57] Agentic Workflows** - [2:19] Workflow agent runs a predefined static workflow; agentic workflow is planned and run dynamically by the agent, clarifies Yogendra Miraje. - [3:25] Enterprises can automate workflows at scale by building agentic workflows on top of existing microservices, says Yogendra Miraje. - [4:12] Yogendra Miraje: "We need to move on from React-based agent to proactive agents. By the way, great philosophy for life as well." - **[5:15] Architecture** - [5:15] FactSet adapted the LLM compiler architecture by adding a blueprint generator, planner, executor, and joiner on LangGraph, says Yogendra Miraje. - **[6:37] Tools** - **[8:01] Blueprint** - [8:01] A blueprint, a natural language breakdown of tasks, reduces the planner's cognitive load and limits the in-context tools passed to it, explains Yogendra Miraje. - **[9:34] Example** - [9:34] Yogendra Miraje demonstrates a blueprint for NVIDIA earning call preparation, breaking it into summarization, data retrieval, reasoning, and report generation. - **[10:55] Evals** - [10:55] Without proper evals, agentic workflows won't work; Yogendra Miraje stresses component, end-to-end, and aspect-based evaluation techniques like LLM-as-judge and code-based checks. - **[11:55] Limits** - **[12:33] Learnings** - [13:11] The six key takeaways from Yogendra Miraje: agentic workflows bring reliability at scale, planning by subgoal division, plan-and-execute architecture, leveraging microservices, adapting research, and treating evals as first-class. - **[14:02] Q&A** - [14:47] Q: What is the primary method of orchestration going forward? A: MCP for a standard across the organization, LangGraph for orchestration, and other frameworks per use case, says Yogendra Miraje. ## Speakers - **Yogendra Miraje** (guest) ## Topics Agent Engineering, Agent Evaluation ## Mentioned FactSet (company), LangChain (product), LangGraph (product), MCP (product) ## Transcript ### Intro **Yogendra Miraje** [0:15] Hi everyone. I'm Yogi, I work at FactSet, a financial data and software company, and today I'll be sharing some of my experience while building agents. In the last few years, we have seen tremendous growth in AI, and especially in the last couple of years, we are on an exponential curve of intelligence growth. And yet, it feels like when we develop AI applications, driving a monster truck through a crowded mall with tiny joysticks. So AI applications have not seen its ChatGPT moment yet. There are many reasons why agents don't behave, but probably one reason that strikes out is it misses theright context. ### Control Problem **Yogendra Miraje** [1:04] And in the case of enterprises, often it means that it does not have knowledge of enterprise-specific workflows. But before that, we will see some common context, and just like agents, humans also need a common context. So let's start with some key definitions. As you know, LLMs are limited by their knowledge at the time of training, so we enhance their functionality by ### Definitions **Yogendra Miraje** [1:35] increasing it by tool. And when you combine this LLM with tool and memory, we call it augmented LLM. When you place this augmented LLM on a static and predefined path, we call it a workflow. And if these augmented LLMs have high autonomy and feedback loop, we call it an agent. Now, workflows are controllable and reliable, while agents have flexibility and they are highly autonomous. So the question is, can we get the best of both worlds? So the answer is yes. With agentic workflows, we can plan and execute the workflows based on the goal, context, and feedback. ### Agentic Workflows **Yogendra Miraje** [2:19] I see these terms being used very loosely and at times interchangeably. So I would like to make a key distinction between workflow agent and agentic workflow. Workflow agent is a predefined workflow run by agent, while agentic workflow is a workflow planned and run by an agent. I know these terms are quite confusing, and in AI we are very bad at naming things. So if you are confused, don't worry. In the case of workflow agent, just remember that workflow is in control and workflow is static. In the case of agentic workflow, agent is always in control and the workflow is dynamic. It is also important to view these systems as agentic systems, as Andrew Ng pointed out correctly. On the agentic spectrum, agentic workflows have more agenticness than workflow agents, generally speaking. So why does all of this matter? Apart from control, reliability, and predictability, for enterprises, agentic workflows provide a way to automate the workflows at scale. And perhaps the most important thing is, enterprises can use their existing enterprise microservices to build on top of it. And in some cases, these enterprises have invested years, if not decades. So before diving deep, I would like to say that even though I'm speaking in terms of enterprise context here, the concepts are generally applicable. So where do we begin? In the last few years, the focus really has been on the React-based agent. And in building agentic workflow, we need to move on from React-based agent to proactive agents. By the way, great philosophy for life as well. So for building agentic workflows, you need tools, memory, and reflection. But more importantly, you will need a design pattern called planning by subgoal division, sometimes also referred to as task decomposition. And it is just a fancy way of saying that take your goal and break it down into simpler steps. Here are some specific agentic architecture and research papers that you will find useful, and each of that has its own pro and cons. And LangChain has done a fantastic job of creating a blog from this and also given the code. So I highly recommend checking it out. So how does it look in practice? So in FactSet, what we have done is we are taking this LLM compiler architecture and trying to adapt for our problems. And you can see some components here that you will also find in your organization, microservices, and you build tools around those microservices. ### Architecture **Yogendra Miraje** [5:40] And when a user question asks, it goes to blueprint generator, and I will get to that in a bit. But consider it as a high-level plan. What we call it is a blueprint that gets fed to planner. Planner is your low-level task. Planner, it gives the plan to the executor, and the executor is supposed to execute it. And joiner combines the outputs from different tasks. Based on your replanning logic, either you do replanning again or you just terminate and give the response back to the user. Sometimes you also set some recursion limits so that your agent just doesn't go into loop. On LangGraph, we are using each of these components as nodes. So blueprint generator, planner, executor, and joiner are all nodes on the LangGraph. ### Tools **Yogendra Miraje** [6:37] When building these tools in your enterprises around your microservices, probably this is where you will spend most of your time. And it's important to consider how this relationship between tools and microservices goes. And here, the relationship is definitely not one-to-one or end-to-end. It's end-to-end. It's up to you how you want to design your tools according to your microservices so that your agent knows how to use this tool. Perhaps this is the most key point here that you need to really put yourself into agents' shoes so that the agent really understands what tool to use and it has that knowledge of your microservices. Always follow standard. I know MCP is everyone's favorite, so build the MCP tool server for your tools. And for providing the tool details, just think from the agent's point of view that you need to provide a tool purpose, description, and input/output contracts. So tool purpose will help you what tools to be selected. Tool detail description will tell you when this tool needs to be invoked, and input/output contracts will tell you how to use this tool. And lastly, add some validation checks, which acts as a break for your agent. Now, I would like to a little bit zoom in into this blueprint because this is one of the key architecture changes that we made. Blueprint is just a series of steps for workflow as per tool capabilities in natural language. ### Blueprint **Yogendra Miraje** [8:15] And it gets fed to planner, but why are we doing it? So what we realized was planner really gets cognitively loaded when you try to just put too much onto it. So introducing a blueprint, which is just a natural language of breaking down of a task, is very helpful. But we also noticed that it brings a lot of other benefits as well. For example, it achieves finder control over task planning. It limits the in-context tool for the planner. So when blueprint, you can select what tools need to be given to the planner. And sometimes this planner has a lot of tool description, and you run all sorts of problems as context window limit and planner getting very much overloaded. So using blueprint, you can limit what tools really go to the planner, and thus it really helps in the planning. It also helps interpreting the agentic behavior. And lastly, when you need to collaborate with non-technical people, it's really helpful because natural language is less intimidating. ### Example **Yogendra Miraje** [9:34] Let's see a concrete example. So in financial research, preparing for a company's earning call is a common workflow. So this is a very, very simplified version of the workflow of preparing for a company's earning call. And for example, we are showing you preparing for NVIDIA's earning call. Now, you can see in the blueprint, there is a tool and there is a task. And in the plan, there is a tool and the function call. So how does it look in the blueprint is you have two tools, and then your first step is summarizing NVIDIA's previous earning call. And the next step is retrieval, gathering some of the financial data for NVIDIA, and then your reasoning, suggesting some questions for the earning call, and finally reporting, generate a comprehensive report from all the information. And there are corresponding function calls, and as you can see, context is being fed from task. A concrete example of the response is before you implement agentic workflow, the response is pretty much vanilla. But after this, it can easily capture your workflow and give a very structured response. So whatever we talked about, none of this will really work without writing proper evals. So always make sure to invest and build and maintain your eval framework. You should have at least component and end-to-end evals. You should really use the correct techniques, like code-based LLM as judge, human-in-the-loop, and more importantly, write evals for metrics that you really care for. ### Evals **Yogendra Miraje** [11:16] Aspect-based eval is something we should really think about. And for example, for blueprint, you can check an aspect like how many blueprints, whether it resembles a golden blueprint or not. And you can use LLM as a judge. If you want to see whether tools are selected correctly or not, you should leverage code-based evals. If you want to check whether a plan is in line with the blueprint or not, LLM as a judge is probably theright technique. And for some cases, leveraging human-in-the-loop is good because report formatting, that's the best approach to deal with report formatting. ### Limits **Yogendra Miraje** [11:55] So when not to use agentic workflows? So in some cases, definitely agentic workflow doesn't make sense. In the case of fixed and repeated tasks, just probably go for ETL pipelines. If your workflow cannot be really captured, you cannot really capture your use case in workflows. Agentic workflows are probably not worth it. And if deterministic outcome is paramount in cases of strict compliance and the safety-critical context, you should probably not go with agentic workflow. And in case of low latency and cost-constrained environment also, you should probably try to avoid agentic workflow. So wrapping up some learnings, start with simple blueprints. Work your way up building a complex RAG system for the blueprints. Use blueprint to reduce the in-context tools and provide the high-level plan to the planner. Design tools from an agent point of view. ### Learnings **Yogendra Miraje** [12:56] Always aim for the tool usage simplicity. Implement safety guardrails and evals, observability, and all the good software engineering that should help you a lot. And from the whole presentation, the key takeaways are agentic workflow is planned and run by agent. Agentic workflows bring the reliability at scale. And planning by subgoal division is a key design pattern. Plan and execute is a key agentic architecture. And build your tools to complement your microservices. Always try to leverage your microservices in the tools and modify your architecture to solve the problems. Don't really shy away from changing, taking research paper, and experimenting on it. And finally, treat your evals like first-class citizens. And with that, thank you very much for your time. ### Q&A **Host** [14:02] Allright. Thank you. Any questions? We have a little bit of time to spare. **Guest** [14:09] I have a question. **Host** [14:10] Sure. **Guest** [14:11] Do you have on top of your mind any GitHub project or reference that we can follow? **Host** [14:20] Sure, sure. So if you just go back here, I kind of shared some of the links for the LangChain. It should have all the code for these research papers. And that's probably the best place to start with this plan and execute kind of agents. **Guest** [14:43] Thank you. **Host** [14:43] Yeah. Any other questions? Allright. I guess one question I would have for you is when you talk about MCP and other forms of orchestration, what do you foresee being the primary method of orchestration going forward? Is it going to be LangGraph or some other? Yeah. I think the answer is probably like everything. MCP, you use it so that you provide a standard across the arc. And MCP will really help for organizations to build once, use it everywhere. You can have oftentimes in organizations, we see that people just trying to just use this functionality in different AI apps. But if you can build an MCP around it, you can keep using it. And obviously, for orchestration, LangGraph is great. And whatever the other tools that you find to solve your problems, that will be also. So the answer is probably there will be multiple things that are useful. It depends on your use case. What is the most optimal framework that you want to use? Amazing. Thank you so much, Yuri. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.