The 100-Tool Agent Is a Trap - Sohail Shaikh & Ankush Rastogi, Prosodica
Jun 28, 2026 · 28:27
Sohail Shaikh and Ankush Rastogi of Prosodica argue that the common 'Fat Agent' architecture—loading every tool schema into every prompt—causes latency, cost, and accuracy collapses as tool catalogs grow. Their benchmarks show tool selection accuracy dropping from 78% at 10 tools to 13.6% at 741 tools, while their Semantic Tool Router pattern stays above 83% across the same range. The router uses Just-in-Time Context Injection, embedding tool descriptions offline, retrieving 3–5 relevant tools per query at runtime, and injecting only those schemas into the model call. This reduces input tokens by up to 99% (from 127k to ~1k tokens) and cuts time-to-first-token latency by up to 90% in high-tool-density scenarios. They present a production checklist and emphasize that routing pays off once beyond 50 tools, with K=5 as a strong default, but warn of router misses and weak descriptions needing fallbacks and monitoring.
