Practical tactics to build reliable AI apps — Dmitry Kuchin, Multinear
Aug 3, 2025 · 14:55
Dmitry Kuchin argues that building reliable AI applications requires reverse-engineering evaluation metrics from real-world user scenarios and business outcomes, rather than relying on abstract data science metrics. He recommends using a strong LLM like o3 to generate specific user questions and criteria from knowledge materials, and building these evaluations at the start of development, not the end. This approach lets teams run frequent experiments, catch regressions, and iterate until they have a baseline benchmark. Examples include customer support bots (where the key metric is escalation rate to humans), text-to-SQL (using mock databases), and call center classifiers. Kuchin emphasizes that the process works with any platform and shares his open-source tool Multineer as one option.
