BotDojo Launch: Enhancing AI Assistants with Evaluations and Synthetic Data
Feb 5, 2025 · 5:47
BotDojo founder Paul Henry demonstrates how his platform uses synthetic data generation and evaluations to productionize AI assistants. He shows a chatbot template with a low-code editor, node tracing, and JSON schema support. By running batch evaluations, he identifies knowledge gaps in the vector database. Then, he generates synthetic test data from live support tickets by extracting question-answer pairs and writing new documents to the index. Running the evaluations again, the scores improve from having red failures to all green, measurably boosting the chatbot's performance. Henry stresses that while hooking up a vector database to an LLM is easy, getting it production-ready requires systematic evaluation and data augmentation.
