Writer

When Vectors Break Down: Graph-Based RAG for Dense Enterprise Knowledge - Sam Julien, Writer

Jul 22, 2025 · 15:47

Sam Julien, Director of Developer Relations at Writer, explains how the company built a graph-based RAG system that achieved 86.31% accuracy on the RobustQA benchmark and sub-second response times, significantly outperforming vector search approaches for dense enterprise knowledge. The team moved from simple vector retrieval to a graph-based approach to handle concentrated data where similar terms appear frequently, solving problems like inaccurate chunking and failure with similar documents. They built a specialized model to convert data into graph structures, stored as JSON in a Lucene-based search engine, and incorporated Fusion-in-Decoder with knowledge graphs to lower hallucination rates. Key decisions included focusing on customer needs over hype, staying flexible based on team expertise, and letting research challenge assumptions, leading to features like multi-hop reasoning and complex data format handling.

Evaluating Domain Specific LLMs for Real World Finance — Waseem Alshikh, Writer

Apr 22, 2025 · 12:01

Writer CTO Waseem Alshikh presents FailSafe, a benchmark for evaluating LLMs in real-world finance, challenging the notion that general-purpose models suffice. Tests on query failures (misspelling, incomplete, out-of-domain) and context failures (missing context, OCR errors, irrelevant context) reveal that reasoning models like o1 and o3 perform 50% to 60% worse on context grounding than smaller domain-specific models, despite higher answerability. Even the best model achieves only 81% combined robustness and grounding, meaning roughly 20% of responses are wrong. Alshikh argues that domain-specific models remain essential, and that full-stack systems—RAG, guardrails, and grounding—are necessary for reliable deployment in high-stakes finance.