Fireworks

Fast Models Need Slow Developers — Sarah Chieng, Cerebras

May 22, 2026 · 18:02

Sarah Chieng from Cerebras argues that the 20x speed increase of models like Codex Spark (1,200 tokens/sec vs. 40-60 for Sonnet/Opus) forces developers to rethink workflows, or risk generating technical debt at unprecedented scale. She presents a practical playbook: validation and linting become free at every step, so must run continuously; developers can generate 75 component variations across five sub-agents and cherry-pick the best; and with context filling in 30 seconds instead of ten minutes, a four-file external memory system (agents, plan, progress, verify) maintains continuity between sessions. Chieng emphasizes real-time collaboration with the model rather than spawning agents and walking away, and advocates using a slower planner model with a fast executor to orchestrate agents effectively.

How fast are LLM inference engines anyway? — Charles Frye, Modal

Jun 27, 2025 · 16:07

Charles Frye presents benchmarks from hundreds of runs on Modal comparing open-source inference engines VLM, SGLang, and TensorRTLM across models like Qwen 3 and Gemma 27B, arguing open weights models have caught up to proprietary ones, making self-hosting viable. He shows, for example, that Qwen 3 (MoE) on VLM achieves ~1 request/sec with 128 input tokens and 1024 output tokens, while switching to a RAG-like workload (1024 in, 128 out) yields a 4x throughput improvement. Frye warns that optimizing for context over reasoning can improve latency without sacrificing quality, and notes that the engines' out-of-the-box performance varies by model—e.g., SGLang underperforms VLM on Gemma due to less optimization. He also highlights the gap between prefill (parallel) and decode (autoregressive) speeds, which a rationalist would expect from transformer architecture. The benchmarks, available at modal.com/llmalmanac, aim to help engineers choose hardware and engines, with contributions welcome for optimized configs like TensorRTLM's knobs.

Customized, production ready inference with open source models: Dmytro (Dima) Dzhulgakov

Feb 16, 2025 · 18:55

Dmytro (Dima) Dzhulgakov, co-founder and CTO of Fireworks AI, argues that open source models are the future for production Gen AI applications, and Fireworks provides a platform to make them customized and production-ready. He explains that while proprietary models like GPT-4 are powerful, they are too large, expensive, and slow for many use cases, whereas open models like Llama or Gemma can be fine-tuned for specific domains to achieve better quality at up to 10x speed and lower cost. The main challenges of using open models—complex setup, performance tuning, and production scaling—are addressed by Fireworks' custom serving stack, which achieves the fastest long-prompt inference and serves SDXL fastest among providers, handling over 150 billion tokens per day. The platform supports fine-tuning and serving thousands of LoRA adapters on the same GPU with serverless pay-per-token pricing. Dzhulgakov also highlights the emerging architecture of compound AI systems, where function calling—exemplified by the open-source Fire Function model—connects LLMs to external tools and knowledge sources, enabling agentic applications like stock querying and chart generation. The episode…

The AI emperor has no DAUs why most devs still don't use code AI: Quinn Slack

Nov 20, 2024 · 18:45

Quinn Slack, CEO and cofounder of Sourcegraph, argues that despite massive hype, only about 5% of professional developers actually use Code AI tools, with total recurring revenue from Code AI sitting at roughly $300 million ARR—a fraction of Salesforce's $36 billion. He cites GitHub's 1.3 million paid Copilot subscribers and just 935,000 yearly active users receiving suggestions, revealing the gap between perception and reality. Slack warns that the entire AI ecosystem—foundation models, infra, and applications—risks collapse if usage doesn't grow, and most revenue in AI flows to NVIDIA and chip makers, not software. From building Cody, the number two Code AI product, he shares lessons: hype fools everyone, autocomplete is a freakishly good feature that spoils expectations, while chat and agents are harder to vet and adopt. He advises builders to use their own product daily, ignore customer demands for buzzwords like fine-tuning, and manually build explicit interactions before adding magic. Slack concludes that the industry must collectively dehype and focus on real daily active users to turn the potential into sustained enterprise revenue.

Build an AI Research Agent: Apoorva Joshi

Oct 25, 2024 · 27:33

In this workshop, Apoorva Joshi, an AI Developer Advocate at MongoDB, teaches how to build an AI research agent using MongoDB as the memory provider and knowledge store, open-source LLMs from Fireworks AI (Fire Function V1) as the agent’s brain, and LangChain to orchestrate the workflow. The agent searches for research papers, summarizes them, and answers questions based on past research, using tools like ArXivLoader and a MongoDB vector store. Joshi explains key agent concepts—planning with chain of thought and react patterns, short-term and long-term memory, and tool creation—then guides attendees through hands-on sections to build the agent step by step. Attendees learn to create agent tools, implement reasoning with react, and add short-term memory persisted to MongoDB. The workshop emphasizes that agents enable complex, multi-step tasks through iterative reasoning, tool use, and memory, trading higher cost and latency for improved accuracy.

Architecting and Testing Controllable Agents: Lance Martin

Oct 11, 2024 · 2:21:54

Lance Martin presents LangGraph, a graph-based framework for building controllable agents that trade some open-ended flexibility for significantly higher reliability compared to classic React agents, achieving 100% consistent tool-calling trajectories even with an 8B local model. He demonstrates self-corrective RAG patterns like Corrective RAG, SelfRAG, and Adaptive RAG, where the agent grades retrieved documents, checks for hallucinations, and routes to web search when needed. Martin also covers three testing loops: in-app error handling with LangGraph, pre-production evaluation using LangSmith to compare agent answers and tool trajectories against ground-truth datasets, and production monitoring with online evaluators that flag retrieval quality, answer relevance, and hallucinations without reference answers. He shares results from a five-question evaluation showing LangGraph agents achieve 80% answer accuracy and 100% tool-trajectory correctness with Fire Function V2, while React agents with GPT-4o degrade in tool reliability. The talk addresses practical concerns like handling many tools (suggesting RAG for tool selection), multi-turn conversations, and the importance of…