Sambanova

Scaling the Next Paradigm of Heterogeneous Intelligence — Adrian Bertagnoli, Callosum

May 24, 2026 · 15:13

Adrian Bertagnoli, founding engineer at Callosum, argues that the era of homogeneous intelligence—scaling single models on identical chips—is ending, and heterogeneous intelligence, which routes tasks to optimal chips and models, is the next paradigm. He demonstrates this with two case studies: on the Ulong benchmark, running recursive language models on Cerebras instead of GPT-5.2 cuts cost by 7x and latency by 5x while matching accuracy; on Video Web Arena, a mixture of Qwen 3 VL8B and Kimi K2.5 beats GPT-5.2 and Gemini 2.5 by 18% and 25%, while costing 3.7x less and running 3x faster. The key insight is that complex problems decompose into subtasks—like zooming on a webpage—which require different intelligence levels; offloading those to smaller models yields 11x speed and 43x cost improvements. Callosum builds an automation layer that predicts the best model and hardware for each subtask, and has secured a £3 million grant with the UK's Arya institute to operate the first heterogeneous co-located cluster.

[Full Workshop] Llama 3 at 1,000 tok/s on the SambaNova AI Platform

Feb 7, 2025 · 1:00:58

Michelle Matern and Petro Milan of SambaNova present their full-stack AI platform, built on the SN40L RDU chip with a three-tiered memory architecture capable of storing up to 5 trillion parameters. They demonstrate Samba1 Composition of Experts (CoE), a trillion-parameter model combining 92 expert models behind a single endpoint, and show Llama-3-8B achieving 1,000 tokens per second with a time-to-first-token of 0.09 seconds and total inference time of 0.65 seconds—far exceeding GPU-based providers. The workshop includes a hands-on basic inference call using LangChain and SambaStudio API, and a RAG-based Q&A system for enterprise search that integrates Unstructured for document loading, E5-large-v2 embeddings, ChromaDB vector store, and the high-speed Llama-3 endpoint. Attendees learn to configure prompts with special Llama-3 tags, set chunk size and overlap, and optionally run embeddings on SambaNova's RDU hardware for faster processing.

Build enterprise generative AI apps using Llama 3 at 1,000 tokens/s on the SambaNova AI platform

Sep 11, 2024 · 54:34

SambaNova’s Michelle Matern and Petro Milan present their full-stack AI platform, demonstrating how the SN40L RDU chip enables Llama 3 inference at 1,000 tokens per second. They introduce Samba-1, a composition of 92 expert models behind a single endpoint, and benchmark it against GPT-3.5 and GPT-4 on enterprise tasks like information extraction and text-to-SQL. The workshop then builds a RAG-based Q&A system using LangChain, Unstructured, E5-large-v2 embeddings, ChromaDB, and Llama-3-8B-Instruct at 1,000 tokens per second. Attendees set up the environment, load documents, and run inference with real-time metrics showing time to first token of 0.09 seconds and total inference time of 0.65 seconds.