Weaviate

The Small Model Infrastructure Nobody Built (So We Did) — Filip Makraduli, Superlinked

May 5, 2026 · 18:30

Filip Makraduli of Superlinked introduces SAI, an open-source inference engine for small models that addresses gaps in embedding infrastructure by enabling dynamic model loading, hot-swapping, and memory-aware eviction on a single GPU. He argues that provisioning separate GPUs for each small model wastes idle capacity, and that the real challenge lies in supporting diverse model architectures (e.g., BERT, Qwen, Colbert) with different attention mechanisms and positional embeddings. The engine re-implements forward passes with variable-length FlashAttention and handles model swapping via a least recently used eviction policy. Makraduli also explains that context management for agents requires small models to pre-process data, referencing Andrej Karpathy’s graph-based knowledge bases and Chroma’s own model. The talk details the infrastructure layer including routing, auto-scaling with Prometheus, and GPU provisioning using spot instances, all open-sourced as SAI (Superlinked Inference Engine) with Helm charts and Docker images.

Graph Intelligence: Enhance Reasoning and Retrieval Using Graph Analytics - Alison & Andreas, Neo4j

Jun 27, 2025 · 1:41:16

Alison Cossette and Andreas Kollegger from Neo4j demonstrate how graph data science algorithms enhance GraphRAG systems by connecting, clustering, and curating unstructured data. They walk through running KNN similarity (K=25) on document embeddings, then applying Louvain community detection to identify clusters of near-identical chunks—one community had 49 documents with 0.98 average similarity—which can be collapsed via APOC to improve retrieval efficiency while preserving lineage. Using page rank and betweenness centrality, they show how to surface influential documents and track conversation paths across communities. The session uses a preloaded Neo4j AuraDB database with 17K nodes and 774K relationships from the Agent Neo project, and includes practical cipher queries for projecting graphs and running algorithms. They emphasize using graph analytics to diversify retrieval, manage content hygiene, and build accountable agent systems.