Google Colab

Training an LLM from Scratch, Locally — Angelos Perivolaropoulos, ElevenLabs

May 4, 2026 · 1:21:26

Angelos Perivolaropoulos from ElevenLabs walks through building a small GPT-2-like LLM from scratch on a local machine, demonstrating that the core techniques used by major labs are accessible in a few hundred lines of PyTorch code. The workshop uses character-level tokenization (65 tokens) on a Shakespeare dataset to enable fast training with limited compute. The model architecture includes multi-head self-attention, MLP layers, residual connections, and layer normalization, totaling 10 million parameters across six transformer blocks with a 256-token context window. The training loop employs next-token prediction with a warm-up cosine decay learning rate schedule and validation loss to detect overfitting. Inference uses temperature sampling (default 0.7) and top-k sampling to improve creativity. Perivolaropoulos explains that audio and multimodal models share the same transformer foundation but differ in tokenization (e.g., mel-spectrograms for audio) and use specialized losses like L2 or KL divergence, while reasoning models result from post-training base models with high-quality chain-of-thought data.

[Full Workshop] Building Metrics that actually work — David Karam, Pi Labs (fmr Google Search)

Jul 29, 2025 · 40:28

Achim and David Karam, former Google Search leaders now at Pi Labs, argue that building reliable AI evaluations requires a multi-signal scoring system rather than single LLM-as-judge metrics. They walk through designing custom metrics starting with simple correlated signals, using a copilot to generate and iterate on dimensions like 'includes insights' or 'title length,' then calibrating them against ground truth thumbs up/down data from users. The workshop demonstrates applying this to a meeting summarizer, integrating the scoring system into Google Sheets to score 120 examples and produce a confusion matrix showing alignment with human feedback. They also show how to use the same scoring system for model comparison (e.g., GPT-1.5 vs 2.5) and online best-of-N sampling, which boosts response quality by generating multiple responses and selecting the highest-scored one.

The RAG Stack We Landed On After 37 Fails - Jonathan Fernandes

Jun 3, 2025 · 18:52

Jonathan Fernandes, independent AI engineer, details the RAG stack his team settled on after 37 failed attempts, covering orchestration (LlamaIndex), embeddings (BAAI BGE small), vector database (Qdrant), LLMs (GPT-4, Qwen, Llama), reranking (Cohere), monitoring (Arize Phoenix), and evaluation (RAGAS). He demonstrates a live prototype in Google Colab using a London railway knowledge base, showing how a naive RAG returns irrelevant results (e.g., suggesting black cabs for "where can I get help at the station"). By swapping components—replacing in-memory storage with Qdrant, using an open-source embedding model, upgrading to GPT-4, and adding Cohere reranking—the answer improves to "go to booth number five next to the Eurostar ticket gates." For production, he deploys via Docker Compose with NVIDIA embedding/reranking models and Ollama for serving. The episode also stresses the importance of tracing latency per component and using RAGAS for systematic evaluation across many queries.

LLM Quality Optimization Bootcamp: Thierry Moreau and Pedro Torruella

Feb 8, 2025 · 53:05

Thierry Moreau of OctoAI demonstrates how to fine-tune Llama 3 8B on a PII redaction task using OpenPipe and OctoAI, achieving 47% better accuracy and a 200x cost reduction (from $30 to $0.15 per million tokens) compared to GPT-4 Turbo. He explains that fine-tuning should follow prompt engineering and RAG, and works best for specialized tasks like function calling. The talk walks through building a fine-tuning dataset from the PI Masking 200k dataset, using OpenPipe to train a LoRA for $40, deploying it on OctoAI, and evaluating it to show the fine-tuned model scores 0.97 accuracy versus GPT-4’s 0.68. Moreau emphasizes that this continuous deployment cycle requires monitoring data drift and retraining, but tools like OpenPipe and OctoAI make it accessible even for teams without deep ML expertise.

Knowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications: Zach Blumenthal

Nov 1, 2024 · 1:39:52

In this workshop, Zach Blumenthal of Neo4j demonstrates how to build a GraphRAG application using Neo4j, OpenAI, and LangChain on an H&M fashion dataset. He argues that combining knowledge graphs with vector search and graph embeddings improves retrieval and personalization for LLMs. The session covers creating a Neo4j sandbox, loading data, performing vector search with OpenAI embeddings, adding collaborative filtering via graph traversal patterns, using graph data science (FastRP) to generate graph embeddings for recommendations, and integrating everything into a LangChain chain that generates personalized marketing emails. Attendees build a Gradio app that, given a customer ID and season, outputs a tailored email with product recommendations, demonstrating how structured graph data enhances RAG systems beyond pure vector search.

Build an AI Research Agent: Apoorva Joshi

Oct 25, 2024 · 27:33

In this workshop, Apoorva Joshi, an AI Developer Advocate at MongoDB, teaches how to build an AI research agent using MongoDB as the memory provider and knowledge store, open-source LLMs from Fireworks AI (Fire Function V1) as the agent’s brain, and LangChain to orchestrate the workflow. The agent searches for research papers, summarizes them, and answers questions based on past research, using tools like ArXivLoader and a MongoDB vector store. Joshi explains key agent concepts—planning with chain of thought and react patterns, short-term and long-term memory, and tool creation—then guides attendees through hands-on sections to build the agent step by step. Attendees learn to create agent tools, implement reasoning with react, and add short-term memory persisted to MongoDB. The workshop emphasizes that agents enable complex, multi-step tasks through iterative reasoning, tool use, and memory, trading higher cost and latency for improved accuracy.

Domain adaptation and fine-tuning for domain-specific LLMs: Abi Aryan

Nov 14, 2023 · 25:09

Abi Aryan's talk covers domain adaptation and fine-tuning for large language models, contrasting prompting, RAGs, and three fine-tuning methods: adaptive, behavioral, and parameter-efficient. Adaptive fine-tuning adds small adapter modules (0.15% of parameters) for new domains like biochemical engineering; behavioral fine-tuning optimizes label space for a single task; and parameter-efficient methods like LoRA and QLoRA reduce model size via low-rank adaptation and four-bit precision, ideal for low-resource devices. Aryan emphasizes data quality—deduplication reduces memorization—and practical tips: batch size of 32 or 64, starting with 100 epochs, using Adam optimizer, gradient checkpointing for memory savings, and in-context learning with dynamic examples to handle drift. Evaluation should combine metric-based (Bleu, Rouge), tool-based (Weights & Biases), model-based, and human-in-the-loop approaches, though full pipeline considerations (data collection, base model choice, storage) are critical for robust applications.