Quen

Running LLMs locally: Practical LLM Performance on DGX Spark — Mozhgan Kabiri chimeh, NVIDIA

Apr 10, 2026 · 10:16

NVIDIA’s Mozhgan Kabiri Chimeh demonstrates that running LLMs locally on the DGX Spark workstation, powered by the GB10 Grace Blackwell superchip and 128GB unified memory, achieves practical performance for models up to 14B parameters. Using a reproducible vLLM benchmarking methodology, she shows that the 14B NVFP4 quantized model delivers 20.19 tokens per second and a time-to-first-token 3.4× faster than the unoptimized 14B base model. The DGX Spark supports the same NVIDIA AI software stack as production environments, enabling local development, fine-tuning, and privacy-sensitive workloads before scaling to the cloud. NVFP4 quantization is highlighted as critical for balancing intelligence and throughput on single-system setups.

Jack Morris: Stuffing Context is not Memory, Updating Weights is

Dec 29, 2025 · 1:02:44

Jack Morris argues that large language models fail at niche, long-tail knowledge tasks, such as optimizing AMD GPU kernels or answering private company queries, because they rely on context windows and RAG, which suffer from quadratic self-attention costs and context rot. He advocates for a third paradigm—training knowledge directly into model weights—using synthetic data generation (e.g., synthetic continued pretraining from Stanford) to expand small datasets and parameter-efficient methods like LoRA or memory layers to avoid catastrophic forgetting. Morris demonstrates that full fine-tuning on a 3M 10-K report causes the model to only regurgitate exact sentences, whereas generating diverse synthetic question-answer pairs enables better generalization. He notes that RL-based fine-tuning (e.g., GRPO) can achieve improvements with as few as 14 parameters, while memory layers offer the best trade-off between learning and forgetting. The episode also explores temporal information handling, federated learning resurgence, and the practical decision boundary between RAG and weight-based injection based on data freshness and volume.

The Future of Qwen: A Generalist Agent Model — Junyang Lin, Alibaba Qwen

Jun 3, 2025 · 25:14

Junyang Lin from Alibaba Qwen presents the latest developments in the Qwen model series, including Qwen3's hybrid thinking mode that combines thinking and non-thinking behaviors in a single model, supporting over 119 languages and dialects. The flagship 235B MOE model activates only 22B parameters and competes with top-tier models like O3-mini, while a 4B model deployable on mobile devices rivals Qwen 2.5 72B. Qwen3 introduces dynamic thinking budgets for inference-time scaling, achieving over 80% on AIME 2024 with 32K thinking tokens, and enhances agent capabilities with MCP support. The vision-language model Qwen2.5-VL excels in benchmarks like MMU and MathVista, and the omni model Qwen2.5-Omni accepts text, vision, and audio inputs while generating text and audio, achieving competitive vision-language performance. Future directions include scaling reinforcement learning with environment feedback, extending context to at least 1M tokens, and unifying understanding and generation across modalities, marking a shift from training models to training agents.