Replicate

Design like Karpathy is watching — Zeke Sikelianos, Replicate

Jul 19, 2025 · 19:26

Zeke Sikelianos of Replicate analyzes Andrej Karpathy's experience building and deploying MenuGen, a vibe-coded web app that turns menu photos into images, to argue that LLMs are now the primary audience for developer tools. He details Replicate's specific failures—rate limiting and outdated docs that blocked Karpathy—and the fixes they implemented: adding LLMs.txt files for Markdown-friendly documentation, promoting curl commands as the LLM-friendly interface, and launching an MCP server built on OpenAPI schemas. Sikelianos advocates for 'boring technology' like SQL and 'good API hygiene' to keep payloads small and information-dense for LLM context windows. He also calls for better payment acceptance and documentation discipline, stressing that unblocking power users like Karpathy (whose CEO intervened) should not require a viral blog post.

The Weekend AI Engineer: Hassan El Mghari

Nov 22, 2023 · 21:49

Hassan El Mghari shares how he built viral AI apps like RoomGPT and AI Commit, arguing that simple off-the-shelf APIs and focused weekend sprints can attract millions. He describes building 11 side projects in a year, which grew from 20,000 visitors to over 8.5 million unique visitors and 2.8 million sign-ups. Key projects include RoomGPT, which used ControlNet for room redesign and reached 6 million visitors, and AI Commit, an open-source CLI tool adopted by 30,000 developers. He emphasizes using tools like the Vercel AI SDK and v0.dev to accelerate development, making apps free and open source to drive growth, spending 80% of time on UI, and launching quickly with minimal fine-tuning.

[Workshop] AI Engineering 201: Inference

Nov 7, 2023 · 1:43:16

Charles Frye, instructor of the Full Stack LLM Bootcamp, leads a workshop on AI engineering inference, focusing on the build-versus-buy decision between proprietary and open models. He argues that proprietary models like OpenAI's GPT-4 and Anthropic's Claude are currently more capable but expensive, while open models like LLaMA 2 are less capable but offer hackability, though they may catch up if capabilities requirements saturate. Frye covers inference on end-user devices, noting that running models locally avoids network latency but faces tight memory and power constraints—e.g., a 7B parameter model requires 14 GB, too large for phone RAM. He explains inference-as-a-service (e.g., OpenAI, Replicate) versus self-serving on cloud GPUs or serverless platforms like Modal, highlighting that memory bandwidth is the bottleneck: GPUs have 1.5 TB/s memory bandwidth vs 312 TFLOPS compute, so batching is crucial for throughput. Frye also discusses inference arithmetic, custom silicon like TPUs which offer ~30% better efficiency but not drastic gains, and containerization challenges for GPU workloads.