How to Build Your Own AI Data Center in 2025 — Paul Gilbert, Arista Networks
Apr 27, 2025 · 23:00
Paul Gilbert, tech lead at Arista Networks, explains the key considerations for building AI data center networks in 2025, emphasizing the stark differences from traditional enterprise networks. He details the backend GPU network (eight 400G ports per H100 server, no over-subscription), the front-end storage network (calmer, 100-200G), and the need for lossless Ethernet with ECN and PFC flow control to prevent packet drops during synchronized GPU bursts. Gilbert highlights power challenges (10.2kW per GPU server, requiring 100-200kW water-cooled racks) and the importance of telemetry like RDMA error monitoring and an AI agent that correlates GPU and network issues. He also covers advanced load balancing (cluster-aware, up to 93% utilization) and smart system upgrades without downtime, while noting the upcoming Ultra Ethernet Consortium (v1.0 in 2025) that shifts congestion control to NICs.
