AI Engineering at Jane Street - John Crepezzi
Mar 28, 2025 · 16:57
John Crepezzi, an engineer on Jane Street's AI Assistants team, explains how they built custom LLM-powered coding assistants for OCaml, a functional language with scarce public training data. To overcome off-the-shelf tool limitations, they trained models on data from workspace snapshots — automated captures of developer workstations every 20 seconds, tracking build status changes from red to green. They used a Code Evaluation Service (CES) for reinforcement learning and evaluation, pre-warming builds to quickly test if model-generated diffs compile and pass tests. Their sidecar architecture, AID, serves thin editor integrations for Emacs (used by 67% of the firm), VS Code, and Neovim, allowing easy model swapping and A/B testing (e.g., sending 50% of users to different models). The talk details the end-to-end process: data collection from features, commits, and snapshots; training with supervised data and RL; and building pluggable infrastructure for domain-specific tools.
