JEPA

How Transformers Finally Ate Vision – Isaac Robinson, Roboflow

May 8, 2026 · 17:05

Isaac Robinson, research lead at Roboflow, explains why transformers ultimately beat convolutional neural networks for vision, arguing that massive ViT-specific pretraining and borrowed infrastructure from LLMs overcame the transformer's lack of inductive bias. He traces the evolution from ViT and Swin (windowed attention reducing complexity to n²) through ConvNeXt (reintroducing convolution with transformer-style blocks) to Hiera (stripping biases and recovering them via MAE pretraining), and back to the simple, scalable ViT. Robinson highlights that pretraining methods like MAE and DINOv3 learn the inductive biases CNNs have built-in, while tools like FlashAttention from the LLM world nullify ViT's n⁴ compute scaling penalty. In practice, this pattern appears in the SAM model series: SAM used a ViT backbone, SAM2 switched to Hiera with MAE, and SAM3 returned to the simple ViT. However, these massive models lack deployment flexibility; Roboflow's RF-DETR uses neural architecture search on a foundation model backbone to generate a family of high-performance models, achieving up to 40× speedup over fine-tuning SAM3 while outperforming real-time convolutional detectors.

The Unbearable Lightness of Agent Optimization — Alberto Romero, Jointly

Nov 24, 2025 · 17:58

Alberto Romero, co-founder and CTO of Jointly, introduces Meta-ACE, a meta-optimization framework that orchestrates multiple adaptation strategies to overcome the limitations of single-dimensional context engineering like ACE. The framework uses a meta-controller to profile task complexity, uncertainty, verifiability, and resource constraints, then allocates strategies across context, compute, verification, memory, and parameter dimensions. Initial results show 8-11% improvement on agent benchmarks, 30-40% reduction in compute costs, and 6-8% gains on domain-specific tasks. Meta-ACE addresses ACE's weak reflector problem with quality gates and multi-signal reflection, feedback brittleness via a hierarchical verification cascade (self-verification, multimodal consensus, execution checks), and task complexity mismatch by dynamically adjusting strategy allocation to save up to 90% compute on simple tasks. Future work includes scaling to multimodal and compound AI systems, with challenges in meta-controller training stability, computational overhead, and verification cascade brittleness.