This video was edited with AI agent. But how?
Feb 22, 2025 · 5:00
Muhteşem from Re-Skill presents the world's first open-source video editing agent, built in collaboration with Diffusion Studio, which uses LLM-generated code to edit videos via a browser-based UI. The agent starts a Playwright browser session and connects to operator.diffusion.studio, a web app built with Diffusion Studio Core that renders video directly in the browser using WebCodecs. Three tools drive the process: VideoEditingTool generates and runs JavaScript code based on user prompts, DocsSearchTool uses RAG to pull relevant information from llms.txt, and VisualFeedbackTool samples one frame per second and decides whether to proceed with rendering or refine further. File transfers between the Python agent and browser happen via Chrome DevTools Protocol, and for scalability, the agent can connect to a remote GPU-accelerated browser through WebSocket. The project emerged from Re-Skill's need for automatic video editing for personalized learning, where they found FFmpeg too limiting and turned to Diffusion Studio Core for its intuitive API and reliable in-browser rendering.
