Z.ai GLM 4.6: What We Learned From 100 Million Open Source Downloads — Yuxuan Zhang, Z.ai

Hello everyone, I'm Zhang Yuxuan from zAI, and I'm very happy to be here to talk about our latest model series, the GLM 4.6 series. Um, let's jumpright in.

First, I'll introduce the GLM series model. GLM 4.6 is not our first open-source model since 2022; starting from the very first GLM 130b, we've been quite serious about open-source our work. Over the years, we've released a whole family of models, such as CHATGMT-6B4 language model and the CogVideo for reasoning and understanding, CogVideo for image generation, and the CogVideo for video generation, and many more across different domains.

On the slide, you can see a map of our open-source models so far. It includes different code, such as white for the language model, the GLM series, and pink for the multimodal understanding, such as the CogVideo and now it's called GLM V.

The green one is for image generation, and the yellow is for video generation.

2025 is our open-source year of, and in this year we added even more models, including the GLM 4.0414 model including LangChain 9B and 32B, and the GLM 4.5, GLM 4.6, and the MOE series model, which is actually our first MOE models family.

So up to now, we have released over 65 models in total, and the closed platform like Hugging Face, ModelScope, and others we have already passed 100 million downloads. If you search for the GLM or CogVideo on GitHub, you'll find 1,500 community projects including Top of Them, and it's much a community-driven ecosystem now.

Let's move to GLM 4.6, which I'll introduce now. So, GLM 4.6 is our latest flagship model. On many public benchmarks, especially in math and coding, GLM 4.6 shows the clear game over GLM 4.5. It also outperformed open-source models released in the same period, like DeepSeek v3.2, and even beat commercial models such as the Claude 3.4 on several benchmarks.

Of course, if we compare it to, like, CogCode 4.5, there's still a noticeable gap. So we're not claiming we've blocked everything, but we're getting closer and closer.

But what makes us especially happy is here is a renowned list benchmark,

which is closer to real user preference. On LMS Arena, GLM 4.6 is trying for number 1 together with GPT-5 and the Claude 3.5, and it's the only open-source model here. And so, I'm very happy, I appreciate it, and I want to thank all our developers who tried our model and voted for it.

So let's move to the CSE bench.

So, besides the usual benchmarks, we also built our own dataset called the CSE bench here. We wanted text agent style coding in a realistic world, not just isolated legal problems. So we built an agent coding test platform based on the Claude code, and on the top of that we created CSE bench version 1.1.

So compared with version 1.1, the new version added 22 hardware coding tasks, and we systematically evaluated CogCode Sonnet, uh, Claude Sonnet 4 and GLM 4.5, like Kimi K2 and the DeepSeek version 3.1 terminus. In total, CSE bench had 74 tasks, covering the front-end development and internal tool development in the data analysis, and also algorithmic implementation.

So for every model, we record the full agent trajectory, the planning stamp, the tool calls, and code edits, and execution. It's fully open-source this benchmark, so you can check all the links below in the Hugging Face. And GLM 4.6 made a clear jump over GLM 4.5, and over- overperformance is close to Claude Sonnet 4 with about 68.6% win rates, while being significantly better than other open-source benchmarks.

So where does the performance come from? A lot of it is, let's talk about GLM 4.6 training. And in this case, we will start from the data and training design. First part is the general pre-training. So we start with about 15 billion, uh, 15 trillion tokens of the general repo poster data.

It includes webpage, books, Wikipedia, and then multilingual, uh, multilingual content, and so on. So this stage is all about building a strong all-rounded best model. The content then here is 4,000 tokens, and the next step is coding the reasoning and continual pre-training.

So on top of that base, we have about 7 trillion tokens of extra code in the reasoning data. So it's part of this co part of this con comes from the high-quality open-source reports, and another part is math, science, and context problems with full state-by-state reasoning.

Then we come to the mid-training. So we move to repo-level code. It includes multiple files, issues, and pull requests, and the difference from the same project. All of this packed into one long context, and the goal is to teach the model to follow the closed file and understand the chains, and to also understand the previous questions, and read the real project structure end to end.

So at this stage, we extend the content to 32,000, and the model can basically see the key file of a medium-sized repo on one shot. Then there's the synthetic reasoning data. We added about 500 billion tokens of synthetic reasoning data.

So it covers math, science, and algorithms with expedited thinking traits. So it's mainly the lay of the groundwork of future agent behavior, like breaking down a task, refre- uh, reflecting on a mistake, and doing long-chain reasoning. The next step is the long context and agent data.

Finally, we use about 100 billion tokens of long context and agent data here. The secret then is now pushed further to 180, uh, 20, uh, 128,000. For GLM 4.6, it's 200,000. So the model can handle full documents, the whole data, uh, code base, and the very long charts.

At the same time, we feed a lot of agent trajectories. So it includes, like, multi-step tool calls, the search, and the code execution, that extra. So in this phase, it improves the model's long-context capability and the agent capability.

Also in this slide, we introduce SLIME. It's our reinforcement learning framework based on SGLang inference stack. In practice, we design an enhanced RL training framework here, and we also open-source it. We found that a different RL task needs very different system design.

For short reasoning tasks, like the math or the code complexion, so the best setup is Claude 3.0 with the average culture. So we train the inference chain in the same GPU, so that the one batch updates the weight, so the next batch immediately samples from the latest policy.

That screens the most of GPU memory and computes. But for agent tasks, for example, the real software engineering,

usually have many steps. Like, for example, open a browser and hit the backend API, and for external response, extra. So if we force every worker to stay in the same list, the whole system will get dragged down by the slowest field task and the GPU is still idle.

So in SLIME, we designed a hybrid architecture to support both

decorators and secrets models. If you look at the diagram, the blue part is magic batch training engine, weight read for data buffer, and update weights. And the green part is high-through-out SGLang interference cluster. So it's a real-time finder for this path request, and in the middle, the data buffer acts like a shared nervous system.

So one side connects training, and the other side is friendly agent environments. For regular reinforcement learning tasks, we keep training and inference on the same GPU pool, using simplest mode: dynamic sampling, instant update, and maximum throughput. Once we switch to complex agent tasks, we move to a decodable and synchronous mode.

So the raw outside points directly to real environments and just generates trajectory and writes them into the buffer, and then the training side consumes the data in its own space up to the math model, and periodically pushes new weights back to the raw worker.

So the nice thing is, even if some tasks are super slow, they don't block the whole training pipeline. So on top of that, we have done a bunch of efficiently optimization, like the main train still runs before 16-step stability, but after each policy update, we do block-wise FDA synchronization on the latest weights and send the FDA version to the raw worker.

So. The most expensive part, the data generation and running FDA, which is much higher through output, while training itself keeps BF16 precision. So in practice, we will get the benefit of both accuracy and speed in this framework.

Now let's zoom in reasoning RL, and the next slide with some clauses. So the first one is about the two-stage curriculum we use. We don't train on a fixed dataset from start to finish. Instead, we use a two-stage difficulty curriculum.

In stage 1, we use medium difficulty problems. In each batch, some answers areright and some are wrong, so the rewards have variance and the gradient are meaningful. When the model gets stronger, we switch to extremely hard problems in stage 2.

But with 512 samples, we can still occasionally get a correct solution. So you can see on the plots, the blue curve is our method. After switching to the hard problems, the curve keeps going up. However, use the medium difficulty way is not on the red curve.

The next picture is about the single-stage reinforcement learning at 64,000 tokens. Some previous works suggest multi-stage reasoning reinforcement learning. The first is, like, for example, it's 15, then 32, then 48, and finally 64. But we found that for a model that has already been trained with 64,000 token SFT, those shorter RL strings actually make it forget its long-context ability.

So average output, latest inks, and the final 64k token stage can't fully recover the loss. So the red curve here is our approach. We start directly with 64,000 tokens and train in one single state to the end. It's clearly outperformed than the blue middle multi-stage curve.

The picture below is about, like, code RL. So on the left bottom plot, we compare two ways of commuting the loss for code RL. So the blue one is classic sequence means loss. Each sequence has one loss value, and the red one is our token weights means loss, which average over token instead of sequence.

The token weights version converts faster and more steadily, and it reduces the chances to generate very short template answers just to get the rewards. On theright, you can see the data. We do get a science reinforcement learning on GPQ layer damage, and the message is almost opposite of "more data is better."

So the red curve is trained to only the small set of expert-verified, but high-quality multiple-choice questions. And the blue curve uses mixed-quality data. So this result, like a small, flat, clean dataset, gives much better performance. So for scientific reasoning, data quality really matters more than raw size.

After talking about GLM 4.6 language model, we move to the multi-model.

GLM 4.5 verse supports both image and video understanding. It is our latest visual understanding model. And on grounding and the image understanding benchmark, it shows strong performance and clear advantages over the other open-source model releases around the same time.

So aggregately, you have the three main parts here.

The one is the version transforming code platforms, and then it's aligned with MLP projector, and the final is GLM 4.5 base model and decoded. So we try hard to keep the virtual input as original as possible. So the model can see the image's native resolution and the asset ratio.

Instead of focusing everything into a fixed square. So this matters a lot of UI screenshots, and also long vertical images and the PowerPoint slides. So for the video, we in so we insert a tiny index token after each time.

Basically, tell the model, "This is the first C, and this is the second C," and let it help it understand the temporal order and the variance. Which is crucial for actual understanding and step-by-step reproduction. We also use our method as we researched before in CogAgent.

Now the GUI agent capabilities also support GLM 4.5 re. So it can, like, it can also help you to control the computer and also, like, the website to control. You can use the mouse or the keyboard touching and to communicate with your browser, also computer, or mobile environments.

So how to use GLM 4.6 or GLM 4.5 re model? The first one is using an open-source weight. As we know, both these two are open-source, so you can use the echelon or VMs or other frameworks to inference it.

Along with the weights, on the release day, we already had SGLang and the vLLM integrate ready. And we also work with various third-party open-source frameworks like Llama Factory or MSW. So thank you to this community. There you have, you can choose any framework you want and to try our model.

But the GLM 4.6 model is a large model with, like, more than 355 billion parameters. So if you don't have that edge 100 or, like, another GPU, there's an easier way to use our model. So in this slide, we show the deploy commands of using SGLang or the vLLM here.

The next slide, we can use the GLM on the z.ai. This is a website, and you can try GLM directly. And you can use it writing code, you can use it to generate PowerPoints, and other so on. And in this demo, it's using one command to write the Google searching in our zr.com demo.

So you can just communicate with it. And also, GLM is famous in coding capability. So we also provide, like, GLM coding plans, which connect GLM with tools and other plugins, like Cloud Code or other coding developed tools, and to provide a very strong coding assistant experience.

We also have a short demo video that shows how to replace a Yodi model in a Cloud Code-like setup with GLM 4.6 here. You can see the you can watch this on YouTube.

Then is our community activity.

Beyond today's talk, we regularly host events, both online and offline. So whenever we release a new model, we usually run a server community session afterwards. It's the first one. There is an AMA in Reddit, and we also have some we also have some offline and onsite

technology sharing, so you can join us.

The final slide is some important links you may need to know. It's about website, as I mentioned before, to try GLM model at z.ai, and also our API on here. Then we also provide GLM 4.6 technical board and GLM 4.5 tech reports.

You can check it. And if you want to join our community, here is the Discord link. And also, the GitHub link is below with the open-source model, including the README to how to deploy on the open-source method. That's all of today.

Thank you very much.