Kimi K3 Open Day: Moonshot AI Puts the Weights, the Paper, and Its Infra on the Table

Everything at once: weights, paper, and toolchain

On July 27, Moonshot AI held an "open day" for Kimi K3. Instead of quietly dropping a model, the team put out three things at the same time: the model weights, a full technical report, and the key infrastructure code used to train it. The weights are already on Hugging Face, free to download; developers can build on them, curious users can poke at them, all within the license.

Not many Chinese labs open up this much of their kitchen, which is why a few people joked that Moonshot had turned into a saint. Jokes aside, the interesting part is what they actually opened up.

What kind of model K3 is

By Moonshot's own account, K3 is a Mixture-of-Experts (MoE) model, roughly three times the size of the previous K2, with a one-million-token context window and visual understanding that wasn't sacrificed for scale. The number worth flagging is training efficiency: the company says large-scale training is about 2.5x more efficient than the last generation, meaning the same compute buys you more model.

You don't need to memorize the specs. One line covers it: bigger, can read images, can swallow very long documents in one go, and this time it's open source.

What's actually worth reading in the report

A technical report usually says more about a company's thinking than the weights do. A few design choices in K3's report stand out:

KDA + AttnRes: they blend their own Kimi Delta Attention with Gated MLA at a 3:1 ratio, to cut the cost of long context without giving up accuracy.

Stable LatentMoE: each token routes through 896 experts, and with SiTU-GLU and Quantile Balancing the training stays stable at that scale. Instability is the thing that kills large training runs.

MoonViT-V2: the vision encoder was pre-trained from scratch with next-token prediction, which the team says beats initializing from SigLIP.

Post-training: the focus is on agents and coding, polished with millions of samples and around twenty internal eval sets.

The names sound intimidating; the gist is plain. They put work into four things: cheaper long context, steadier training, sharper vision, and getting more done.

The rarer move: open-sourcing the infra

Most companies stop at the weights and keep the training scaffolding locked away. This time Moonshot released three pieces of infrastructure too:

MoonEP: expert-parallel communication for very large MoE models, so shuttling data between experts doesn't drag the compute down.

FlashKDA: a high-performance kernel for Kimi Delta Attention. Moonshot's numbers put it 1.72–2.22x faster than mainstream flash-linear-attention on the H20, and it's a drop-in replacement.

AgentEnv: an agent training environment built with KVCache.ai, strongly isolated, supports parallel forks, and where K3's post-training gets its high-quality data.

All three repos are up. For any team trying to train its own model, that's far more useful than a lone weights file, because it explains part of how the thing was actually made.

Why this one is worth noting

Moonshot's stated reason is "standing on the shoulders of giants": they took from the open-source community and are choosing to give back. It sounds nice, but it also holds up in today's context. After DeepSeek, the contest among Chinese models is shifting from "who scores highest" toward "who's more open, whose ecosystem is more alive." Releasing weights, paper, and infra together lowers the bar again, letting more people start from the same line.

The CATAI take

We've always thought the endpoint of AI discovery isn't how many tools you can list; it's helping people find the capability that gets the job done. A move like K3's makes that capability easier to reach: you no longer have to lean on one vendor's API to wire a strong model into your own work. More openness means more choices, and that's always good news for users.

---

About this piece: figures on parameters and performance come from Moonshot AI's official announcement and the published technical report, which we've summarized and interpreted; for exact numbers, refer to the official release.