Apps & Guides

What's new in Qwen 3

The global AI race is accelerating. Research institutions, private corporations, and even entire nations are now competing for leadership in the AI domain. Broadly speaking, this race can be divided into several phases. The first stage involved the creation of narrow AI. Existing neural network models such as GPT, MidJourney, and AlphaFold show that this stage has been successfully achieved.

The next step envisions the evolution of AI into AGI (Artificial General Intelligence). AGI should match human intelligence in solving a wide range of tasks, from writing stories and performing scientific calculations to understanding social situations and learning independently. As of the time of writing, this level has not been reached yet.

The ultimate stage in AI development is referred to as ASI (Artificial Super Intelligence). It would far exceed human capabilities in all areas. This would make it possible to develop technologies we can’t even imagine today and to manage global systems with a precision beyond human capabilities. However, this might only become a reality after decades (or even centuries) of continuous advancement.

As a result, most AI race participants are focused on reaching AGI while retaining control over it. The development of AGI is closely tied to a host of complex technical, ethical, and legal challenges. Still, the potential rewards far outweigh the costs, which is why corporations like Alibaba Group are investing heavily in this area.

The release of Qwen 3 marks a significant milestone not only for one company’s neural networks but also on the global stage. Compared to its predecessor, the model introduces several important innovations.

Features

Qwen 2.5 was pretrained on a dataset of 18B tokens, while the new model has doubled that amount to 36B tokens. The largest dataset has significantly improved the base model’s accuracy. Interestingly, in addition to publicly available internet data gathered through parsing, the system was also trained on PDF documents. These are typically well-structured and knowledge-dense, which helps the model provide more accurate answers and better understand complex formulations.

One of the most promising directions in AI development is building models capable of reasoning, which can expand the task context through an iterative process. On one hand, this allows for more comprehensive problem-solving, but on the other hand, reasoning tends to slow the process down considerably. Therefore, the developers of Qwen 3 have introduced two operational modes:

Thinking mode. The model builds up context step-by-step before providing a final answer. This makes it possible to tackle complex problems that require deep understanding.
Non-thinking mode. The model responds almost instantly but may produce more superficial answers without in-depth analysis.

This manual control over model behavior enhances user experience for handling many routine tasks. Reducing the use of thinking mode also significantly lowers GPU load, allowing more tokens to be processed within the same time frame.

In addition to this binary choice, there’s also a soft-switching mechanism. This hybrid behavior allows the model to adapt to context using internal weighting mechanisms. If the model deems a task difficult, it will automatically trigger reasoning or even self-verification. It can also respond to user cues such as “Let’s think step by step”.

Another significant improvement is expanded multilingual support. While Qwen 2.5 supported only 29 languages, version 3 can now understand and generate text in 119 languages and dialects. This has greatly improved instruction following and contextual comprehension. As a result, Qwen 3 can now be effectively used in non-English environments.

In addition, Qwen 3 is now significantly better integrated with MCP servers, giving the model tools to dive deeper into problem-solving and execute actions. It can now interact with external sources and manage complex processes directly.

Model training

Pre Training

Such a substantial leap forward wouldn’t have been possible without a multi-stage training system. Initially, the model was pretrained on 30B tokens with a 4K context length, allowing it to acquire general knowledge and basic language skills.

This was followed by a refinement stage using more scientific and well-structured data. During this stage, the model also gained the ability to effectively write applications in multiple programming languages.

Finally, it was trained on a high-quality dataset with extended context. As a result, Qwen 3 now supports an effective context length of 128K tokens, that’s roughly 350 pages of typed text, depending on the language. For instance, Cyrillic-based languages often have shorter tokens due to morphology and use of prefixes, suffixes, etc.

Reasoning Pipeline

Building reasoning-capable models is a fascinating but labor-intensive process that combines various existing techniques aimed at simulating human thought. Based on publicly available information, we can assume that Qwen 3’s reasoning training involved four main stages:

Cold start for long chains of thought. Training the model to break problems into multiple steps without prior adaptation. This helps it learn iterative thinking and develop a basic layer of reasoning skills.
Reinforcement learning based on reasoning. At this stage, rewards depend not only on the final answer but also on how well the model constructs logical, interpretable, and structured reasoning chains. The absence of errors and hallucinations is also evaluated.
Merging reasoning modes. Humans typically rely on two thinking styles: fast (intuitive) and slow (analytical). Depending on the task type, the neural model should learn to both switch between and integrate these styles. This is usually done using examples that mix both styles or through special tokens indicating which style to apply.
General reinforcement learning. This final stage resembles a sandbox environment where the model learns to interact with tools, perform multi-steps tasks, and develop adaptive behavior. Here, it also becomes attuned to user preferences.

Conclusion

Qwen 3 is a major milestone for Alibaba Group. Its training quality and methodology make it a serious contender against established players like OpenAI and Anthropic. The improvements over the previous version are substantial.

An added benefit is its open-source nature, with the codebase publicly available on GitHub under the Apache 2.0 license.

Further development of the Qwen model family will help strengthen its position in the global AI arena and narrow the gap with closed-source commercial models. And all current achievements are, in one way or another, steps toward humanity’s progress in building AGI.