IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
A. Jian Yang, An Huang, Bo Dan, Chen Mu, Chengran Yang, Chuan Shi and 171 more · IQuest
Collected · snapshot shipped with the site build, not a live feed
Abstract · as written by the authors
We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.
This is the original abstract from arXiv. It has not been summarised, translated or rewritten.
Terms in plain language
AutomaticMade by matching the words of the title and abstract against a fixed list of definitions. It does not read the paper and does not change the abstract above.
- Agentic
- an AI system that plans and takes several steps, often by calling tools, towards a goal.
- Mixture-of-Experts
- a model made of many specialist parts, of which only a few run for each input.
- pre-training
- the first, large training stage on broad data, before any training for a specific task.
- RL
- training by trial and error: good actions earn a reward, and the model adjusts to earn more.