OpenThoughts: Data Recipes for Reasoning Models
Abstract
Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best train- ing recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available. To address this, the goal of the OpenThoughts project is to create open-source datasets for training reasoning models. Our OpenThoughts2-1M dataset led to OpenThinker2-32B, the first model trained on public reasoning data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such as AIME and LiveCodeBench. We then improve our dataset further by systematically investigating each step of our data genera- tion pipeline with 1,000+ controlled experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples and using QwQ-32B as teacher yields our OpenThinker3-7B model, which achieves state-of-the-art results: 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Dia- mond – improvements of 15.3, 17.2, and 20.5 percentage points compared to the DeepSeek-R1-Distill-Qwen-7B. All of our datasets and models are available on openthoughts.ai.
Cite
Text
Guha et al. "OpenThoughts: Data Recipes for Reasoning Models." International Conference on Learning Representations, 2026.Markdown
[Guha et al. "OpenThoughts: Data Recipes for Reasoning Models." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/guha2026iclr-openthoughts/)BibTeX
@inproceedings{guha2026iclr-openthoughts,
title = {{OpenThoughts: Data Recipes for Reasoning Models}},
author = {Guha, Etash Kumar and Marten, Ryan and Keh, Sedrick and Raoof, Negin and Smyrnis, Georgios and Bansal, Hritik and Nezhurina, Marianna and Mercat, Jean and Vu, Trung and Sprague, Zayne Rea and Suvarna, Ashima and Feuer, Benjamin and Chen, Leon Liangyu and Khan, Zaid and Frankel, Eric and Grover, Sachin and Choi, Caroline and Muennighoff, Niklas and Su, Shiye and Zhao, Wanjia and Yang, John and Pimpalgaonkar, Shreyas and Sharma, Kartik and Ji, Charlie Cheng-Jie and Deng, Yichuan and Pratt, Sarah M and Ramanujan, Vivek and Saad-Falcon, Jon and Acharya, Stutee and Li, Jeffrey and Dave, Achal and Albalak, Alon and Arora, Kushal and Wulfe, Blake and Hegde, Chinmay and Durrett, Greg and Oh, Sewoong and Bansal, Mohit and Gabriel, Saadia and Grover, Aditya and Chang, Kai-Wei and Shankar, Vaishaal and Gokaslan, Aaron and Merrill, Mike A and Hashimoto, Tatsunori and Choi, Yejin and Jitsev, Jenia and Heckel, Reinhard and Sathiamoorthy, Maheswaran and Dimakis, Alex and Schmidt, Ludwig},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/guha2026iclr-openthoughts/}
}