arXiv:2611.14912v1 Announce Type: new Abstract: Large language models trained with Reinforcement Learning (RL) with verifiable rewards exhibit strong reasoning ability and broad generalization, whereas models trained with Supervised Fine-Tuning (SFT) are often viewed as more prone to memorization and limited transfer. This paper rethinks this distinction through the lens of SFT training data on widely used Qwen2.5 and Qwen3 series models. First, we study the role of data source and show that it is critical: a carefully mixed SFT dataset substantially outperforms data generated solely by a larger model.
Machine Learning · 2026-11-13
Rethinking "RL Generalizes, SFT Memorizes": The Role of SFT Data
arXiv:2611.14912v1 Announce Type: new Abstract: Large language models trained with Reinforcement Learning (RL) with verifiable rewards exhibit strong reasoning ability and broad generalization, whereas models trained with Supervised Fine-Tuning (SFT) are often viewed as more prone to memorization and limited transfer.
Source and attribution
This is an attributed editorial digest. For the complete study and original reporting, follow the source link.
Reader feedback
0 likes · 0 dislikes