The academic paper introduces Self-play Reinforcement Learning (SeRL), a framework engineered to enhance the reasoning capabilities of Large Language Models (LLMs) specifically in scenarios lacking extensive, high-quality labeled data. SeRL consists of two core, complementary modules: the self-instruction module generates new and diverse training problems from a small seed dataset, ensuring data quality and appropriate difficulty via an online filtering strategy. Simultaneously, the self-rewarding module bypasses the need for external supervision by estimating response rewards using a stable majority-voting mechanism among sampled outputs. This integrated approach facilitates sustained, unsupervised reinforcement learning across multiple training iterations. Experiments demonstrate that SeRL is highly effective, consistently outperforming existing self-play methods and matching the performance levels achieved by models trained on full datasets with verifiable rewards.

Source:

https://openreview.net/pdf?id=ZF93vyH9He