Key points
- JOLT trains a single model in dual roles as both teacher and student during post-training.
- The method matched GRPO accuracy on MATH benchmarks using 6.5 times fewer processed tokens.
- Testing covered models including Qwen3 variants and DeepSeek-V2-Lite, with JOLT+ showing high pass rates.
The short version
- Researchers from Perplexity and NVIDIA introduced JOLT to improve efficiency in reinforcement learning post-training.
- The approach uses a shared model for teaching and learning, keeping the teacher's guidance aligned with the student.
- Results showed performance gains on complex tasks like AppWorld while drastically cutting token usage compared to GRPO.
The details
Perplexity and NVIDIA researchers introduced JOLT, a post-training method for language models that trains one model in two roles simultaneously. According to MarkTechPost, the technique matches the accuracy of GRPO on the MATH benchmark while processing 6.5 times fewer tokens. JOLT stands for Joint On-policy Learning and Teaching. It addresses the issue of sparse rewards in outcome-reward reinforcement learning by having the teacher score each student token. Unlike previous methods, JOLT trains the teacher alongside the student to ensure its feedback remains useful and aligned with the student's capabilities.
The method was tested on several models, including Qwen3-0.6B, Qwen3-4B-Instruct, Qwen3-8B, Qwen3-30B-A3B-Instruct, and DeepSeek-V2-Lite. The team did not disclose the specific training hardware used. In every setting tested, JOLT+ posted the highest mean score against GRPO, OPSD, and pi-Distill. On the AppWorld state-test, JOLT+ achieved a pass rate of 72.09 percent, which is 9.9 points higher than GRPO's 62.20 percent. However, JOLT without student reward scored 51.92 percent on AppWorld, falling below GRPO's performance.
The context
Reinforcement learning post-training often struggles with sparse signals. Methods like GRPO provide only one reward per attempt, which does not indicate which specific tokens contributed to success. This makes learning slow for long agentic tasks where successes are rare. On-Policy Self-Distillation (OPSD) attempted to fix this by using a stronger teacher to score tokens. OPSD removed the separate teacher model, allowing the same model to act as a teacher when given privileged context, such as a verified solution.
The research team argued that OPSD is insufficient because a teacher holding the answer can take shortcuts the student cannot follow. This causes the teacher's guidance to sit beyond the student's reach. In their paper, the team noted that teacher-student disagreement grows during MATH training with OPSD. JOLT aims to create a reward-consistent teacher by training both roles of the shared model at the same time. The teacher learns from outcome rewards and a KL penalty, while the student learns from the teacher's token-level distillation signal.
What to watch
- Monitor how JOLT performs on non-mathematical benchmarks and real-world agentic tasks beyond AppWorld.
- Watch for open-source releases of the JOLT implementation or detailed hardware requirements for training.
- Observe whether other vendors adopt this joint teacher-student approach in their own model post-training pipelines.
- Track if the token efficiency gains hold up when scaling to much larger foundation models.
What it means for organisations
Teams managing AI training pipelines should evaluate how this method impacts their compute budgets and training timelines. Organizations should ask whether their current reinforcement learning setups suffer from sparse reward signals that JOLT could mitigate. Decision-makers need to understand if the efficiency gains justify retraining existing models or adjusting their post-training strategies. Internal teams should discuss how to integrate these findings into their model development roadmaps without disrupting current production workflows.
What to do and how to stay safe: Perplexity
- Review current reinforcement learning post-training methods to identify inefficiencies in token usage.
- Assess whether your models benefit from joint teacher-student training approaches like JOLT.
- Monitor research updates from Perplexity and NVIDIA for further details on implementation.
- Plan for potential compute savings if you adopt more efficient post-training techniques.
Step-by-step guide: Secure Home Routers for Small Business Networks
General security guidance from the TechSec Headlines newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What models were tested with JOLT?
The method was tested on Qwen3-0.6B, Qwen3-4B-Instruct, Qwen3-8B, Qwen3-30B-A3B-Instruct, and DeepSeek-V2-Lite.
How does JOLT compare to GRPO on the MATH benchmark?
JOLT matched GRPO's accuracy on MATH while using 6.5 times fewer processed training tokens.
What is the main advantage of JOLT over OPSD?
JOLT trains the teacher and student simultaneously to keep the teacher's guidance aligned with the student's capabilities.
