Liquid AI Open-Sources Antidoom: A Final Token Preference Optimization (FTPO) Method that Reduces Doom Loops in Reasoning Models
0
0
Liquid AI has released Antidoom, an open-source method that targets a common failure mode in reasoning models. That failure mode is the doom loop. In a doom loop, a model emits a span. It then repeats that span again and again. The output continues until the context window is exhausted. Small reasoning models are more prone to this, especially on long thinking traces and hard problems.
On an early checkpoint of LFM2.5-2.6B, 10.2% of completions on hard math and coding prompts produced repetitive loops. After Antidoom training, that rate fell to 1.4%. Eval scores improved across the board, attributable entirely to the reduced looping.
TL;DR
- Antidoom reduces doom loops by retraining only the first loop-start token.
- FTPO spreads probability across multiple coherent alternatives, not one replacement.
- LFM2.5-2.6B looping fell 10.2% to 1.4%; Qwen3.5-4B fell 22.9% to 1%.
- The pipeline runs in a few hours, and the full stack is open source.
What is Antidoom?
Antidoom is a targeted fix, not a broad sampling change. It finds the exact token that begins a loop. It then trains the model to prefer coherent alternatives at that single position. The rest of the distribution stays largely untouched.
The method adapts Antislop. It trains on chosen/rejected pairs that represent a single completion token. The training algorithm is Final Token Preference Optimization (FTPO), which is similar to DPO.
The training teaches the model nothing new about math or code. It clears the looping that blocked answers the model could already produce.
Anatomy of a Doom Loop
Liquid AI team attributes doom loops to three mechanisms working together:
Mechanism 1: overtrained tokens plus uncertainty. Some tokens are more likely to be selected in general. Well-known examples in the wild include ‘delve’ and ‘testament.’ Liquid AI team notes this can trace back to synthetic data in the training set. In reasoning traces, high-prior continuations often include discourse markers such as ‘Wait’ or ‘Alternatively.’ These tokens are not inherently bad. They can mark a useful change of strategy, a verification step, or a branch. When the model is uncertain or stuck, they instead become attractive fallback continuations.
For an early LFM2.5-2.6B checkpoint, the most common loop-starting tokens were the following.
| Token | Share of loop starts |
|---|---|
the |
11.39% |
So |
4.51% |
Alternatively |
3.22% |
Wait |
2.56% |
But |
2.46% |
Mechanism 2: prior context reinforces the loop. Each repetition pushes every token in the span toward probability that Duan et al. study this in their work on circular reasoning. They link it to a “V-shaped” attention pattern. They find that semantic repetition precedes textual repetition.
Mechanism 3: greedy sampling. Reasoning models usually run at low temperature for stable, reproducible traces. At temperature 0, the most likely token is always selected. A locally reinforced loop then has no exit. Liquid AI reports significant looping even at temp=0.67. Lower temperatures exacerbate the problem.
How Antidoom Locates the Failure
Antidoom generates completions on a prompt mix designed to elicit looping, at low temperature. That mix ships as the LiquidAI/antidoom-mix-v1.0 dataset. A loop is detected when a section repeats at least four times, over at least 60 characters.
The method then targets the first token of the first repeat. At that position, it takes the base model’s top-k log-prob alternatives. It filters short or non-alphanumeric noise. It keeps up to 20 plausible substitutes as chosen tokens.
Each training row is a tuple of prompt prefix, one rejected token, and one or more chosen tokens. The chosen and rejected distributions are regularised before training. Otherwise a few culprits like Wait, So, and the would dominate and over-suppression would degrade reasoning.
The detection rule itself is simple to state in code. The snippet below is illustrative.
# A loop = a unit repeating >=4 times, spanning >=60 characters.
# Returns the index of the first token of the first repeat (the target), else None.
def find_loop(text, min_repeats=4, min_chars=60):
n = len(text)
for span in range(1, n // min_repeats + 1):
start = 0
while start + span * min_repeats <= n:
unit = text[start:start + span]
repeats = 1
pos = start + span
while text[pos:pos + span] == unit:
repeats += 1
pos += span
if repeats >= min_repeats and span * repeats >= min_chars:
return start + span # first token of the first repeat
start += 1
return NoneEach detected loop then becomes one training row. The structure is a simple tuple.
# One FTPO training row, per the post's [prefix, rejected, chosen] format.
row = {
"prompt": prefix_up_to_the_loop, # text before the first repeat
"rejected": " Wait", # the single token that started the loop
"chosen": [" So", " Since", " The", " Therefore"], # up to 20 alternatives
}Final Token Preference Optimization (FTPO)
FTPO is a preference-optimization algorithm similar to DPO. A training sample has a prompt, a chosen continuation, and a rejected continuation. It is built to change a handful of tokens, with minimal disturbance to the model otherwise.
FTPO differs from DPO in four ways:
- Final token training: It trains only the trailing token of a sequence that is midway through generation.
- Multiple chosen tokens per sample: It spreads probability across a group of alternatives, so one overtrained token is not simply replaced by another.
- KL-like loss in logit space: It omits the softmax and computes divergence from reference in logits, avoiding pressure on unrelated tokens.
- Two-part regularization: Chosen and rejected logits move more freely, while the remaining vocab stays tightly constrained.
In the Antidoom implementation, the model trains for one epoch with LoRA. High LoRA ranks of 128-256 gave the best results. Training covers all attention and MLP projections, plus lm_head. Learning rates land around 4e-6 to 2e-5.
Training uses early stopping on chosen_win, the share of samples where chosen tokens beat rejected. Stopping at chosen_win=0.35 cut doom-loop rates from 20-30% down to 1-2%. Training longer tended to degrade the model.
For the early LFM2.5-2.6B checkpoint, training-set generation took about one hour on 8x MI325 GPUs. Training then took about one to two hours on a single MI325 GPU. Generation stops after collecting 20k pairs.
How Antidoom Compares to the Usual Fixes
| Approach | What it changes | Cost profile | Reported drawback |
|---|---|---|---|
repetition_penalty |
Reweights the output distribution | Inference-time, cheap | Band-aid; can degrade performance |
| Reinforcement learning | Policy via rewards | Calibrated rewards, costly online rollouts | Setup and compute overhead |
| DPO (final-token) | One chosen token per sample | Offline training | Coarse beta; updates a single token |
| Antidoom (FTPO) | First loop token → many chosen tokens | ~1h gen (8x MI325) + 1-2h train (1x MI325) | Can expose new loops; may need extra rounds |
Results
After training, the doom-looping rate on the early LFM2.5-2.6B checkpoint dropped from 10.2% to 1.4%. Eval scores improved across the board, attributable entirely to the reduction in looping.
Liquid AI team also ran the pipeline on Qwen3.5-4B, which is known to loop during reasoning. Its doom-looping rate dropped from 22.9% to 1% under greedy sampling. Eval scores increased markedly.
The eval score changed inversely with the doom-loop rate as temperature rose. After training, both models showed a performance drop near temp=1.0. This is expected, since higher-temperature sampling can favor less-preferred tokens. Once looping is removed, near-greedy sampling gave the strongest scores in the models tested.
Liquid AI team flags a related point about common practice. The belief that higher temperatures aid reasoning may be conflated with the effect of doom-looping. In their tests, once loops are gone, near-greedy sampling performs best.
Multiple rounds can help. The first round rejects loop-causing tokens and reweights toward alternatives. That can expose new failure points, which a second round then targets.
Interactive Explainer
Use Cases with Examples
- On-device reasoning models: Sub-1GB reasoning models like the LFM2.5 family can stall mid-proof on hard prompts. Antidoom recovers the accuracy those loops were costing.
- Small coding agents: A 4B coding model can loop on a hard debugging trace and burn its context window. Removing the loop lets it reach the fix it already knew.
- Agent pipeline cost control: Loops consume tokens until context exhaustion. Cutting them reduces wasted tokens and latency across long agent runs.
- Post-training repair. Teams shipping fine-tuned reasoning checkpoints can run Antidoom as a cleanup pass in a few hours.
Strengths and Challenges
Strengths:
- Targeted: it edits the first loop token and leaves the rest of the distribution largely intact.
- Fast: the whole pipeline runs in a few hours.
- Measured: LFM2.5-2.6B fell 10.2% to 1.4%; Qwen3.5-4B fell 22.9% to 1%.
- Open source: generation, detection, and the FTPO trainer are all released.
- Recovers, not teaches: it restores answers the model could already produce.
Challenges:
- It can expose new failure points, so multiple rounds are sometimes needed.
- Over-training degrades the model, so early stopping on
chosen_winis required. - Reported results cover LFM checkpoints and Qwen3.5-4B, both small reasoning models.
- Performance can drop near temp=1.0 after training.
- Each model needs its own generated looping dataset.
Check out the Technical details and GitHub Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Liquid AI Open-Sources Antidoom: A Final Token Preference Optimization (FTPO) Method that Reduces Doom Loops in Reasoning Models appeared first on MarkTechPost.
MarkTechPost
