Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models

Generative AI & LLMs
Published: arXiv: 2510.15061v1
Authors

Samuel Paech Allen Roush Judah Goldfeder Ravid Shwartz-Ziv

Abstract

Widespread LLM adoption has introduced characteristic repetitive phraseology, termed ``slop,'' which degrades output quality and makes AI-generated text immediately recognizable. We present Antislop, a comprehensive framework providing tools to both detect and eliminate these overused patterns. Our approach combines three innovations: (1) The Antislop Sampler, which uses backtracking to suppress unwanted strings at inference time without destroying vocabulary; (2) An automated pipeline that profiles model-specific slop against human baselines and generates training data; (3) Final Token Preference Optimization (FTPO), a novel fine-tuning method that operates on individual tokens, surgically adjusting logits wherever a banned pattern has appeared in an inference trace. We demonstrate that some slop patterns appear over 1,000$\times$ more frequently in LLM output than human text. The Antislop Sampler successfully suppresses 8,000+ patterns while maintaining quality, whereas token banning becomes unusable at just 2,000. Most importantly, FTPO achieves 90\% slop reduction while maintaining or improving performance in cross-domain evals including GSM8K, MMLU, and creative writing tasks. In contrast, DPO suffers significant degradation in writing quality and lexical diversity despite achieving weaker suppression. We release all code and results under MIT license: https://github.com/sam-paech/auto-antislop.

Paper Summary

Problem
Language models (LLMs) have become widespread, but they often produce repetitive and overused patterns in their output, known as "slop." This can make AI-generated text easily recognizable and degrades its quality. The problem is that existing approaches to suppress unwanted patterns are either brittle or ineffective.
Key Innovation
The research presents a comprehensive framework called Antislop, which provides tools to detect and eliminate repetitive patterns in language models. The framework combines three innovations: 1. The Antislop Sampler, which uses backtracking to suppress unwanted strings at inference time without destroying vocabulary. 2. An automated pipeline that profiles model-specific slop against human baselines and generates training data. 3. Final Token Preference Optimization (FTPO), a novel fine-tuning method that surgically adjusts logits wherever a banned pattern has appeared in an inference trace.
Practical Impact
The Antislop framework has significant practical implications for language models. By suppressing repetitive patterns, it can improve the quality and coherence of AI-generated text, making it more difficult to distinguish from human-written text. This can be particularly important in creative writing, where the goal is to produce engaging and original content. Additionally, Antislop can help reduce the perception of repetition and overuse, making language models more suitable for applications such as customer service chatbots, where human-like conversation is essential.
Analogy / Intuitive Explanation
Imagine you're trying to write a poem, but every time you start to describe a sunset, you end up using the same phrase: "a tapestry of color." You might want to vary your language to make the poem more interesting, but you don't want to lose the essence of the description. Antislop is like a tool that helps the language model avoid using overused phrases like "a tapestry of color" and instead find more creative ways to express the same idea. By doing so, it can produce more engaging and original text.
Paper Information
Categories:
cs.LG cs.CL
Published Date:

arXiv ID:

2510.15061v1

Quick Actions