Paragraph inference for ASR
Small models that do one thing to speech-to-text output: decide where the paragraphs go, and change nothing else.
The task
Modern speech recognition already handles punctuation and capitalisation well, and sentence boundary detection is a widely adopted, well-understood step. What most engines still return is one unbroken block of text. For a two-minute voice note that block is hard to read, and the fix is not more punctuation: it is paragraphs.
Paragraph inference takes flat, punctuated text and inserts paragraph breaks. It is meant to be deterministic in the sense that matters for a pipeline: the words, punctuation and casing that go in are exactly the words that come out. Only whitespace changes.
Why a dedicated model?
ASR post-processing models are already very good. Superwhisper's S1 Mini, for example, is a 0.6B normaliser that removes fillers, resolves self-corrections, punctuates and writes numbers and dates in written form. A general LLM cleanup pass can do all of that and paragraphing too.
The idea here is narrower: a very small module that does only paragraph inference, so that it can be
- dropped into an ASR pipeline as its own step, before a deterministic or LLM post-processing stage, or as the last step;
- tested and tuned in isolation, with its own metric (break-position precision and recall) instead of being one behaviour buried inside a larger rewrite;
- safe by construction: it cannot drop, add or reword anything, so the step after it (or the reader) gets exactly what the ASR model heard.
This is a prototype. The models are trained on synthetic English voice memos in one person's paragraphing style, and they are published to test the idea, not as finished components.
Where it fits: intended pipelines
A is the minimal deployment: Parakeet does not emit a newline token, so its output is always one block, and the paragraph model is the only step after it. The transcript stays verbatim.
B adds a generic cleanup step afterwards that tidies the text (fillers, numbers, formatting) without doing its own paragraph inference, so each stage owns one job and can be swapped or evaluated separately.
The models
Three repos, two models. All score break-position precision and recall on the same 26 held-out, hand-reviewed synthetic memos (290 breaks).
| Model | Type | Size | F1 | Notes |
|---|---|---|---|---|
| MiniLM-L12-ASR-Paragraph-Breaks | Token classifier | 33M · 32 MB int8 ONNX | 0.85 | Recommended. Labels each word as a paragraph start or not, so it cannot change the text. ~0.65 s for a 340-word memo on a laptop CPU. |
| Gemma-3-270M-ASR-Paragraph-Breaks | Generative | 270M | 0.78 | First attempt. Retypes the text, so every output is checked and falls back to the input if anything but whitespace changed (2 of 26). |
| Gemma-3-270M-ASR-Paragraph-Breaks-ONNX | ONNX export | 1.66 GB fp32 · 417 MB int8 | – | ONNX Runtime build of the Gemma model; the int8 quant is experimental. |
Baseline for comparison: SaT segment-any-text/sat-3l-sm in paragraph mode scores F1 0.65
(precision 0.48, recall 1.00) on the same set.
Data: danielrosehill/ASR-Paragraph-Breaks · Try it: demo Space · Code: GitHub · Write-up: blog post
Prior art and related work
Paragraph segmentation of speech isn't a new idea. This is a short, non-exhaustive map of what's already out there, collected 2026-10-25.
Closest to this project
- Paragraph Segmentation Revisited: Towards a Standard Task for Structuring Speech
· paper · Retkowski & Waibel, LREC 2026
Makes the same case: ASR transcripts arrive as unstructured streams, and paragraphing is the missing step. Introduces the TEDPara and YTSegPara benchmarks, constrained decoding so LLMs insert breaks without altering the transcript, and a compact model (MiniSeg). - Segment Any Text (SaT) / wtpsplit
· models + library · 2024
Multilingual sentence segmentation models that predict newline probability, withdo_paragraph_segmentation=Truefor paragraphs. The baseline used above (sat-3l-sm). Papers: WtP (2023), SaT (2024). - superwhisper/s1-mini
· model · 2026
A 0.6B ASR normaliser (fillers, self-corrections, punctuation, inverse text normalisation). It isn't built for paragraph inference. Its card mentions blank lines only for a greeting, body and sign-off in email context.
ASR API features
- Deepgram
paragraphs=true· API feature: paragraphs from punctuation, speaker changes (with diarization) and channel changes. - AssemblyAI paragraphs endpoint · API feature: returns a finished transcript split into paragraphs.
Text and topic segmentation
- Text Segmentation by Cross Segment Attention · paper · Lukasik et al., EMNLP 2020: Cross-segment BERT.
- Text Segmentation as a Supervised Learning Task · paper · Koshorek et al., 2018: introduces the Wiki-727K dataset.
- Self-Adaptive Sliding Window for Spoken Document Segmentation · paper · 2021: segmentation aimed at spoken documents and ASR output.
- dennlinger/bert-wiki-paragraphs · model · 2021: BERT that decides whether adjacent paragraphs belong together (paper, dataset).
- Automatic Paragraph Identification · paper · Sporleder & Lapata, 2004: early work on paragraph boundaries across languages and domains.
- TextTiling (Hearst, 1997) and C99 (Choi, 2000) · classical: unsupervised topic-shift detection from lexical cohesion.
Adjacent step
- oliverguhr/fullstop-punctuation-multilang-large · model: punctuation restoration for unpunctuated ASR. This is the step before paragraphing, for engines that don't punctuate.