Prosody as Language

Speech conveys meaning using both words and prosody – the melodic component of speech. The central time scale of prosody is the Intonation Unit (IU), a prosodically coherent group of words delimited by measurable acoustic cues. Across languages, despite morphosyntactic, phonological, and prosodic differences, speech unfolds through IU sequences at a similar pace. IUs may take several syntactic forms to encapsulate processable chunks of information (e.g., a clause, ‘he likes cocoa pops’; a noun phrase, ‘a bag of potato chips’; or a prepositional phrase, ‘on toast’). IUs also help regulate communication by combining these substantive pieces of information into coherent messages via conjunctions, e.g. ‘but’, ‘and’, and ‘so’, managing their exchange via response tokens, e.g., ‘yeah’ and ‘uh-huh’, and indexing participants' cognitive states via hesitation markers, e.g., ‘uh’ and ‘um’ or change-of-state tokens, e.g., ‘oh’. 

We recently identified two core language properties in prosody: a vocabulary and a simple syntax. Unsupervised analysis of large spontaneous-conversation datasets yielded ~200 groups of pitch contours that speakers recurrently use. The average (or center) of each such group is an idealized pitch contour which represents one elementary building block of the prosodic vocabulary. Since under our framework these prototype patterns are the smallest independent elements of prosody that carry meaning, we call them "prosodic words".

A prosodic word is typically characterized by a single attitudinal meaning (e.g., ‘enthusiastic’ or ‘indifferent’). It also performs a few (2-5) well-defined linguistic functions (e.g. ‘yes-no question’, ‘affirmation’, or ‘contrastive statement’) that depend on the text with which it is coupled and the context in which it appears. Unlike earlier theoretical attempts to define a prosodic vocabulary, our data-driven approach both delimits a finite inventory, viz. the set of ~200 prosodic words, and provides its real-use frequency distribution, which is amenable to statistical analysis.

Second, we found a Markov-like syntax that relates a prosodic word to the one before and after it. We did not find evidence for a recursive syntax or a complex generative grammar of prosody, though this may be due to the limited amount of data that we analyzed. These invaluable insights into the underlying mechanisms of conversation empirically inform existing theoretical concepts of prosody. The methodology we present, combining unsupervised analysis of large datasets of spontaneous speech with manual sampling of the results, constitutes a broad basis for expanding our findings to other languages, genres, and contexts.

Once we understand that prosody is language-like, it is natural to ask how it conveys information and coordinates with text in real-time speech. If prosody merely reflects the text, it should be largely predictable from it. If the channels are independent, they should show no systematic relation. A third possibility is that they divide labor, and that their information is partially coupled.

Our current work conjectures that text and prosody are systematically coupled but only partially redundant. Associating IUs also with semantic prototypes – by clustering their text instead of their prosody – gives us a novel tool for probing the text-prosody relation. We find that the mutual information between text and prosody is low and varies with IU length, and that the two channels can divide labor or move in step, depending on the communicative function of the IU.

We also demonstrate that the prosodic vocabulary exhibits statistical regularities characteristic of lexical vocabularies, following Zipf's rank-frequency law and a markedness-based analogue of the law of brevity. These findings provide strong evidence that prosody is language-like precisely because it is not simply text-like. It is a structured communicative channel whose organization reflects, and is shaped by, its division of labor with textual words at the IU level.

Structure in conversation: Evidence for the vocabulary, semantics, and syntax of prosody
Matalon N., Weinreb E., Freche D., Volk E., Biron T., Moses E. & Biron D.
(2025) PNAS. 122 (17) e2403262122.
PDF

Automatic detection of prosodic boundaries in spontaneous speech
Biron T., Baum D., Freche D., Matalon N., Ehrmann N., Weinreb E., Biron D. & Moses E.
(2021) PLoS ONE. 16, 5, e0250969.
PDF