Learning grammars using Minimum Description Length

Abstract

I will discuss an approach to learning based on the principle of Minimum Description Length (MDL) and show how it can help us (a) account for the learning of grammars from positive evidence alone, and (b) choose between competing proposals for grammatical architecture in some cases where adult judgments alone are insufficiently informative.

MDL considers both the size of the grammar and that of the description of the data given the grammar and attempts to minimize their sum. By doing so, it guides the learner to hypotheses that balance between generality and the need to fit the data. MDL appears to match subjects’ generalization patterns in a variety of tasks and is arguably less stipulative than alternative approaches that have been proposed in the literature. Moreover, in several different domains it has yielded the first implemented learners that assume both realistic linguistic theories and realistic input data.

I will review these properties of MDL and show how theoretical linguistics can use it to compare between competing architectures in cases where it is difficult to make progress based on adult grammaticality judgments alone. I illustrate this with a phonological case study concerning constraints on underlying representations (also known as morpheme-structure constraints), which were central to early generative phonology but rejected in Optimality Theory. Evidence bearing directly on the question of whether the grammar uses constraints on URs has been scarce. I will show, however, that if children are MDL learners, then they will succeed in learning patterns such as English aspiration if they can use constraints on URs but run into difficulties otherwise. While the case study that I will discuss is phonological, the methodology is very general, and I will outline a second case study, In semantics, where MDL allows us to extract divergent empirical predictions from two competing hypotheses from the literature concerning the representation of semantic denotations.