Frequencies vs. Biases: Machine Learning Problems in Natural Language Processing - Abstract

Abstract

This chapter discusses the mismatch between natural language and finite-state devices. Noam Chomsky, an American linguist, has presented convincing arguments for the mismatch between natural language and finite-state devices. Chomsky's argument and his later work underlines the claim that the ability of a learner to distinguish between grammatical and ungrammatical sentences depends crucially on strong innate constraints on possible natural languages, that is, on restrictions on the hypotheses classes from which candidate concepts of grammaticality are drawn. This claim is based on a poverty of the stimulus argument, according to which the supervision that children receive when learning to speak their native language is far less than would be required by a learner without a strong initial bias. For practical applications, what is often needed is not a model of grammaticality, but rather a model of the relative likelihoods of different utterances, whether grammatical or not.