Abstract: The paper reports on experiments with a population of visually grounded robotic agents capable of bootstrapping their own ontology and shared lexicon without prior design nor other forms of human intervention. The agents do so while playing a particular language game called the guessing game. We show that synonymy and ambiguity arise as emergent properties in the lexicon, due to the situated grounded character of the agent-environment interaction, but that there are also tendencies to dampen them so as to make the language more coherent and thus more optimal from the viewpoints of communicative success, cognitive complexity, and learnability.

Abstract: This dissertation explores the way in which English-speaking children acquire the meaning of sentences containing negation and quantified noun phrases (QNPs). This investigation is based on a series of psycholinguistic experiments designed to assess children’s comprehension of sentences like ‘Every horse didn’t jump over the fence’ or ‘Cookie Monster didn’t eat two slices of pizza’ among others. The major finding is that children around the age of 5 do not interpret these sentences the way adult speakers of English do. This finding raises the following questions (a) How and why do children’s interpretations of sentences containing negation and quantified noun phrases differ from those of adults? (b) How do children manage to converge onto the adult system of interpretation? Regarding the first question, it appears that children’s non-adult interpretations are nevertheless systematic, i.e. governed by principle. Specifically, children (unlike adults) are found to map overt syntactic relations between QNPs and negation and their relative semantic interpretation isomorphically. This, however, is just a descriptive generalization. The observation of isomorphism is treated as an epiphenomenon, derived from the interplay between a universally encoded dichotomy splitting the class of QNPs and learnability considerations. Regarding the second question, I show that children can move from their system of interpretation to the adult system solely on the basis of positive evidence and thus, that the observed difference does not create a learnability problem. In summary, this dissertation uncovers a new area where the linguistic behavior of children and adults diverge: the comprehension of sentences containing negation and quantified noun phrases. The rest of the dissertation is a methodological statement, namely that it is not only desirable but also possible to account for the observed difference between children and adults without invoking any differences between the two groups beyond minimal conceptual necessity. To the extent that this goal is achieved, the present investigation emphasizes the role played by the theory of Universal Grammar and language learnability in helping us understand language development and its biological basis. Comments University of Pennsylvania Institute for Research in Cognitive Science Technical Report No. IRCS-99-01. This thesis or dissertation is available at ScholarlyCommons: http://repository.upenn.edu/ircs_reports/40 UNIVERSAL GRAMMAR AND THE ACQUISITION OF SEMANTIC KNOWLEDGE: An Experimental Investigation into the Acquisition of QuantifierNegation Interaction in English.

Abstract: 1 I The Issues 2 Investigating Second Language Grammars: Some Conceptual and Methodological Issues in Generative SLA Research (by Klein, Elaine C) 3 II Current Perspectives on Generative L2A Studies 4 Common Methodological Issues In L1 And L2 Research (by Cairns, Helen) 5 The Second Language Acquisition Of The Split CP Structure (by Prevost, Philippe) 6 Activating AgrIOP in Second Language Acquisition (by Montrul, Silvina A) 7 Null subjects in non-native grammars: The Spanish L2 of Chinese, English, French, German, Japanese and Korean speakers (by Liceras, Juana M) 8 Selectivity in the Acquisition of Complex NP Islands (by Perez-Leroux, Ana Teresa) 9 III New Directions in Generative L2A Studies 10 Psych verbs In Second Language Acquisition (by White, Lydia) 11 Just Parsing Trough: Notes On The State Of L2 Processing Research Today (by Klein, Elaine C) 12 Processing Strategies in Second Language Acquisition: Some Preliminary Results (by Fernandez, Eva M) 13 Language Mixing and Second Language Acquisition: Some Issues and Perspectives (by Bhatia, Tej K) 14 Codeswitching, Grammar, and Sentence Production: The Problem of Light Verbs (by Ritchie, William C) 15 Multiple-Specifiers and wh-island Effect in L2 Acquisition: A Preliminary Study (by Yusa, Noriaki) 16 Losing the V2 Constraint (by Robertson, Daniel) 17 Learnability Theory: Triggers for Parsing With (by Fodor, Janet Dean) 18 List of Authors 19 Name Index 20 Subject Index

Abstract: The paper reports on experiments in which autonomous visually grounded agents bootstrap an ontology and a shared lexicon without prior design nor other forms of human intervention. The agents do so while playing a particular languagegame called the guessing game. We show that synonymy and polysemy arise as emergent properties in the language but also that there are tendencies to dampen it so as to make the language more coherent and thus more optimal from the viewpoints of communicative success, cognitive complexity, and learnability.

Abstract: The approach of ordinal mind change complexity, introduced by Freivalds and Smith, uses (notations for) constructive ordinals to bound the number of mind changes made by a learning machine. This approach provides a measure of the extent to which a learning machine has to keep revising its estimate of the number of mind changes it will make before converging to a correct hypothesis for languages in the class being learned. Recently, this notion, which also yields a measure for the difficulty of learning a class of languages, has been used to analyze the learnability of rich concept classes. The present paper further investigates the utility of ordinal mind change complexity. It is shown that for identification from both positive and negative data and n ⩾ 1, the ordinal mind change complexity of the class of languages formed by unions of up to n + 1 pattern languages is only ω ×0 notn(n) (where notn(n) is a notation for n, ω is a notation for the least limit ordinal and ×0 represents ordinal multiplication). This result nicely extends an observation of Lange and Zeugmann that pattern languages can be identified from both positive and negative data with 0 mind changes. Existence of an ordinal mind change bound for a class of learnable languages can be seen as an indication of its learning “tractability”. Conditions are investigated under which a class has an ordinal mind change bound for identification from positive data. It is shown that an indexed family of languages has an ordinal mind change bound if it has finite elasticity and can be identified by a conservative machine. It is also shown that the requirement of conservative identification can be sacrificed for the purely topological requirement of M-finite thickness. Interaction between identification by monotonic strategies and existence of ordinal mind change bound is also investigated.

Abstract: An index for an r.e. class of languages (by definition) is a procedure which generates a sequence of grammars defining the class. An index for an indexed family of languages (by definition) is a procedure which generates a sequence of decision procedures defining the family. Studied is the metaproblem of synthesizing from indices for r.e. classes and for indexed families of languages various kinds of language learners for the corresponding classes or families indexed. Many positive results, as well as some negative results, are presented regarding the existence of such synthesizers. The negative results essentially provide lower bounds for the positive results. The proofs of some of the positive results yield, as pleasant corollaries, subset-principle or tell-tale style characterizations for the learnability of the corresponding classes or families indexed. For example, the indexed families of recursive languages that can be behaviorally correctly identified from positive data are surprisingly characterized by Angluin's condition 2 (the subset principle for circumventing overgeneralization).

Abstract: The study of language acquisition involves determining how children learn the linguistic phenomena of their first language. This thesis analyzes the Triggering Learning Algorithm, which describes how children set the parameters which define the linguistic variation seen in the syntax of natural language. This thesis describes enhancements to an implementation of this algorithm and the results found from the implementation. This implementation allows for more careful study of the algorithm and of its known problems. It allows us to examine in more depth the solutions which have been proposed for these problems. The results show that the theory correctly predicts some principles expected to be true of language acquisition. However, many of the proposed principles are shown to make incorrect predictions about the learnability of certain kinds of languages. This thesis discusses a second potential solution and ways in which it can be tested as well as other ways to test the predictions of the theory. Thesis Supervisor: Robert C. Berwick Title: Professor of Computer Science and Engineering

Abstract: We consider the following classes of quantified formulas. Fix a set of basic relations called a basis. Take conjunctions of these basic relations applied to variables and constants in arbitrary ways. Finally, quantify existentially or universally some of the variables. We introduce some conditions on the basis that guarantee efficient learnability. Furthermore, we show that with certain restrictions on the basis the classification is complete. We introduce, as an intermediate tool, a link between this class of quantified formulas and some well-studied structures in Universal Algebra called clones. More precisely, we prove that the computational complexity of the learnability of these formulas is completely determined by a simple algebraic property of the basis of relations, their clone of polymorphisms. Finally, we use this technique to give a simpler proof of the already known dichotomy theorem over boolean domains and we present an extension of this theorem to bases with infinite size.

Abstract: Automated inductive learning is a vital part of machine intelligence and the design of intelligent agents. A useful formalization of inductive learning is the model of PAC-learnability. Nevertheless, the ability to learn every target concept expressible in a given representation, as required in the PAC-learnability model, is highly demanding and leads to many negative results for interesting concept classes. A new model of learn-ability, called Universal Learnability or U-learnability, recently has been proposed as a less demanding, average-case variant of PAC-learnability. This paper uses the U-learnability model to analyze a top-down decision tree induction algorithm. Speciically, this paper proves that an idealized variant of the well-known decision tree learning algorithm CART|one of the most successful existing machine learning algorithms|is a U-learner under a natural set of assumptions regarding target hypotheses. (The motivation and description of these assumptions is best delayed until the U-learnability model is described.) Equally interestingly, various related PAC-learning algorithms such as those for k-DNF cannot be used to U-learn under the same assumptions. Finally, the paper raises a number of 1 related open questions and general research directions; open questions include not only U-learnability questions, but also several new PAC-learnability questions and one question regarding a general property of propositional logic.

Abstract: The article focuses on four questions for a theory of semantics: the ontological, semantic, learnability and communicative questions. It is shown how different realist, cognitivist and constructivist semantic theories answer the questions.

Abstract: The validity of a grammatical framework can be verified in at least the three ways: (a) by showing its applicability to wide range of linguistic phenomena, (b) by demonstrating the soundness of its formal foundations, and (c) by verifying its compatibility with experimental evidence. As for Optimality Theory (OT; Prince and Smolensky 1993), option (a) has been pursued extensively in the recent phonological literature (and to a lesser extend in the syntactic literature). Also option (b) has been the the topic of some research (e.g., Ellison 1994; Karttunen 1998). However, no attempts have been made so far to test the concepts and mechanisms assumed in OT against experimental evidence. The present paper attempts to fill this gap by testing OT against evidence from what is probably the most natural empirical domain for a linguistic framework: grammaticality judgments. More specifically, we focus on the phenomenon of gradience in linguistic data. We argue that gradient data can serve as a tool for evaluating the status of suboptimal candidates in OT, an approach that allows to scrutinize OT’s concepts of constraint ranking and constraint interaction. The experimental data we present show that constraint violations are cumulative, and that two types of constraints have to be distinguished: hard and soft ones. These results lend limited support to the notion of constraint ranking assumed in OT, and seem compatible with OT’s concept of strict domination of constraints. The second part of this paper deals with the theoretical issues arising from an attempt to model gradient linguistic data in OT. We show that a naive model that equates relative grammaticality with relative optimality is not tenable, and propose an alternative approach based on the concept of selective constraint re-ranking. This approach, which is grounded in OT learnability theory, predicts the cumulativity of constraint violations, and allows to model the distinction between hard and soft constraints, thus accounting for the experimental findings.

Abstract: In this thesis we analyze the learnability of rational expectiations equilibria in three general equilibrium business cycle models. The economic example business cycles models comprise the basic real business cycles model, an increasing returns model and a model with both static and dynamic complementarities. In these models the business cycles are driven by both shocks that affect the production technology and by taste shocks that affect the marginal rate of substitution between consumption and labor. In the two latter models we also analyze the existence and learnability of sunspot rational expectations equilibria.

Abstract: We study the learnability of first order Horn expressions from equivalence and membership queries. We show that the class of range restricted Horn expressions, where every term in the consequent of every clause appears also in the antecedent of the clause, is learnable. The result holds both for the model where interpretations are examples (learning from interpretations) and the model where clauses are examples (learning from entailment). The paper utilises a previous result on learning function free Horn expressions. This is done by using techniques for flattening and unflattening of examples and clauses, and a procedure for model finding for range restricted expressions. This procedure can also be used to solve the implication problem for this class.

Abstract: We prove a new combinatorial characterization of polynomial learnability from equivalence queries, and state some of its consequences relating the learnability of a class with the learnability via equivalence and membership queries of its subclasses obtained by restricting the instance space. Then we propose and study two models of query learning in which there is a probability distribution on the instance space, both as an application of the tools developed from the combinatorial characterization and as models of independent interest.

Abstract: Generative linguistics aims to provide an analysis of the grammar-forming capacity that individuals bring to the task of learning their native language (Chomsky 1965, 1981, 1991, 1995) Pursuing this goal amounts to developing a linguistic theory that achieves maximum universality and generality in its premises, while at the same time offering explicit, limited means for representing possible interlinguistic variation The term “Universal Grammar” is used to refer to the system of principles defining what the grammar of a human language can be Optimality Theory (Prince and Smolensky 1993) asserts that Universal Grammar provides a set of general, universal constraints which evaluate possible structural descriptions of linguistic objects These constraints are assumed to be strongly universal, in the sense that they are present in every grammar; they must be simple and general if they are to have any hope of universality The structural description that is grammatical for a linguistic object in a given language is the one, among all possible structures assignable to that object, which is optimal, in the sense that it best satisfies the universal constraints, given the defining characteristics of that object The theory builds from a notion of “best satisfaction” — optimality rather than perfection — because constraints are often in conflict over the well-formedness of a given candidate analysis, so that satisfying one constraint entails violating others to varying degrees Indeed, an optimal structural description will typically violate some (or many) of the constraints, because no possible description satisfies all of them Differences between languages then emerge as the different ways allowed by the theory for resolving the conflicts that are inherent in the universal constraint set The theory is therefore one of constraint interaction: the effects of any constraint are determined both by its intrinsic demands and by its relation to the other constraints; fixing the relations between the universal constraints defines the grammar of a particular language Interactionism is the heart and soul of the theory: it places a heavy empirical burden on the positing of constraints, since all interactions must yield possible grammars; it leads to a pattern of explanation in which many universally observed properties of language follow not from hypothesized special principles designed to encode them directly, but from nothing more than the interaction of more general constraints which are not specifically concerned with those particular properties; it opens the way to tremendous simplification of the constraints themselves, since they need not contain codicils and complications that emerge from interaction; and, at the methodological level, it enforces the research ethic that posited constraints must not contain such codicils and complications, thereby guiding the search for a general understanding of the nature of the constraints active in human language This essay presents an overview of selected work making use of Optimality Theory, with the goal of illuminating these general ideas Section 1 presents and illustrates the central principles of Optimality Theory Section 2 gives an example of syntax within the theory Section 3 examines possible implications of Optimality Theory for studies of language processing, discussing work on the computability of Optimality Theoretic grammars, as well as some conceptual similarities between Optimality Theory and work in connectionism and dynamical systems Section 4 discusses work on language learnability and acquisition within Optimality Theory

Abstract: The present work introduces and justifies the notion of hyperrobust learning where one fixed learner has to learn all functions in a given class plus their images under primitive recursive operators. The following is shown: This notion of learnability does not change if the class of primitive recursive operators is replaced by a larger enumerable class of operators. A class is hyperrobustly Ex-learnable iff it is a subclass of a recursively enumerable family of total functions. So, the notion of hyperrobust learning overcomes a problem of the traditional definitions of robustness which either do not preserve learning by enumeration or still permit topological coding tricks for the learning criterion Ex. Hyperrobust BC-learning as well as the hyperrobust version of Ex-learning by teams are more powerful than hyperrobust Ex-learning. The notion of bounded totally reliable BC-learning is properly between hyperrobust Ex-learning and hyperrobust BC-learning. Furthermore, the bounded totally reliably BC-learnable classes are characterized in terms of infinite branches of certain enumerable families of bounded recursive trees. A class of infinite branches of a further family of trees separates hyperrobust BC-learning from totally reliable BC-learning.

Abstract: Functional dependencies play an important role in the design of databases. We study the learnability of the class of minimal covers of functional dependencies (MCFD) within the exact learning model via queries. We prove that neither equivalence queries alone nor membership queries alone suffice to learn the class. In contrast, we show that learning becomes feasible if both types of queries are allowed. We also give some properties concerning minimal covers.

Abstract: We prove a new combinatorial characterization of polynomial learnability from equivalence queries, and state some of its consequences relating the learnability of a class with the learnability via equivalence and membership queries of its subclasses obtained by restricting the instance space. Then we propose and study two models of query learning in which there is a probability distribution on the instance space, both as an application of the tools developed from the combinatorial characterization and as models of independent interest.

Abstract: The present paper motivates the study of mind change complexity for learning minimal models of length-bounded logic programs. It establishes ordinal mind change complexity bounds for learnability of these classes both from positive facts and from positive and negative facts. Building on Angluin's notion of finite thickness and Wright's work on finite elasticity, Shinohara defined the property of bounded finite thickness to give a sufficient condition for learnability of indexed families of computable languages from positive data. This paper shows that an effective version of Shinohara's notion of bounded finite thickness gives sufficient conditions for learnability with ordinal mind change bound, both in the context of learnability from positive data and for learnability from complete (both positive and negative) data. More precisely, it is shown that if a language defining framework yields a uniformly decidable family of languages and has effective bounded finite thickness, then for each natural number m > 0, the class of languages defined by formal systems of length ≤ m: - is identifiable in the limit from positive data with a mind change bound of ωm; - is identifiable in the limit from both positive and negative data with an ordinal mind change bound of ω × m. The above sufficient conditions are employed to give an ordinal mind change bound for learnability of minimal models of various classes of length-bounded Prolog programs, including Shapiro's linear programs, Arimura and Shinohara's depth-bounded linearly-covering programs, and Krishna Rao's depth-bounded linearly-moded programs. It is also noted that the bound for learning from positive data is tight for the example classes considered.

Abstract: Usability issues are traditionally associated with user interfaces rather than with agent frameworks. We argue that the metaphors and models used in a framework will affect the thinking of the developer, and will influence the application design. Therefore, usability is of central importance for successful software development, and for reducing development and maintenance costs. We discuss the design and implementation of a lightweight agent framework for interactive multi-agent applications. A lightweight framework is advantageous for distributed interactive applications, for instance applications running on hand-held devices with limited memory. The design is based on minimalism and simplicity. We present the results from a usability study of the framework, where issues such as learnability and attitude have been evaluated. The study shows that minimalist design principles are useful for achieving understandable and navigable frameworks.

Abstract: Designing the representation languages for the input and output of a learning algorithm is the hardest task within machine learning applications. Transforming the given representation of observations into a well-suited language LE may ease learning such that a simple and efficient learning algorithm can solve the learning problem. Learnability is defined with respect to the representation of the output of learning, LH. If the predictive accuracy is the only criterion for the success of learning, the choice of LH means to find the hypothesis space with most easily learnable concepts, which contains the solution. Additional criteria for the success of learning such as comprehensibility and embeddedness may ask for transformations of LH such that users can easily interpret and other systems can easily exploit the learning results. Designing a language LH that is optimal with respect to all the criteria is too difficult a task. Instead, we design families of representations, where each family member is well suited for a particular set of requirements, and implement transformations between the representations. In this paper, we discuss a representation family of Horn logic. Work on tailoring representations is illustrated by a robot application.

Abstract: In this paper, we look at lexical categories and their predictability from a machine learning perspective. Starting from linguistic intuitions about predictability in three different domains, we show how standard techniques for analyzing classification tasks arive at a similar predictability scale. In the second part of the paper, we carry out machine learning experiments covering these domains and relate learnability results to the previous analysis. The IB1-IG classifier is found to be capable of learning in all three domains, although with varying degrees of success.

Abstract: A fundamental question in the field of artificial neural networks is what set of problems a given class of networks can perform (computability). Such a problem can be made less general, but no less important, by asking what these networks could learn by using a given training procedure (learnability). The basic purpose of this paper is to address the learnability problem. Specifically, it analyses the learnability of sequential RAM-based neural networks. The analytical tools used are those of Automata Theory. In this context, this paper establishes which class of problems and under what conditions such networks, together with their existing learning rules, can learn and generalise. This analysis also yields techniques for both extracting knowledge from and inserting knowledge into the networks. The results presented here, besides helping in a better understanding of the temporal behaviour of sequential RAM-based networks, could also provide useful insights for the integration of the symbolic/connectionist paradigms.

Abstract: Shinohara, Arimura, and Krishna Rao have shown learnability in the limit of minimal models of classes of logic programs from positive only data. In most cases, these results involve logic programs in which the "size" of the head yields a bound on the size of the body literals. However, when local variables are present, such a bound on body literal size cannot directly be ensured. The above authors achieve such a restriction using technical notions like mode and linear inequalities. The present paper develops a conceptually clean framework where the behavior of local variables is controlled by nonlocal ones. It is shown that for certain classes of logic programs, learnablity from positive data is equivalent to limiting identification of bounds for the number of clauses and the number of local variables. This reduces the learning problem finding two integers. This cleaner framework generalizes all the known results and establishes learnability of new classes.

Abstract: This study uses the foreign language classroom to examine students' beliefs about learning, perceptions of goal attainment, and motivation to continue language study. Survey and interview results indicated students’ attributions for success and failure and their expectations for certain subjects’ learnability played a role in the relationship between goal attainment and volition. It appears that over-effaciousness negatively affected student motivation. For other students who felt they were "bad at languages," their negative beliefs increased their motivation to study. Suggestions for how these results apply to other disciplines and interventions for increasing student motivation are offered.