Generalization (learning)

Abstract: To select effective actions in complex environments, intelligent agents need to generalize from past experience. World models can represent knowledge about the environment to facilitate such generalization. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many potential ways for deriving behaviors from them. We present Dreamer, a reinforcement learning agent that solves long-horizon tasks purely by latent imagination. We efficiently learn behaviors by backpropagating analytic gradients of learned state values through trajectories imagined in the compact state space of a learned world model. On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance.

Abstract: The 1st part of this article evaluates the extent to which 2 elemental theories of conditioning, stimulus sampling theory and the Rescorla-Wagner (1972) theory, are able to account for the influence of similarity on discrimination learning. A number of findings are reviewed that are inconsistent with predictions derived from these theories, either in their present form or in various modified forms. The 2nd part of the article is concerned with developing an alternative, configural account for discrimination learning. In contrast to previous configural theories, the present version is set within the framework of a connectionist network. The studies of stimulus generalization by Pavlov (1927) provided the first demonstration of the influence of similarity on conditioning. Once a dog had been trained to salivate to a tone of a given frequency, it was found that tones of other frequencies would also elicit this response but to a lesser degree. The magnitude of this generalization decrement was directly related to the extent of the difference between the training and the test stimuli. Pavlov's findings have been replicated on many occasions with a wide range of species. There have also been a number of attempts to understand these findings.

Abstract: Statistical learning is a rapid and robust mechanism that enables adults and infants to extract patterns embedded in both language and visual domains Statistical learning operates implicitly, without instruction, through mere exposure to a set of input stimuli However, much of what learners must acquire about a structured domain consists of principles or rules that can be applied to novel inputs It has been claimed that statistical learning and rule learning are separate mechanisms; in this article, however, we review evidence and provide a unifying perspective that argues for a single statistical-learning mechanism that accounts for both the learning of input stimuli and the generalization of learned patterns to novel instances The balance between instance-learning and generalization is based on two factors: the strength of perceptual and cognitive biases that highlight structural regularities, and the consistency of elements’ contexts (unique vs overlapping) in the input