Abstract: What determines what we see? In contrast to the traditional "modular" understanding of perception, according to which visual processing is encapsulated from higher-level cognition, a tidal wave of recent research alleges that states such as beliefs, desires, emotions, motivations, intentions, and linguistic representations exert direct top-down influences on what we see. There is a growing consensus that such effects are ubiquitous, and that the distinction between perception and cognition may itself be unsustainable. We argue otherwise: none of these hundreds of studies - either individually or collectively - provide compelling evidence for true top-down effects on perception, or "cognitive penetrability". In particular, and despite their variety, we suggest that these studies all fall prey to only a handful of pitfalls. And whereas abstract theoretical challenges have failed to resolve this debate in the past, our presentation of these pitfalls is empirically anchored: in each case, we show not only how certain studies could be susceptible to the pitfall (in principle), but how several alleged top-down effects actually are explained by the pitfall (in practice). Moreover, these pitfalls are perfectly general, with each applying to dozens of other top-down effects. We conclude by extracting the lessons provided by these pitfalls into a checklist that future work could use to convincingly demonstrate top-down effects on visual perception. The discovery of substantive top-down effects of cognition on perception would revolutionize our understanding of how the mind is organized; but without addressing these pitfalls, no such empirical report will license such exciting conclusions. Language: en

Abstract: Numerous single-image super-resolution algorithms have been proposed in the literature, but few studies address the problem of performance evaluation based on visual perception. While most super-resolution images are evaluated by fullreference metrics, the effectiveness is not clear and the required ground-truth images are not always available in practice. To address these problems, we conduct human subject studies using a large set of super-resolution images and propose a no-reference metric learned from visual perceptual scores. Specifically, we design three types of low-level statistical features in both spatial and frequency domains to quantify super-resolved artifacts, and learn a two-stage regression model to predict the quality scores of super-resolution images without referring to ground-truth images. Extensive experimental results show that the proposed metric is effective and efficient to assess the quality of super-resolution images based on human perception.

Abstract: Psychological research results have confirmed that people can have different emotional reactions to different visual stimuli. Several papers have been published on the problem of visual emotion analysis. In particular, attempts have been made to analyze and predict people's emotional reaction towards images. To this end, different kinds of hand-tuned features are proposed. The results reported on several carefully selected and labeled small image data sets have confirmed the promise of such features. While the recent successes of many computer vision related tasks are due to the adoption of Convolutional Neural Networks (CNNs), visual emotion analysis has not achieved the same level of success. This may be primarily due to the unavailability of confidently labeled and relatively large image data sets for visual emotion analysis. In this work, we introduce a new data set, which started from 3+ million weakly labeled images of different emotions and ended up 30 times as large as the current largest publicly available visual emotion data set. We hope that this data set encourages further research on visual emotion analysis. We also perform extensive benchmarking analyses on this large data set using the state of the art methods including CNNs.

Abstract: Time is a universal psychological dimension, but time perception has often been studied and discussed in relative isolation. Increasingly, researchers are searching for unifying principles and integrated models that link time perception to other domains. In this review, we survey the links between temporal cognition and other psychological processes. Specifically, we describe how subjective duration is affected by nontemporal stimulus properties (perception), the allocation of processing resources (attention), and past experience with the stimulus (memory). We show that many of these connections instantiate a "processing principle," according to which perceived time is positively related to perceptual vividity and the ease of extracting information from the stimulus. This empirical generalization generates testable predictions and provides a starting-point for integrated theoretical frameworks. By outlining some of the links between temporal cognition and other domains, and by providing a unifying principle for understanding these effects, we hope to encourage time-perception researchers to situate their work within broader theoretical frameworks, and that researchers from other fields will be inspired to apply their insights, techniques, and theorizing to improve our understanding of the representation and judgment of time. (PsycINFO Database Record

Abstract: How we perceive what we see depends on the context in which we see it, such as what we are doing at the time. For example, we perceive a park landscape differently when we are running through it than when we are sitting on a park bench. Behavior can also alter neuronal responses in the brain. Indeed, the neurons in the part of the brain that receives information related to vision (known as the visual cortex) respond differently to visual stimuli when an animal is moving compared to when the animal is still. However, while some recent studies revealed that specific types of neurons become more or less responsive during movement, others reported the opposite results. One hypothesis that would explain these contradictory findings would be if the way that behavior, in this case movement, affects neuronal responses also depends on the external context in which the movement happens. Now, Pakan et al. have tested this hypothesis by imaging the activity of different types of neurons in the primary visual cortex of mice that were either running on a treadmill or staying still. The experiments were conducted in two different contexts: in total darkness (in which the mice could not see) and in the presence of display screens (which provided the mice with visual stimulation). Pakan et al. confirmed that running does indeed affect the activity of specific neurons in different ways in different contexts. For example, when the mice received visual stimulation, the three main classes of neurons that send inhibitory signals in the visual cortex became more active during running. However, when the mouse ran in the dark, two of these neuron types became more active during running while the third type of neuron was unresponsive. This finding reveals more about the dynamic nature of inhibitory activity that strongly depends on the animal’s behaviour. It also shows how these neurons influence the excitatory neurons in the visual cortex, which send information to the rest of the brain for further processing towards perception. The next step will be to identify what precise mechanism makes these neurons respond differently in unique contexts, and to tease apart how these movement-dependent signals affect the way animals perceive visual stimuli.

Abstract: The authors find that activity in rodent visual cortex can depend on the animal's location in a virtual environment and can predict upcoming visual stimuli. Omitting a stimulus that a mouse expects to see results in a strong mismatch signal, implying that visual cortex compares visual signals to expectations in familiar environments.

Abstract: Models of predictive coding frame perception as a generative process in which expectations constrain sensory representations. These models account for expectations about how a stimulus will move or change from moment to moment, but do not address expectations about what other, distinct stimuli are likely to appear based on prior experience. We show that such memory-based expectations in human visual cortex are related to the hippocampal mechanism of pattern completion.

Abstract: The processing of interoceptive signals in the insular cortex is thought to underlie self-awareness. However, the influence of interoception on visual awareness and the role of the insular cortex in this process remain unclear. Here, we show in a series of experiments that the relative timing of visual stimuli with respect to the heartbeat modulates visual awareness. We used two masking techniques and show that conscious access for visual stimuli synchronous to participants' heartbeat is suppressed compared with the same stimuli presented asynchronously to their heartbeat. Two independent brain imaging experiments using high-resolution fMRI revealed that the insular cortex was sensitive to both visible and invisible cardio-visual stimulation, showing reduced activation for visual stimuli presented synchronously to the heartbeat. Our results show that interoceptive insular processing affects visual awareness, demonstrating the role of the insula in integrating interoceptive and exteroceptive signals and in the processing of conscious signals beyond self-awareness.

Abstract: Research on aging and vision has increased dramatically over the past few decades. Changes in our visual capacities in later adulthood have the potential to impact our ability to perform common everyday visual tasks such as recognizing objects, reading, engaging in mobility activities, and driving, thus influencing the quality of our life and well-being. Here, we discuss several common visual problems in older adults that cause performance problems in the visual tasks of everyday living and when exacerbated are related to the development of common eye conditions and diseases of aging.

Abstract: A fundamental issue in visual attention is the relationship between the useful field of view (UFOV), the region of visual space where information is encoded within a single fixation, and eccentricity. A common assumption is that impairing attentional resources reduces the size of the UFOV (i.e., tunnel vision). However, most research has not accounted for eccentricity-dependent changes in spatial resolution, potentially conflating fixed visual properties with flexible changes in visual attention. Williams (1988, 1989) argued that foveal loads are necessary to reduce the size of the UFOV, producing tunnel vision. Without a foveal load, it is argued that the attentional decrement is constant across the visual field (i.e., general interference). However, other research asserts that auditory working memory (WM) loads produce tunnel vision. To date, foveal versus auditory WM loads have not been compared to determine if they differentially change the size of the UFOV. In two experiments, we tested the effects of a foveal (rotated L vs. T discrimination) task and an auditory WM (N-back) task on an extrafoveal (Gabor) discrimination task. Gabor patches were scaled for size and processing time to produce equal performance across the visual field under single-task conditions, thus removing the confound of eccentricity-dependent differences in visual sensitivity. The results showed that although both foveal and auditory loads reduced Gabor orientation sensitivity, only the foveal load interacted with retinal eccentricity to produce tunnel vision, clearly demonstrating task-specific changes to the form of the UFOV. This has theoretical implications for understanding the UFOV.

Abstract: We experience the world as a seamless stream of percepts. However, intriguing illusions and recent experiments suggest that the world is not continuously translated into conscious perception. Instead, perception seems to operate in a discrete manner, just like movies appear continuous although they consist of discrete images. To explain how the temporal resolution of human vision can be fast compared to sluggish conscious perception, we propose a novel conceptual framework in which features of objects, such as their color, are quasi-continuously and unconsciously analyzed with high temporal resolution. Like other features, temporal features, such as duration, are coded as quantitative labels. When unconscious processing is "completed," all features are simultaneously rendered conscious at discrete moments in time, sometimes even hundreds of milliseconds after stimuli were presented.

Abstract: Categorization is our ability to flexibly assign sensory stimuli into discrete, behaviorally relevant groupings. Categorical decisions can be used to study decision making more generally by dissociating category identity of stimuli from the actions subjects use to signal their decisions. Here we discuss the evidence for such abstract categorical encoding in the primate brain and consider the relationship with other perceptual decision paradigms. Recent work on visual categorization has examined neuronal activity across a hierarchically organized network of cortical areas in monkeys trained to group visual stimuli into arbitrary categories. This has revealed a transformation of visual-feature encoding in early visual cortical areas into more flexible categorical representations in downstream parietal and prefrontal areas. These neuronal category representations are encoded as abstract internal cognitive states because they are not rigidly linked with either specific sensory stimuli or the actions that the monkeys use to signal their categorical choices.

Abstract: Humans assimilate information from the traffic environment mainly through visual perception. Obviously, the dominant information required to conduct a vehicle can be acquired with visual sensors. However, in contrast to most other sensor principles, video signals contain relevant information in a highly indirect manner and hence visual sensing requires sophisticated machine vision and image understanding techniques. This paper provides an overview on the state of research in the field of machine vision for intelligent vehicles. The functional spectrum addressed covers the range from advanced driver assistance systems to autonomous driving. The organization of the article adopts the typical order in image processing pipelines that successively condense the rich information and vast amount of data in video sequences. Data-intensive low-level “early vision” techniques first extract features that are later grouped and further processed to obtain information of direct relevance for vehicle guidance. Recognition and classification schemes allow to identify specific objects in a traffic scene. Recently, semantic labeling techniques using convolutional neural networks have achieved impressive results in this field. High-level decisions of intelligent vehicles are often influenced by map data. The emerging role of machine vision in the mapping and localization process is illustrated at the example of autonomous driving. Scene representation methods are discussed that organize the information from all sensors and data sources and thus build the interface between perception and planning. Recently, vision benchmarks have been tailored to various tasks in traffic scene perception that provide a metric for the rich diversity of machine vision methods. Finally, the paper addresses computing architectures suited to real-time implementation. Throughout the paper, numerous specific examples and real world experiments with prototype vehicles are presented.

Abstract: Psychological research results have confirmed that people can have different emotional reactions to different visual stimuli. Several papers have been published on the problem of visual emotion analysis. In particular, attempts have been made to analyze and predict people's emotional reaction towards images. To this end, different kinds of hand-tuned features are proposed. The results reported on several carefully selected and labeled small image data sets have confirmed the promise of such features. While the recent successes of many computer vision related tasks are due to the adoption of Convolutional Neural Networks (CNNs), visual emotion analysis has not achieved the same level of success. This may be primarily due to the unavailability of confidently labeled and relatively large image data sets for visual emotion analysis. In this work, we introduce a new data set, which started from 3+ million weakly labeled images of different emotions and ended up 30 times as large as the current largest publicly available visual emotion data set. We hope that this data set encourages further research on visual emotion analysis. We also perform extensive benchmarking analyses on this large data set using the state of the art methods including CNNs.

Abstract: In this study, we concurrently investigated 3 possible causes of dyslexia-a phonological deficit, visual stress, and a reduced visual attention span-in a large population of 164 dyslexic and 118 control French children, aged between 8 and 13 years old. We found that most dyslexic children showed a phonological deficit, either in terms of response accuracy (92.1% of the sample), speed (84.8%), or both (79.3%). Deficits in visual attention span, as measured by partial report ability, affected 28.1% of dyslexic participants, all of which also showed a phonological deficit. Visual stress, as measured by subjective reports of visual discomfort, affected 5.5% of dyslexic participants, not more than controls (8.5%). Although phonological variables explained a large amount of variance in literacy skills, visual variables did not explain any additional variance. Finally, children with comorbid phonological and visual deficits did not show more severe reading disability than children with a pure phonological deficit. These results (a) confirm the importance of phonological deficits in dyslexia; (b) suggest that visual attention span may play a role, but a minor one, at least in this population; (c) do not support any involvement of visual stress in dyslexia. Among the factors that may explain some differences with previously published studies, the present sample is characterized by very stringent inclusion criteria, in terms of the severity of reading disability and in terms of exclusion of comorbidities. This may exacerbate the role of phonological deficits to the detriment of other factors playing a role in reading acquisition. (PsycINFO Database Record

Abstract: Ensemble coding supports rapid extraction of visual statistics about distributed visual information. Researchers typically study this ability with the goal of drawing conclusions about how such coding extracts information from natural scenes. Here we argue that a second domain can serve as another strong inspiration for understanding ensemble coding: graphs, maps, and other visual presentations of data. Data visualizations allow observers to leverage their ability to perform visual ensemble statistics on distributions of spatial or featural visual information to estimate actual statistics on data. We survey the types of visual statistical tasks that occur within data visualizations across everyday examples, such as scatterplots, and more specialized images, such as weather maps or depictions of patterns in text. We divide these tasks into four categories: identification of sets of values, summarization across those values, segmentation of collections, and estimation of structure. We point to unanswered questions for each category and give examples of such cross-pollination in the current literature. Increased collaboration between the data visualization and perceptual psychology research communities can inspire new solutions to challenges in visualization while simultaneously exposing unsolved problems in perception research.

Abstract: Visual working memory (VWM) has a limited capacity. This limitation can be mitigated by the use of focused attention: if attention is drawn to the relevant working memory content before test, performance improves (the so-called retro-cue benefit). This study tests 2 explanations of the retro-cue benefit: (a) Focused attention protects memory representations from interference by visual input at test, and (b) focusing attention enhances retrieval. Across 6 experiments using color recognition and color reproduction tasks, we varied the amount of color interference at test, and the delay between a retrieval cue (i.e., the retro-cue) and the memory test. Retro-cue benefits were larger when the memory test introduced interfering visual stimuli, showing that the retro-cue effect is in part because of protection from visual interference. However, when visual interference was held constant, retro-cue benefits were still obtained whenever the retro-cue enabled retrieval of an object from VWM but delayed response selection. Our results show that accessible information in VWM might be lost in the processes of testing memory because of visual interference and incomplete retrieval. This is not an inevitable state of affairs, though: Focused attention can be used to get the most out of VWM. (PsycINFO Database Record

Abstract: The nature of the visual representation for words has been fiercely debated for over 150 y. We used direct brain stimulation, pre- and postsurgical behavioral measures, and intracranial electroencephalography to provide support for, and elaborate upon, the visual word form hypothesis. This hypothesis states that activity in the left midfusiform gyrus (lmFG) reflects visually organized information about words and word parts. In patients with electrodes placed directly in their lmFG, we found that disrupting lmFG activity through stimulation, and later surgical resection in one of the patients, led to impaired perception of whole words and letters. Furthermore, using machine-learning methods to analyze the electrophysiological data from these electrodes, we found that information contained in early lmFG activity was consistent with an orthographic similarity space. Finally, the lmFG contributed to at least two distinguishable stages of word processing, an early stage that reflects gist-level visual representation sensitive to orthographic statistics, and a later stage that reflects more precise representation sufficient for the individuation of orthographic word forms. These results provide strong support for the visual word form hypothesis and demonstrate that across time the lmFG is involved in multiple stages of orthographic representation.

Abstract: Interpreting visual scenes typically requires us to accumulate information from multiple locations in a scene. Using a novel gaze-contingent paradigm in a visual categorization task, we show that participants' scan paths follow an active sensing strategy that incorporates information already acquired about the scene and knowledge of the statistical structure of patterns. Intriguingly, categorization performance was markedly improved when locations were revealed to participants by an optimal Bayesian active sensor algorithm. By using a combination of a Bayesian ideal observer and the active sensor algorithm, we estimate that a major portion of this apparent suboptimality of fixation locations arises from prior biases, perceptual noise and inaccuracies in eye movements, and the central process of selecting fixation locations is around 70% efficient in our task. Our results suggest that participants select eye movements with the goal of maximizing information about abstract categories that require the integration of information from multiple locations.

Abstract: Prior expectations have a powerful influence on perception, biasing both decision and confidence. However, how this occurs at the neural level remains unclear. It has been suggested that spontaneous alpha-band neural oscillations represent rhythms of the perceptual system that periodically modulate perceptual judgments. We hypothesized that these oscillations instantiate the effects of expectations. While collecting scalp EEG, participants performed a detection task that orthogonally manipulated perceptual expectations and attention. Trial-by-trial retrospective confidence judgments were also collected. Results showed that, independent of attention, prestimulus occipital alpha phase predicted the weighting of expectations on yes/no decisions. Moreover, phase predicted the influence of expectations on confidence. Thus, expectations periodically bias objective and subjective perceptual decision-making together before stimulus onset. Our results suggest that alpha-band neural oscillations periodically transmit prior evidence to visual cortex, changing the baseline from which evidence accumulation begins. In turn, our results inform accounts of how expectations shape early visual processing.

Abstract: A number of studies have demonstrated that individuals with autism spectrum disorders (ASDs) are faster or more successful than typically developing control participants at various visual-attentional tasks (for reviews, see Dakin and Frith in Neuron 48:497–507, 2005; Simmons et al in Vis Res 49:2705–2739, 2009) This “ASD advantage” was first identified in the domain of visual search by Plaisted et al (J Child Psychol Psychiatry 39:777–783, 1998) Here we survey the findings of visual search studies from the past 15 years that contrasted the performance of individuals with and without ASD Although there are some minor caveats, the overall consensus is that—across development and a broad range of symptom severity—individuals with ASD reliably outperform controls on visual search The etiology of the ASD advantage has not been formally specified, but has been commonly attributed to ‘enhanced perceptual discrimination’, a superior ability to visually discriminate between targets and distractors in such tasks (eg O’Riordan in Cognition 77:81–96, 2000) As well, there is considerable evidence for impairments of the attentional network in ASD (for a review, see Keehn et al in J Child Psychol Psychiatry 37:164–183, 2013) We discuss some recent results from our laboratory that support an attentional, rather than perceptual explanation for the ASD advantage in visual search We speculate that this new conceptualization may offer a better understanding of some of the behavioral symptoms associated with ASD, such as over-focusing and restricted interests