We maintain that the key to intelligent fusion of disparate sensory information is to provide an effective model of sensor capabilities A sensor model is an abstraction of the actual sensing process It describes the information a sensor is able to provide, how this information is limited by the environment, how it can be enhanced by information obtained from other sensors, and how it may be improved by active use of the physical sensing device The importance of having a model of sensor performance is that capabilities can be estimated a priori and, thus, sensor strategies developed in line with information requirements

Sensor models and multisensor integration

A solution to the correspondence problem for stereopsis is proposed using the differences in the complex phase of local spatial frequency components. One-dimensional spatial Gabor filters (Gabor 1946; Marcelja 1980), at different positions and spatial frequencies are convolved with each member of a stereo pair. The difference between the complex phase at corresponding points in the two images is used to find the stereo disparity. Disparity values are combined across spatial frequencies for each image location. Three-dimensional depth maps have been computed from real images under standard lighting conditions, as well as from random-dot stereograms (Julesz 1971). The algorithm can discriminate disparities significantly smaller than the width of a pixel. It is possible that a similar mechanism might be used in the human visual system.

Stereo disparity computation using Gabor filters

Understanding dynamic scenes

Despite the fact that image understanding and natural language processing constitute two major areas of AI, there have only been a few attempts toward the integration of computer vision and the generation of natural language expressions for the description of image sequences. In this contribution we will report on practical experience gained in the project Vitra (VIsual TRAnslator) concerning the design and construction of integrated knowledge-based systems capable of translating visual information into natural language descriptions. In Vitra different domains, like traffic scenes and short sequences from soccer matches, have been investigated.

/pdf/visual-translator-linking-perceptions-and-natural-language-541x4zccml.pdf

VIsual TRAnslator: Linking perceptions and natural language descriptions

This thesis describes a connectionist model which learns to perceive spatial events and relations in simple movies of 2-dimensional objects, so as to name the events and relations as a speaker of a particular natural language would Thus, the model learns perceptually grounded semantics for natural language spatial terms
Natural languages differ--sometimes dramatically--in the ways in which they structure space The aim here has been to have the model be able to perform this learning task for terms from any natural language, and to have learning take place in the absence of explicit negative evidence, in order to rule out ad hoc solutions and to approximate the conditions under which children learn
The central focus of this thesis is a connectionist system which has succeeded in learning spatial terms from a number of different languages The design and construction of this system have resulted in several technical contributions The first is a very simple but effective means of learning without explicit negative evidence, taking positive instances of other concepts as implicit negative instances of the concept being learned, and deliberately weakening the evidence from these implicit negatives so as to reduce the effect of false implicit negatives in cases where concepts overlap This method is shown to be effective even in situations involving a high degree of overlap among concepts In addition, this thesis presents the notion of partially-structured connectionism, a marriage of structured and unstructured network design techniques capturing the flexibility afforded by unstructured networks, and the tractability in learning and improved generalization ability that result from highly structured network design Finally, the idea of learning within highly specialized structural devices is introduced, with the purpose of restricting the search during learning to a set of options known in advance to be of possible relevance
Scientifically, the primary result of the work described here is a computational model of the acquisition of visually grounded semantics This model is shown to successfully learn terms for spatial events and relations from a range of languages with widely differing spatial systems, including English, Mixtec (a Mexican Indian language), German, Bengali, and Russian The model exhibits prototype effects which roughly match human intuitions, and extensions to the core system model the linguistic phenomena of polysemy and deixis While no claims are made regarding structure-to-structure correspondences between architectural elements of the model and structures in the brain, the model is informed by a number of neuroscientific results And perhaps most importantly, the model does more than just recapitulate the data; it also generates a number of falsifiable linguistic predictions regarding the sorts of semantic features, and combinations of features, one might expect to find in lexemes for spatial events and relations in the world's natural languages

The acquisition of lexical semantics for spatial terms: a connectionist model of perceptual categorization

LandScan (LANguage Driven SCene ANalysis) is presented as an integrated vision system which covers most levels of both vision and natural language processing. Computations are both data-driven and query-driven. In the report we focus on the design of the vision and control modules. Future work will investigate in more detail the design of the natural language interface. The data-driven system employs active control of stereo cameras for image acquisition, and dynamically constructs a surface model from multiple aerial views of an urban scene. The query-driven system allows the user's natural language queries to focus analysis to pertinent regions of the scene. This is different than many image understanding systems which present a symbolic description of the entire scene regardless of what portions of that picture are actually of interest.

/pdf/landscan-a-natural-language-and-computer-vision-system-for-1b65jdt5d3.pdf

LandScan: a natural language and computer vision system for analyzing aerial images

Visual hyperacuity, acuity an order of magnitude finer than the optical resolution of a diffraction-limited visual system, is described and analyzed in the terms of information theory. It is shown that in principle, the computation and representation of both luminance and edge features can be performed with a precision commensurate with hyperacuity thresholds and human abilities. Luminance features are represented as the centroid of the intensity distribution. Edge features are represented as zero-crossings in the difference of two filtered intensity distributions; this method is shown to be very economical. Algorithms are formulated in accord with the different representations, and tested with vernier acuity tasks. The results indicate that both methods can extract relative position with an accuracy better than 1 s arc.

Visual hyperacuity: representation and computation of high precision position information

This paper addresses the problem of merging multiple views of a static scene into a common coordinate frame, explicitly considering uncertainty. It assumes that a static world is observed by an agile vision system, whose movements are known with a limited precision, and whose observations are inaccurate and incomplete. It concentrates on acquiring uncertain three-dimensional information from multiple views, rather than on modeling or representing the information at higher levels of abstraction. Two particular problems receive attention: identifying the transformation between two viewing positions; and understanding how errors and uncertainties propagate as a result of applying the transformation. The first is solved by identifying the forward kinematics of the agile camera system. The second is solved by first treating a measurement of camera position and orientation as a uniformly distributed random vector whose component variances are related to the resolution of the encoding potentiometers, then treating an object position measurement as a normally distributed random vector whose component variances are experimentally derived, and finally determining the uncertainty of the merged points as functions of these variances. Comments University of Pennsylvania Department of Computer and Information Science Technical Report No. MSCIS-88-11. This technical report is available at ScholarlyCommons: http://repository.upenn.edu/cis_reports/465 INTEGRATING MULTIPLE UNCERTAIN VIEW OF A STATIC SCENE ACQUIRED BY AN AGILE CAMERA SYSTEM Eric Krotkov Ralf Kories MS-CIS-88-11 GRASP LAB 135 Department of Computer and Information Science School of Engineering and Applied Science University of Pennsylvania Philadelphia, PA 191 04

/pdf/integrating-multiple-uncertain-views-of-a-static-scene-4qpa28ighl.pdf

Integrating Multiple Uncertain Views of a Static Scene Acquired by an Agile Camera System

We present a practical method for computing range from stereo disparities with verging cameras, a theoretical model of the expected range errors, and an analysis of their experimentally determined components. To compute stereo disparities we first extract linear features and then match them using a hypothesize-and-verify method. To compute range we derive the relationship between object distance, vergence angle and disparity, and determine its parameters by a calibration procedure. The expected range errors are due to (i) residuals in the calibrated coefficients, (ii) feature localization errors, and (iii) mistaken matches. Overall, the range of objects lying between 1 and 3 m from the cameras and with a restricted field of view can be computed to 2.5 percent/m, excluding mistaken matches. Including mistaken matches results in performance an order of magnitude worse, leading us to suggest some practical methods to identify them.

Ranging Errors with Verging Stereo Cameras

This paper is a first step toward integrating sensor measurements of distance. Its major contribution is to identify and present qualitative models for the errors and mistakes introduced in three particular computer vision distance measurements: range from focus, range from point-based stereo, and range from line-based stereo.

Models of errors and mistakes in machine perception: first results for computer vision range measurements

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Eric Krotkov

Author Tools

Chat about Author