Abstract: The structure from motion problem has been extensively studied in the field of computer vision. Yet, the bulk of the existing work assumes that the scene contains only a single moving object. The more realistic case where an unknown number of objects move in the scene has received little attention, especially for its theoretical treatment. We present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images. The method does not require prior knowledge of the number of objects, nor is dependent on any grouping of features into an object at the image level. For this purpose, we introduce a mathematical construct of object shapes, called the shape interaction matrix, which is invariant to both the object motions and the selection of coordinate systems. This invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects. Once the structure is computed, it allows for segmenting features into objects by the process of transforming it into a canonical form, as well as recovering the shape and motion of each object. >

Abstract: Presents an algorithm for determining 3D motion and structure from correspondences of line segments between two perspective images. To the author's knowledge, this paper is the first investigation of use of line segments in motion and structure from motion. Classical methods use their geometric abstraction, namely straight lines, but then three images are necessary for the motion and structure determination process. In this paper the author shows that it is possible to recover motion from two views when using line segments. The assumption used is that two matched line segments contain the projection of a common part of the corresponding line segment in space, i.e., they overlap. Indeed, this is what the author uses to match line segments between different views. This assumption constrains the possible motion between two views to an open set in motion parameter space. A heuristic, consisting of maximizing the overlap, leads to a unique solution. Both synthetic and real data have been used to test the proposed algorithm, and excellent results have been obtained with real data containing a relatively large set of line segments.

Abstract: A common framework for rapid scene analysis for detecting scene changes in compressed motion JPEG and MPEG videos is proposed. We develop algorithms to detect both abrupt and gradual scene changes. The algorithms operate directly on the DC sequence which can be easily extracted from motion JPEG and MPEG compressed video without decompression. The DC images capture most of the essential "global" information, but is of a small fraction of the original data size. Operating on these images offers significant computation savings. Experimental results show that the proposed algorithms are fast and effective in detecting abrupt scene changes and gradual transitions.

Abstract: The accuracy of cine phase-contrast magnetic resonance (MR) imaging for motion analysis was evaluated. By using a rotating phantom and postprocessing algorithm for phase tracking, errors arising during data acquisition were identified and compensation methods were developed. A spatially varying background phase offset in the velocity images was found to be due to eddy current-induced fields. The magnitude of the offset was in the range of 0–20 cm/sec, which is of the same order of magnitude as cardiac contractile velocities. Background offset is thus an important source of error in tracking cardiac motion. Study of different tracking algorithms revealed the need for an integration scheme using motion terms higher than velocity. Also, considerable improvement in the accuracy and stability of the predicted trajectories was obtained by averaging the trajectories proceeding both forward and backward in time from the starting point. With the algorithm developed, the motion of the phantom was tracked through a complete rotation of the phantom to an accuracy of 2 pixels.

Abstract: A novel active stereo vision system with a pair of foveated wide angle lenses is presented. The projection curve of the lens is designed so that it facilitates active vision algorithms for motion analysis, object identification, and precise fixation. A pair of such lenses are mounted on a specially designed active stereo vision platform. It is compact and light so that it can be mounted on a mobile robot or a manipulator. A real time stereo tracking system is constructed using the platform, a dual-processor servo controller, and pipelined image processors with a multi-processor backend.

Abstract: A new high-speed system for recording, processing, and analyzing vocal fold vibrations has been developed. Results obtained with this system are discussed from the technical and the medical points of view. Laryngeal movement can be recorded with a digital high-speed camera, at a maximum speed of about 5 600 frames/s and the sequence can contain as many as 8 192 single frames. Application specific software for adaptive, semiautomatic, motion analysis is used to calculate and plot the glottograms for selected points on each vocal cord. From the data obtained, we can calculate speed, acceleration rates, the fundamental frequency, amplitudes, and perturbation parameters. This approach to laryngeal examination, based on the digital films, motion plots and characteristic statistics, is a practicable method that promotes the possibilities of quantitative and graphic analysis of the moving vocal folds and overcomes the disadvantages of the currently common examination methods.

Abstract: Since general automatic assembly motion planning using a geometric CAD model is computationally difficult, automation of a general assembly task is not feasible. However, if a human operator roughly specifies an assembly motion, the remainder of the process except for planning can be automated. We construct a teaching system for assembly tasks according to the above concept. This paper describes our teaching system for assembly tasks by using a position/force simulator. We show how to construct the position/force simulator and analyse the hybrid position/force control which is needed when dealing with general rotational motion. We show the example of extracting a sequence of contact state transitions from the motion which the operator performs in the position simulator of our system. The sequence of contact states automatically extracted from the motion shown by the operator makes it feasible to achieve an error-tolerant automated assembly motion. If a sequence of contact states is obtained, studies of the automatic assembly task system can progress based upon this information.

Abstract: This paper presents a detailed analysis of the differential approach for motion estimation in video image sequences. The models considered are defined either by a parametric approach, or by a physical approach (in terms of parameters of the pick-up equipment, movement and object structures). The relationships between the 2D and the 3D approaches are examined. The ambiguities inherent in a physical interpretation of a set of descriptors identified from image sequences are underlined. The critical points when using differential estimators are discussed, in particular, we study several classical image processing tools which improve the convergence rate of these estimators (hierarchical analysis, multiresolution, spatial interpolation of the luminance…). Definition and tuning of gains, initialization stage, cross-dependence between image segmentation and identification of the parameters associated to each region (as well as the duality between top-down and bottom-up approaches), which partly condition the behavior of the algorithms, are studied too. Results in terms of motion and segmentation maps, images predicted from one or several previous images by motion compensation, convergence curves of some of the proposed iterative algorithms illustrate this paper. Finally, we draw from these theoretical developments and the associated simulations some key elements for an effective implementation of a complete motion estimation scheme. We conclude by some perspectives for future work.

Abstract: In the future, robots may perform cooperative tasks with humans in daily life In this paper, the authors focus on a hand-over motion as an example of cooperative work between a human and a robot, and propose an algorithm which enables a robot to perform a human-like motion First the authors analyze trajectories and velocity patterns of a hand-over motion performed by two humans The experimental results show that a receiver's motion during hand-over has some typical characteristics The authors then confirm that a human-like motion can be generated using these characteristics Finally, the authors plan the robot's motion considering these results Initially, two kinds of potential fields are used to generate a motion command which leads the robot along a trajectory similar to that followed by the human In addition, more precise motion is considered at the end of the hand-over operation to guarantee accurate positioning and to soften the shock of contact Simulation results show the validity of the proposed method

Abstract: This paper deals with the problem of analysing sequences of images of rigid point objects taken by uncalibrated cameras. It is assumed that the correspondences between the points in the different images are known. The paper introduces a new framework for this problem. Corresponding points in a sequence of n images are related to each other by a fixed n-linear form. This form is an object invariant property, closely linked to the motion of the camera relative to the fixed world. We first describe a reduced setting in which these multilinear forms are easier to understand and analyse. This new formulation of the multilinear forms is then extended to the calibrated case. This formulation makes apparent the connection between camera motion, camera matrices and multilinear forms and the similarities between the calibrated and uncalibrated cases. These new ideas are then used to derive simple linear methods for extracting camera motion from sequences of images.

Abstract: Image stabilization is a key preprocessing step in dynamic image analysis and deals with the removal of unwanted image motion in a video sequence. This paper presents an integrated algorithm for the problem of image stabilization. The algorithm combines various visual cues such as points and horizon lines, and relies on an extended Kalman filter for the estimation of parameters of interest. We study both calibrated and uncalibrated stabilization cases, and consider the problem of the selection of model dynamics for the estimation of warping parameters. Experimental results from video sequences generated from off-road vehicle platforms show good performance of stabilization algorithm.

Abstract: A 6 DOF motion system using a new parallel link mechanism is developed for purposes of evaluating human motion sensation. This motion system is made up of three five-bar mechanisms, each of which has 2 degrees of freedom (DOF) and is driven by AC servo motors. Therefore, this system is very small and low cost, but its motion area is larger and its response is better than previous hydraulic Stewart platform. Furthermore, we have developed a high-speed parallel signal-processing controller for motion control and a new 6 DOF acceleration sensing system based on the parallel sensing concept. Combining the new motion and sensory system with multivideo system, we have built up a new analytical system for evaluating wide band-width of motion sensation in getting on such vehicles as automobile or airplane.

Abstract: Human vision, machine vision, and image coding, each demand representations that are useful and efficient. The best-established techniques today are based on low-level processing. Future systems for image analysis and image coding will increasingly use image representations that involve such concepts as surfaces, lighting, transparency, etc. These representations fall in the domain of "mid-level" vision, and there is accumulating evidence of their importance in human vision. By representing images with these more sophisticated vocabularies we can increase the flexibility and efficiency of our vision and image coding systems. We are developing systems that decompose image sequences into overlapping layers, rather like the "cels" used by a traditional animator. These layers are ordered in depth, sliding over one another and being combined according to the rules of transparency and occlusion. Using the layered representation we can achieve greatly improved motion analysis and image segmentation. By applying layers to image coding we can achieve data compression far better than MPEG, and achieve frame-rate independence as a side benefit. Moreover, the image sequence is decomposed in a meaningful way, which allows flexible image editing and access.

Abstract: This paper presents an efficient, biologically-inspired early vision architecture, the dynamic retina, that is well-suited to highly active and responsive vision platforms. The dynamic retina exploits normally undesirable camera motion as a necessary step in detecting image contrast, by using dynamic receptive fields instead of traditional spatial-neighborhood operators. We analyze the continuous miniature “noise” movements made by active imaging systems, and show that they can be exploited to detect contrast. We then develop an appropriate photoreceptor response function, based on light-adaptation models for vertebrate receptors. Together, the movements and response function over time compute image contrast. The dynamic retina is also useful for motion analysis, since moving objects processed by the system leave a clear signature from which motion parameters can be extracted. Results from a number of experiments with real video sequences demonstrate the effectiveness of the system for both contrast detection and motion analysis.

Abstract: This paper describes a motion-analysis system, applied to the problem of vehicle tracking in real-world highway scenes. The system is structured in two stages. In the first one, a motion- detection algorithm performs a figure/ground segmentation, providing binary masks of the moving objects. In the second stage, vehicles are tracked for the rest of the sequence, by using Kalman filters on two state vectors, which represent each target's position and velocity. A vehicle's motion is represented by an affine model, taking into account translations and scale changes. Three types of features have been used for the vehicle's description state vectors. Two of them are contour-based: the bounding box and the centroid of the convex polygon approximating the vehicles contour. The third one is region-based and consists of the 2-D pattern of the vehicle in the image. For each of these features, the performance of the tracking algorithm has been tested, in terms of the position error, stability of the estimated motion parameters, trace of the motion model's covariance matrix, as well as computing time. A comparison of these results appears in favor of the use of the bounding box features.

Abstract: Digital image analysis appears to be more and more relevant to the study of physical phenomena involving fluid motion, and of their evolution over time. In that context, 2D deformable motion analysis is one of the important issues to be investigated. The interpretation of such deformable 2D flow fields can generally be stated as the characterization of linear models provided that first order approximations are considered in an adequate neighborhood of so-called singular points, where the velocity becomes null. This paper describes an efficient method, based on a statistical approach, which explicitly addresses these problems, and allows us to locate, characterize and track such singular points in an image sequence. It does not require the prior computation of the velocity field. The method has been validated by experiments carried out with synthetic and real examples corresponding to meteorological image sequences. In fact, the described approach can be of interest in different applications dealing with the characterization of vector fields.

Abstract: Efforts to understand human motion have been increasing in number and complexity, and will most likely prove to be a key component in human-computer interfaces. One key feature of motion in general, human motion in particular, is its dynamic nature. The present work seeks to model human motions in a manner amenable to leaning and recognition. For such application, hidden Markov models (HMMs) are employed to model semantically meaningful human movements. The data used for modeling the human motions is an approximate pose derived from a sequence of camera images. An HMM is learned for each motion class and employed as a maximum likelihood recognizer. Experiments show promising results for a set of six sport actions.

Abstract: We present a simple frame-adaptative procedure for minimizing the total rate in classical block based, motion-compensated lossless video coders. Our method can be applied to compress digital video for applications where no distortion is tolerated in the original video data. In motion-compensated video coding, motion vectors are used to improve the prediction of the current frame to be coded and must themselves be encoded with bits. We find an expression for the total rate of the current frame as a function of the number of bits allocated to the motion vectors. Then, we analytically derive the optimal motion bit allocations that minimize the total rate. We implement the video coder and present results on real video sequences, and discuss the potential applications of this work to the more interesting case of lossy video coding.

Abstract: For pt. I see ibid., p. 1386-93 (1995). An approach applying artificial neural net techniques to 3D nonrigid motion analysis is proposed. The 3D nonrigid motion of the left ventricle of a human heart is examined using biplanar cineangiography data, consisting of 3D coordinates of 30 coronary artery bifurcation points of the left ventricle and the correspondences of these points taken over 10 time instants during the heart cardiac cycle. The motion is decomposed into global rigid motion and a set of local nonrigid deformations which are coupled with the global motion. The global rigid motion can be estimated precisely as a translation vecto and a rotation matrix. Local nonrigid deformation estimation is discussed. A set of neural nets similar in structure and dynamics but different in physical size is proposed to tackle the problem of nonrigidity. These neural networks are interconnected through feedbacks. The activation function of the output layer is selected so that a feedback is involved in the output updating. The constraints are specified to ensure stable and globally consistent estimation. The objective is to find the optimal deformation matrices that satisfy the constraints for all coronary artery bifurcation points of the left ventricle. The proposed neural networks differ from other existing neural network models in their unique structure and dynamics. >

Abstract: We propose a novel approach for fusing multi-resolution stereo and motion (both rigid and non-rigid) analysis in order to complement each other's performance. A hierarchical frame-work is presented to couple motion correspondences and stereo correspondences in order to generate accurate disparity map and motion parameters. One scenario for such system is the analysis of time-varying multi-spectral observations of clouds from meteorological satellites. Our experiments involve such time-varying remote sensing stereo data sets, and the motion is typically non-rigid as the clouds undergo shape changes. Rigid motion matching may still be performed for initial fusion, and gradually raised to non-rigid motion matching as in a coarse-to-fine strategy. Both stereo disparities and motion correspondences are estimated using such multi-resolution coarse-to-fine strategy to a sub-pixel accuracy. Experimental results using time-varying data of visible channel from two satellites in geosynchronous orbit is presented for the Hurricane Frederic.

Abstract: "Weak perspective" represents a simplified projection model that approximates the imaging process when the scene is viewed under a small viewing angle and its depth relief is small relative to its distance from the viewer. We study how to generate dynamic models for estimating rigid 3D motion from weak perspective. A crucial feature in dynamic visual motion estimation is to decouple structure from motion in the estimation model. The reasons are both geometric-to achieve global observability of the model-and practical, for a structure independent motion estimator allows us to deal with occlusions and appearance of new features in a principled way. It is also possible to push the decoupling even further, and isolate the motion parameters that are affected by the so called "bas relief ambiguity" from the ones that are not. We present a novel method for reducing the order of the estimator by decoupling portions of the state space from the time evolution of the measurement constraint. We use this method to construct an estimator of full rigid motion (modulo a scaling factor) on a six dimensional state space, an approximate estimator for a four dimensional subset of the motion space, and a reduced filter with only two states. The latter two are immune to the bas relief ambiguity. We compare strengths and weaknesses of each of the schemes on real and synthetic image sequences. >

Abstract: Particle-based models and articulated models are increasingly used in synthetic image animation applications. This paper aims at showing examples of how Evolutionary Algorithms can be used as tools to build realistic physical models for image animation.

Abstract: This paper describes efforts to extract motion characteristics of a scene directly from the gray-scale data. The measurements are, by the very nature of the sampling of image data, integral values. The approach solves the ill-posedness of differentiation. A complete class of spatiotemporal operators which concisely captures the local spatiotemporal information tip to any order in space and time is proposed. Spatial and temporal scale are treated as free parameters. The operators are used to extract spatiotemporal features and to estimate the velocity field. In the estimation of the velocity field we use the generalized optic flow constraint equation, in which the signal over a region which may be subject to the flow field is conserved rather than the gray-value associated with a voxel. Examples on test images and MR-data of the Left Ventricle are shown

Abstract: Proposes an approach applying artificial neural net techniques to 3D rigid motion analysis based on sequential multiple time frames. The approach consists of two phases: (1) matching between every ...

Abstract: This paper addresses different issues of motion analysis with stereovision. The stereo system can be either strongly calibrated in the classical sense, or weakly calibrated in the sense that only the epipolar geometry is known, or even not calibrated at all so we must seek for information only from the surrounding environment by moving the cameras in it (self-calibration).

Abstract: Some of the recent image processing methods proposed for detecting object moving in 3-D space form a spatiotemporal image by placing consecutive camera images side by side in temporal order. This paper presents one such method, the directional temporal plane transform method (DTT), an image processing method that is not influenced by environmental changes. Assuming that the objects are moving on a nearly constant path, DTT first extracts significant data about these objects from each image, and then makes a 1-D data stream by projecting the data long a directional axis parallel to the moving loci. After that, by placing the 1-D data from individual frames side by side in temporal order, the spatiotemporal image is transformed into a 2-D image on a directional temporal plane. Since the object motion is represented by regions of this 2-D image, moving objects can be detected by simple 2-D image processing. This method was used on an experimental basis to detect vehicles running along a road. Experimental results show the effectiveness of the method. An advantage of this method is its ability to detect vehicles using the same processing during both daytime and nighttime-a feature that has not been achieved with conventional methods.

Abstract: This dissertation presents a comprehensive and systematic investigation into some fundamental issues relating to creating an autonomous wheeled mobile robot (WMR). The forms of WMRs with various structures developed in the past are first classified into four groups according to the method of steering and powering. The four groups are: a. ordinary car-like robots (including passenger cars, single unit trucks, single unit buses and articulated trucks); b. dual drive robots (dual drive motors with various casters); c. synchro drive and steering robots; and d. omnidirectional robots. The concepts .of inverse and direct kinematics widely used in non-mobile. manipulators are, for the first time, introduced to WMRs, and a unified treatment of the kinematics for the four kinds of WMRs is presented. A motion feasibility and smoothness analysis for each of them is carried out, revealing the motion characteristics resulting from each of the different mechanical structures. This provides a better understanding of their motion characteristics and forms the basis for discussing the path planning problem. The concept of deviation angle interval is defined and used to explain the strange phenomenon of a pirouette. The conditions and formula for pure translation, pure rotation, straight line motion and circular motion are developed. In order to verify the correctness and to illustrate the advantages of the developed kinematic model, the simulation results from the present model are compared with the existing standards from other kinematic models. Path planning is essential for creating an autonomous robot. Various methods for dealing with the find-path problem have been developed in the past. Based on the motion. analysis of the four kinds of WMRs, a critical review of the presently available algorithms for moving a WMR among known static obstacles from a givenstart location to a given goal location is presented, and the suitability of the existing algorithms to each of the four kinds of WMRs is examined. The study shows that most of these algorithms suffer from the fundamental drawback that kinematics of the robot has not been taken into account, and thus there is no guarantee that the paths generated by these algorithms are always