Controlled active vision

TL;DR: The framework of "controlled active vision" is presented which states that a controlled motion of a vision sensor can maximize the performance of any robotic algorithm that involves a moving vision sensor.

Abstract: Current robotic systems lack the flexibility of dynamic interaction with the environment. The use of sensors can make the robotic systems more flexible. Among the different types of sensors, vision sensors play a critical role. In this thesis, we present the framework of "controlled active vision" which states that a controlled motion of a vision sensor can maximize the performance of any robotic algorithm that involves a moving vision sensor. The research in this thesis is a bridge between conventional vision and control theory research. One of the applications of this framework is real-time robotic (eye-in-hand configuration) visual servoing around static or moving arbitrary 3-D objects. We formulate the problem of visual tracking and servoing as a problem of combining control with computer vision. Algorithms that incorporate the use of multiple windows and numerically stable confidence measures are combined with stochastic controllers in order to provide a satisfactory solution to the tracking problem. The special relations between the characteristics of computer vision and control algorithms are highlighted through a series of experimental results. One of the most important contributions of this work is the introduction of adaptive control techniques in order to compensate for inaccurate modeling of the environment, such as depth estimation. One example is in the case of inaccurate knowledge of the features' depth where the displacements are fed to adaptive SISO controllers that estimate the desired robot motion. Our recent work extends the algorithms to full 3-D tracking and servoing. In one of the experiments, the manipulator repositions itself with respect to the static target. In another experiment, the manipulator tracks the full 3-D motion of the target without having accurate information about the system's parameters. The human operator can be integrated with the autonomous servoing modules through a sophisticated system architecture. The performance of the proposed algorithms has been tested on two real systems, the CMU DD Arm II and the TROIKABOT.