TL;DR: This paper presents the first successful autonomous completion on a real RC helicopter of the following four aerobatic maneuvers: forward flip and sideways roll at low speed, tail-in funnel, and nose- in funnel using differential dynamic programming (DDP), an extension of the linear quadratic regulator (LQR).
Abstract: Autonomous helicopter flight is widely regarded to be a highly challenging control problem. This paper presents the first successful autonomous completion on a real RC helicopter of the following four aerobatic maneuvers: forward flip and sideways roll at low speed, tail-in funnel, and nose-in funnel. Our experimental results significantly extend the state of the art in autonomous helicopter flight. We used the following approach: First we had a pilot fly the helicopter to help us find a helicopter dynamics model and a reward (cost) function. Then we used a reinforcement learning (optimal control) algorithm to find a controller that is optimized for the resulting model and reward function. More specifically, we used differential dynamic programming (DDP), an extension of the linear quadratic regulator (LQR).
TL;DR: The control approach described in this paper is robust since it explicitly deals with unmodeled state-dependent disturbances and forces without needing any prior knowledge of the same.
Abstract: The dynamics of a quadrotor are a simplified form of helicopter dynamics that exhibit the same basic problems of underactuation, strong coupling, multi-input/multi-output design, and unknown nonlinearities. Control design for the quadrotor is more tractable yet reveals corresponding approaches for helicopter and UAV control design. In this paper, a backstepping approach is used for quadrotor controller design. In contrast to most other approaches, we apply backstepping on the Lagrangian form of the dynamics, not the state space form. This is complicated by the fact that the Lagrangian form for the position dynamics is bilinear in the controls. We confront this problem by using an inverse kinematics solution akin to that used in robotics. In addition, two neural nets are introduced to estimate the aerodynamic components, one for aerodynamic forces and one for aerodynamic moments. The result is a controller of intuitively appealing structure having an outer kinematics loop for position control and an inner dynamics loop for attitude control. The control approach described in this paper is robust since it explicitly deals with unmodeled state-dependent disturbances and forces without needing any prior knowledge of the same. A simulation study validates the results obtained in the paper.
TL;DR: This paper first fit a stochastic, nonlinear model of the helicopter dynamics, then uses the model to learn to hover in place, and to fly a number of maneuvers taken from an RC helicopter competition.
Abstract: Autonomous helicopter flight represents a challenging control problem, with complex, noisy, dynamics. In this paper, we describe a successful application of reinforcement learning to autonomous helicopter flight. We first fit a stochastic, nonlinear model of the helicopter dynamics. We then use the model to learn to hover in place, and to fly a number of maneuvers taken from an RC helicopter competition.
TL;DR: In this paper, a tracking controller for a class of underactuated mechanical systems, based on a backstepping procedure, is presented. But this controller operates directly in the configuration manifold of the vehicle.
Abstract: In this paper we present a tracking controller for a class of underactuated mechanical systems, based on a backstepping procedure. This class includes an approximation of small helicopter dynamics. The need to avoid artificial singularities due to the attitude representation is the main driver behind the control design presented in this paper: to achieve this goal, we will operate directly in the configuration manifold of the vehicle. The control design provides asymptotic tracking for an approximate model of small helicopters, and bounded tracking when more complete models are considered. Simulation examples, including both point stabilization and aggressive maneuver tracking, are presented and discussed.
TL;DR: Three baseline models are described and it is shown that they are significantly outperformed by the ReLU Network Model in experiments on real data, indicating the power of the model to capture useful structure in system dynamics across a rich array of aerobatic maneuvers.
Abstract: We consider the problem of system identification of helicopter dynamics. Helicopters are complex systems, coupling rigid body dynamics with aerodynamics, engine dynamics, vibration, and other phenomena. Resultantly, they pose a challenging system identification problem, especially when considering non-stationary flight regimes. We pose the dynamics modeling problem as direct high-dimensional regression, and take inspiration from recent results in Deep Learning to represent the helicopter dynamics with a Rectified Linear Unit (ReLU) Network Model, a hierarchical neural network model. We provide a simple method for initializing the parameters of the model, and optimization details for training. We describe three baseline models and show that they are significantly outperformed by the ReLU Network Model in experiments on real data, indicating the power of the model to capture useful structure in system dynamics across a rich array of aerobatic maneuvers. Specifically, the ReLU Network Model improves 58% overall in RMS acceleration prediction over state-of-the-art methods. Predicting acceleration along the helicopter's up-down axis is empirically found to be the most difficult, and the ReLU Network Model improves by 60% over the prior state-of-the-art. We discuss explanations of these performance gains, and also investigate the impact of hyperparameters in the novel model.