Reducing critical failures for control algorithms using executable assertions and best effort recovery

TL;DR: Fault injection experiments show that the percentage of value failures with severe consequences was reduced to 3% when the state variables were protected with executable assertions and best-effort recovery mechanisms.

Abstract: Systems that use f+1 computer nodes to tolerate f node failures ordinarily require that the computer nodes have strong failure semantics, i.e. a node should either produce correct results or no results at all. We show that this requirement can be relaxed for control applications, as control algorithms inherently compensate for a class of value failures. Value failures occur when an error escapes the error detection mechanisms in the computer node and an erroneous value is sent to the actuators of the control system. Fault injection experiments show that 89% of the value failures caused by bit flips in a CPU had no or minor impact on the controlled object. However, the experiments also show that 11% of the value failures had severe consequences. These failures were caused by bit flips affecting the state variables of the control algorithm. Another set of fault injection experiments showed that the percentage of value failures with severe consequences was reduced to 3% when the state variables were protected with executable assertions and best-effort recovery mechanisms.