An Efficient Algorithm for Computing the HHSVM and Its Generalizations

TL;DR: A generalized coordinate descent (GCD) algorithm for computing the solution path of the hybrid Huberized support vector machine (HHSVM) is proposed, which takes advantage of a majorization–minimization trick to make each coordinatewise update simple and efficient.

Abstract: The hybrid Huberized support vector machine (HHSVM) has proved its advantages over the l1 support vector machine (SVM) in terms of classification and variable selection. Similar to the l1 SVM, the HHSVM enjoys a piecewise linear path property and can be computed by a least-angle regression (LARS)-type piecewise linear solution path algorithm. In this article, we propose a generalized coordinate descent (GCD) algorithm for computing the solution path of the HHSVM. The GCD algorithm takes advantage of a majorization–minimization trick to make each coordinatewise update simple and efficient. Extensive numerical experiments show that the GCD algorithm is much faster than the LARS-type path algorithm. We further extend the GCD algorithm to solve a class of elastic net penalized large margin classifiers, demonstrating the generality of the GCD algorithm. We have implemented the GCD algorithm in a publicly available R package gcdnet.