Sparse Feature Learning

TL;DR: In this chapter, some sparse representation problems with different norm regularizations are reviewed, and the classical sparse learning method, i.e., Lasso, and its variations are introduced, which reduce the affection caused by outliers and produce the sparsity of the outputs.

Abstract: The traditional linear feature extraction methods focus l2, 1on data global structure information or data local structure information. Although these learning methods perform well in some real applications to some extent, they still have some limitations. In this chapter, some sparse representation problems with different norm regularizations are reviewed. Then the classical sparse learning method, i.e., Lasso, and its variations are introduced, which reduce the affection caused by outliers and produce the sparsity of the outputs. Some sparse feature learning methods based on generalized regression are presented, including generalized robust regression (GRR), robust jointly sparse regression (RJSR) and locally joint sparse marginal embedding (LJSME). These methods not only preserve the local structure but also enhance the model robustness due to using norm and feature selection. In addition, the traditional projection matrix is divided into a new representation, i.e., a projection matrix and an orthogonal rotation matrix, and thus the small-class problem can be overcome.