1. What is the focus of the paper in the Introduction section?
The focus of the paper in the Introduction section is on Feature Selection (FS) in the general setting of sparse optimization. The paper aims to tackle Feature Selection (FS) by minimizing a specific objective function that includes the l 0 pseudo-norm. The l 0 pseudo-norm counts the number of nonzero components of any vector, and the paper discusses the use of different norms, such as the l 1-norm and polyhedral k-norms, to ensure sparsity in the solution. Additionally, the paper explores the application of sparse optimization in SVM classification and proposes a novel model based on the polyhedral k-norm. The paper also introduces a multi-objective reformulation of the feature selection model, considering SVM classification models as multi-objective optimization problems and obtaining a set of Pareto optimal solutions instead of a single solution.
read more
2. What is binary classification?
Binary classification is a task where two classes of individuals are represented by two finite sets A and B, and the goal is to classify an input vector x as a member of either set A or B. The training set consists of labeled points in R^n, with labels +1 for set A and -1 for set B. The functional dependency f(x) determines the class membership of a given vector x, and the separating hyperplane P = x: xTw = g(w) separates the two sets, with open halfspaces *P1 and *P2 containing most of the points belonging to A and B, respectively. The convex hulls of A and B must be disjoint for the sets to be linearly separable. Feature selection in SVM aims to suppress as many components of w as possible.
read more
3. What is the objective of feature selection in SVM?
The objective of feature selection in SVM is to construct a separating plane that gives good performance on the training set while using a minimum number of problem features. This is achieved by finding a normal vector to the separating hyperplane with the smallest possible number of nonzero components. A sparsity enforcing term is added to the objective function to achieve this. Feature selection is primarily performed to select informative features, and has become an important issue in machine learning. The goal is to minimize classification error and maximize the separation margin, with the trade-off defined by the parameter C. Additionally, the LASSO approach, which suppresses as many elements of the vector w as possible, is obtained by replacing the l2-norm with the l1-norm.
read more
4. What is the objective of sparse SVM?
The objective of sparse SVM is to control the number of nonzero components of the normal vector to the separating hyperplane while maintaining satisfactory classification accuracy. This is achieved by minimizing the number of misclassified training data and the number of nonzero elements of vector w. Sparse optimization in SVM involves feature selection, where the problem is to minimize the sum of misclassified training data, the number of nonzero elements of vector w, and the number of nonzero components of any vector. This can be represented by the parametric program (9) with the step function s, which selects features based on their contribution to the model. The l1-norm can be used as a simplification, resulting in model (10) that exhibits good sparsity properties of the solution.
read more