Academic

Consider a two-class classification problem where we observe samples generated from groups 0 and 1. Suppose a new sample from the same mixture is observed, our goal is to estimate which group it comes from. If the two groups have the same population covariance matrices but different population mean, then the problem has been discussed in many works. However, usually the covariance matrices are also different, and provide some information for classification.  When the number of features is small, then Quadratic Discriminant Analysis (QDA) is applied to achieve the optimal rate. However, when each sample has numerous of features but many of the features are useless for classification, the classical QDA method needs some modification. We set up a rare and weak model for both the mean vector and the precision matrix (inverse of the covariance matrix). We further propose a QDA with feature selection method for this case. We have derived the boundary separating the region of successful classification from the region of unsuccessful classification of the new QDA approach. We also show that when both the mean vector and the covariance matrix are known, all classification methods will fail in the unsuccessful region. Hence, the new QDA method achieves the optimal classification results. 

This presentation is based on joint work with Jingjing Wu and Zhigang Yao.