Using micro-documents for feature selection: the case of ordinal text classification

Most popular feature selection (FS) methods for text classification (TC) such as information gain (a.k.a. mutual information), chi-square, and odds ratio, are based on binary information concerning the presence/absence of the feature in each training document. As such, these methods do not exploit a rich source of information, namely, the information concerning how frequently the feature occurs in each training document (term frequency). In order to overcome this drawback we break down each training document of length k into k training "micro-documents", each consisting of a single word occurrence and endowed with the same class information of the original training document. This move has the double effect of (a) allowing all the original FS methods to be still straightforwardly applicable, and (b) making them sensitive to term frequency. We study the impact of this strategy in the case of ordinal TC, using four recently introduced FS functions, two SVM-based learning methods, and two large datasets of product reviews. The experiments show that the use of this strategy substantially improves the accuracy of ordinal TC.