Feature selection for ordinal text classification

Ordinal classification (also known as ordinal regression) is a supervised learning task that consists of automatically determining the implied rating of a data item on a fixed, discrete rating scale. This problem is receiving increased attention from the sentiment analysis / opinion mining community, due to the importance of automatically rating increasing amounts of product review data in digital form. As in other supervised learning tasks such as (binary or multiclass) classification, feature selection is needed in order to improve efficiency and to avoid overfitting. However, while feature selection has been extensively studied for other classification tasks, is has not for ordinal classification. In this paper we present four novel feature selection metrics that we have specifically devised for ordinal classification, and test them on two datasets of product review data against three metrics previously known from the literature, using two learning algorithms from the "support vector regression" tradition. The experimental results show that all four proposed metrics largely outperform all of the three baseline techniques, on both datasets and for both learning algorithms.