Learning From Data: A Tutorial With Emphasis on Modern Pattern Recognition Methods

The purposes of this tutorial are twofold. First, it reviews the classical statistical learning scenario by highlighting its fundamental taxonomies and its key aspects. The second aim of the paper is to introduce some modern (ensembles) methods developed inside the machine learning field. The tutorial starts by putting the topic of supervised learning into the broader context of data analysis and by reviewing the classical pattern recognition methods: those based on class-conditional density estimation and the use of the Bayes theorem and those based on discriminant functions. The fundamental topic of complexity control is treated in some detail. Ensembles techniques have drawn considerable attention in recent years: a set of learning machines increases classification accuracy with respect to a single machine. Here, we will introduce boosting, in which classifiers adaptively concentrate on the harder examples located near to the classification boundary and output coding, where a set of independent two-class machines solves a multiclass problem. The first successful applications of these methods to data produced by the Pico-2 electronic nose (EN), developed at the University of Brescia, Brescia, Italy, will also be briefly shown.