Efficient diversification of Web search results

In this paper we analyze the efficiency of various search results diversification methods. While ecacy of diversification approaches has been deeply investigated in the past, response time and scalability issues have been rarely addressed. A unied framework for studying performance and feasibility of result diversification solutions is thus proposed. First we dene a new methodology for detecting when, and how, query results need to be diversied. To this purpose, we rely on the concept of "query renement" to estimate the probability of a query to be ambiguous. Then, relying on this novel ambiguity detection method, we deploy and compare on a standard test set, three dierent diversification methods: IASelect, xQuAD, and OptSelect. While the rst two are recent state-of-the-art proposals, the latter is an original algorithm introduced in this paper. We evaluate both the efficiency and the effctiveness of our approach against its competitors by using the standard TREC Web diversification track testbed. Results shown that OptSelect is able to run two orders of magnitude faster than the two other state-of-the-art approaches and to obtain comparable figures in diversification effctiveness.