Programas Nacionales para la Inclusión Social del Reino de España 2006 2008 y 2008-

Consider again the setting that we obtain relative judgments about documents of the form of preferences similar to

di q dj, (3.12)

which indicate that di is more relevant than dj to query q. In general, the prefer-

ences we collect may be noisy: some may be correct while others will be incorrect. A standard approach to modeling noise in pairwise comparisons is to assume that the probability of observing a particular preference is determined by the Bradley-Terry model (Bradley & Terry, 1952):

P (di q dj) =

rel(di, q)

rel(di, q) + rel(dj, q)

, (3.13)

where rel(di, q)is a measure of the relevance of di to query q. The Bradley-Terry

model can be reparameterized setting rel(di, q) = 10µ q

i/σ where σ is a known, global and fixed parameter. Further, suppose that each document dihas some

true relevance µqi to the query q. We write the set of document relevance values

for all documents in a collection C as

M = (µq₁, . . . , µq_|C|). (3.14)

Assuming the pairwise judgments are independent, we can define the proba- bility of observing some set D of relative relevance statements as

P (D|M = (µ1, . . . , µ|C|)) = Y diqdj∈D P (di q dj|µi, µj) (3.15) = Y d d ∈D 1 1 + 10−(µi−µj)/σ (3.16)

νi ← νi+ q 1 σ2 i +_δ12 g(σ2_j)(si− E(s|νi, νj, σj2)) (3.17) σ_i2 ←  1 σ2 i + 1 δ2 −1 (3.18) where q = log 10 400 g(σ2) = 1 p1 + 3q2_σ2_/π2 E(s|νi, νj, σ2j) = 1 1 + 10−g(σj2)(νi−νj)/400 δ2 = 1 q2_g(σ2 j)2 × 1 E(s|νi, νj, σj2)(1 − E(s|νi, νj, σj2))

Figure 3.2: The Glicko update equations, which describe how the estimated relevance νi and estimated variance σi2 for document di should be updated

following a comparison to document dj.

Given this likelihood model, and assuming a Gaussian prior over the values µq_i, Glickman showed that the values µqi can be estimated iteratively from a set

of preferences provided as training data. Given an initial estimate of document relevance (player ability in the context of chess) νiand error in the estimate σi, this

algorithm provides a set of approximate online update equations for maintaining the estimated relevance and error as pairwise preferences are collected. The update to the estimates for di following a single comparison to dj (where siis 1 if

3.4

Summary

This chapter has described typical performance measures used to evaluate rank- ing systems, and described five classes of learning algorithms commonly used to learn them. In particular, we have seen that there are many different measures used for evaluating ranking performance. However, for most it is unclear how well they relate to ranking performance from a user perspective. This suggests an open question in what measures should in fact be optimized. Further, we also discussed how relevance data in different forms can be used to learn to rank. We saw that there are many algorithms that are amenable to learning from pairwise preference judgments as can be collected from implicit feedback. Finally, two algorithms were presented in some more detail, namely the Ranking SVM and Glicko chess rating system.

CHAPTER 4

LEARNING FROM IMPLICIT FEEDBACK ENCODED IN QUERY CHAINS

In this chapter, we consider further the question of obtaining relevance judgments from users of a search system. We will show how to extend previous approaches for learning from clickthrough data to obtain substantially more useful training data. In addition to evaluating the validity of the data collected in this new way, we will show that it can then be used to learn an improved ranking function in a real search engine. The work presented in this chapter was originally published in (Radlinski & Joachims, 2005b).

4.1

Introduction

As we have seen in earlier chapters, there has been substantial work in learning to rank documents. This is largely motivated by the difficulty in manually constructing well performing ranking functions. For this learning task, training data can be collected in at least two ways. The first approach, most commonly used in the past, has relied on asking expert judges to rate the relevance of documents to queries. However, as we have seen earlier in this thesis, such data is expensive and slow to collect, with numerous difficulties to overcome. The alternative of recording user interactions and inferring relevance judgments from this implicit feedback has received less attention.

In particular, previous research in learning to rank from clickthrough data has considered each query independently. However, studies of user behavior in Web search engines (for instance by Lau and Horvitz (1999) and by Silverstein

et al. (1998)) have observed that users often run more than just one query when they visit a search engine. Rather, users tend to perform a sequence of queries for any given question. This likely indicates that the results shown for the first query often do not satisfy the users’ information needs. Sessions typically consist of 2 to 3 queries (as observed by Beeferman and Berger (2000); Cucerzan and Brill (2004); Jones and Fain (2003); Lau and Horvitz (1999)). Query chains were also observed in the eye tracking study performed by Granka et al., where mean query chain length was 2.2 queries (although the particular questions asked and the laboratory environment would be expected to have an influence on this value). This suggests that when working with implicit feedback, we should consider user behavior over entire sessions rather than just a single query.

We will show that ignoring query chains when learning from clickthrough data ignores valuable information that is hidden in the sequence of queries and clicks in a search session. For instance, if we repeatedly observe the query “special collections” followed by another for “rare books” on a library search system, we may deduce that Web pages relevant to “rare books” may also be relevant to “special collections”. Moreover, when queries are considered independently, we can only infer implicit feedback on a few results at the top of the result set for each query because users very rarely look further down the list. The advantage of using query chains is that we can also deduce relevance judgments on the many more documents seen during an entire search session.

This chapter will describe how to interpret user behavior as preference judg- ments that provide relative relevance information about documents within indi- vidual query result sets, and between documents returned by different queries within the same query chain. The method used to generate the preference judg-

ments is validated using a controlled user study. We will then show how an adapted ranking SVM can be used to learn a ranking function from the preference judgments, evaluating on a real-world Web search system, the Cornell University library4Web search.

An important innovation in this chapter is that we also learn a more general ranking function than previous work, by learning an association between query words and specific documents. Such a general approach has been used previously to learn to generate abstracts by Scholer and Williams (2002), but not to learn ranking functions. Prior approaches for learning to rank cannot learn to associate “new” documents with a given query because they usually only combine or re-order results obtained from one or more static ranking functions. In particular, given a query q, they cannot learn to retrieve any document not originally returned for q. Coming closest to solving this limitation previously, the method presented by Kemp and Ramamohanarao (2002) could be extended with query chains. However, they assume implicit absolute feedback, making their approach more likely to be susceptible to bias and noise, as seen in Chapter 2 and will be reviewed again later in this chapter.