ENTRAMADO DE SIGNIFICADOS

In the early days of information retrieval, relevance expressed a criterion for assessing effectiveness in retrieval of information. Therefore in traditional IR models, relevance was considered a property of the system. That is, a system was judged how it acquired, represented, organized and matched the texts to the information need. Therefore, tra- ditional IR systems focused on evaluating different approaches or algorithms based on how well they retrieved relevant results. Majority of IR evaluation studies, from Cran- field studies in the late 50’s and early 60’s to Text Retrieval Conference (TREC) eval- uations in 1990s, are based on this framework for considering the nature of relevance [Saracevic,1996b].

The Cranfield model [Cleverdon et al.,1966] is a classic example of the system-oriented approach to IR system performance and effectiveness evaluation [Hildreth,2001]. Cran- field made two main assumptions: first, the users prefer to view results (documents) that are relevant to their search queries; and second, the document relevance to a query is perceived as the property of the document.

For decades, the notion of relevance was used as a measuring criteria for effectiveness of a retrieval system, and users’ interactions with the system was not considered. However, interactive information retrieval (IIR) researchers highlighted the importance of users’ role during information seeking process [for example,Ingwersen,1996;Mizzaro,1998;

Robins,2000;Borlund,2003].

Cosijn and Ingwersen [2000] argued that the cognitive model for interactive IR com- prises three of elements – systems, users, and the environment. The system involves documents or information objects, which are organized in different ways. The user typ- ically has a problem or a work task to perform. The socio-organizational environment provides the context or situational framework influencing the activities of the user. Since the traditional IR model did not reflect interaction, therefore in later research, a number of efforts were devoted to the development of IR models that incorporated the rich and complex nature of IR interactions. The prime weakness of the system-based relevance is that it was completely one-sided. It did not incorporate in any way anything from the users’ interaction, except the query. It did not consider elements, variables, and context related to the user and their use, nor did it reflected the dynamic, interactive na- ture of IR as practiced. Therefore, situational, psychological, motivational relevance etc emerged as a reaction and challenge to the system-based relevance approach [Saracevic,

1996b].

2.1. The concept of Relevance

expressed relevance as a relation, and suggested that different theories (which he refers to as manifestations) of relevance encompass different relations. He listed five manifes- tations of relevance [also called as “attributes” of relevance by,Cosijn and Ingwersen,

2000] based on different relations as follows:

• System or algorithmic relevance: The relation between a query and information objects (texts) which are retrieved or failed to be retrieved by a given procedure or algorithm.This manifestation of relevance focuses on system or algorithm mea- sures, whereas, the following four take users’ interactions into account.

• Topical or subject relevance: The relation between the subject and topic ex- pressed in a query, and topic or subject covered by retrieved texts. This relation is also system-oriented largely because the success of the relation depends on the system’s input policy, as well as its indexing and searching ability to retrieve rele- vant objects. However, success also depends on the formulation of the request by the user, transformed into a query by the system [Cosijn and Ingwersen,2000]. • Cognitive relevance or pertinence: The relation between the state of knowledge

and cognitive information need of a user, and the information retrieved. Cognitive correspondence, informativeness, novelty and information quality are criteria by which cognitive relevance is inferred.

• Situational relevance or utility: The relation between the situation, task, or problem at hand, and texts retrieved by a systems or in the file of a system, or even in existence. Usefulness in decision-making, appropriateness of information in resolution of a problem, reduction of uncertainty, and the like are criteria by which situational relevance is inferred.

• Motivational or affective relevance: The relation between the intents, goals, and motivations of a user, and texts retrieved by a system or in the file of a system, or even in existence. Satisfaction, success, accomplishment, and the like are criteria for inferring motivational relevance.

Mizzaro[1998] further built uponSaracevic[1996b]’s concept of relevance as the mani- festations of topical, cognitive and situational relevance, and suggested four-dimensions of relevance. He defined relevance as a four-dimensional relationship between an in- formation resource(surrogate, document, and information) and a representation of the user’s problem(query, request, real information need and perceived information need). Which is then judged according to one or more of the following components: topic,

2.1. The concept of Relevance

task, or context, at a particular point in time [Cosijn and Ingwersen, 2000]. Graphical representation ofMizzaro[1998]’s four-dimensional representation of relevance can be seen in figure2.1. Topic Task Context PIN RIN InfRes Repr Information Document Surrogate Query Request

FIGURE2.1: The Various kinds of relevance proposed by [Mizzaro,1998]

Therefore, dimensions of relevance can be seen as the “attributes” based on which, the relevance of the result to the given information need is determined. So far in the web search, relevance is often estimated based on single attribute; topic, theme, time, category, and sometime based on the overlap of two or more attributes. For instance, location and theme as in Geographic Information Retrieval [Bucher et al.,2005]. How- ever, within the context of the aggregated search, what attributes to the relevance of a result is not known. The aim of this research is to obtain an understanding of relevance within the context of aggregated search.