In order to overcome the potentially negative influences of the user aspect in reputation systems (low number of votes, biases in voting), a different approach in supporting users in credibility evaluation may be taken, namely automated support. Instead of considering user opinions, algorithms could be used to determine the credibility of information, which can in turn be communicated to the user, as shown in Figure 2.
A vast body of research is available on (automated) DSS (see Lee & See, 2004, for a review). One returning topic is the reliance of the user on the support. When a support system is not 100% reliable, which is mostly the case in real-life systems, the user should rely appropriately on the advice of the system. Appropriate reliance implies that the level of trust in the support system matches its actual performance. Two pitfalls can be identified; first, a user may overly rely on the support. In this case, trust in the support exceeds the actual capability of the support, leading to misuse (Parasuraman & Riley, 1997). Second, a user may under-rely on the support. This happens when the capability of the system exceeds trust of the user, leading to disuse.
Lee and See (2004) have proposed a model which explains how a reliance action is formed. First, a user assimilates the information needed to form a decision. Second, the influence of this information on trust is evaluated. Third, a reliance intention is formed and fourth and last, a reliance action is taken.
An important challenge lies in the information assimilation stage of this process. The information may be gathered from the context in which the support is used (being individual, organizational, cultural or environmental) and from the support system itself. In order to optimize appropriate reliance, Lee and See (2004) advise that support systems should be made easy to understand, using simple algorithms clear to the user. In this case, in situations where the support fails, the user also has the opportunity to understand why, leading to more appropriate reliance. The advantages of understandability has also been demonstrated by Ye and Johnson (1995). Participants in their experiment had a greater belief in the truthfulness and reasonableness of the support system after they were given a clear explanation of the advice.
On the other hand, more complex support systems may perform better, especially in complex task settings such as credibility evaluation, where many factors may contribute to credibility in various ways. However, such support systems may be harder to understand
by the user, if at all. Lee and See (2004) name this challenge the trade-off between trustable and trustworthy advice. The first supplies understandable information about how it functions, which makes the support trustable, whereas the latter performs better, and is thus more trustworthy.
When considering DSS in the domain of online credibility evaluation, an important difference with traditional DSS can be seen. In many cases, support systems are designed to assist professionals, who are motivated to perform a very specific task to the best of their abilities. Examples are support for air traffic control (Hilburn, Jorna, Byrne, & Parasuraman, 1997) or in the naval domain (Van Maanen, Lucassen, & Van Dongen, 2011). However, in online credibility evaluation, not much is known about the user. Motivation levels are questionable (Metzger, 2007), as are information skills (Lucassen et al., 2013, Lucassen & Schraagen, 2011b). This has the following important implications for the design of a support system:
1) Because of the widespread user group (Internet users), extensive training with a support system is virtually impossible. Therefore, the support should be very easy to use.
2) Motivation may be (very) limited, using the support should not take too much time and effort. Ideally, users should be even faster in evaluating credibility with a DSS than without.
3) The issue of credibility evaluation may not be salient to all Internet users. The mere presence of a support system may influence this attitude of the user; why would there be a support system if the information is always credible?
A good example of a highly relevant domain for the application of credibility evaluation support systems is Wikipedia. The overall information quality of this online encyclopedia is very high (Giles, 2005). However, its information quality fluctuates heavily between the articles (Denning, Horning, Parnas, & Weinstein, 2005), due to its open-editing model. A few attempts have already been made to implement automated support systems in the domain of Wikipedia. An example of a support system focusing on understandability is WikipediaViz (Chevalier, Huot, & Fekete, 2010).This is a system for Wikipedia that presents various statistics about an article (e.g., number of words, contributors and internal links) to the user. Because of the fact that only raw statistics are presented, without further processing by the support, it is very clear to the user what this system actually does. However, for at least some of these statistics, it may be unclear what their
130 |
Chapter 7relationship to credibility is. For instance, as far as the number of authors is concerned, it is unknown whether articles with a lot of authors are to be trusted more or less than articles with fewer authors.
Adler et al. (2008) have introduced a more sophisticated (and complex) algorithm to support credibility evaluation called WikiTrust. Based on ideas proposed earlier (Cross, 2006; McGuinness et al., 2006), this support system calculates a trust value on a single word basis. First of all, the credibility of a particular word is based on how long this word has been in the article. When a new word is added, it is assigned the credibility of its author, which in its turn is based on the average credibility (survival duration) of his additions. The credibility of each word rises over time as the article is edited but this word is kept unchanged. Credibility is visualized by coloring the background of each word (instead of the words themselves) in a shade of orange. Dark orange indicates low credibility; white implies high credibility (the actual algorithm is slightly more complex than described here).
An indication of how well such a complex support system could perform in terms of user acceptance can be found in Lucassen and Schraagen (2011a). They showed that college students have difficulties incorporating the extra information by WikiTrust into their own credibility evaluations. It can be argued that this is due to differences between the mental model of trust of the participating students and the features used by the support. This notion is supported by Hilligoss and Rieh (2008), who claim that the user’s personal definition of trust (e.g., features involved) is highly influential on the way credibility is evaluated by this user. When a support system is based on features that are not deemed important for credibility by the user, it is likely not to be accepted. Hence, next to understandability of support, agreement with the user about the features used by the support is a vital requirement for user acceptance.