DRENAJE DE TALUDES

Online Q&A communities are not only a popular type of support communities, but also provide a tangible way to derive success metrics from their main goal:

solving questions. Stack Exchange¹⁸ (SE) is a popular multi-purpose Q&A plat-form for hobbyists and professionals, where members discuss a wide variety of topics from cooking and bicycle repair to software-related questions in the pop-ular Stack Overflow site. From the Stack Exchange platform, we downloaded the publicly available data dump from June 2016, which contains 8 years of data in 152 forums that vary from a few hundred to almost three million users who produced up to 30 million posts (see Table8for details).

Because of the ample availability of many different Q&A platforms, we are able to source user interaction data from a second Q&A site for our analyses.

This allows us to investigate whether successful user behaviour is transferable between communities of the same type but from different hosting platforms.

The SAP Community Network¹⁹ (SCN) is a corporate Q&A forum for customers of SAP, where members mainly discuss technical questions revolving around SAP’s software products. The SCN data contains community activity from 2003 to 2011 in 95 forums from just 5 to 11,022 members, who produced up to 181,128 posts (see Table8). The data was made available to us as part of the EU project ROBUST.²⁰

Both the Stack Exchange and the SAP Community Network platforms are or-ganised in a discussion-forum like structure, where each forum is defined by a topic, and people contribute to it by posting questions and answers around the given topic. The given topic is usually also the title of the forum. Analogue to existing literature, we refer to the forums in our Q&A data sets as online com-munities [TMB+09;ZZS09;ARA11;WL11;RA12;RFA+13;WRT+14;HAF15]. By this definition, communities are overlapping groups of users that are topically pre-defined by the host or manager of the Q&A platform. But whereas users can belong to several communities, their individual posts belong to exactly one com-munity each. The typical interaction structure in Q&A communities is described

18 http://stackexchange.com/

19 http://scn.sap.com/

20 http://robust-project.eu/

t

Figure 3: Typical interaction structure in Q&A communities. Each thread consists of an original post (a question), which receives responses in the form of answers.

Many Q&A communities also allow the question asker to explicitly mark the best answer as “accepted” (green).

in Figure 3, where a forum is divided into individual threads, and each thread consists of an original post (i.e. the question) and any number of responses in the form of answers. If the question received answers and one of them pro-vides a solution to the question, the asker might then explicitly²¹ “accept” this solution, thus marking the best answer. This functionality can be found in most common online Q&A communities.

User satisfaction has been recognised as an important element to the success of online communities [DM92; DM03; San05; LFW+07;Lin08; ZZS09]. While Q&A communities – like any other kind of online communities – are also subject to non-functional aspects, such as member integration, trust and interface usability [IL09], their main goal and their performance towards that goal can be measured as a function of successfully solved questions. Liu et al. [LBA08] and Hiscock et al. [HAF15] defined that information seekers are satisfied if their questions are sufficiently solved, which they indicate by explicitly selecting a best answer. His-cock et al. added a time constraint to ensure that solutions arrive in a time that is relevant for the question asker. We use the notion of asker satisfaction based on timely solved questions, and translate it to the level of the whole community by formalising solved and tsolved as follows:

solved: When a question asker explicitly accepts a solving answer, the ques-tion is marked as “solved”. The metric solved then measures the global pro-portion of solved questions for a given community (Equation 2). In our data,

21 People do not always explicitly mark the solving answer, which affects the reliability of mea-suring success in Q&A communities. However, existing work aims at automatically identifying when a question has been sufficiently solved, e.g. [BHA12].

Community statistics Stack Exchange SAP Community Network 152communities 95communities Age

Youngest 1.5 months 2.6 months

Median 4.6 years 4.0 years

Mean 4.4 years 3.9 years

Oldest 7.9 years 7.3 years

Size(Number of users)

Smallest 101 5

Median 3,501 742

Mean 30,722.7 1,975.7

Biggest 2,808,774 11,022

Activity(Number of posts)

Least active 347 7

Median 16,053.5 4,487

Mean 258,315.8 19,976.8

Most active 30,444,143 181,128

Table 8: Q&A community statistics

the range of questions that are explicitly marked as solved by the asker is 17%

– 78% for SE (44% average), and 0% – 56% for SCN (20% average). Burel et al. reported that about 50% of questions were marked as solved on a subset²² of our data [BHA12]. We can confirm a similar average for the Stack Exchange sites, however, the average of our SCN forums is considerably lower when tak-ing the greater number of communities into account. With Q as the set of posed questions in a community, we define S as the set of solved questions as follows:

S ={q : q ∈ Q, q has an accepted answer} (1)

solved = |S|

|Q| (2)

t_solved: Response time has been recognised as a relevant aspect of Q&A com-munity success [HAF15]. We measure the average time between questions and their accepted best answer in hours as per Equation 3. In our data, the average

22 Burel et al.’s data covers two out of our 152 SE sites and 33 out of our 95 SCN forums.

solving delay ranges from 15 hours to 45 days (SE) and from 11 minutes to 5 months (SCN). Questions that had no explicitly selected best answer at the time of data recording do not contribute to tsolved, as their solving time is not defined (and could be potentially infinite).

t_solved =

P{h : time between s ∈ S and its accepted answer}

|S| (3)

The solved and tsolved metrics are not the only options to measure success in Q&A communities, although they are good baselines as they are measurable in most Q&A platforms. Alternative success criteria can include user votes, knowl-edge increase of question askers or a reduction in spending on public or private services. Stack Exchange is an example of a community site that supports users voting on each others’ content, and a high average voting score (normalised by the community size) could indicate that the community is performing well in terms of solving questions. However, this information is not available every-where. Other goals of Q&A communities can be to increase the knowledge of the participants or to help people save money by reducing the need of paid services, such as paid help lines or computer technicians. These are examples where the outcome is usually not directly measurable by the community provider. Yet, in cases where this information is available, a community provider should consider using it to gauge the success of their community.