4.3.1 Interview Design
The purpose of the interview was twofold. First, based on the responses provided by the respondents in the survey, I followed up with the interview participants to better understand the context behind their behaviors. Secondly, employing the Critical Incident Technique (CIT) (Flanagan, 1954) to help facilitate our discussion and improve participants’ recall of events, I explored their motivations behind sharing articles on Twitter, as well as the impact of the shared articles to these researchers. The interview was designed to be semi-structural to allow the pursuit of unanticipated lines of inquiry. The interview guide is attached in Appendix II. This guide is organized with both specific questions serving as topic anchors and probes as potential follow-up questions used to gather additional detail and clarify responses when needed.
48
In the first section of the interview, the main goal was to follow up with questions from Section II of the survey – to ask about the reasons for having the different sentiment of opinions on the article they tweeted, retweeted, replied to, and liked. The questions in this section aimed to elicit the ways these researchers come to understand, account for, and take actions on Twitter in their naturalistic settings.
In the second section of the interview, the participants were presented with ten of their tweets and retweets. Each of the tweets and retweets contained a link to a scholarly article. In other words, each of the tweets and retweets were related to an incident of sharing a scholarly article on Twitter. Questions were asked specifically relevant to the articles in these tweets and retweets identified. The major goals were to discuss in more depth their motivations behind sharing these articles and the impact of these articles on them.
For the critical incidents, I also asked participants about the occurrences of other scholarly activities associated with the specifically identified articles. The answers to this question were compared to the answers provided by the participants in Section III of the survey, which was their general memory and perception of how sharing fit in their research process. The discussions about these comparisons are included in the Reflections on the Methods section.
To evaluate and improve the interview questions and procedures, three pilot studies were conducted via Zoom (see Appendix III) on an iterative basis. Both the content and instrumental aspects of the interview were improved according to the pilot participants’ reactions and
answers, as well as feedback provided in the cognitive interviewing after the interview.
4.3.2 Interview Data Collection
The first step in the interview data collection was to identify interview participants. Based on the survey responses, I created a pool of potential interview participants. To account for the
49
variety of demographics and characteristics that might affect their use of Twitter, I used quota sampling and selected participants from different ages, genders, regions, education backgrounds, positions, and disciplines. In this selection process, their activeness on Twitter (using the counts of tweets of scholarly articles they had posted or reposted on Twitter during the period of October 2011 to February 2017) was also taken into consideration. This ongoing selection accompanied the recruiting process in order to ensure a reasonable spread of participants across all demographics and characteristics mentioned above.
The second step was to identify ten tweets or retweets that contained a link to a scholarly article for each of the participants to use in Section II of the interview. In Priem and Costello (2010), this was defined as a “first order Twitter citation” (pp. 2). The guidelines in selecting these tweets and retweets were as follows:
1. For each participant, the number of original tweets was no less than seven; the number of retweets was no more than three.
2. There were diverse types of tweets: they were of various lengths; there were diverse patterns of Twitter affordances uses; they were associated with different sentiments; they seemed to be associated with different purposes of sharing.
To be able to select tweets that seemed to be associated with different purposes of sharing, a preliminary content analysis of the tweet text was conducted. This was assisted by my content analysis experiences in two of my previous studies investigating the motivations behind sharing scholarly articles on microblogging platforms (Xu, Yu, Hemminger, & Dong, 2018; Yu, Xu, Xiao, Hemminger, & Yang, 2017). For each participant, I reviewed 20-100 tweets or
50
These tweets were displayed in a ten-page .pdf file generated from PowerPoint slides. On each page, the link of the tweet or retweet is located at the top of the page. The screenshot of the tweet or retweet is located in the middle of the page. By clicking on the link of a tweet or
retweet, the participants could further explore the tweet content, including the links, mentions, and hashtags included in them. Before the interview, participants were asked to spend a few minutes to examine these tweets.
All interviews were conducted via Zoom. Consent forms were signed either digitally or print-sign-scanned. For each participant, I sent out a Zoom link to them before the interview. Two participants joined the meeting on the phone; eighteen participants joined the meeting via the Internet. All conversations were recorded via the recoding functionality of Zoom. The interviews were conducted from the beginning of August to the end of August, except for one in early September.
4.3.3 Interview Data Analysis
The interviews were initially transcribed using the Google Speech-to-Text tool. In this process, I chose to opt-out of their data logging program to protect the privacy of the
participant’s interviews. This would prevent Google from having access to the study data for the purpose of improving their algorithms. Then, I manually transcribed the interviews based on the transcriptions obtained in the initial step.
The coding process started from open coding to focused coding. In the pre-coding phase, the in vivo coding technique was used in the first reading of the data (Saldaña, 2008). This approach emphasized the voice of the participants, which could better express participants’ opinions and feelings in their own language. Then, the official coding process began with descriptive coding by inductive reasoning. In this process, codes were developed and constantly
51
compared to one another to describe relevant concepts and their dimensions and properties (Davies, 2008; Stake, 2010).
A second coder helped in the coding of 50% of the data. First, two coders independently coded 25% of the data. After initial codes and tentative categories were formed, we had a
discussion about our coding process, the meaning and boundaries of codes, as well as the level of specificity. We refined our codes and categories by adding, subtracting, combining or splitting the codes, and reached a reasonable agreement after this discussion. In the second round of coding, we recoded the first 25% of the data and continued to code another 25% of the data. We compared our coding results again after the second round of coding. A high level of agreement was reached after this discussion. There was not an established criterion of the reliability of the coding of critical incidents, but according to Andersson and Nilsson (1964), an acceptable agreement level was if independent raters could correctly classify 75% to 85% of the incidents into the categories and 60% to 70% into the sub-categories. In our coding of the 100 critical incidents, we reached an agreement on 97 (97%) incidents in the sub-categories of motivations behind sharing; and the agreement on the category level was 99%. We reached an agreement on 93 (93%) incidents in the subcategories of impact of the articles on the researchers; and the agreement on the category level was 98%. The codebooks are attached in Appendix VI and VII.
Therefore, based on our agreement, I continued to code the remaining 50% of the data. Then, I had another discussion with the second coder about a few extracts where I had
uncertainty in coding, as well as one minor addition and one minor revision to the sub- categories. The second coder agreed on these two changes. At last, I re-examined the codes carefully to ensure they were coded in a systematic and consistent fashion across the entire dataset.
52