REDISEÑO DEL PROCESO DE PLANIFICACIÓN DE DESPACHO DE GLP

Issues of validity and reliability of the methodological design are addressed in this section.

3.4.1. Internal Validity

Whether a study possesses internal validity depends on whether it properly

demonstrates a causal relation between the independent and dependent variables (Brewer, 2000). The key question that should be asked when considering internal validity is “are we measuring what we think we are measuring” (Kerlinger, 1984, p. 417).

completely relies on the potential survey subjects to volunteer to participate in the study and may skew the findings by not being able to generate a representative sample of the population. However, as stated in the section of “methodological specifics”, the current population of chat reference practitioners is unidentifiable, and hence, none of the random sampling methods would be effective enough to create a representative sample for the study either. Thus, given the unknown size and demographics of the population, it is unlikely to ensure the certainty of a representative sample. Under such circumstances, the self-selection method was the most appropriate resort in sampling the chat reference population. Still, in order to reduce the selection bias to the least extent, a number of approaches were taken so that the surveys could be made accessible to as many chat reference practitioners as possible:

o The invitation email was sent to a total number of eleven listservs where chat

reference librarians are most likely to be subscribers;

o In the invitation email, recipients were encouraged to forward the message to

chat reference practitioners they knew if they themselves did not work with chat;

o About two weeks after each survey was launched, a follow-up message was

sent to all the listservs again in the attempt of generating more responses; and

o For each survey, a $100 reward was promised for a randomly selected winner.

Another concern with regard to internal validity is how the surveys were administered. Both surveys were published via a Web based survey service –

stated in the “methodological specifics” section, the study was targeting chat reference practitioners whose job involves substantial use of computers and the Web, and access to Web-based surveys should not be a problem to them. Subsequently, the concern of coverage bias can be alleviated.

Thirdly, the questionnaires for both surveys were pilot tested by four experienced chat reference librarians to ensure the clarity of the wording and for each competency and training technique listed on the survey, a definition was provided so that the participants can retain a consistent understanding of the survey. These definitions were piloted tested as well. All these efforts were made to guarantee that the questionnaires would convey the same message to every respondent with the least possible confusion and

misunderstanding.

As far as data analysis is concerned, only valid responses were taken into consideration in the examination of competency and training technique ratings. In the competency survey, in addition to the seven point Likert scale, respondents were offered the option of “N/A” if they did not think a rating of the competency was applicable. Then, in the data analysis, the “N/A” responses were all excluded because they were not valid in terms of rating the importance of surveyed chat reference competencies. In the survey on training techniques, respondents were asked whether or not they had experienced a particular training technique before they could vote on its effectiveness. Sometimes, respondents who answered “no” to this question still provided a rating for the training technique, and such a response would be deemed invalid and excluded from the analysis of training technique ratings. Only ratings from those who answered “yes” were

To sum it up, in order to achieve the internal validity of the study, a number of different measures were taken to make sure that data collection and data analysis were carefully and accurately implemented in this study so that the relationship between different variables could be correctly delineated.

3.4.2. External Validity

External validity refers to the generalizability of a study, that is, the results from the study could hold across different experimental settings, procedures and participants (Brewer, 2000). The goal of this study is to create a framework of chat reference

competencies and training techniques that can be applicable in multiple contexts. A total number of 883 responses were received for the two surveys and this large size of the sample, although lacking certainty in representing the chat reference population, indicates generalizability to some extent. In the meantime, the researcher made every effort to provide a detailed description of the setting, procedures and content of the study, so that others might determine how generalizable the findings would be in other situations. It is possible that some or all of the findings can be tested in other settings in the future.

3.4.3. Reliability

Reliability, or repeatability, refers to the consistency of the measuring instruments in an experimental setting. Since this study employed mostly quantitative methods in seeking the perceptions on chat reference competencies and training techniques, it is expected that replication of the study may result in similar results unless there is a change in the landscape of chat reference service and hence, in chat reference practitioners’

As a matter of fact, the two surveys conducted in the study shared the same sampling and administering method, and the demographic information of the respondents in both surveys displayed similar patterns. Thus, the reliability of the measuring

instrument has been supported by the consistent demographic characterization of the survey respondents. It is safe to assume that if the methods employed in this study are to be repeated under similar circumstances, consistency in the findings can be anticipated.