There are two major criteria for evaluating measurements, these are reliability and validity.
4.9.1 Reliability & Validity
Reliability can be defined as the degree to which measures are free from error and therefore yield consistent results. Thus, reliability is obtained when similar results are presented over time and across situations. (Zikmund, 2000) For example, in order to obtain high reliability the results should be independent of the researcher and the selected respondent. (Wiedersheim & Eriksson, 1982).
Validity can be defined as the ability of a measuring instrument to measure what is intended to measure. (Ibid)
In order to illustrate the differences between validity and reliability Zikmund (2000) uses three rifle targets, figure 24. An marksman fires an equal amount of bullets with one old and with one modern rifle. The shots from the old rifle are scattered whilst the shots from the modern rifle are closely scattered. Consequently, the variability of the old rifle compared to the modern indicates that it’s less reliable. When the marksman shot the rifle target to the right in the figure the sun dazzled him. His shot is still consistent but he doesn’t manage to hit the bull’s-eye. Accordingly, a result can be considered as reliable but it doesn’t necessarily imply that the result is valid.
Figure 24: An illustration of the differences between validity and reliability. ( Zikmund, 2000)
In order to obtain valid and reliable results in this study several precautions were made. When developing the research questions I was careful about covering all aspects of the research problem, and the research questions were also approved by both supervisors. The questions included in the interview guide were based on the research questions, which ensures that the questions are relevant for the study. In addition, the interview guide were reviewed by both supervisors.
I used multiple sources of information, both documentation and interviews, which is important when collecting data. It is also important to collect data from several respondents, but due to the time limitations for this study the number of respondents that could be interview were restricted. However, I managed to make interviews with several respondents at the two customer companies and with two retailers
In order to receive reliable answers the respondents must be experienced within the current area of interest and I believe that the appropriate respondents were found. They were very knowledgeable about their companies and their needs. It could have been desirable that the respondents at Company A & Company B were more knowledgeable about the CRM approach. However, since the answer regarding research question one primly is based on the theoretical frame work the respondents lack of knowledge doesn’t constitute any problem.
When performing interviews there is a risk of misinterpretations and loss of information. In order to avoid this I used a tape recorder during the interviews which implied that I could check the answers with the notes I had made. In addition the probing technique was used, which means that the interviewer asks for clarification or further expansion of answers that are unclear or concise. This further improved my interpretation of the answers given from the respondents. In addition, the answers from the interview were typed down shortly after the interview had took place, this way it was still fresh in the memory. However, the validity of the answers may have been undermined by the fact that the respondents may have misinterpreted some of the theoretical concepts used during the interview.
The personal interviews with Company A were conducted in English since the head respondent is from England. When conducting interviews in English the risk of not being able to talk freely and to make misunderstandings is greater. However, since English is the mother tongue for the head respondent and everyone else was used to
down the data collected. However, the risk for misunderstandings was reduced regarding the interviews with Company B. This since the respondent received a presentation of the data before the analysis was performed, a few corrections were made and then the respondent approved the presentation. Due to lack of time it was not possible to send the other data presentations to the other respondents.
As mentioned earlier it is very important to put the right questions to the respondents. Thereby. I asked several questions during the first interviews with Company A and B in order to be able to put the right questions during the second interview occasion. For example, during the first interview I asked several questions about the steps in the sales process in order to, during the second interview, be able to ask questions about the CRM requirements at the different steps.
When making the literature review there is a risk that certain areas not are covered. Thereby I also performed an analysis of three CRM systems. This way I could increase the likelihood of covering all areas of CRM functionality and thereby develop more appropriate questions to the respondents.
When performing interviews there is a risk of influencing the respondents. In order to receive the right answers I tried to keep neutral and not influence the respondents. However, sometimes I found it hard to stay objective since there were occasion were I had a different view from the respondents.