Sesgos y Obstáculos de la evaluación por competencias

The current study followed the qualitative research approach wherein the demand for trustworthiness became an absolute necessity. Dimmock and Lam (2012: 188) postulate that trustworthiness has to do with issues of validity and reliability. For this study I will adopt four criteria as noted by Shenton (2004: 64) citing Guba (1981) for use to ascertain trustworthiness. These are credibility (in preference to internal validity), transferability (in preference to external validity/generalisability), dependability (in preference to reliability) and confirmability (in preference to objectivity). From the next paragraph I indicate how each one of them was achieved in the case of the current study.

The first element by which to ascertain trustworthiness in research is credibility. I contend that credibility was unavoidable if this study was to be authentic. The term credibility refers to the factual accuracy of the research findings (Fitzgerald, 2012: 301). In the case of the current study it relates to the validity of the data within the confines of Solusi University. This implies that there needed to be a continuous examination or review of the information that would be coming from the interviews and the documents. In line with this I adopted the following strategies as suggested by Leedy and Ormrod (2010: 100) as a way of ensuring trustworthiness of my findings:

a. Extensive time in the field- apart of my ten years of academic experience at Solusi University, I used the second semester of 2015 (September to December) to uphold or correct certain tentative insights that I had formed over the years. I did this by comparing these with the information that was given by the various respondents and groups. The varieties of responses were compared with each other. The determination was based on the accuracy, applicability and truthfulness of the evidence.

b. Negative case analysis- I used the semester to continually improve upon my existing insights and assumptions by eliminating the contradictory ones. This largely involved a continuous comparison of the various components of my on- going analysis. Each episode and the distinct responses or evidences from the

109

documents were treated separately and compared with each other. This was so in order to establish consistency in my analysis. The inconsistent ones were either revised or abandoned completely.

c. Thick description- I acquired sufficiently rich and informative detail from my investigations. The resultant data was presented in a detailed mode that is transparent and self-explanatory. This should enable any reader to comprehend the data (see Sections 5.1 and 5.2).

d. Feedbacks from others- Conclusions have been drawn from the data that was presented. These needed feedback so as to regulate and modify their credibleness. There are other lecturers and senior staff such as those in the Registrar’s office who were not part of the participants in the current study. As such I did interface with such colleagues so as to get their opinion on my treatment of data.

e. Respondent validation- The lecturers and students who were interviewed form the actual players of the situation being described. I did communicate my findings to them in order to get their opinion on the correctness and truthfulness of my conclusions.

The second element by which to ascertain trustworthiness in research is transferability. One must be able to carry into action the conclusions of the research. This refers to the possibility of having other readers to draw inferences from the research findings after applying them to their own contexts and situations (Dimmock and Lam, 2012: 202). In other words, my conclusions will need to be valid in other stations or organisations other than Solusi University. This however does not necessarily mean that the conclusions of any qualitative study can be generalized to all situations. The distinctive characteristic of qualitative research is particularity (Creswell, 2009: 193). As such the descriptions and themes in this research have been developed in the context of a specific site, namely Solusi University. The readers of the conclusions to this research will decide for themselves whether the results speak to their situation and experience (Pitney, 2004: 27). Hence the breadth of my research has to be plausible for it to be valid externally.

Thirdly, dependability is another element by which to ascertain trustworthiness in research. Dependability together with confirmability has to do with issues of reliability of the findings of the study. According to Anney (2014: 278), this involves participants evaluating the findings of the study to make sure that they are all supported by the data received from the

110

informants of the study. This is possible where at least two methods have been used to collect data from the participants (Shenton, 2004: 71). In the current study the use of the face-to-face interviews with the focus groups and the individual lecturers ensured that dependability was achieved. The documents that I collected from the lecturers, namely copies of course outlines, quizzes, tests and assignments also served the same purpose.

The fourth element by which to ascertain trustworthiness in research is confirmability. This ensures that the findings of the research have been derived from the data that was collected and not from the biased notions of the researcher’s experience (Anney, 2014: 279). A common feature between dependability and confirmability involves issues of consistency and solidity of the findings of the study. Therefore, I also used a common approach to ensure that dependability and confirmability were established. From start to finish I maintained the raw data as I collected more information. Then I made it as a rule to go over the data that I was capturing from the interviews and documents, making comparisons and double checking with the participants.

In addition to all of the above four measures I factored in two concepts which have been recognised by scholars in dealing with validity and reliability in qualitative research. These are triangulation and member checking. They are considered to be part of the most powerful and common ways of ensuring that trustworthiness is met ((Dimmock and Lam, 2012: 202). In other words, they may be applied to the whole exercise of making certain that there is trustworthiness in research.

Triangulation is common and relevant to the applicability of the four elements namely credibility, transferability, dependability and confirmability of qualitative research. It augments the validity and reliability of research information by use of more than one source of data (Bush, 2012: 84). This was applied to the current study by the interviews as well as the document analysis that were done. Interviews were done with four different lecturers and also four different focus groups. Document analysis was done on four different sets of documents in the form of course outlines and quizzes, tests and assignments. These were able to provide a variety of data sources.

Member checks are also relevant and common to the applicability of the four elements namely credibility, transferability, dependability and confirmability of qualitative research.

111

According to Kornbluh (2015: 397), they consist in making follow ups with the participants to verify whether the researcher’s perception of the data is compatible with their input and intended meaning. As such the process of data collection, recording, categorising and classification of themes was on-going from the beginning. During that process I conferred with the participants to check if they went along with what I had captured and given a certain meaning. This ensured trustworthiness of the study.