Interpersonal trust is one of the important factors in the performance evaluation process. In the current study, the variable is operationalised as follows.
4.9.1 The Instrument
The measure of interpersonal trust was derived from Read (1962). However, the four questions in Read (1962) were tailored into a five-item, five-point Likert- scaled instrument agreement statement that reflects the BSC setting. Respondents were asked to indicate their agreement with each of the statements in the survey regarding their interpersonal trust with the parties involved in the performance measurement process. One such example is: “My senior manager takes advantage of opportunities that come up to further my interest by his/her actions and decisions”. Each item had a five-point response scale of agreement as explained in Section 4.6.1. The complete instrument is presented in Appendix I – Part A.
4.10 Managerial Performance
The latent construct of managerial performance in this current study is divided into two constructs, which are: 1) managerial performance based on division (business unit) manager perception of their performance (division manager’s
self-assessment); and 2) managerial performance based on division manager’s view of senior manager’s perception of performance. The instruments of the two constructs are discussed below.
4.10.1 Division (Unit) Manager Perception of Their Performance
The nine dimensional five-point Likert scaled employed by Mahoney et al. (1965) is a self-rating measure used in this study to evaluate the managerial performance variable. The scale comprises of eight performance dimensions and one overall effectiveness dimension. This self-rating measure is chosen because it has been used extensively in earlier studies (Heneman, 1974; Brownell, 1982; Brownell and Hirst, 1986; Brownell and McInnes, 1986; Brownell and Dunk, 1991).
The self-rating scales have been criticised for their tendency for respondents to be too lenient on themselves, thereby resulting in a small range in the score being observed (Prien and Liske, 1962; Thornton, 1968; Mia, 1989). However, according to Brownell (1982), it has the advantage of overcoming the problem of “halo error”. “Halo error” is the tendency to evaluate “globally” or, in other words, to evaluate only one cognitive dimension (Brownell, 1982).
A high inter-correlation among separate dimensions is evidence of “halo error”, which seems to occur with the ratings of senior managers (Thornton, 1968). Additionally, the nine dimensional structure of the measure clearly captures the multi-dimensional nature of performance without introducing the problem of excessive dimensionality (Brownell, 1982). Independent assessments of reliability and the validity of the Mahoney instrument provide supportive evidence of the measure’s sound development (Heneman, 1974).
The nine dimensional structure of the Mahoney measure includes a single overall performance rating, together with ratings on eight sub-dimensions. Respondents were asked to rate their “performance as division (unit) manager” on the following dimensions: planning, investigating, coordinating, evaluating, supervising, staffing, negotiating, and representing. Each item had a five-point
response scale with the endpoints 1 (extremely poor) and 5 (excellent), and responses were summed. In this survey, the 0 (zero) response is also provided for the response “No Basis for Answering” which allow respondents who lack of knowledge in this area to respond to the statement, and ensure that the data collected are valid. The complete instrument is presented in Appendix I – Part A.
4.10.2 Managerial Performance Based on Division Manager’s View of Senior Manager’s Perception
A self-rating single-item measure of a five-point Likert scale is developed to measure the division (unit) manager’s performance based on the division manager’s view of senior manager’s perception. The self-rating single-item is chosen to confirm the self-rating measure of division (unit) manager’s performance based on the division (unit) manager perception.
The single-item approach has been criticised by many researchers (Oshagbemi, 1999; Nagy, 2002). Wanous, Reichers and Hudy (1997) divided the single-item measure into two categories: (a) single-item that measures self-reported facts such as age, gender, education and so on; and (b) a single-item that measures psychological constructs such as job satisfaction. Furthermore, Wanous et al. (1997) argue that measuring self-reported facts with a single-item measure is a common and acceptable practice. However, using single-item measures to measure psychological constructs is usually discouraged.
The problems regarding the use of single-item measures have centred on the difficulty in establishing internal consistency and reliability (Oshagbemi, 1999; Nagy, 2002). However, prior research found that a single-item measure is acceptable so long as it represents and measures the constructs (Nagy, 2002), implies their use, or when situational constraints limit or prevent the use of multiple-item measures (Wanous et al., 1997). It is obvious that establishing internal consistency and reliability can be important to evaluate the validity of an instrument; however, having an instrument that is more inclusive of the construct is even more important (Nagy, 2002).
Moreover, based on their examination of single-items to measure job satisfaction, Wanous et al. (1997) identified several reasons why single-item measures may be preferable. They include: (a) single-item measures take less space in questionnaires compared to multiple-item measures; (b) single-item measures can cost less to develop; (c) single-item measures are likely to increase face validity, since a single-item is usually easier to understand than a multiple-item. Additionally, respondents may dislike being asked questions that appear to be repetitive; and (d) single-item measures are better for measuring changes in job satisfaction.
In the present study, a single-item was developed to capture the construct division (unit) manager’s performance based on division manager’s view of senior manager’s perception. The single-item instrument was “In my most recent performance evaluation my senior manager rated my managerial performance as:…”. That item had a five-point response scale with the endpoints 1 (extremely poor) and 5 (excellent). The response scales and zero response statement follow those mentioned previously.
In order to reinforce the respondent’s answer about division (unit) manager’s performance based on the division manager’s view of senior manager’s perception, two-item instruments were developed to measure the respondent’s agreement of their senior manager’s perception of their performance. The two- item instruments are derived from Korsgaard and Roberson (1995). They are: (1) I agree with the way my senior manager rated my managerial performance; and (2) I agree with my final rating. Each item has a five-point response scale with the endpoints 1 (strongly disagree) and 5 (strongly agree). The response scales and zero response statement follow those mentioned previously.
4.11 Summary
In this chapter the research question which guides the research has been explored. This research question led to the development of the research framework which will be used to guide the research. The framework makes explicit the link between: participation in the development of performance
measures and the common-measure bias in the BSC; the fairness perception of the performance measure; the trust between parties involved in the performance evaluation process; and managerial performance. In the next part of the chapter, the set of hypotheses which ultimately answer the research question were formalised. This was followed by a discussion of the operationalisation of the key constructs along with the development of the indicators of each construct. The development of the indicator for the variables was based on prior instruments where possible, or developed by the researcher based on a thorough literature review. Qualitative tests to assess the scales were employed to ensure that the constructs were both valid and reliable. This was achieved by asking experts in the area of performance measurement using BSC and procedural justice. The measurement of each variable was discussed. In the next chapter, the research methodology issues will be explored in detail, including justification of the survey method used to facilitate the investigation in this research.