Previous literature indicates that the extent of CSRD can be measured through various methods, either through weighted or non-weighted techniques (see Arshad et al., 2012; Naser, Al-Khatib and Karbhari, 2002; Gao et al., 2005). Non-weighted index is the simplest way in which the extent of CSRD is measured by counting the number of CSR-related instances or items that are disclosed in annual reports (see Gao et al., 2005; Janggu et al., 2007; Menassa, 2010). This form of measurement, however, was criticised for not considering the extent of information disclosure (Branco and Rodrigues, 2008) which indicates the significance and weight of each disclosed item (Zeghal and Ahmed, 1990). Therefore, to address this problem, weighted index with the use of word count (see Gao et al., 2005; Menassa, 2010), sentence count (see Amran and Devi, 2008; Janggu et al., 2007), or page count (see Lynn, 1992; Thompson and Zakaria, 2004) have been applied to measure the extent of CSRD in some previous studies.
Appropriate units of measurement are also debated in the literature. For instance, while scholars, such as Gray et al., (1995a) and Milne and Adler (1999), support the use of page count with the argument that page count reflects the total space given for a specific topic; and therefore, indicates its importance, this type of measurement was criticised by Lavers (1993). The author argued that the use of page count tends to be subjective due to the differences in front size, graphics and margins (Hamid, 2004). Regarding the use of word count, in contrast to the justification that the volume of disclosure can be examined in greater detail with the use of word count, some studies claimed that individual words do not provide any meaning without its context (El-Bannany, 2007; Milne and Adler, 1999).
Similarly, the use of sentences as units of measurement can also be problematic as it ignores the fact that two different writers can convey the same message with different numbers of sentences using differing grammatical styles of (Unerman, 2000). Therefore, previous literature on this issue has shown that different measurements can have their own strengths and weaknesses. As a result, the use of an appropriate unit of measurement should be based on the context of study. In this study, the use of non-weighted index is considered more suitable as the use of weighted index (word count, sentence count or page count) could create subjectivity and bias during the analysis
167
process as the format and presentation of each annual report can be very different from another, considering the cross-national context. Moreover, previous literature shows that the choice of weighted or un-weighted index would not result any significant difference in findings (Chow and Wong-Boren, 1987; Firth, 1980; Hossain and Hammami, 2009). Finally, as the simplest type of measurement, the use of non-weighted index can be easily replicated in future studies.
As discussed in chapter 2, section 2.2, information on CSRD is collected based on a checklist of items adapted from the study of Branco and Rodrigues (2008). The checklist classifies CSR information into four main categories: environmental disclosure (ED), human resources disclosure (HRD), products and consumers disclosure (PCD) and finally community involvement disclosure (CD). The total number of items on the checklist is 30, with 11 items in ED, 9 items in HRD, 5 items in PCD, and 5 items in CD. With the use of non-weighted index, a company is awarded 1 if an item in the checklist is disclosed and 0 if it is missing. The CSRDI is calculated by the ratio of actual scores awarded to the maximum score that a firm could achieve. In most of the cases, the maximum score is 30 as there is 30 items on the checklist. However, in order to avoid penalising firms for not disclosing the items unrelated to their business, certain items are excluded for firms from the industries of healthcare services, banking and financial services, hotels, leisure and entertainment. The excluded items are presented as follows:
Healthcare services: items such as pollution from business conduct, pollution created from the use of company’s products, prevention or repair of environmental damage, environment aesthetics, energy efficiency of products, discussion of specific environmental laws and regulations, product safety, and disclosing of consumer safety practices are excluded, making the maximum number of items for this industry is 22. Banks and financial services: pollution created from the use of company’s products,
prevention or repair of environmental damage, environmental aesthetics, energy efficiency of products, discussion of specific environmental laws and regulations and product safety are excluded. The maximum number of items for this industry is 24 Hotels, leisure and entertainment: pollution arised from the use of company’s products,
prevention or repair of environmental damage, environmental aesthetics, and energy efficiency of products and discussion of specific environmental laws and regulations are excluded, making the maximum number of items is 25.
168
Software and computer services: pollution created from the use of company’s products, prevention or repair of environmental damage, environmental aesthetics, energy efficiency of products, discussion of specific environmental laws and regulations, product safety, and disclosing of consumer safety practices are excluded, resulting with a maximum number of 23 items.
4.6.1.2 Reliability and Validity of the Construction Index
(i) Reliability
Reliability refers to the question of whether a measurement is stable (Bryman, 2016) and able to provide the same results in separate occasions (Howell, 2013). This concept is important for a content analysis study to address the issue of reliability of not only the instruments but also the data collected using those instruments to ensure inferences drawing from the data are replicable and valid (Milne and Adler, 1999). In order to ensure the reliability of the construction index and the results of content analysis, following the suggestion of Krippendorff (2013), three elements of reliability, stability, reproducibility and accuracy, are examined in this study. However, as the accuracy element which is preferred to as the issue of determining coding performance against an expert built predetermined standard or from previous studies (Krippendorff, 2013) can be related to the validity of the construction index, only two elements, stability and reproducibility, are discussed in this sub-section.
The first element, stability, also known as intra-coder reliability, is mentioned as the degree to which the process of content analysis remains unchanged and produces the same results over time. Assessing stability requires the involvement of test and retest procedure in which a sample of the data should be coded again by the same coder after a period of time. High correlation from the test and retest procedure is a sign of stability in measurement. In this study, to examine the stability of the construction index, a sample of 30 companies is re-analysed after three-months. The results between the two times of coding were relatively similar without any major difference. As a result, the intra-coder reliability can be assured.
Regarding the reproducibility of measurement, the main aim of this reliability type is to measure the extent to which the coding results are the same when there are multiple coders (Milne and Adler, 1999). In this study, a second coder is used to ensure the reproducibility of measurement.
169
The second coder is a postgraduate researcher who has previous experience in content analysis and a competent understanding of CSR concept but not involved in any other aspect of this study rather than re-coding a small sample. This requirement helps to ensure the independence of coder to allow freedom of judgement without the influence of the researcher. A process of training was applied prior the coding assignment as it allows coders to familiarise themselves with definitions, coding rules and scheme (Kolbe and Burnett, 1991). The second coder was explicitly explained the categories, meaning of each item and when an item is considered disclosed. After fully understanding the process, the second coder was asked to analyse one document to test whether he perceived the coding procedure correctly. At the end of the training, the second coder was provided a code book with detail description of each item (Appendix 2) and asked to choose a sample of 30 companies with 5 randomly chosen companies from each country. When the coding is finished, the two coders met and discussed any difficulties raised from the coding process and further develop description for some vague terms in the items. When comparing the results of two coders’ analysis did not show any major difference. Therefore, inter-coder reliability can be guaranteed.
Furthermore, in order to assess internal consistency of the construction index, the Cronbach’s alpha test is employed. Internal consistency measures the extent to which the items in the checklist measure the same construct and hence are internally consistent (Bryman and Cramer, 2005). Some previous disclosure studies (see Botosan, 1997; Khan et al., 2013a; Aribi and Gao, 2010) have used the Cronbach’s coefficient alpha to examine the degree that correlation amongst categories of the disclosure index are reduced as a result of random error. According to previous literature (Field, 2013, p.709; Gul and Leung, 2004, p.360), the alpha value above 0.70 is considered acceptable. In this study, the coefficient alpha value of four disclosure categories in the CSRD index is 0.715, which exceeds the cut-off point and higher than some previous studies of Gul and Leung (2004), Khan et al. (2013a) and Aribi and Gao (2010). Therefore, the statistics indicate that the disclosure items in the index captures the same construct. Table 9 presents the result of the Cronbach’s test.
170
Table 9: Reliability test of the CSRDI
Cronbach’s Alpha Value No. of items
0.715 4
(ii) Validity
According to Matthes and Kohring (2008), validity’s indicator is the extent to which the measuring instrument measures the concept it is supposed to measure. Validity plays an important role in ensuring inferences drawn from the coding procedure are supported by empirical evidence and theoretical rationale (Rourke and Anderson, 2004). In this study, different procedures have been adapted to ensure the validity of the construction index. Firstly, as mentioned in the previous chapters, the index was adapted from the study of Branco and Rodrigues (2008) which was developed based on well-recognised CSRD instruments. Therefore, the appropriateness and validity of the index could be ensured. Moreover, according to Rourke and Anderson (2004), a valid protocol does not either miss out behaviours that should be included or include behaviours that should be ignored. The CSRDI includes 30 items classified into four main categories, ED, HRD, PCD and CD. Compared with other indexes in some previous CSRD studies (see Anas et al., 2015; Esa and Ghazali, 2012; Sobhani et al., 2009), the index adapted in this study is more exhaustive with most mentioned items included and diverse aspects of CSR covered. After a trial analysis, however, the researcher recognised that the items mentioned in the checklist are too general and may cause confusion and/or inconsistency during the coding process. As a result, a detailed coding guideline with detailed descriptions for each item was developed based on previous studies that Branco and Rodrigues (2008) adapted, including Gray et al. (1995a); Hackston and Milne (1996); and Williams and Pei (1999) (Appendix 2). Besides, the coding guideline is also modified after the trial analysis as to include some relevant points. The coding guideline, then, is discussed with the two supervisors for further alterations. Furthermore, the instruction index has also been tested through the presentation and submission in various national and international conferences. Therefore, the validity of the construction index can be well-assured.
171
4.6.2 Independent Variables