During the planning and review meeting held at ICRISAT from July 9-11, 2008, Dr. Hans P Binswanger suggested that we examine the ‘treatments effects in Village-level Studies’. His suggestion was to undertake an analysis for ascertaining whether the VLS activity has had any effect on improving the welfare of the people in the study villages. Data from 1975 to 2007-08 was examined for this purpose. Two types of effects were considered in order to examine the hypothesis that continuous engagement through surveys of households in a village may expand the horizons of the respondents and make them significantly more progressive and knowledgeable than others in the community. For example, respondent households who interact with the investigator once or twice every month may gain a further edge in terms of their knowledge of technologies and may be leading the technology adoption process in the village. Another possible effect relates to interactions between villagers and interviewers resulting in the entire village becoming more progressive than the neighboring villages. This set of hypotheses was tested by measuring certain indicators of treatment effects such as income, consumption expenditure, wealth status, rate of growth in asset values, size of holding and incidence of poverty. The data available from the VLS household panel enabled us to assess the treatment effects. The census data collected in 1985, 1989, 2000/2002 and 2007 were also analyzed to study the differences between respondents and nonrespondents.
Using the above information, the treatment effects were tested in two stages. In the first stage, the respondent households were compared with the nonrespondents to get an idea about the first level of treatment effect. At the second stage, households with varying lengths of exposure to the investigators were analyzed to determine whether direct interaction for different lengths of time influenced the degree of progressiveness. In the VLS sample, some households were tracked for 17 years, others for 7 years, and the remaining for 3-4 years. The possible third-level effect was not assessed because of lack of survey information about the villages neighboring the VLS villages.
The results suggest that there is no treatment effect noted in the six sample VLS villages. In 2007 census data, it was found that both the old VLS households as well as the sample since 2005 were not representative of the village and had a higher average ownership holding than the nonrespondents. The households that were added in 2001-02 were largely drawn from poorer households to make the sample more representative of the population. These new sample households had a lower average size of holding. The sample became biased again when the split-offs from the original VLS households were added to the sample in 2005. However, the differences in the size of ownership holding were attributable more to bias in the sample than to treatment effects resulting from different lengths of exposure to investigators and data collection process. The results obtained from t-test also proved that there is no significant difference in the mean own land holdings between sample and non-VLS households.
holding was slightly higher in the case of non-sample households, while in Shirapur, Kanzara and Kinkheda, sample households reported a slightly higher ownership holding than non-sample households. The bias present in the sample during the first generation VLS (with more representation to cultivators, less representation to landless households and zero representation to nonagricultural households) was corrected to a large extent by making the sample more representative of the village population. However, while the sample was drawn using operational holding as the criterion, the treatment effects were studied using ownership holding, which is a better indicator of household wealth. The small differences noted in the average ownership holdings of the respondent and non-sample households were attributable more to the change in the concept of land rather than to any treatment effect. The t-test results also lend support to this statement.
In the 1989 Census, the weighted average size of own land holding of VLS sample households was higher than that of non-VLS households in Shirapur, Kalman and Kinkheda, but lower than that of non- VLS households in Aurepalle and Kanzara. Due to the arbitrary decision of giving equal weightage to the four categories of households, the sample was not representative of their populations. Hence, the first-level treatment effect among them could not be studied.
Similarly, in the 1985 Census, among the three continuous villages for which data were analyzed, the VLS sample households were better off in terms of having higher own land holding than the non-VLS households. However, the limitation of nonrepresentative nature of the sample still exists. The results clearly showed that, in all the three villages, the landless labor households were under-represented with respect to their actual shares in the population. This phenomenon led to excessive representation of cultivator households in the sample. Hence, the differences in ownership land holding are more due to distorted samples than to treatment effect. However, the differences in their mean own land holding values were not statistically significant.
A comparison was made between the continuous and closed pairs of villages in the three districts of Mahabubnagar, Solapur and Akola. If any treatment effect has to be seen, it is expected that the continuous villages will report a faster rate of growth in assets and incomes than the closed villages. The continuous villages had constant interaction with investigators and research staff of ICRISAT for 10 years, while this interaction was limited to 3-5 years in the closed villages. However, the closed villages in Maharashtra reported a faster appreciation of assets as well as growth in incomes than the continuous villages. The sample households in Kalman reported faster growth in assets and income than those in Shirapur. Similarly, the sample households in Kinkheda recorded a faster growth in assets and incomes than Kanzara village. In the Mahabubnagar villages, sample households in Aurepalle reported a faster growth in incomes than the same in Dokur during the period 1975-84. However, both the villages reported similar growth rates with respect to asset values. Thus, we believe that the households in different villages responded to the development opportunities available to them and reported different rates of growth in assets and incomes. No systematic effect in favor of continuous villages was found even during the first generation VLS. The results obtained using t-test also conclude that the differences in their mean values were not statistically significant from each other. Similarly, the regression results also prove that the treatment dummy showed a negative relationship with differences in the respective indicator mean values (in most of the cases). So we can safely conclude that there is no treatment effect in VLS villages.
In fact, when the growth percentage in asset values of new VLS households that were added in 2001- 02 were compared with that of the old VLS households for the period 2001-02 to 2007-08, it was higher for the new VLS households which had much shorter exposure when compared with the old VLS
households with much longer exposure. The evidence that was analyzed did not give any indication that there are any treatment effects in the six VLS villages.
The analysis of census and sample data from both the second generation as well as the first generation VLS led us to conclude that there are no treatment effects due to engagement of households or villages by the VLS research team.