As a matter of fact, the 1990 census had already made a clear conclusion on the notorious program statistics. When the SFPC finally decided to resort to the SSB data source in 1991, it seemed that it had every reason to abandon this statistical rubbish. However, given its availability at each administrative level, the SFPC and its subordinates continued to collect these statistics and devoted much effort to improving the quality. This situation is quite like what Scharping (2003: 207) pointed out: ‘the system continues to be maintained, because discarding it would impair accountability and evaluation of family planning performance even more’.
3.7.1 SFPC countermeasures in improving the quality of program statistics
Since the early 1990s, the SFPC have taken a series of measures to strengthen the statistical work, including (1) highlighting the problem of data manipulation and falsification; (2) updating the stipulations for family planning statistics; (3) requiring provincial commissions to invest in the building of statistical capability at the basic level; and (4) directly conducting random quality checks, mainly village surveys, through household interviews. At nationwide conferences during the 1990s, Peng Peiyun, the then SFPC Minister (1988-1998) repeatedly criticised the underreporting problems and highlighted the importance of collecting accurate statistics (Peng 1992; Peng 1993b). Moreover, the SFPC successfully lobbied the top leaders to address this issue. For example, in the 1994 annual central forum on family planning, the then President Jiang Zheming praised the 1993 SFPC village surveys in Hebei and Hubei province and asked provincial leaders to pay attention to the improvement of statistical quality. Similarly, when SFPC survey team revealed serious problems in the family planning work in rural Shanxi province in 1999, they reported to the then
Premier Zhu Rongji, who seriously criticised the provincial leaders and asked them to make improvements (SFPC 2000a).
It was suggested that the one-veto-down system introduced after the 1991 Decision increased the pressure on local cadres to manipulate statistics (see Yu and Xie 2000; Zhang 1995). But the 1991 Decision also stipulated that one serious statistical falsification could also lead to wiping out other performance, being an important component of the one-veto-down system, which (CPC Central Committee and State Council 1991). This requirement may not ensure improvement in data quality, but at least it could deter potential statistical manipulation occurred at higher administrative levels, the provincial and prefecture levels. Many provincial commissions, such as those in Hunan, Jiangsu and Shandong, also frequently organised random surveys to target local manipulation in statistics (Cai and Zhang 2000).
The imperfect statistical procedures and requirements were considered by the SFPC to be one cause of the inaccuracy of statistics. The disorganisation of statistical offices, and the lack of specific persons for compiling statistics, were also recognised as being responsible. Consequently, the SFPC in collaboration with the SSB reformed the statistical procedures in the program statistics (Peng 1997). The SFPC repeatedly asked provincial commissions to ensure the establishment of statistical institutions and personnel recruitment, and further the necessary statistical training for personnel and funding for statistical investigations. In fact, one task of the SFPC village surveys was to push local governments to attach importance to statistical work in rural villages, in addition to inspecting the quality of local statistics.
Until the early 1990s, the SFPC had realised that the problems in the program statistics, the worst data falsification, occurred at the rural township and village levels. The problems in the 1992 survey further supported this conclusion (Jiang et al. 1996). Some provincial family planning commissions, if not involved directly, were suspected of at least turning a blind eye to local manipulations or falsifications. It was a long way to go to form the national statistics through many administrative levels from the registrations at village up to the SFPC at the top. Undoubtedly, the more numerous the layers up which the statistical data were transmitted, the more serious the distortion of the data. Therefore, the SFPC concluded that probably the
most effective countermeasure was to obtain data directly from respondents through household interviews, avoiding any possible interference from intermediaries.
Subsequently, the SFPC decided to directly investigate the rural population situation. These investigations relied upon its own staff, and recruited some additional personnel only when it was really necessary, such as those with geographic familiarity, or who could speak local dialects, but from provinces other than those sampled, to ensure that the survey remained unknown to local cadres. The first two of these series of carefully designed surveys were conducted in rural Hubei and Hebei provinces in October 1993. As expected, the village surveys revealed that local family planning statistics in rural Hebei and Hubei underreported births by an astonishing 35 percent in 1992 (Wang and Wang 1995). Two SFPC statisticians involved in the 1993 surveys described the whole process two years later. To better understand why such surveys could reveal concealed births, it is worthwhile to quote a description here.
At the dawn of the enumeration date, the investigation team directly drove to targeted villages. Once reaching there, two team heads were responsible for seeking out village heads and other cadres to introduce the purpose to them, more importantly, ask them to provide village-held hukou registration booklets, women’s information cards, statistical reports of births, immunisation records for newborns after 1992, etc. They also asked them to provide statistical figures of total households, number of persons in each household, lists of women of reproductive age, marriage and birth information after 1992, and then copied these materials.
At the same time, other team members directly conducted household interviews family by family, without omitting any family or any person in the village. After reaching each family, the investigation members used all kinds of interview skills to inquire for information on marriages, births and adoptions, particularly from those women who had these events after 1992. Meanwhile, they also asked interviewees to present necessary materials, such as ID cards, hukou registers, marriage certificates, childbirth certificates and other documents. If they had any suspicions about births, they insisted on seeing the children in person and managed to persuade those women who were afraid to tell the truth. If some women did leave the village before the survey, interviewers would confirm their status by questioning at least three neighbours to make sure there was no discrepancy at all (Wang and Wang 1995: 28).
In the following years until 1999, similar village surveys were conducted primarily in rural areas in another nine provinces. The majority of them are populous agricultural provinces in middle China, which had difficulty in reducing fertility and high occurrence of statistical manipulation and falsification (SFPC 1993-2000). Not surprisingly, these village surveys were successful in revealing underreporting in local program statistics, ranging from less than 5 percent in Shangdong province in 1994, to 30 percent in Gansu province in 1995 and in Guizhou province in 1996 (Table 3.7).
Table 3.7 The extent of underreporting of births in rural family planning statistics in selected provinces in China, detected by the SFPC village surveys, 1993-1999
Year Province Location
Program performance before the survey
Underreporting extent in local program statistics in specified reference period Hebei 11 rural counties Average 35 percent (1992)
1993
Hubei 8 rural counties Average 35.5 percent (1992) Henan 11 rural counties Average Around 20 percent (1993) 1994
Shandong 24 rural counties Lower than average Lower than 5 percent (1993) Gansu 11 rural counties Better than average Very serious
(about 30 percent in 1994) 1995
Hainan 18 rural counties Average Very serious
(about 30 percent in 1994) Guizhou 18 rural counties Better than average 22.6 percent (1992-95) 1996
Beijing Floating population - Mean CEB less than national average
Shanxi 23 rural counties Better than average 16.37 percent (1994-98) 1998
Sichuan 18 rural counties Better than average 9.46 percent (1993-98) Shannxi 15 rural counties Average Good
1999
Hubei 15 rural counties Average Good Source: The SFPC village survey series are from the SFPC (1993-2000).
Note: The SFPC did not give exact figures of underreporting extent in 1994, 1995 and 1999.
3.7.2 Implications of the SFPC village surveys
The SFPC village surveys were widely considered to be successful in revealing underreporting of births in rural areas. However, the detected extent of
underreporting of births did not necessarily translate into the same extent in other statistics, particularly in censuses and surveys; it is misleading to directly relate findings in these surveys to other analyses. There are many important implications of these surveys for the general understanding of underreporting of births and data quality, but unfortunately, these have remained unnoticed until recently.
As discussed in Section 3.5, the family planning statistics are compiled and reported by local program workers, who can manipulate them at will when necessary. Recognising this problem, the most important procedure of village surveys was to exclude the interference from local cadres. The SFPC made every effort to ensure direct contacts between the interviewers and respondents. The survey results demonstrated that this strategy was successful. It is likely that respondents could still conceal out-of-plan births, but the surveys had succeeded in revealing many of them. For example, in Hebei province alone, the upward-reporting of births by the survey- sampled villages in 1992 missed out-of-plan births by an astonishing 35 percent. This underreporting was achieved by inflating the in-plan birth rate and misreporting higher-order births as first-order births. The proportion of third and higher-order births detected in the village survey was 14.7 percent, 18 times the 0.8 percent reported in local statistics (Wang and Wang 1995).
It was generally believed that couples who had out-of-plan births would conceal these births once they were questioned in surveys and censuses. However, the village surveys suggested that when the interviewers questioned individual couples in person, these respondents appeared very co-operative. Indeed, a number of field studies, undertaken by trained anthropologists or demographers, suggested that peasants were co-operative in telling of their out-of-plan births (Chen 2003; Greenhalgh 1994; Murphy 2003; Zhang 1998). It is very likely that the general perception that out-of-plan births would not appear did not come from field experience.
Many authors cited the astonishing underreporting of births in the program statistics to suggest data deterioration in other statistics, such as surveys and censuses (e.g. Goodkind 2004; Merli and Raftery 2000). However, it is extremely important to keep in mind that the detected extent of underreporting represented only the problems in
the program statistics themselves. This is well illustrated by the two separate SFPC village surveys in Hubei province. In 1993, the SFPC first-round village surveys revealed that the local program statistics in rural Hubei underreported births by 35 percent in the year 1992 (Wang and Wang 1995). Facing strong pressure after being exposed this problem, the family planning commissions in Hubei province paid much attention to improving the program statistics. In 1999, the SFPC carried out another unnoticed random village survey in Hubei. The exact extent of underreporting in local program statistics was not revealed, but the SFPC seemed satisfied with the provincial program progress achieved, and their comment of ‘good’ indicated that the extent of underreporting was probably around 5 percent (SFPC 2000a: 105-106).
Another important implication is that one should be very cautious in using the detected degree of underreporting in these provinces to extrapolate to the problems in other provinces or the national aggregate statistics. When the SFPC decided to randomly investigate these provinces, their accumulated problems in program performance and population statistics had already been recognised. In fact, the surveyed results just confirmed the SFPC’s long-held concerns. Moreover, these special surveys revealed only the underreporting in birth registration of surveyed areas, much different from those already adjusted at the provincial level. It is easy to overestimate the degree of underreporting at the national level, if directly employing the surveyed extent of underreporting. In a widely cited paper Zeng (1996: 32) estimated the number of births as 25.96 million in 1991 and 24.40 million in 1992, using the findings from village surveys in Hebei and Hubei province. However, his estimation of the number of births exceeded any other available source. The SSB adjusted figures based on the annual surveys were 22.58 million in 1991 and 21.19 million in 1992, while the estimated figures from the 2000 census using the reverse- survival method were only 20.87 and 19.48 million, respectively.