These problems with the initial sample are only compounded by the absence of any effective mechanism to rebalance the panel as members are lost to non-response in later waves or by passing out of scope. So far the only proposal for a new refresher sample has related to securing a higher representation of newly arrived migrants, and even this remains only a suggestion (Watson and Wooden 2007: 228). It is possible that some of the more mobile sub-populations may be less affected over time by the geographic bias in the original sample because CSMs who leave an originally sampled household are tracked to their successive households, thereby including in the sample the other members of their new households. Some reduction in geographic bias may also be expected if sampled households themselves move to new locations. However, it seems doubtful whether these mechanisms alone will be sufficient to bring about any worthwhile adjustment over the short period for which data are so far available.
For the purposes of the present research, the main interest lies in those members of the panel who are currently in the active labour force and employed, and who complete the relevant sections of the SCQ. These effectively “leave” the sample for these purposes when they retire. At the other end, they are replaced by household members who pass the age of 15 and thus become eligible to respond to the individual and self-completion questionnaires (i.e. pass from being simply “enumerated persons” to “responding persons”). Given that this threshold is at least one year below the minimum legal school leaving age in all Australian States, a lag of some kind can be expected before these “recruits” become employed in the primary labour market, and that lag is likely to be considerably greater
before any of them settle into jobs requiring substantial entry-level qualifications; to the extent that they are employed in between, it is likely to be in casual or part-time student jobs which are not representative of their longer-term career paths. In any event, their jobs in these initial years will almost certainly be very different from those left by retiring panel members, and logic suggests that the differences will be especially marked in respect of skill content, learning and task discretion. There is no reason to expect that these changes in the occupational balance of the panel, affecting only the extreme points in the age spectrum of the active labour force, will be representative of those experienced by the employed workforce in general.
To this “natural” attrition must be added the impact of attrition proper, i.e. loss of sample through members becoming untraceable or unwilling to participate in later waves. The HILDA attrition rate is considered acceptable or even creditable by the standards of comparable household panel surveys in other countries (Watson and Wooden 2004: 345), but it has certainly been more than trivial. In particular, between the first and second waves some 13% of households in the original panel dropped out, though many of them returned in later waves. Even accepting that the rate of repeat response (both between adjacent waves and from members who were non-respondents in earlier waves) has grown steadily (Wooden and Watson 2007: 215), by 2006 the panel had lost almost 28% of its original members, while only 63% have responded in all six waves (Watson N 2008: 118). The managers of the survey acknowledge that attrition bias has not been random; in particular, so far as the purposes of this research are concerned, it has meant a declining proportion of respondents who have lower levels of education or work in lower-skilled jobs (Watson N 2008: 123).
These losses were offset by the recruitment of some 1429 new entrants, equivalent to 11% of the 2006 sample, over Waves 2-6 (Watson N 2008: 122). However, the only ways of being recruited to the panel are to join a sampled household, either as a new member from the outside population through marriage, partnering, joining a group house, etc., or by being born or adopted into it, or else to be part of a household which a continuing sample member joins. Given that births are unlikely to affect the relevant variables for at least another fifteen waves, any “natural” rebalancing must occur through the other mechanisms, or else through non-sampling mechanisms such as children passing the age of 15 or individual members moving to different jobs.
The limited impact of such mechanisms is apparent when the 2006 Census data are used to compare changes since 2001 in the composition of the working population with those that occurred in the composition of the HILDA sample. More detail on this aspect is provided in Chapter 9. Indeed, given the amount of attrition and its non-random distribution, it is surprising to find that the representativeness of the panel in 2006 has not notably
deteriorated. Neither is it significantly better than in 2001; it is simply different. The over- representation of workers in Education has increased to 29%, while that of Professionals and Agriculture remains strong though somewhat diminished (17% and 34% respectively) and that of Accommodation, Cafes & Restaurants has increased from 3% to 9%. Managers, Labourers, Electricity, Gas & Water Supply and Construction have moved from over- to under-representation, while Intermediate Clerical, Sales & Service workers have moved in the opposite direction. With those exceptions, it is generally the trades and the lower- skilled occupations that remain under-represented. Substantial growth in the proportion of the population engaged in Mining has not been captured in the sample, nor has a substantial decline in Cultural & Recreational Services. Thus, even apart from the representativeness
of the sample in each individual Census year, the changes in the composition of the sample between 2001 and 2006 do not accurately reflect those that occurred in the composition of employment as measured by the Census.
To the extent that it is possible to generalise, these considerations taken together suggest that the HILDA data for both years probably overstate the average amount of skill
demanded across all Australian jobs. The discrepancies reflect both the selection bias in the original sample and attrition bias working against lower-skilled occupations, and together would suggest that the stable or gently declining trend which appears in the main indicators of skill-intensity and task discretion across the first six waves cannot be simply an artefact of the sample. This question is examined fully in Chapter 9.
More generally, the evidence on these first two sources of error indicates a need for caution in taking changes in the overall occupational composition of the panel over successive waves as indicative of changes in the composition of employment. The evidence also dictates some caution in assuming that findings from the same analyses undertaken on the full panel in successive years accurately mirror change in the same population.
At first sight it might appear that many of these problems have been offset by the inclusion in recent HILDA releases of longitudinal population weights to compensate for changes in sample composition on a number of key parameters that include occupation (Watson N 2008: 86). However, their usefulness appears, so far as can be deduced from this and other documentation (Watson 2004), to be limited by the fact that they are derived from a single model in which occupation is only one among a number of benchmarks, the others being sex by broad age, State by part of State, State by labour force status, and marital status. Most of these are demographic, and the priority given by the sample designers to demographic representativeness may itself be one reason for the errors in occupational balance in the original sample. In any event, even if it is assumed that application of these weights would produce a balanced panel that accurately mirrored the industry/occupational composition of the base-year sample, such a panel would self-evidently be useless for tracking the incidence or impact of changes that actually occurred across the six waves in any of these parameters.