MoD gives work to 70,000 people in the UK defence equipment industry. An approach to DESO Head of Technical Services elucidated that the number of defence industry employees was 110,000 ± 5,000 for the latest year of DESO statistics (97/98), of which 70,000 were involved in MoD equipment expenditure; the remainder in exports. MoD’s purchase of electronic equipment in the same year was £982M, out of a total spend of £10,159M, but this excludes electronics embedded in platforms. In 1993 a Seaking airborne early warning helicopter comprised by value 25% airframe, 25% engine and 50% electronics. The tin box of an MBT represents only 25% of the total of £2M Challenger 2 cost. The remainder covers the engine, armament and electronics. It is thus reasonable to suggest that as much as half of platform costs are for electronic equipment. As platform cost for that year, including guided w e^ons, amounted to £3,840M, 50% suggests a further electronics expenditure of £1,920M, giving a grand total for all MoD electronic equipment of £2,902M (£982M + £I,920M) - 28% of the total procurement budget.
70,000 people work on MoD defence equipment, at an average turnover of £100K per employee, gives a total turnover of £7B, The out turn for defence production and repair in the UK was £7,200M - a good level of agreement. Thus it is reasonable to assume that the defence electronics industry (including COTS IT suppliers) employs some 28,000 people (2,902 7,200 x 70,000). Making an assumption that around 10% of those employed in industry are engineers, this suggests a total engineering population of 7,000 ± 330. Looking at the value statistics, if 28% of the total budget is spent on electronic, this suggests that 28% of 70,000 (19,600) of the population were involved in electronics, of which some 1,960 are engineers. This suggests that some 16% of the total electronic engineers were sent questionnaires and some 5% responded.
For the defence industry, there is no possibility of taking a large sample; the numbers involved are simply too great. In this case, the 319 sent questionnaires represents only a small percentage but can still provide a reasonable level of accuracy.
7.4 Analysis methodology
All analysis of the data gathered from the returned questionnaires involved the use of the computer program SPSS for Windows, release 7.5.1. This was an invaluable tool used for the calculation of mean values, variances and standard deviations as well as the automatic production of bar graphs and pie charts. A very few graphs had to be done in Excel 2000. It is also worth reiterating that the term OR is used in place of Equipment Capability Customer throughout this section and Section 8, as the change in title came only after all the graphs had been produced.
Missing data are included in the database, including the ‘insufficient knowledge’ category, but have been excluded in the analysis. At one extreme, there was only 4% missing data in response to the statement ‘an advantage of COTS IT is that it is familiar and thus easy to use’ while the percentage for COTS IT can be used in satellites’ was 30.3%. For platforms, many people only filled in boxes relevant to their particular role. In the case of terms and conditions, those isolated
from contractual w ork - OR staff - had an above average m issing rate, w hile on technical issues, it w as those with the least engineers - again O R - w hich had the largest ratio o f m issing data. Typical m issing data values are shown graphically in Figure 21.
Missing tanks Missing terms & conditions
Missing Missing
Missing viruses
Missing
Missing COTS IT in tanks
200 150
L
h DPA 100 ■ OR 501 1
1 O D E R A 0 Data Missing ■ IndustryMissing terms & conditions
200 DPA DERA Industry Data Missing Missing virus
Figure 21. Samples o f missing data rates.
7.4.1 Bias
One o f the key problems to be faced is that o f bias. Bias applies in terms o f the author, who has spent m ost o f his w orking life in industry, following a period as an RAF pilot. There is also the problem o f bias in gathering data from people who are to be interviewed or sent a questionnaire. In particular, the following points are recognised as areas where bias is likely to arise:
Bias issue
Selection o f the audience to be questioned. Responses only from those who have time. People saying what they think answers should be.
Reluctance to discuss problems in current systems.
G o-aw ay responses. M ischievous answers. Self-fulfilling prophecies.
Bias when interpreting the responses.
Response
Alm ost everyone in M oD departm ents selected. High rate o f return makes this fairly unlikely. Not found in subsequent interviews.
A surprising degree o f openness found in interviews.
Unlikely from arm ed forces/civil servants. Unlikely from arm ed forces/civil servants. Always a risk.
A logical statistical approach has been taken.
Each point has been examined in turn although it is recognised that any bias review is both subjective and judgm ental. To minimise this, a senior m em ber o f D EC staff, not involved in supervision o f this research, assisted in the review and made some constructive suggestions.
The approach to data gathering for this research has been to send people a questionnaire to complete and to interview a small sample of those who returned the questionnaire to provide an extended range of views on the subject. People in four different organisations were mailed the questionnaire between autumn 1997 and summer 1999. Each questionnaire carried an individual reference number to provide anonymity, apart from my own ‘confidential’ reference table of names against reference numbers. This is excluded from the thesis to maintain the promised confidentiality to those who completed the questionnaire. The number of questionnaires mailed to each group is shown in Figure 22, together with the return rates.
Organisation Sent
Estimated
received Replied % replied
% of estimated received MoD DPA 278 250 174 62 70 OR 327 284 173 55 61 DERA 274 247 99 36 40 Industry 319 287 101 32 35 Total 1198 1068 547 46 51
Figure 22. Questionnaires sent out to and received back from the various organisations.
One difficulty is that individuals in MoD are typically posted/promoted to a new position every three years and, in addition, frequent re-organisations add to the rate of staff change. Because of these factors, it has been assumed that 10% of those addressed have changed jobs and their successor has forwarded rather than looked at the questionnaire, and 214% are not involved in work where COTS IT is employed. Thus 1214% of questionnaires sent are unlikely to have produced any response.
Since the defence industry mailing list was compiled from a more diverse series of sources than the MoD one, it is likely that the addressees will have not received some letters because:
1. The address is incorrect because the company has moved or been taken over. 2. They have moved to another company.
3. They have changed jobs within their existing company. 4. They are not involved in military projects
5. They are not involved in projects using COTS IT.
Because of these factors, it has been assumed that 214% of addresses are incorrect, 1214% of those addressed have changed jobs or their job no longer exists and 10% are not involved in military work. As a result, it is assumed that 25% of questiormaires sent are unlikely to have provoked any response. Figure 22 shows the response rates both on the basis of questionnaires sent and on the estimate of questionnaires that are assumed actually to have reached the addressee. In calculating the error rates, the former figures only were used.
The difficulties of avoiding bias in gathering data start with the list of questions to be asked. These problems are accentuated by the very different backgrounds, experience, motivations and understanding of the issues of the people involved.
A total of eighteen questions (20 for the people in industry) were arranged into three distinct groups. The first deals with the use of COTS IT by UK MoD and considers the possible areas where COTS IT might have an application. The second examines its purchase and the third looks at the impact this use will have. Thus the following list of statements was used to prompt people to indicate their views. These views allowed five possible responses from ‘strongly agree’ to ‘strongly disagree’. An additional category - insufficient knowledge - was also included. When looking at means, standard deviations and variances, the following numerical values were used:
Strongly agree 5 Slightly disagree 2
Slightly agree 4 Strongly disagree 1