DIABETES MELLITUS

Written evidence submitted by Market Research Society (4STATS 01)

I am writing on behalf of The Market Research Society (MRS) in response to the PASC fourth study on communicating and publishing statistics. No part of the MRS submission needs to be treated as confidential.

With members in more than 70 countries, MRS is the world’s largest association representing providers and users of market, social, and opinion research, and business intelligence. All individual MRS members and MRS Company Partners agree to self-regulatory compliance with the MRS Code of Conduct. Full details regarding MRS and its activities can be found via: www.mrs.org.uk

MRS has an advisory board, the MRS Census and Geodemographics Group (CGG), which recommends MRS policy in relation to geographic, demographic and census information and which has advised MRS in the formulation of this response. CGG has been operating continuously for the last twenty years and has been a leading voice representing business users. It includes members with experience in the creation and use of population statistics, census data and geographic information. MRS and CGG have worked with the UK Census Offices, via the Office for National Statistics, representing the needs of the research sector in consultations on the 1991, 2001 and 2011 censuses.

MRS and CGG welcome the commitment of Government to the principles of Open Data and transparency. We believe the commitment to improved accessibility and use of government data will provide long term benefits to the wider economy.

We have attached an Annex that contains our responses to the questions raised by PASC. The issues that MRS and CGG consider particularly important are:

— Official statistics are widely used in marketing and research.

— But private sector needs have low priority, which runs counter to government policy to help grow the economy by use of its data in business.

— MRS’s top priority is better access to data, essential to good communication and to realising full value from the investment in official statistics.

— The supply of official statistics as open data is a major advance by the government which is warmly welcomed by MRS and its members.

— There are many opportunities to get more value from official statistics, such as a self-service facility allowing for ad hoc output within defined parameters and disclosure control rules.

— It could be counter-productive to try to put all official statistics on one website, better search services are a more feasible option.

— Insufficient resources are devoted to communicating data and messages in the data. This issue should be a priority for the Inquiry.

Annex

MARKET RESEARCH SOCIETY RESPONSE TO THE PASC PROGRAMME OF WORK ON STATISTICS

Fourth Study: Communicating and Publishing Statistics

Summary of the Market Research Society (MRS) Response — Official statistics are widely used in marketing and research.

— But private sector needs have low priority, which runs counter to government policy to help grow the economy by use of its data in business.

— MRS’s top priority is better access to data, essential to good communication and to realising full value from the investment in official statistics.

— The supply of official statistics as open data is a major advance by the government which is warmly welcomed by MRS and its members.

— There are many opportunities to get more value from official statistics, such as a self-service facility allowing for ad hoc output within defined parameters and disclosure control rules.

— It could be counter-productive to try to put all official statistics on one website, better search services are a more feasible option.

— Insufficient resources are devoted to communicating data and messages in the data. This issue should be a priority for the Inquiry.

Importance of official statistics

Official statistics are widely used in marketing and market, social and opinion research both directly and to help structure and analyse other data. Figures for very local areas, mainly from the population census, applied ingeodemography, help in understanding patterns of customer preferences and in effective location of services. Priorities for marketing and research

The broad scope of the Inquiry is welcome and necessary, as effective communication through all media is essential to realise the full value of the investment in official statistics. There are many opportunities for improvement, with a top priority of better access to data, and deployment of relatively modest additional resources can bring a high return from greater use.

Open data, a major advance in communication

The supply of statistics as open data with minimal restrictions on re-use is major advance especially appreciated in marketing and research. It facilitates adding value though further products and services, ultimately adding to effective communication of the statistical data. However, restrictive terms of use and cost of key supporting data such as Ordnance Survey address gazetteers incorporating data from the Royal Mail Postcode Address File discourage use, for example in the coding of data to match with statistical geography, and diminishes potential benefit from open statistics.

There is a strong case for extending the scope of open data, and the Government’s transparency and open data initiatives are welcome as is inclusion of this important area later in this Inquiry.

The Committee’s Questions

Q1.How well are the practices for the release of official statistics, and pre-publication access to those statistics (“pre-release access”) working?

1.1 The publication of schedules for the release of official statistics, as under current practice, is important when planning use in commercial activities where timetable slippage is costly, and particularly where services to third parties are involved. The arrangements are working, although lead times once the actual dates for release are announced seem short.

1.2 Pre-release access is generally less important, provided that there is no commercial gain from any privileged access.

Q2. (a)Has the right balance been struck between the resources devoted to producing statistics and those devoted to communicating them?

2.1 Insufficient resources are devoted to communicating data and messages in the data.

2.2 This is illustrated by the Census, and although there is not yet a publicly available breakdown of expenditure for the 2011 Census, the indications are that, given for example the lengthy timetable by international standards for publication of results and a lack of innovation in access via the Web, little has changed since the 2001 Census when only 3% of the budget was devoted to preparing and disseminating results. The Committee could helpfully ask for more information on the current position.

Q2. (b)What should be the future of the National Statistics publication hub, particularly in the context of the proposed single Government domain for communicating data?

2.3 The publication hub is convenient but not essential for commercial users who generally are familiar with relevant sources, and there are better ways of providing access.

2.4 It would not be the best way forward for statistics to become part of a monolithic Government domain, and indeed could be counter-productive, on the evidence of the long standing shortfalls of the smaller scale ONS website intended to integrate all output. A less restrictive “umbrella” would be more appropriate to the very varied nature of statistical output—from single key indicators to millions of Census figures.

Q3.In what circumstances should the UK Statistics Authority comment on the use or misuse of statistics? 3.1 UK Statistics Authority should act to ensure supply of “full and frank” commentary to help pre-empt criticism, for example of the discontinuation of a series, as well as ensuring compliance with the Code of Practice for Official Statistics and responding to perceived abuse.

Q4. (a)To what extent are the requirements of users taken into account in decisions on the communication and publication of official statistics?

4.1 Experience shows that private sector needs tend to have low priority, unless shared with the public sector, with communication and publication being no exception, which runs counter to government policy to help grow the economy by use of its data in business.

Q4. (b)How easy is it for users to find the official statistics they need, whether via the Office for National Statistics website, other websites, the National Statistics publication hub or elsewhere?

4.2 It is not easy to find data on the ONS website or publication hub. Making it easier would be the single greatest contribution to better access and communication.

4.3 Search facilities are poor, often giving long, undifferentiated lists. First and foremost the need is for a search service which covers all relevant sites—rather than trying to put all data on one site. The objective should be to reach a point where all official statistics sites have identical search engines so that results for a specific search are identical whichever site is used.

4.4 Further measurable objectives should be set to help improve access and meet user needs:

— Making all data easy to download, and developing ways such as Application Programming Interfaces (APIs) to download updated data automatically into users’ systems.

— Providing clear and consistent guidance and metadata, targeted at a range of user communities and expertise from schools to businesses.

— Providing clear and consistent explanation of methodology.

Q4. (c)How well do the formats used for releasing official statistics meet user needs, both in terms of publication type (compendia, reports, press releases) and data format (pdf, excel, open data standards), and what changes, if any, should be made?

4.5 In one key area—the supply of large Census datasets in CSV format—needs have only been accepted after repeated requests.

Q4. (d)How effectively does the UK Statistics Authority engage with the user community to understand its requirements, and what, if anything, should it do differently?

4.6 The activities of UK Statistics Authority (UKSA) tend not to come to the notice of users outside government who may also not be clear about the distinction between the Authority and statistics producers— an area for improved communication and promotion. UKSA should put more emphasis on strategic, cross government issues such as the publication of comparable output across all parts of the UK—a major requirement for businesses operating nationally.

Q5.How well are trends in, and limitations of, official statistics (produced by the Office for National Statistics and elsewhere) explained to users? Do these explanations have the right level of detail and how balanced are they?

5.1 There is need for better explanations, issued concurrently with statistics, and at a more detailed geographical level.

5.2 Users in marketing and research, as well as elsewhere, need quality measures to indicate the degree of confidence which can be put on results. Otherwise there is a risk that figures will be taken as perfectly correct. Quality measure should be published concurrently and be well publicised to help minimise this risk. As quality is not consistent in a set of output, measures should also be published for all levels of area aggregation. Q6.How well do producers of official statistics respond to ad hoc requests for data, and what should be the publication policy be in respect of responses to these requests?

6.1 Ad hoc output has not been very significant for commercial users. The technical process of producing such output has had limitations, timetables can be long and uncertain, definitions used in official statistics may differ from those used elsewhere, statistical disclosure control tends to restrict detail, and poor value for money may discourage commissions.

6.2 Policy should be to actively exploit the many opportunities to get greater value from existing datasets, avoiding the cost of collecting new data, with better and more flexible production systems, co-operative producer-user partnerships, and generally higher priority for ad hoc output. Self-service systems should be used more widely to allow users to produce their own ad hoc output within defined parameters and disclosure control rules.

6.3 An example of the way datasets can be opened up to yield more value is the coding of data in major official surveys by type of small area—the classification itself developed in a producer-user partnership—after encouragement by geodemographers, thus adding a completely new geographical dimension to the analysis of the survey data.

official statistics?

7.1 In the context of the websites of large organisations, the Office for National Statistics (ONS) and other producers have embraced internet technology relatively well in principle, with examples of innovation such as the work of the small ONS team producing interactive Web content, but there is a need for significant improvement in structure and function as commented in response to Question 4.

7.2 However, it is not easy for outsiders to get a full picture of what is realistically possible, and the Committee would help greatly by focussing on this highly relevant area.

October 2012

Written evidence submitted by Tom King (4TATS 02)

Summary

Statistics can of necessity be complex in their formulation; where this is the case the assumptions behind them need to be clearly explained. Such explanations need not be written anew for new releases but could be designed to support a particular class of release and simply linked to each new iteration.

Central user engagement is dominated by large, fixed uses; a shift from users to uses should be the next step in the user engagement strategy. It should be possible to engage individuals in research and evaluation environments for this purpose.

UKSA is known for criticising statistical use but should balance this by commending good practice and developing good communication. Statistics without good metadata invite problems of interpretation so communicating statistics without linking to appropriate sources of information should be controlled.

Finding statistics and what information may be held is harder than it should be, maintaining a substantial barrier to use. As other users suggest developing resources including some derived from ONS data, there may be partnership opportunities but certainly insights into user needs.

Public understanding of statistics has wide variations but for unfamiliar applications it will need support. The value of statistics in providing accountability should be researched not presumed. Consideration could be given to those who do not use statistics but may benefit.

The text roughly follows the ordering of the questions paper until paragraph 18 when more general issues are discussed.

1. The sheer number of breaches in relation to release and pre-release access and the lack of evidence of problems caused by these demonstrate that the purpose of pre-release access and a publication timetable are not well understood by anyone.

2. There may be a more indirect effect of appreciation of the unaccountable nature of unequal access to information. An example comes from a recent Public Accounts hearing on diabetes: A person giving evidence wished to cite the latest performance statistics which were to be published later that week in the belief that these would be more useful. It was readily understood that the NAO had not had access to these data and so referring to them was inappropriate. Equal access to data facilitates accountability by agreeing certain facts without privileging the producer of the data.

3. Not enough is understood of the use made of statistics to judge whether their communication is particularly limiting. However, it is certainly true that statistics would find more uses were they communicated better and that many users would request further statistics if they were better communicated.

4. The UKSA is well known for criticising political misrepresentation of statistics and little known for anything else. It would do better to promote serious use and understanding of statistics by instigating production of explanations of technical aspects where they are being misunderstood. (Where commentary exists, it focuses on release-to-release trends, not underlying issues.) Criticism can foster a lack of trust if other issues are not taken in hand.

5. Two particular issues where interventions might be made are misrepresentation of the precision of official statistics, and to dispute “zombie statistics” which have assumed the status of official fact. These issues might require its position on statistics which were published before the Act was passed to be clarified. An example from inquiries into Mid Staffordshire Foundation Hospitals Trust illustrates all three issues.

6. A leaked inquiry draft report included an observation on mortality of roughly 800–1200 above the average for similar hospitals in that period, which might statistically be attributed to poor standards of care. This value was never published and an estimate of around 400–600 over a three year period was the official confidence interval for the excess mortality. Media reporting subsequently took on the interval 400–1200, or “as many as 1200”. These misrepresent the precision and vivify the value of 1200. The origin of the 1200 value was confusing enough that the independent inquiry struggled to identify its origin.

7. There is an underlying problem that the needs of users are not well understood. More details are given from paragraph 18 but publication and communication rarely facilitate information integration in the sense of composite statistic construction.

8. Finding simple statistics is such an ordeal for novice users that there are repeated suggestions that online resources be independently developed for teaching in schools and universities (eg http://rsscse.org.uk/images/ stories/ts_pedagogy_full_report.pdf). The premise is that it is too difficult to expect teachers to do this to support topical lessons.

9. Recurring needs of large users are easier to address and engage with and various user groups exist to facilitate this. However, they have not been successful in coalescing concerns into changes in known underlying problems in methodology (eg RPI/CPI).

10. Much more metadata to support releases and more proximate links to other, related (in the minds of users) datasets would be very helpful.

11. UKSA would do better to change its focus from the demands of large users to establish the uses of statistics in end users which are often not evidenced.

12. Initiatives such as the MAC are excellent but there is no option for users to contribute to the agenda. Thus the integrity of the statistical system as a whole has no forum for user consultation as opposed to specific surveys and datasets which have consultations.

13. There needs to be further consideration of public understandings of statistics: misconceptions of assumptions, resistance to quantification and unilateral acceptance/rejection of findings. A presentation by the GCSA to the ESRC RMF 2012 showed that statistical understanding is essential when considering future policy. Statistics will be fundamental to citizenship and therefore governance but UKSA has not looked to see whether these uses are being served by present dissemination of official statistics.

14. Trends and limitations can be difficult to understand and can be hard to explain (PISA is a recent