Executives and senior staff of Fortune 1000 companies were surveyed annually from 2012 to 2018. Details of the design of the first survey can be found in the methodology chapter. The questions posed aimed to obtain insight and to benchmark big data initiatives at Fortune 1000 companies in the USA. The results over the period 2012 to 2018 have been consolidated for the purposes of this secondary data analysis.
Table 5-5 indicates the number of respondents from each of the annual surveys:
Table 5-5. Number of respondents
2012 2013 2014 2015/6 2017 2018 Number of respondents to
the survey Over 50 Over 50 59 44 50 Nearly
60
Since 2012, the respondents have grown from around 50 to nearly 60. The author, Thomas Davenport, in the foreword of the Newvantage Big Data Executive Survey (2012), suggested that this survey was “one of the few I have seen that focuses on large organizations and offers responses from C-level executives” (Newvantage Partners, 2012, p.2). The respondents surveyed are, thus, senior leaders in their organisations, which makes the survey extremely valuable, since big data initiatives are decided by and investments are made by the executives of large organisations. The growth in the executives participating over the period shows the importance of big data and being data driven in Fortune 1000 companies in the USA.
Table 5-6 gives the responses for the various surveys regarding the big data initiatives introduced in the respondents’ organisations.
Table 5-6. Big data initiatives
2012 2013 2014 2015/6 2017 2018
Big data initiatives
Not indicated.
85% had plans
31.4% 48.2% 62.5% 87.1% 97.2%
Currently, in 2018, 97.2% of companies surveyed have big data initiatives, such as using big data in business decision making, etc, underway in their organisations, whereas in the 2013 survey only 31.4% of the organisations that responded had big data initiatives implemented in their organisations. In 2012, when these surveys were conducted for the first time, none of the companies indicated that they had big data initiatives in their organisations; however, 85% had plans to introduce such initiatives.
One of the reasons for having big data initiatives implemented in the organisations surveyed, is to create a data-driven culture within the company. The 2015/6 survey shows that only 9.3% of the respondents indicated that a business driver of big data is to create a data-driven culture. Results from the 2017 and 2018 surveys show that 85.5% and 98.6%, respectively, indicated that their companies aspired to be more data driven.
The results, with reference to the question on whether organisations are moving towards a data-driven culture, are shown in Table 5-7:
Table 5-7. Data-driven culture
In 2018, only 1.4% of the organisations surveyed did not have the creation of a date-driven culture as a priority, whereas in 2017, 14.5% did not have it as a priority. This is a big change in over just one year. In 2017, even though 85.5% aspired to have a data-driven culture, only 37.1% felt that they had been successful in this. For 2018, the success rate among those surveyed reduced to 32.4% with 98.6% of the respondents aspiring to have a data-driven culture. Some of the challenges for the poor success rates cited are people (48.5%), process (32.4%), and technology (19.1%), according to the 2018 Newvantage Big Data Executive Survey (Newvantage Partners, 2018). Having a data-driven culture at a large organisation means that big data initiatives will be implemented and taken seriously and will ultimately generate financial benefits for the organisation.
It is the responsibility of the CDO to create a data-driven culture in an organisation (Patil and Mason, 2015). Newvantage Partners (2018) agrees with this and suggests that the CDO has emerged as a key role player in changing the culture of the organisation to that of a data-driven culture over the years. Because of this, many organisations have introduced the role of CDO in their organisations (Newvantage Partners, 2018).
Table 5-8 shows the various organisational roles that participated in the survey, which over time included more from the C-level of the organisation, thus showing interest in the importance of big data from these executives. The proportion of C-executive respondents dramatically increased from only 30% in 2012 to 93.1% in 2018.
Table 5-8. Organisational roles of the survey respondents
*Senior Information Technology and Data Experts within the company
The number of CDOs who responded to the survey increased from 30% in 2012 to 55.6% in 2018. This shows the increasing role of the CDO, which is part of the argument of the research. This stakeholder is critical in the data-driven environment in the era of big data. Over time, the presence of the CDO as a respondent to these surveys implies the importance of this role and is shown in this growth in the number of CDOs in the USA.
The companies surveyed by NewVantage Partners have shown a steady increase in the percentage of companies with a CDO. It increased from 12% in 2012 to 63.4% in 2018.
The results are shown in Table 5-9 overleaf.
Table 5-9. Companies with a chief data officer
2012 2013 2014 2015/6 2017 2018
Companies with a
chief data officer 12% 26% 43% 54% 55.9% 63.4%
The growth in the rise of the number of CDOs over the past years is in agreement with the qualitative research and is also supported by the literature. In the USA only 6% of the companies surveyed have a CDO (JLL™, 2016). In Europe, 13% of companies have CDOs, whereas a CDO exists in only 2% of companies in Middle Eastern and North African countries and by 2019, 90% of large organisations would have CDOs to gain competitive advantage and to make use of their data assets (JLL™, 2016).
Even though many companies in the USA and across the globe are increasingly appointing CDOs in their organisations, it appears that even though there is evidence that the role of the CDO is recognised (Newvantage Partners, 2018), there is no consensus of what the mandate of the CDO should be, according to Bean (2018).
To understand whether there is a link between the growth in the number of companies with CDOs and companies with big data initiatives, a correlation analysis was performed to establish whether there is such a link. The correlation analysis, based on data from 2013 to 2018, showed a result of 0.938, which indicates a strong positive correlation (a linear relationship) between the number of CDOs and the number of big data initiatives. This suggests the increasingly important role of the CDO, but a deeper exploration is needed to establish whether there is a causal relationship between the presence of a CDO and big data initiatives at that company.
The secondary data analysis is complementary to the online survey conducted with companies in Cape Town, as well as with some retailers from outside South Africa,
where collaboration and the aspects influencing collaboration were explored. This data analysis corroborates the findings from the additional qualitative studies, where concepts like the role of the CDO, as well as big data in a data-driven organisation, was explored.