EVALUACIÓN DEL PROGRAMA

In a sense, data protection law aims to reduce the risks of complex, tightly coupled systems. The big data ecosystem consists of millions of devices, each generating or processing hundreds of transactions every day; every transaction can generate personal data. This data is then transmitted, stored, analysed, sorted, queried and delivered for reuse over numerous pathways, relayed through many diferent nodes, to large numbers of endpoints. Every pathway, node and endpoint is an avenue for the introduction of risk where errors can occur, or where data can be intercepted, corrupted or abused. Protection mechanisms are “brittle”: a single failure, sometimes in another area of a system, may expose large quantities of data. These failures are not always accidental: many data leaks are a result of targeted hacking attempts or calculated “leaks from the inside.” Indeed, especially in the big data ecosystem, protection mechanisms can introduce new pathways for error.333

“Normal accidents” concerning personal data appear to occur frequently. A very concrete example of complexity and tight coupling is the Diginotar case, where the hacking of a root certifcate authority in the Netherlands resulted not only in the supposed spying on individual Gmail users in Iran and the potential spying on all Gmail users, but also to the revocation of security credentials for almost all Dutch government servers.334 _{In another case, a vendor of PC hardware intercepted all}

network trafc from new laptops to serve personalised advertisements to its customers, exposing all data sent to and from the computer to third parties. Consumers had actually “agreed” to this practice by clicking through a license

333 _{Daniel Nunan and Marialaura di Domenico, ‘Big Data: A Normal Accident Waiting to}

Happen?’ [2015] Journal of Business Ethics 1, 5.

334 _{Axel Arnbak and others, ‘Security Collapse in the HTTPS Market’ (2014) 57}

Communications of the ACM 47, 48–49 <http://dl.acm.org/citation.cfm?

doidd2661061.2660574> accessed 19 March 2019; Hans Hoogstraaten and others, ‘Black Tulip: Report of the Investigation into the DigiNotar Certifcate Authority Breach’ (Fox-IT BV 2012) 45 <https://roselabs.nl/fles/audit_reports/Fox-IT_-_DigiNotar.pdf> accessed 13 February 2019.

agreement for the software that provided this function.335 _{Other possibilities for}

attacking data security, confdentiality or integrity are deep packet inspection (at the network level) or the leaking of sensitive login information, like in the Ashley Madison server hack.336 _{Nunan and Di Domenico give more examples of these “data}

accidents” and call them “normal accidents waiting to happen”.337 _{The information}

processing industry appears to agree on the necessity of systematically applying “best practices” to prevent these incidents: information security is the subject of a series of ISO standards, describing best known practices for information security management systems.338

Besides providing a framework for accident analysis, NAT provides a vocabulary that can be used for preventive analysis at all levels, possibly including the level of entire societies, for example to analyse whether free speech, equality and a democratic culture may be afected by permanent surveillance.339 _{Seen in Perrow’s terms,}

datafcation is causing data subjects to increasingly live in a complex, tightly coupled environment: it can “bring classifcation to an increasing range of human activity.”340

Especially through profling, it can have efects in seemingly unrelated areas: Havard and O’Neil have indicated education, advertising, predictive policing, employment, credit, insurance and the efects on civil society as areas of concern.341 _{In this way, it}

can cement or exacerbate the discrimination of marginalised groups. Peppet is wary about the onset of a “signalling economy”, where citizens feel pressure to reveal ever more details about their personal lives to avoid giving the impression that they are

335 _{Tim Ring, ‘Keeping Tabs on Tracking Technology’ (2015) 2015 Network Security 5, 6}

<http://www.sciencedirect.com/science/article/pii/S1353485815300477> accessed 13 February 2019; Perlroth (n 204).

336 _{Christopher Mele, ‘No Anonymity for Plaintifs Suing Ashley Madison Over Hack, Judge}

Rules’ The New York Times (21 April 2016)

<http://www.nytimes.com/2016/04/22/technology/no-anonymity-ashley-madison-hack- case.html> accessed 20 March 2019.

337 _{Nunan and di Domenico (n 333).}

338 _{See, for example, NEN-ISO/IEC 27001:2013 en: Information technology – Security}

techniques – Information security management systems – Requirements and the standards referred therein.

339 _{Weick (n 311) 29; Neil M Richards, Intellectual Privacy: Rethinking Civil Liberties in the}

Digital Age (Oxford University Press 2015) 153.

340 _{Dwork and Mulligan (n 224) 35.}

341 _{Cassandra Havard, ‘“On the Take”: The Black Box of Credit Scoring and Mortgage}

Discrimination’ (2011) 20 Boston University Public Interest Law Journal 241, 287

<http://papers.ssrn.com/sol3/papers.cfm?abstract_idd1710063> accessed 13 February 2019; O’Neil (n 268) chapters 3-10.

hiding embarrassing or disqualifying information.342 _{“Scores can become self-fulflling}

prophecies, creating the fnancial distress they claim merely to indicate” or they could “trap us in patterns that perpetuate the basest or narrowest versions of ourselves.”343

Complexity is further increased due to the fact that many data controllers, especially operators of online platforms like social media and search engines, operate in two- sided markets: a data controller can ofer a service or product to consumers for free or at a reduced price, making a proft from selling user information to third parties. Individual behaviours having efects in unrelated areas of life is a clear indicator of complexity, and — like in the case of ongoing release of persistent pollutants — the permanent “emission” of personal data in the context of datafcation means that there may be no real possibility to “recover” from profling, indicating tight coupling. This complexity and tight coupling and the possibility of large-scale accidents can be fertile grounds for fantasy documents to persuade lawmakers, data subjects and data controllers that the risks are acceptable.

4.4

Application of Risk Society Theory and Normal Accident