Contramedidas y Plan de Administración Ambiental .1 Contramedidas

A range of data sources, discussed in chapters 3 and 4 have been used to construct key reference modes. All raw data were processed using a combination of data analysis tools MySQL 5.4.11 community server, Mathworks Matlab 2016b, Python 2.7.13 and R – Statistical computing.

During the quantitative data collection process three key limitations were observed about the data. These limitations include incomplete data, methods of data collection at source and source bias. In the first instance data that has been collected is almost certainly incomplete, hence the abductive nature of this research with incomplete data is a regrettable feature within the sciences, more so when undertaking an exploratory investigation of an online phenomenon such as vulnerability discovery and disclosure. It is however possible to compensate for the lack of complete data by imputing, or calculating any missing data with approximate replacement values.

The second limitation is centred on the collection methods and quality assurance that is used by third parties when collecting data from end users (e.g. ExploitDB). These methods are unknown and therefore must be viewed with an appropriate level of confidence. Again, it is possible to compensate for issues that arise from insufficient quality control of collected data. Where possible data that is used to inform state variable construction has been compared with at least one other similar dataset, and checked for consistency. Where this has not been possible, this is highlighted. Furthermore, it is reasonable to assume that third parties have made every effort to correct and deal with omissions and errors, and the quality of the data that has been collected from third parties is of reasonable quality.

Finally, a general limitation to all collected quantitative data used is the nature of the phenomena under investigation - it is globally distributed and opaque. As such, data sources that have been selected are the most prevalent, or popular, and therefore taken as the most representative of the feature that is being observed. For example, data representing sentiment toward software originators was collected from social media platforms such as Hackernews.com

and Reddit.com as this is where the richest data was located. Other sources of data exist, however are potentially less concentrated or not publicly available. Where data is missing or there is a potential bias or error within the data, statistical techniques that are used to compensate for these issues are noted. The data sources that have been collected are outlined below in Table 18, along with sources, associated variables and online location.

Table 18 outlines several properties about data that is used within the EDA framework. Variable name represents the descriptive aspect of the behaviour, and is a direct link to previous themes. Data source, lists the name of any locations of raw data, and appropriate detail. Location is the online web URL, and date accessed shows the time the data was collected and processed.

Variable Name Data Source Location Dates Accessed or Collected Variable A: Researcher

sentiment

Hackernews.com; Reddit.com; Online blogs (See chapter 4 for details)

Online November 2014 –

April 2016 Variable B: Number of

Vulnerability Discoverers within System

ExploitDB.com www.exploitDB.com December 2016

Variable C: Vulnerability Removal Rate National Vulnerability Database https://nvd.nist.gov/ December 2016 Variable D: Full disclosure and Coordinated Disclosure Ratios

Open bug bounty trading platform

www.openbugbounty.org Sept 2015 -Jan 2017

Variable E: Time to fix vulnerability from coordinated disclosure

Multiple academic studies Full Disclosure Email Archive http://seclists.org/fulldisclosure/ February 2015 – February 2017 Variable F: Monitory reward Hacker One Bug Crowd www.hackerone.com www.bugcrowd.com January 2014 - December 2016 Variable G: Number of Bug Bounty Schemes

Hacker One Bug Crowd www.hackerone.com www.bugcrowd.com November 2013 – Apr 2017 Variable H: Vulnerability Discoverer Participant Activity ExploitDB

Open bug bounty reporting platform

www.exploitdb.com www.openbugbounty.org

January 2017

Variable I: Software Originator Market Share

W3Cschools Wikimedia Foundation Statcounter http://gs.statcounter.com/ www.w3cschools.com https://analytics.wikimedia.org/d ashboards/browsers/ March 2017

Meta Variable: Time and Delays

Full disclosure email distribution list

Vulnerability Policies

http://seclists.org/fulldisclosure/ Multiple

5.1.1

The System Dynamics modelling process requires the identification of key state variables that represent the phenomena under investigation. Therefore, extracting representative variables from the identified themes that best describes the VDDS is a critical step in completeness of theory and models. The pulling out of variables which numerically express the state of a system or sub-system is known as state variable exposition. State variables are used to describe the state of a system at a specific time, or over a defined period of time (Palm, 2010, p.229). Within System Dynamics these behaviours are known as reference modes, and describe how the state variable changes over time (Sterman, 2000, p.90). Variables, and therefore reference modes, are drawn from within the five themes identified previously (Perception of Punishment, Disclosure Stance, Vendor Interactions, Motivation for Discovery, Emergence of Markets) and are a direct representation of the structure of the VDDS and relationships within it. The Identified themes also provide a rich narrative upon which to both base dynamic changes of the VDDS and build the structure of the model (Coyle, 1996, p.26; Morecroft, 2015, p.60).

State variables were selected based upon identified themes, with three key criteria influencing the choice; necessity, aggregation and directionality (Albin et al., 2001, p.10). Each state variable characterises factors within each theme as it impacts the VDDS. For example, in the case of vulnerability interaction time, this variable has a direct causal link to the sentiment within the VDDS. Furthermore, a key factor within the VDDS is time. Time is represented as a both an intrinsic aspect of the variables (i.e. the behaviour is mapped to elapsed time) and within variables themselves. Thus time represents how the theme evolves, and is represented by relationships and a number of processes steps as opposed to state variables. Variables such as number of quantity of vulnerabilities, the disclosure route that is taken and quantity of active discoverer are all provided as level of activity, initialisation parameters and associated rates of those activities. A mapping of identified themes and derived state variables is given in Figure 13.

Figure 13 - Systemic Themes and Variable Grouping.