This section describes the different game events that can occur during game play. A random number is generated during the play of the game; an event is generated when this random number falls in between specified ranges (Table 7-5). The triggered event is displayed to the user and affects the user’s score, if the user does not have the corresponding item in the inventory list. Items in the inventory list can be obtained through the accumulation of points and points are earned by answering questions correctly.
For example, if the game event triggered is a virus infection then the user needs to have the anti-virus item to avoid losing points. Otherwise, 500 points will be deducted. The user can obtain these items through the accumulation of points by answering questions correctly. If enough points have been acquired during the play of the game, then the user
can use these points to obtain available items. For example, the user needs to accumulate 1000 points to obtain the “Password Training” item.
Table 7-5: Game Events participants. This section describes the data gathered during the collection process. The participants were undergraduate students from the University of Venda (South Africa) who are registered for a degree programme in Computer Science and Information Systems.
The students who graduate from the course will have sufficient knowledge, skills and competence to conduct development work within the information systems business sector.
The layout of the course addresses many aspects of information systems which include Introduction to Computer Systems, Database Fundamentals, Data Communication and Computer Networks, Software Engineering and Artificial Intelligence (University of Venda 2013). However, no module within the course addresses security within information systems or networking. This implies that the participants will not have been exposed to the wide variety of security-related topics covered by the information security awareness program.
In spite of the lack of security-related modules, some of the participants could have directly or indirectly learned about some of the information security topics. To elaborate, the use of privacy controls on social networking sites might have been emphasised by
security-related events propagated by various news sources. For example, the social networking site Facebook had to compensate some users after their privacy was violated (Miller 2013). In this way the participant could have indirectly learned about the use of privacy controls to manage private information. Another example of indirect learning would be the creation of strong passwords; many websites provide users with feedback to enforce the use of strong passwords.
In direct learning, the participant chooses to learn about security-related topics. For example, the participant could have an interest in computer security and learned about the different aspects from websites that focus on security-related topics like Net-Security (http://www.net-security.org/), Hackin9 (http://hakin9.org/) and Security Magazine (http://www.securitymagazine.com/). In addition, the students that form part of the sample have formal education course outcomes which are aligned with most of the tertiary institutions within the Gauteng province in South Africa. The Tshwane University of Technology (TUT) only provides an information security module during the fourth year of studies. The University of Pretoria (UP) also addresses information security during the latter part of the studies. The undergraduate degree at University of Johannesburg (UJ) does not contain a computer security module, but offers certificate courses that focus on cybersecurity from the first year of study. Furthermore, the University of South Africa (UNISA) includes computer security as part of postgraduate studies. The participants from the University of Venda were selected due to their availability to partake in the study as well as their opportunity to transfer knowledge to other rural communities.
During the literature review, it was shown that quantitative and qualitative data could be used for information security awareness research. Dey (2003) described qualitative data as a measure of relative worth based on the evaluation from a general observer, for example, how a person emotionally experiences a movie or book. In other words, data that can be observed but not measured. The most common method is the qualitative research interview, but other forms of the data collection can also include group discussions, observation and reflection field notes, various texts, pictures, and other materials (Major &
Savin-Baden 2013). Qualitative data was not included in this study because limited resources were available to complete the study. The use of interviews, group dialogue and reflection would have provided more data in conjunction with the quantitative data to form an accurate conclusion.
Quantitative data is measurable data, for example the height of people, colour of hair, and number of correct answers. The most common method used in quantitative research is the use of surveys or quizzes (Balnaves & Caputi 2001). The use of surveys and quizzes to determine the awareness levels of a target group has been considered in Section 7.2 and is listed in Table 7-1. This study used quantitative data with the limited resources available and it was anticipated that this would provide enough evidence to prove gaming to be effective to reinforce learning.
Quantitative data was captured from two sources; the surveys which served as quizzes to measure the knowledge levels and the online game to track the progress of the participant and reinforce the learning process. The data collected from these two sources are described in the following two sections.
7.3.3.1 Surveys
Online surveys were used to collect data from participants in the form of quizzes. The questions were displayed to the participant together with possible answers. Only one of the possible answers presented to the participant was correct, while all the other answers were incorrect. The quantitative format allowed the analysis of the data to be conducted with the use of Microsoft Excel (Dretzke 2005) and R (R Core Team 2013). R is a free statistical analysis language while Excel forms part of Microsoft Office. Both these software packages not only provide powerful statistical analysis capabilities but also graphing of results.
The online survey web application stored the answers in a database and allowed the researchers to export the data into different formats including comma separated values (CSV) and Microsoft Excel spreadsheets. This allows the data to be imported into other software analysis tools. The answers resemble the option selected as the most viable option from the available list by the participant. The answer is depicted by an integer number that corresponds to the option: for example, the third option would be represented by a “3”. The questions were also categorised to resemble the different topics. The answers provided would also have the same sequence, thus correlating with the different categories. Other data include a unique identifier that correlates with the answers provided by the participant. The main objective of the questionnaire data is to determine the current knowledge levels of the participants: this is achieved by determining if the answer provided is correct.
7.3.3.2 Online Game
The data generated in the online game was used for multiple purposes, which included tracking the progress of the participants and measuring the understanding of the information security awareness topics. The data generated during the game play was stored in a database. The online game also used the stored data to display the questions, each of which belonged to a category related to an identified security topic. The random events were generated by the online game and stored as part of historical data and enhanced the features of the game. The responses created by the participants were stored and compared against the stored correct answers. This was used as a mechanism to score points to determine the leader within the sample, and also to determine which categories (topics) the participants performed better in: in other words, the identification of which information security-related topics needed to be focused on to improve the total information security awareness within the identified group. The tracking data was used to determine how long the participant took to answer each questions and how many questions were answered. This was achieved programmatically by having a timer running from when the question is displayed to when the question is answered. Network latency did not affect the study due to the location of the participants: the study was conducted in the same location, and hence network latency-related issues would affect all the network latency would affect the tracking feature of the game, which determines how long the participant takes to answer a question. In addition, having participants from multiple locations accessing the game during any period of the day introduces uncertainty in the results because participants could have received help to answer the questions from friends or other online resources.
The use of quantitative data limited the findings to the results from the questionnaires.
Multiple-choice questions have an inherent flaw because it is possible for participants to guess the answer correctly. If four options are available then the participant has a 25%
chance to randomly select the correct option. The use of qualitative data could have
supported the hypothesis that gaming could reinforce learning by repeating the content to the participant. Furthermore, interviews could also have been used to support the findings from the quantitative data. For example, if the quantitative data showed that information security knowledge had increased, interviews could have been used to support the findings.
The data was collected in one day. The amount of knowledge transferred during the limited time could have affected the knowledge retained. Although breaks were given to assist in concentration management, many participants could have lost interest and focus as the day progressed. Conducting the data collection over a period of at least two days would have been ideal. This would have allowed for assessing the retention of knowledge more accurately. The first day would consist of the initial pre-assessment and first post-assessment data collection, while the next day would be used to conduct a final data collection after the game play to compare against the findings after the last assessment on day one.
The use of one sample does limit the findings to a specific population. It would have been beneficial to collect data from another sample and compare the findings between the two groups. The size of the current sample also influenced the accuracy of the findings. Having a control group during the day of the collection who did not partake in the online game could have proven valuable, as this would have shown the significance of motivation. One sample was selected for the study due to the number of participants available for the study. Also, lack of time and funding limited the collection of data from the sample to one day.
A profile of the sample population is absent from the current study. The profiling would have provided a more accurate representation of the sample and could have been used to correlate with the pre-assessment data and other personality traits. For example, it could have determined what motivation indicators the group has. This would have been beneficial to understand what the effect of providing a reward for the game play would have had.