Diagnóstico

In order to evaluate our designed artifact, composed of the “as-is” situation of the three lines of defense, the “to-be” situation of the three lines of defense and the migration plan we worked via several iterations in which an alpha, beta and final artifact was created. During the creation of the alpha version of the artifact several experts of the internal audit field from within Ernst & Young and clients of Ernst & Young were interviewed. In these interviews interviewees were also asked several questions concerning the constructs of the Technology Acceptance Model of Morris et al. [41] and if the models were correct according to their expertise. Interviewees were of course also questioned about expected information management improvements from the “to-be” model and migration plan. The beta version of the artifact was then re-evaluated in a discussion with three experts of Ernst & Young, using again the TAM framework [41] and information management aspects [37], [15] as well as redesign aspects in the questioning. The final artifact resulting the final discussion can be found in appendix L. The following findings resulted from the interviews and the discussion regarding the aspects of the TAM framework [41] and the information management aspects [37], [15]:

• Experts during the interviews and the discussion indicated that the topic of this paper is very relevant to the practice situation in the Dutch financial sector. As such all interviewees and discussion participants were very interested in the results and artifact of the paper.

• The discussion participants agreed that the design goals proposed in chapter seven and the research methodology derived from this were relevant and adequate to provide this paper with a structured approach. • The discussion participants acknowledged that the literary topics used in this paper were relevant to the field and have been previously used by practitioners in several assignments. The mix of connections between the literary topics however and the focus on information management was rather new. Expert practitioners were very positive about this new approach.

• During the discussion of the beta version of the artifact experts indicated that the three models of the artifact were built logically and structurally and incorporated all the important aspects of the three lines of defense field.

• During the interviews and discussions experts pointed out that they expected the information management performance in the “to-be” situation to go up in comparison to the “as-is” situation. In addition to this the majority of experts believed that following the migration plan in a transition from the “as-is” to the “to-be” situation would maximize the information management performance of the “to-be” situation.

• During the interviews and the final discussion experts were tested positively on the aspects of the

Technology Acceptance Model of Morris et al. [41]. Meaning that both client and internal expert practitioners of Erns & Young perceived the artifact of this paper to be useful, perceived the artifact to be relatively easily usable, were willing to work with the artifact and were also planning to do this in the near future.

• Interviewees and discussion participants were therefore also positive about the future potential of the artifact to be designed in terms of using it as a consultancy product within companies. For this purpose the artifact has to be tailored per company to apply to specific company needs.

10.2.

Limitations and Suggestions for Future Research

During this research several limitations were discovered, which will be discussed in this chapter. The limitations are of the following nature:

• Technical structure limitations within companies. • Organizational structure limitations within companies. • Qualitative approach used in this research.

• The generic nature of the artifact

The first limitation has to do with the technical structure of companies in the financial sector. The nature of our artifact is based heavily on information technology and as such the “to-be” situation and migration plan propose some big changes to the current technical structure of companies. Our paper uses a set of XES standard extensions as minimum requirements for event data capture in the “to-be” situation, but in literature [60], interviews and the discussion experts have already indicated that companies poses a lot of systems that are unable to log event data according to the requirements proposed in this paper. Many of those systems will be legacy systems and will most often be present in critical systems. It was indicated multiple times during the interviews that the cost and

organizational extension of the XES minimum requirements will probably be hard to use. The vast amounts of data stored because of the “to-be” situation also means that companies will have to invest in the right infrastructure for this. More research into enabling event data capture in systems might solve some of these issues.

In addition to this the artifact of this paper proposes the development of event data-driven dashboards for all the three lines of defense, based on the controls used in the line. The focus for this is on the internal audit department because it will provide them with real-time data needed to develop internal audit objectives and perform internal audits. In practice however it might prove very hard to develop those event data-driven dashboard and not all companies might have the employees with the right skills to enable this. More research into developing event data-driven dashboards for internal audit purposes might solve some aspects of these problems.

From an organizational perspective the artifact also proposes some challenging changes. First of all the role of data analysis within the three lines of defense will increase in importance. Experts of clients of Ernst & Young indicated that at the moment not enough employees of their three lines of defense poses the right skills to do data or process analysis activities. Acquiring skilled employees will be hard for companies willing to implement the “to-be” situation. The “to-be” situation also proposes a whole new way of working for employees, which will take some time to take effect. Humans are often more inclined to work via the well-known previous way of working and fear change. Making sure the “to-be” model is followed will be challenging for companies.

Another limitation of this research is that validation and evaluation of the artifact has been done via interviews and a discussion. Preferably we would have implemented the “to-be” situation at a client company of Ernst & Young and monitored the information management performance during one audit year. This was however not possible within the limited time available for this master thesis. Therefore we propose that further research should be conducted in this area to strengthen the research results.

The last limitation resides in the fact that this research is generic of nature. This means that the artifact will not be one- on-one applicable on financial companies in the field without tailoring the artifact to the needs and the structure of a company. Form interviews and the discussion we derived that experts did believe that tailoring the artifact to match company need was not that hard and that the artifact represented a solid baseline, but for some companies tailoring the artifact will still be challenging. For this reason we have included this in the limitation list. Further research into tailoring a data-driven organizational framework to business needs so be conducted to solve this issue.

10.3.

Contributions