La noción de competencias

4.2.1 DF-C²M² Introductory Workshops and Reviews

Participants from each lab attended one of two initial introductory workshops to introduce the DF-C²M² Framework and design goals. Capability Maturity was introduced and how it should be applied to the People, Processes, and Tools domains of an organisation was described. During this introduction, it was noted that many of the participants had not considered Capability Maturity as a requirement within Digital Forensics. Many participants had assumed that ISO compliance and typical KPIs such as volume of data, number of devices, and cases examined were sufficient indicators of efficiency for the laboratory and/or an examiner. This finding reaffirmed the theory that ‘false performance indicators’ are often used as measures of efficiency, and by virtue of that, assumed Capability Maturity, as stated in Chapter 1.

Participants were invited to complete the DF-C²M² survey and list challenges they faced from both a lab management and practitioner perspective. The majority of the challenges stated re-enforced the original DF-C²M² initial assumptions as defined in Chapters 1 and 2.

4.2.2 DF-C²M² Assessment Tool Review Workshop

Participants were introduced to the DF-C²M² Framework, Body of Knowledge, and assessment tool. Participants were able to review the standard operating procedures, workflows, and forms. Standard operating procedures and workflows – specifically those related to technical subjects as defined within the Technical Manual -- created the most interest in both groups of participants, understandably as the majority of the participants were technical, which also reflects the general staffing ratio of technical to non-technical staff within most digital forensic laboratories.

For a more comprehensive evaluation of the DF-C²M², select key personnel from each lab were given copies of the Body of Knowledge to perform a more thorough review and to provide more detailed feedback and evaluation of the DF-C²M².

Overall, the Body of Knowledge was well-received by technical and non-technical participants alike. Key observations include:

85

1. The workflows proved to be of most interest to the student participants who found the Body of Knowledge to be ‘voluminous’, possibly reflecting their interest in methods of grasping new content in the simplest and quickest way available – which the workflows helped them to do.

2. Managerial lab personnel were most interested in the Body of Knowledge components related to Lab Operations, Training, Quality Management, Assessment Tool Reports including Skills Matrices, and Capability Maturity Ratings.

3. The ethos of the Digital Forensics Body of Knowledge supported and maintained by a community of practitioners was generally seen as a bonus aspect of the model.

4.2.3 DF-C²M² Assessment Tool – Process and Objectives

Participants were presented with an overview of the Assessment Tool and the Assessment Process. The key stages of the assessment, which were designed to follow those found in ‘conventional’ ISO audits, were followed, and the method used to validate and determine answers was briefly discussed.

Initially, it was thought that the duration of the assessment would require discovery of evidence and that witnessing of tests would require two days for the assessor to complete; however, it was realised during discussions with participants that perhaps three to four days with the fifth day to present findings was probably more realistic (dependent on the size of the lab).

The assessment tool included ISO 17025 audit requirements structured along the lines of People, Processes, and Tools, and it was noted during this presentation that less-experienced delegates needed help on how to assess or address each question or criterion in the assessment tool.

This oversight in the design of the Assessment Tool, which assumed that those using the tool for in-house assessments would be well-versed with standard ISO 17025 audit procedures on how to validate answers or support findings with evidence, meant that tips on what to look for would need to be added to each question/criterion within the assessment tool to help guide the assessor and to address this rather important requirement, as it would affect the ability of newer labs to conduct self-assessments effectively, and thus affect the perceived benefits to newer and less experienced labs.

86

Based on the feedback from an ISO 17025 digital forensics auditor, it was agreed that existing ISO 17025 and ASCLD/LAB assessors would be able to conduct DF-C²M² assessments using the Assessment Tool, the existing ISO 17025, and their digital forensics knowledge and experience, and that the DF-C²M² would facilitate more detailed internal ISO 17025 audit assessments and preparations.

87

Table 7 identifies key DSRP design stages of the research where practitioners were directly involved:

Table 7: Practitioner Involvement

DSRP DESIGN STAGE Practitioner

Involvement 1 An initial problem definition and identification. -

2 Defining objectives of the research. -

3 Research existing standards, models, and best practices generally used in Digital Forensic Laboratories.

-

4 Research requirements for accreditation under ISO 17025/ASCLD-LAB and perceive the challenges of attaining and maintaining accreditation.

-

5 Designing initial DF-C²M² tools, methods, processes and project plan for assessments and research.

-

6 Conduct survey and interviews with participating digital forensic practitioners, managers, and investigators.



7 Analyse feedback and findings. -

8 Perform Current State Assessment, and SWOT analysis of current offerings, benefits and challenges.



9 Design DF-C²M², revised assessment tools, workflows, knowledge base goals and criteria.

-

10 Conduct workshops and seminar on draft DF-C²M² for review, and solicit feedback and areas for improvement from interviewees.



11 Conduct an audit/assessment of an existing ISO 17025 digital forensic accredited lab against DF-C²M² requirements. Discuss and review findings with participating lab.



12 Plan and implement updates to the accredited lab to bring it in-line with DF-C²M² requirements.



13 Solicit evaluation on the model from participating labs/practitioners



14 Incorporate changes/updates as may be required into DF-C²M².

88

Participatory design was used at several key stages throughout this research as highlighted above. Key areas where practitioners were most influential were in assessing the DF-C²M² Body of Knowledge, Assessment Tool, Skills Assessment/Needs and Service Catalogue.

Practitioner from participating labs were involved in reviewing the DF-C²M² Model and Body of Knowledge during the workshops, and assessments, with more senior practitioners involved in detailed reviews and discussions around the practicalities and ‘nice to have’ aspects that they wished could be included. Certain decisions made with regards to the scope and components within the model were based on the researcher’s experience/assumptions, whilst other were made based on a consensus opinion amongst practitioners from participating labs. In all such instances the justification or rationale behind certain decisions is indicated.

Key areas where participatory design input has significant effect on the research elements:

1. The need to include a means of measuring customer’s satisfaction within a given lab over time.

2. Inclusion of certain value-add services within the Service Catalogue, and revision of Service Descriptions, and limitations.

3. Inclusion of ASCLD-LAB supplemental requirements within the Assessment Tool specifically related to validation of tools/methods, and proficiency testing.

4. Determining Service Catalogue prerequisite services/tools and processes prior to delivery of a service.

5. Determining annual management reporting requirements as per ISO 17025 requirements.

6. Determining challenges as they affected managers, examiners, and junior personnel.

7. General feedback on usability and content of the Body of Knowledge. 8. Feedback on the usefulness, viability and roadmap of the model.

89