The OIG staff established guidelines for implementation of the assessment, including procedures that ensured that the scientists would
document their conclusions on the checklists provided by the OIG. In addition, the scientists were instructed to base their conclusions solely on their analysis of the written materials above. In other words, when gauging vulnerability within the DNAUI, the scientists were to consider the contents of the protocols only, not factoring in any understanding they might have of the FBI’s DNA methods or their observations and conversations with DNAUI staff members and management during the tour of the DNAUI. OIG staff intended the guidelines to enable the scientists to take a fresh look at subjects with which they were obviously familiar, so that they could find weaknesses in the internal controls that others may have missed.
The scientists reviewed the protocols in two sequential phases. OIG staff worked with the scientists to divide the document sections into two groups for review, referred to simply as Phase 1 and Phase 2. Phase 1 covered pre-
analysis protocols; Phase 2 focused on the remaining protocols, including those related to the actual analysis of DNA samples. The checklists used by the scientists reflect the division that was made. See Appendix 6.
The scientists were given a period of time to review the protocol
documents for each phase and then met with OIG staff to discuss and record the vulnerabilities identified. The meetings generated a consensus on the impact and risk ratings that should be assigned to each protocol section. In addition, the OIG recorded the underlying concerns for the agreed-upon ratings assigned by the scientists to ensure that the fieldwork conducted later and the conclusions and recommendations ultimately reported were an accurate
reflection of the specific underlying weaknesses.
At the conclusion of the Phase 2 meeting, OIG staff asked the scientists to assist them in devising fieldwork to identify the actual work practices of DNAUI staff members. In preparation for fieldwork design, OIG staff
summarized the protocol sections in which the scientists had identified key vulnerabilities, and analyzed the results to determine if any of the sections pertained to similar subject matter. In addition, we analyzed the comments voiced by the scientists for recurring themes and categorized them into key concern areas. From this analysis, we designed fieldwork to verify actual laboratory practices for protocols deemed vulnerable, and to assess whether these practices served to mitigate the vulnerabilities identified.
OIG staff conducted this fieldwork from March 12 through 21, 2003. The fieldwork generally was comprised of a tour of the new DNAUI facility in
Quantico, Virginia, and a series of interviews of staff members from within the DNAUI and the Laboratory Division. Where possible, interview responses and
observations made during the tour were checked against supporting documentation for verification.
Fieldwork interviews served as our primary source of insight into the DNAUI’s operations. We recognized that it would be important to collect information from a broad cross-section of personnel, since we intended to analyze their responses for consistency with the protocols, with others of the same operational position, and with respondents in different positions. Therefore we took the following steps to ensure variety in our sources of information.
Since the DNAUI staff function as teams, with each team generally consisting of a Serologist, a PCR Biologist, and an Examiner, we interviewed multiple staff members in each of these positions.
We also recognized that the amount of time that a person had held a position could affect his or her fluency in describing certain processes.
Consequently, we interviewed the most senior and the most junior employees in each position, and judgmentally picked a third person in that same role. For the third Serologist and Biologist interviewees, we selected a staff member who was currently in training for a different team position and thus would have a level of familiarity with the duties performed in both roles.
This interviewing scheme was expanded to include a fourth interviewee from among the Examiners, so that our interviewees would include, in addition to the most senior and most junior Examiners, the Examiners who also
supervised the key programs within the DNAUI: the Examiner-Supervisor of the Serology Program and the Examiner-Supervisor of the PCR (STR) Program.
Finally, we interviewed DNAUI management, including the Unit Chief, the Assistant Unit Chief, and the Quality Assurance Manager; and Laboratory Division management, including the Laboratory Director, the Deputy Director, and the Chief of the Scientific Analysis Section.
We also reviewed documentation and interviewed key personnel regarding: 1) the factors considered in the design of the new DNA facility; 2) the training curriculum and methods used within the DNAUI, along with various staff training records; and 3) the status of development of the
Laboratory Information Management System (LIMS), a computerized tracking system for evidence, samples, and other information. However, we did not include in our fieldwork design an analysis of case file documentation for two reasons:
• Blake’s misconduct persisted undetected due to the DNAUI’s policy that GeneScan® data produced during electrophoresis did not need
to be included in the case file and therefore did not need to be reviewed by the Unit’s Examiners. Consequently, a review of the case files would not shed additional light on Blake’s misconduct, nor would we be able to detect similar misconduct by other staff members from a case file review.
• In April 2002, OIG auditors had reviewed approximately 150 DNAUI case files as part of an audit of the DNAUI and DNAUII’s compliance with standards governing their CODIS participation.81
We reviewed these case files to determine if the DNA profiles from each case, as reflected in NDIS, were complete and accurate.82
Further, we reviewed the case files to determine if the profiles and supporting documentation complied with applicable Forensic Standards and NDIS Requirements.83 This review identified no
deviations from the applicable audit standards. While this work was performed prior to the OIG’s knowledge of Blake’s
misconduct,84 the review did serve as an indicator of the results we
could expect from a case file review.
In addition, we relied upon the work performed by DNAUI management and staff members, as described in Chapter Four, Section III.A, to determine whether other DNAUI Biologists had failed to process the negative controls
81 To select case files for review, we first obtained a list of identification numbers for all
of the profiles that the DNAUI had submitted to NDIS. The list was provided by the FBI
Laboratory’s FSSU, currently referred to as the CODIS Unit, which oversees the NDIS database. From this listing we selected a random sample of 142 profiles from a universe of 1,693 profiles, and requested that the DNAUI make available for our review the supporting case file
documentation.
82 A DNA profile was considered complete if all the analysis results obtained were
reflected in the profile uploaded to NDIS. When the results in the uploaded profile matched those on the Examiner’s worksheets, the profile was considered accurate.
83 We considered the DNAUI case files and resulting profiles compliant with the
Forensic QAS if the required steps in the analysis process were completed and documented, including the quantification of each sample’s DNA, and if both technical and administrative reviews of the analysis work were performed. We concluded that the DNA profiles we reviewed complied with the NDIS requirements if the profile qualified for inclusion in NDIS. The NDIS requirements prohibit a laboratory from uploading profiles to NDIS that clearly match the DNA profile of the victim or another known person, unless the known person is a suspected
perpetrator.
84 We determined during our vulnerability assessment that one of the 142 cases
included in our file review was identified by the FBI as a case on which Blake worked and failed to complete the negative controls. Our review could not have discerned Blake’s misconduct from this case file because it did not include GeneScan® data per DNAUI policy. See discussion
prior to the discovery of Blake’s misconduct. That work determined that the controls were completed as required.
We analyzed the results of our fieldwork and compared them with the concerns voiced and vulnerabilities detected by the scientists to discern whether information gathered during fieldwork confirmed the extent and nature of the scientists’ conclusions. We then conducted a follow-up meeting with the scientists to discuss the fieldwork results and to adjust, if necessary, their earlier conclusions that had been based strictly on the document review. The scientists made only a few minor updates to their earlier observations to reflect the information obtained during fieldwork. Generally, they did not change their conclusions regarding protocols previously identified as vulnerable.
II. DNA UNIT I PROTOCOLS AND PRACTICES IDENTIFIED AS
VULNERABLE TO ABUSE