Trabajo futuro

After understanding the data quality issues associated with pavement surface data collection, it is next desired to develop a set of quality assurance plans which INDOT can apply in its operations. Quality assurance procedures for IRI data can be viewed as:

• Ensuring that image data collection vehicle has been certified fit for testing and raters are properly trained (Chapter 4).

• Ensuring that data collection vehicles used in network level routine survey produce accurate and precise results on test sections (Quality Control/Quality Assurance)

• Ensuring that data collected from the routine surveys is complete before importing to PMS database (Chapter 6).

• Ensuring that network-level IRI are corrected to project-level IRI when the interest of pavement management applications is at the project level.

These pointers provide the overall framework of the quality assurance program for pavement roughness data. The following subsections shall discuss these in further details.

8.4.1 Quality Control and Certification of Data Collection Vehicles

The first step of the quality assurance program is to ensure that the data collection vehicle used by the vendor to collect network-level IRI data conforms to industry standards and protocols. Chapter 5 described the different standards and protocols to which the testing equipment must conform and provided a set of quality control guidelines which vendors are expected to follow. Besides the data collection vehicle, it is also necessary to ensure that raters in the back-end office are properly trained to identify the distresses and that some form of quality control measures are put in place by the vendor. Chapter 5 also discussed the quality control issues related to back-end pavement distress identification. Since the quality control (QC) tests for testing vehicles is the responsibility of the vendor and inter/intra rater quality control when evaluating pavement distress conditions is predominantly under the control of vendors, proper documentation and certification of the adopted QC tests are recommended. This should be completed before the pre- project phase of the data collection season (see Figure 5.1).

8.4.2 Quality Assurance Tests on Highway Sections

Once the data collection vehicles used for network routine surveys have passed the quality control tests, quality assurance certification must be performed independently by the agency. Typically, this can be performed by subjecting the vendor’s laser profiler to testing on a highway section against a benchmark manual survey as described in earlier sections. The quality assurance tests also ensure that that the level of precision is of the desired standard for the agency and “assures” the surface distress data quality collected by the vendor.

The set of performance measures proposed in earlier sections is used in the quality assurance test for pavement surface distress data. They can be used as a means to check if the data collected by the different contractors, equipment, or procedures are within the benchmarks set by the agency. For example, the illustration shown in Table 8.6 allows the validation of data collected by contractors to a manual benchmark survey on 90 asphalt pavement sections. Automated techniques can be easily tested to see if the data quality is within desirable confidence limits or thresholds (in terms of Δ PCR, θ -value, and _κ -values for individual distresses), as demonstrated in the earlier section.

Moreover, these performance measures can aid the agency in determining quality control and assurance guidelines. An example would be the case of an agency desiring to determine an optimal data processing sampling rate for automated pavement data collection. The goal could be to achieve an overall PCR variation of ± 5 points from the “truth,” a mean _θ of most 20%, and a kappa statistic of at least 0.6 (i.e., substantial agreement) for cracks. From Tables 8.6 to 8.9, it was found that the minimum data processing sampling rate is 50%. Conversely, the performance measures should allow agencies to evaluate their data quality based on existing sampling procedures. For the given illustration, if the current data processing sampling rate is 20%, we can be certain that the PCR is within a ± 5 points range of the “true” PCR, the mean cumulative differences in PCR over entire range _θ is approximately 30%, and the kappa statistic is at least 0.4 (moderate agreement) for cracks. Thus, the combined use of these three performance measures would allow INDOT to further develop and refine quality assurance guidelines related to pavement surface distress data collection.

8.4.3 Completeness of Pavement Surface Distress Data

Before importing the collected PCR and individual surface distress data into the database, the following logic and completeness checks must be performed to ensure that there are no missing data.

• Codd’s Integrity Constraints • Free-of-Error Checks • Completeness Checks • Consistency Checks

Ratings were developed to evaluate these criteria in Chapter 6, which can be applied to the PCR and surface distress data to evaluate data quality from the information management perspective.

8.4.4 Treatment for Project-Level PMS Applications

Once the PCR and surface distress data are assured to be accurate, precise, and complete, they are entered into the PMS database for application. For network level pavement management decision-making, it is sufficient to use the vendor’s quality assurance data without major problems (i.e. for guidance purposes). However, for project level applications, we established that there can be a problem in PCR accuracy due to the mean-variance dependency effect of the PCR and the effect of the sampling rate choice. In fact, it is better to perform project level PCR and surface distress identification. Alternatively, a visual benchmark survey could be performed to calibrate network level PCRs and develop relationships such as Equation (8.10) to correct the network-level PCR data to project-level PCR data. Confidence bands for the project-level PCR can also be further developed to allow probabilistic-based decision-making at the project level.

8.5 Chapter Findings

This chapter presented an evaluation of PCR and surface distress data quality (in terms of accuracy and precision) at the project (manual benchmark survey) and network levels. Techniques to determine the PCR and individual surface distress data quality were discussed. A set of performance measures capable of evaluating the overall pavement condition data quality as well as the quality of individual distress data were proposed. The hypothesis testing of Δ PCR

can provide a quick and efficient way to evaluate data quality, but suffers from the effects of mean-variance dependency. To overcome the mean-variance dependency effect, the cumulative difference in PCR over the entire range _θ was proposed. This parameter complements the traditional approach of using Δ PCR by accounting for the data quality over the entire range of PCR. The Cohen’s kappa statistic allows evaluating the data quality of individual distresses. This set of performance measures were found to be useful in comparing the quality of data obtained from automated data collection method against manual benchmark surveys and in assessing the effect of sampling on data quality. Using the developed performance measures, it was found that there is an error of ± 20% between the network- and project-level PCRs using INDOT testing procedures. It was also noted that certain early age distresses could not be detected by the automated method at network level. Quality assurance procedures were then developed for INDOT to better manage their PCR and surface distress data collection practices and applications. Quality assurance tests on warranty pavement sections or selected pavement sections are possible alternatives to aid the agency in better managing their PCR data quality. It was recommended that network level PCR should only be used for guidance purposes and cannot be taken to be the same as project level PCR. A visual survey is required to accurately determine project level PCR or to calibrate network level PCR as part of the quality control and assurance process.

CHAPTER 9: CONCLUSIONS AND RECOMMENDATIONS FOR FUTURE