Población y muestra

Data are only as good as the measurement system that measures them (see Figure 2.18).

Data collection is a process, and as with any process, there will be variations in it. It is important that the measurement process is accurate and that the variation in the

measurement process is minimized and consistent over time. The ability of a data collector to get the same response time after time and the degree of accuracy of a measurement process must be determined and quantified for both discrete and continuous data collection plans (see Figure 2.19). The question under consideration is, “Is the variation (spread) within my measurement system too large to allow me to study the current level of process variation?”

The goal is to minimize the measurement variation so that the actual process variation can be observed.

Figure 2.18 Overview of Analysis Steps

Figure 2.19 Variability

Why Do We Need to Understand Measurement System Variation?

If measurement system variation is excessive, there is an increased risk of:

• Good service being rejected (a cost issue)

• Bad service being accepted (a quality issue)

It is important that we know how much of the measured variation of a process is due to the variation in the actual process and how much is due to variation in the measurement

system.

Elements of Measurement System Analysis

There are two types of measurement system errors: those involving accuracy and those involving precision.

• Accuracy. Do the measurements match the actual value or expert data?

• Precision. This type of error has two subcategories:

Repeatability. Do we get the same results repeatedly when the same person makes the same measurement on the same unit with the same measuring equipment?

Reproducibility. Do we get the same results when two or more people measure the same characteristic on the same unit with the same measuring equipment?

To find the answers to these questions, we must first determine whether the measurement system tests will be conducted on discrete or continuous data. The analysis will depend on the data type.

The following is an example of attribute measurement system analysis.

Compliance with Form Codes

The operations team at a brokerage firm completes the account opening process once all the documents have been received from the branches. There have been complaints from the branches that accounts are not being opened on time, but the operations team is voicing concern that the incoming documents are incomplete (leading to rework). The Lean Six Sigma team needs to improve this process. However, before it can investigate the root cause of the issue, it first has to ensure that the measurement system is adequate. In this case, the operations employees who receive the documents count the number of errors. The

measurement system analysis will help ensure that the error rate that the operations team is calculating is a true reflection of the number of mistakes being made by the branches.

Calculating Accuracy

Accuracy is the difference between the observed measurement and a master, standard, or expected value. For discrete data, it is calculated by counting the number of instances in which the “wrong” answer was observed.

The Lean Six Sigma team created a set of the documents that the operations team would typically receive from the branches. It deliberately made specific errors and omissions on the documents in order to help create a “master standard” for the measurement system analysis.

The Lean Six Sigma team then asked a member of the operations team to review the master standard documents and count the number of errors. Table 2.5 shows the comparison.

Table 2.5 Accuracy

In this case, for two documents, new account and home equity loan application, the error count was incorrect.

Calculating Precision: Repeatability

Repeatability is the variation that occurs when one person repeatedly measures the same unit with the same measuring device. For discrete data, this is calculated by counting the number of times the same result is achieved for a given unit (percent agreement) for each person.

The Lean Six Sigma team asked two operations team members to review the documents twice, in a difference sequence. The number of errors and omissions for each document was

to be recorded each time. The question is, “Can the operations person match his or her results each time?” It is important to note that repeatability reflects consistency, not correctness. Thus, an operations person may have counted the number of errors on a

document incorrectly, but as long as he or she counts them incorrectly every time, while we have an accuracy issue, we don’t have a repeatability issue. Table 2.6 shows the results.

Table 2.6 Repeatability

While there is only one instance in which Adam’s results don’t match, there are three in which Sarah’s don’t.

Repeatability = agreement within operator/total units Adam’s overall repeatability = 9/10 = 90%

Sarah’s overall repeatability = 7/10 = 70%

Calculating Precision: Reproducibility

Reproducibility is the variation that occurs when two or more people measure the same unit with the same measuring device. For discrete data, this is determined by the number of times all persons achieved the same result (percent agreement). In this case, the team asked two people from operations to review the same set of documents and record the number of errors per document. Table 2.7 shows the results.

Table 2.7 Reproducibility

Based on the Lean Six Sigma team’s findings, in three instances, the two team members’

results did not match.

Reproducibility = agreement between operators/total units Reproducibility = 7 / 10 = 70%

Putting the Whole Picture Together

To calculate the adequacy of the measurement system, we need to determine the overall repeatability, reproducibility, and accuracy (R&R&A), otherwise stated, How many times overall did both operations people match themselves, each other, and the master in both trials? Table 2.8 shows the results.

Table 2.8 Measurement Adequacy

In total, there are four instances in which Adam and Sarah don’t match themselves, the master, or each other.

Is 60 percent measurement accuracy good enough? This means that every time someone records an error in a document, there is a 40 percent chance that the report is not correct. The general guideline is that there needs to be a 90 percent match.

Summary of Step 8: Validate the Measurement System

Process data will be the basis for the Lean Six Sigma team’s analysis and its ultimate list of improvement recommendations. While this step is often overlooked, it is critical for the team to ensure that the data that are being collected and used for analysis are accurate and reliable.

This means that the process and the system used to generate the data are adequate—the variability is understood, measured, and within tolerance. As previously stated, the risks of using data from a measurement system that has unacceptable levels of variability, which translates into poor accuracy, are that good services or products are rejected (a cost issue), or that bad services or products are accepted (a quality issue).