…if one is to make a formal measurement, one must accept some responsibility for making some effort to define one’s purpose. As in many other types of activity, this step is too often taken for granted. The result is that someone else who wants to use the measurement may have to struggle to interpret the data gathered by the original investigation. It even happens that the researcher himself forgets and changes his viewpoint, so that he later makes use of data in a way that is difficult to justify.
In Section 3.1, we stressed the importance of identifying key process issues and the things you would like to know to better manage those issues. You must now decide what measures to use to shed light on the issues. As you think about this, you will realize that identifying issues is like the “tip of the iceberg.” Selecting measures requires more than simply stating an issue. For instance, you want to know
• What processes have effects on the issue? • What products are affected by the issue?
• What product entities and attributes should be measured? • What process entities and attributes should be measured? • What resource entities and attributes should be measured? • How will the measurements be used?
• How will the measurements be analyzed? • Who will use the measurement results?
Selecting measures can be a hectic and time-consuming activity if you do not have a clear understanding of the factors that can influence your selection. The following procedure can help you develop an understanding of many of these factors, so that you can choose measures that shed light on the issues. Keep in mind that the goal is to find measures that provide information relevant to the issues you identified in the initial stage of your planning efforts.
1. Clarify the issue. Make sure that you understand all facets and
dimensions of the issue. Put the issue in context by - listing the questions that are being asked
- listing the questions that need to be answered (even if they are not asked)
- identifying who is asking and why (to ensure that you understand the perspectives of those using the measurement results)
- identifying the time elements relative to the issue (Is the issue periodic, transient, or event based?)
- identifying the purpose of the measures relative to the issue (What does the issue suggest in terms of data analysis and action? Will the data be used to understand, compare, predict, control, assess, or improve some aspect of the process?)
2. Identify the processes encompassed by the issue. You have done
some of this already when you identified the critical process issues that concern you. You may now see other processes whose results affect the issue.
3. Review the relationships among the processes to determine which
processes need to be measured. Some processes, although encompassed by the issue, may contribute little or no risk or uncertainty relative to the issue. Those processes can often be ignored. Sketches that make explicit
the mental models for your processes will be of significant help, both here and in the steps that follow.
4. For each process, select the entities and attributes to be measured and the data elements to be collected. That is, select attributes whose
measures will be most influential or dominant in determining product quality or in meeting process objectives. Your knowledge of the process will be a significant factor in selecting entities and attributes that are important. You should try to identify entities and attributes that have direct relationships to process results. If you are interested in process cost, product size, number of defects found, and so forth, measure these attributes directly. Do not rely on indirect relationships to measure process results. For example, do not measure defects to determine complexity, or size to determine cost, unless you have a theory or data to support the validity of a particular cause-effect or predictive relationship.
5. Test the potential usefulness of the selected measures by sketching
indicators, especially charts and graphs, that show how you propose to use the measurements you will obtain. Do not be surprised if this leads you to sharper, more focused questions and definitions.6
When selecting entities, attributes, and data elements for measurement, it is helpful to frame your selections in the context of a model that describes the process of interest. Visual models are especially helpful. With a concrete (explicit) model of the process as your guide, you can solidify your understanding of the process as well as communicate with others when selecting entities to measure. Good models will help you identify the products and by- products that are produced at all stages of a process.
Some examples of the kinds of entities often found in processes are listed in Figure 3-7. You may want to include some of these elements in your mental models and consider measuring one or more of them when seeking to understand what your processes are doing and why they are doing it.
We suggest that you tuck Figure 3-7 away for future use as a checklist. It gives an extended (but by no means complete) set of entities to consider when looking for useful product and process measures. Note that characterizations of process performance will usually be based on attributes of entities found in the four right-most columns. Measures of process compliance, on the other hand, will usually address entities found in the two left-most columns. (Entities in the second column can be of interest in both cases.)
Figure 3-8 is similar to Figure 3-7. It gives examples of attributes of process entities that you may want to consider as candidates for measurement when studying process performance.
6Examples of indicators and their relationships to measurement definitions and mental models, are