delay for parts
STORES EFFECTIVENESS Inventory turns, stockouts,
delay for parts
WORK PROCESS EFFICIENCY
Planned to Total Work, Overtime as a Percentage of Total Hours, etc.
WORK PROCESS EFFICIENCY Planned to Total Work, Overtime as a Percentage of Total Hours, etc.
PROGRAM EFFECTIVENESS
Faults Detected Prior to Failure, Avoided Cost
PROGRAM EFFECTIVENESS Faults Detected Prior to Failure,
Avoided Cost RELIABILITY MANAGEMENT
MTBF, MTBR, MTTR
RELIABILITY MANAGEMENT MTBF, MTBR, MTTR
INDUSTRY PERFORMANCE / EFFECTIVENESS
Cost as a Percentage of RAV, EDC, EFOR, Manufacturing Cost/Unit
INDUSTRY PERFORMANCE / EFFECTIVENESS Cost as a Percentage of RAV, EDC,
EFOR, Manufacturing Cost/Unit
COST EFFECTIVENESS
Labor effectiveness, cost / unit, RAV
COST EFFECTIVENESS Labor effectiveness, cost / unit, RAV
CORPORATE
RONA, ROCE, TRR
CORPORATE RONA, ROCE, TRR
OPERATING EFFECTIVENESS
OEE, Asset Utilization, RTY, COPQ
OPERATING EFFECTIVENESS
OEE, Asset Utilization, RTY, COPQ CAPITAL EFFECTIVENESSCAPITAL EFFECTIVENESSSpare parts inventory % RAVSpare parts inventory % RAV
STORES EFFECTIVENESS
Inventory turns, stockouts, delay for parts
STORES EFFECTIVENESS Inventory turns, stockouts,
delay for parts
Figure 9.2 Metrics Hierarchy
Some separate metrics in three classes. Quantitative metrics described in this chapter; milestone metrics that define achievements by dates keeping the timing, tempo and accountabilities in place and qualitative metrics. The latter include objectives that can’t be quantified but describe the “nature or quality” of an improvement. Reference tables are advisable to add structure and orientation to qualitative improvements. An example might be the performance of an asset optimization steering team, Chapters XVII and XVIII. Working these three classes of metrics simultaneously provides balance and real accountability — e.g., measuring the performance and staying power of management driving the improvement program. Grahame Fogel
Definition and Comparison
A valid comparison of metrics within and between corporations requires precise, strictly defined terms. Beginning in the chemicals industry in the early 1990’s several metrics used to measure effectiveness have been expressed as a percentage of Replacement Asset Value (RAV) or Estimated Replacement Value (ERV). Maintenance cost as a percentage of RAV / ERV is one example. RAV / ERV was originally proposed as a broadly applicable, neutral measure that could be used as a normalizing factor across plants and processes. By the early 2000’s cost as a percentage of RAV / ERV had gained visibility at the senior management level and was being applied across a broad segment of industry as the top-level benchmark for maintenance spending.
RAV / ERV is defined as the current expenditure that would be required to replace production capacity / output. Note that RAV / ERV is not the same as depreciated value.
With increasing emphasis on cost as a percentage of RAV, many at the working level have begun to question whether a single Reliability and Maintenance (R&M) spending benchmark value can / should apply across industry or is the most accurate benchmark value industry, process and perhaps even site specific? What are R&M expenditures for optimum results? Can a single value of cost / RAV accurately identify expenditures that are too high in every case? What about too low for sustainability? Should maintenance cost per RAV be the final arbiter? If not, is there a method to determine an optimum level of R&M spending for best sustainable production effectiveness?
Companies with a great deal of experience with the metric will state that RAV / ERV calculations are often inconsistent within the same company and are probably misleading as comparative metrics across companies and industries.
Furthermore, optimum maintenance costs clearly vary by process; a water purification plant certainly should have significantly lower maintenance costs as a percentage of RAV than a facility processing sulfur or hydrochloric acid. This is seen in survey numbers where maintenance costs as a percentage of RAV that are routinely achieved by one facility are clearly out of the question for another.
Plant age is another factor. Although some claim otherwise, an older plant typically has higher maintenance costs compared to a new plant as soon as the initial deficiencies have been corrected. Finally, geographic location will have an effect. Parts costs are typically higher in the Middle East and Southeast Asia locations, labor costs are much lower compared to North America.
Many corporations / facilities use RAV / ERV as the denominator for key performance / spending metrics. Recognizing there are inconsistencies in calculation between companies, many limit RAV / ERV based metrics to initial objectives and measuring progress. Once an initial objective is established, the site must only assure that RAV / ERV is calculated consistently for the site throughout the improvement process.
The conclusion: RAV / ERV metrics can be used for guidance as relative metrics to measure improvements, however, they are not absolutes that can be applied with confidence across industries and processes.
For all these reasons any benchmark derived from an average claiming to represent best practice across a range of industries, locations and processes must be viewed with a great deal of skepticism.
MTBF is another example. There are many definitions of “failure” that significantly affect the value of MTBF calculated by different companies.
One company reported they spent days attempting to define a failure without reaching agreement. Another substituted a strictly defined “event” for failure. With some specified exceptions, anything that requires “lock out” is considered an “event”. They calculate Mean Time Between Events (MTBE) as their primary measure of reliability.
Definitions should conform to industry conventions. They must include the metric itself, all of its components, and the method of calculation. Those who will use the metric and are accountable for results must understand and agree on the definition. Definitions should be printed and readily available to ensure consistency of application. Appendix A. lists the definitions utilized in this Handbook.
Comparing metrics also requires consistency of mission, overall intensity, location / environment and even age.(62) Consider a new race car and an older passenger car. Even though both have an internal combustion engine, four wheels and operate on a road, major differences in mission, operating intensity and age significantly impact the ability to compare performance with measures such as fuel consumption and operating cost. This is the same caveat regarding metrics based on RAV / ERV discussed in greater detail earlier.
As this is written in mid 2006, The Society of Maintenance and Reliability Professionals (SMRP) is establishing a consensus definition of frequently used terms.
In addition to sharing a common definition of terms and metrics, facilities must assure metrics are measured and calculated in accordance with common industry practice. Metrics should be measured under consistent conditions within the process. For example, a determination of the average drive time from home to work may or may not include the time required one day a week to fill-up with gasoline. If it did, you might find yourself five minutes early on the days you didn’t purchase gasoline and ten minutes late on the day you did.
This simple example highlights an important issue that is all too often overlooked. Metrics must represent the process. If the process changes, the performance metrics must also change. In the simplified example, the metric could be made more descriptive of the process; average driving time to work without accident delays, stops for gasoline, groceries or other errands. From this basis, one could calculate the average time required to fill up with gasoline, get groceries, etc. and estimate how many times per week errands / delays occurred to arrive at an accurate metric for both the normal process and the exceptions. There also must be consensus on the exact starting and stopping point in the process. Most frequent airline travelers have experienced this issue. Airline on-time departures are measured on the basis of pushback from the gate compared to scheduled departure. Planes are often pushed back ten feet where they remain for an hour or more due to traffic, weather or other delays. However, by the airline’s rules, the flight made an on-schedule departure!
In the industrial world, metrics must be seen as a positive force driving improvements that are beneficial to all and not to attach blame. This is an essential step in tracking progress toward the necessary
improvements. Otherwise metrics will be manipulated like aircraft pushback’s to ensure that performance, representative or not, is as high as possible.
BENCHMARKING
Enlightened managers understand that optimizing asset performance and utilization is a key component of business success and a contributor to — rather than a detractor from — production effectiveness and profitability. In the long run, top-down, peer and working-level support are all essential for success. Benchmarking reveals the comparative performance required to attain organizational buy-in.
Benchmarking — a systematic process for measuring “best practice” (to be) and comparing the results to current plant performance (as is) in order to establish performance objectives and identify opportunities for improvement.(35)
Benchmarking enables companies to accurately identify world-class performance from unbiased, published values and assess their current performance relative to the “best”.(123)
Almost every aspect of business can be improved. Industry leaders understand that while they have pockets of excellence they also have gaps where improvements are necessary. No single facility is excellent in every area.(22) Leaders also recognize that understanding the tools, techniques, and results from other companies is a vital part of the improvement process. Critical self-examination must occur on an ongoing basis to recognize opportunities for improvement and take full advantage of changing state-of- the-art techniques.
Benchmarking is a means to gain insight into competitive realities and define the objectives and measures of performance needed to initiate positive change and to manage for greatest effectiveness.(6)
Numerical reference standards — benchmarks — that define best performance in key aspects of the Asset Optimization process must be established in order to recognize the need and opportunities for improvement. A realistic benchmarking effort is the first step toward understanding real conditions and recognizing opportunities for improving the effectiveness of an Asset Optimization program. Thus, benchmarks establish comparative standards from which performance objectives are set.
Benchmarking must be done continually. Results of benchmarking must be incorporated in objectives for day-to-day operations so that organizations can begin to think differently about how they work and solve problems.”(123)
By its nature the benchmarking process has many advantages including:
Promotes awareness and adherence to performance metrics, establishes objectives
accountability for results
Forms the basis for data driven decisions, minimizes bureaucratic objections
Provides the basis for continuous improvement as expectations are defined, met and repeatedly elevated
The continuous improvement process becomes ingrained in the organization’s standard operating
procedures and eventually the culture Categories
Benchmarks are typically in four categories: industry, process, functional and program.(35) Benchmarks are obtained from surveys, exchanges with similar organizations or they may represent organizational goals and objectives. Fortunately, industry-best benchmarks for nearly every asset and work effectiveness area are published and readily available. Figures 9.10 and 9.11 contain a convenient tabulation of commonly used benchmarks distilled from multiple sources.
External industry-wide and process benchmarks are valuable to assess performance. Each industry has generally accepted overall “world class” benchmarks that are useful for determining comparative performance, e.g., cost and value of spares inventory as a percentage of RAV / ERV or EDC, tons of steel production per availability hour and assembly hours per automobile.(108) From there, internal corporate or facility metrics can be established for each element within the overall process to prioritize and drive the improvement program.
Regardless of the source, care must be taken to ensure that objective values set from benchmarks are realistic and attainable. Objectives that are unrealistically high will not gain the commitment necessary for
success. Leaders report that allowing working-level implementation teams to establish performance objectives typically leads to ambitious goals, strong ownership and a high level of commitment to their attainment.
Benchmarking Process
The general benchmarking process, illustrated in Figure 9.3, has six essential steps:(9, 92)
1. Establish a comprehensive set of performance parameters that indicate contribution and compliance to principal business objectives.
2. Completely define each parameter; assure consistency with industry definitions. Establish “best practice” benchmark objectives; Figures 9.10 and 9.11 both contain typical measures and published benchmark values. Establish methodology for collecting own data.
3. Collect, measure and calculate current performance for each benchmark category from site information
– Validate accuracy of the information – Establish current (as is) performance
4. Compare current performance measurements with “best practice” benchmarks; establish Gaps to best performance.
5. Assess and analyze results.
6. Identify areas of greatest opportunity for improvement based on potential value created by closing gaps
Select key performance / effectiveness parameters to be benchmarked Select key performance / effectiveness
parameters to be benchmarked
Establish definitions and “best practice” values for each parameter Establish definitions and “best practice”
values for each parameter
Collect facility / site data for each parameter, establish current performance Collect facility / site data for each parameter,
establish current performance
Compare current performance to best practice — Establish Gaps
Compare current performance to best practice — Establish Gaps
Assess and analyze results Assess and analyze results
Identify areas for improvement Identify areas for improvement Select key performance / effectiveness
parameters to be benchmarked Select key performance / effectiveness
parameters to be benchmarked
Establish definitions and “best practice” values for each parameter Establish definitions and “best practice”
values for each parameter
Collect facility / site data for each parameter, establish current performance Collect facility / site data for each parameter,
establish current performance
Compare current performance to best practice — Establish Gaps
Compare current performance to best practice — Establish Gaps
Assess and analyze results Assess and analyze results
Identify areas for improvement Identify areas for improvement
Figure 9.3 General Benchmarking Process(10)
One company elected to benchmark internal quality, performance and effectiveness in six areas:(93) 1. Leadership
2. Planning and scheduling
3. Preventive and Condition-Based Maintenance 4. Reliability improvement
5. Spare parts management
6. Contract maintenance management
Results of this process are illustrated in Figure 9.13. Industry best performance is represented by the outer circle, site best by the double line, team performance by the solid single line. Gaps between industry and site best are measured along a radial for each area.
Power generating companies have traditionally benchmarked in areas, such as:(6)
Reliability, Availability, Maintenance (RAM) non-fuel operating and maintenance costs Fuel costs and heat rates
Capital effectiveness
Successful benchmarking begins with consistent definitions. Activity Based Accounting, Chapter VII and a chart of accounts delineating equipment, and preferably component level, costs are essential. Accurate records of availability, downtime and cause must be available or constructed from logs. Data requirements should be matched to data available for comparison. There is often a tendency to want too much data.(6)
Benchmarking methodology must be: (6)
Plant, process and / or equipment specific
Cognizant of and account for data availability, reliability and accuracy Easily understood
Repeatable
Capable of demonstrating cause, effect and results for a resulting improvement strategy Useful for follow-on monitoring
Figure 9.4 illustrates a corporate level dashboard with drilldown capability to individual plant, system, program, equipment and component metrics. With this information deviations from objective performance can be quickly spotted for correction.
KPI’s Plant/Business Unit Pareto to identify distribution Effectiveness by Production Unit/Line Equipment MTBF affects availability and cost Corporate RONA/ROCE RONA/ROCE RONA/ROCE Cost Cost Prod. Effect. Prod. Effect. Availability
Availability YieldYield QualityQuality
MTBF MTBF Cost/Unit Cost/Unit Cost/RAV Cost/RAV Work Metrics RONA Orders Backlog Share Price Safety
Index Environmental Index
Efficiency Efficiency Program Work Effectiveness Work Effectiveness Reliability Reliability KPI’s Plant/Business Unit Pareto to identify distribution Pareto to identify distribution Effectiveness by Production Unit/Line Equipment MTBF affects availability and cost Corporate RONA/ROCE RONA/ROCE RONA/ROCE Cost Cost Prod. Effect. Prod. Effect. Availability
Availability YieldYield QualityQuality
MTBF MTBF Cost/Unit Cost/Unit Cost/RAV Cost/RAV Work Metrics RONA Orders Backlog Share Price Safety
Index Environmental Index
Efficiency Efficiency Program Work Effectiveness Work Effectiveness Reliability Reliability
Figure 9.4 Corporate Dashboard with Drilldown to Performance and Effectiveness Metrics Identifying and Prioritizing Improvements
The comparison to “best practice,” often called a Gap analysis (see Chapter XVIII), leads to an array of potential improvements directed to achieving “best practice” levels of effectiveness.(55) Within the Asset Optimization program, gap identified potential improvements are risk adjusted and prioritized in order of potential increased value.
A more detailed diagram of the benchmarking process and how benchmarking fits into an overall improvement process is illustrated in the following figure.(35)
Form Benchmarking Team Form Benchmarking Team
Set Benchmarks Set Benchmarks Measure Current Performance Measure Current Performance
Compare Against Benchmarks Compare Against Benchmarks
Analyze Variances Analyze Variances Identify Improvements Identify Improvements Monitor Results Monitor Results Compare Against Benchmarks Compare Against Benchmarks
Implement Full Improvements Implement Full Improvements Corporate Objectives Performance Gap? Performance Gap? Improvement OK? Improvement OK? Recalibrate Benchmarks? Recalibrate Benchmarks? Recalibrate Benchmarks Recalibrate Benchmarks Implement Trial Improvements
Implement Trial Improvements
Enhance Trial Improvements Enhance Trial Improvements No
No Yes
Yes
Yes Identify Key Performance
Indicators
Implement Continuous Improvement ProgramNo Form Benchmarking Team
Form Benchmarking Team Set Benchmarks Set Benchmarks Measure Current Performance Measure Current Performance
Compare Against Benchmarks Compare Against Benchmarks
Analyze Variances Analyze Variances Identify Improvements Identify Improvements Monitor Results Monitor Results Compare Against Benchmarks Compare Against Benchmarks
Implement Full Improvements Implement Full Improvements Corporate Objectives Performance Gap? Performance Gap? Improvement OK? Improvement OK? Recalibrate Benchmarks? Recalibrate Benchmarks? Recalibrate Benchmarks Recalibrate Benchmarks Implement Trial Improvements
Implement Trial Improvements
Enhance Trial Improvements Enhance Trial Improvements No
No Yes
Yes
Yes Identify Key Performance
Indicators
Implement Continuous Improvement ProgramNo
Figure 9.5 Detailed Benchmarking and Improvement Process(35)
In a multi-business enterprise there may be broad variations in performance resulting from factors such as design, type and intensity of processing. Industry leaders recognize that efforts should not necessarily be focused on the largest apparent Gaps. Value potential, difficulty, cost and time required to close a Gap all are considered during a risk assessment. Industry leaders utilize the risk assessment process to adjust opportunities for improvement and evaluate initiatives to determine which have the highest probable returns.(129) Typically there are far greater opportunities to increase profits than resources, and this process assures that resources are allocated to gain greatest risk balanced improvement.(129)
One facility recognized the need for and potential value of permanent improvements to a spared pair of pumps that continuously cavitated causing accelerated seal and bearing failures as well as impeller and casing erosion. The pumps sounded terrible in operation, failed frequently and were a nagging concern to Operations. Operations had written numerous Work Orders to maintenance as well as requests for corrective action to Engineering. A detailed study disclosed that the solution required digging a pit to lower the pumps five feet with safety concerns or raising the tank from which the pumps were supplied by about the same amount. Neither solution was practical or cost effective. They simply had to suffer the inadequate design.
Gaps to benchmark performance must be followed by a comprehensive improvement plan. The plan must