errors addressed, consideration of decision-making) and in terms of the general approach they use for quantifying human failure events (HFEs) for a PRA, among other aspects. The general focus of the methods reflect, at least to some extent, the evolving knowledge-base of HRA (e.g., the need to better address the decision-making process), while the different approaches to
quantification reflect more pragmatic concerns (e.g., ease of use), and tend to be represented all along the temporal continuum. Table 2 provides a list of the different methods that have been developed over the years (may not be a complete list) and discusses their general focus and their general approach to quantification. More detailed discussions of ten of these methods that have frequently been used in the U.S. can be found in NUREG-1842 [47].
As mentioned above and noted in Table 2, following the earliest work on the simplest of errors (i.e., slips and lapses), considerable expansion occurred to create methods that would address either mistakes like those performed at TMI, or generally all types of errors, within the method framework. Another issue was the extent to which earlier methods tended to consider a safety related human action to be similar to an item of equipment that either succeeds or fails in its intended function. That is, while a few factors may be taken into account by the methods in estimating the likelihood of failure to take the action, the analysis focuses on just the success or failure of actions typically defined by systems analysts as important to safety.
Many analysts, particularly those trained in the behavioral sciences have criticized this approach as gravely over-simplistic (e.g., Woods et al. [16]; Dougherty [48]; Hollnagel [5]).
Even NRC’s review of HRA in NUREG-1050 [49] identified the limitations in the earlier
methods as an important weakness in PRA. Many of the more recent HRA models recognize that people behave in very complex ways and are capable of creating new conditions (not simply failing to accomplish system-demanded tasks) or are subject to influences in more complex ways than those implied by a few simple performance shaping factors. The development of some of the most recent methods incorporates explicitly some kind of a model of human cognitive behavior that takes account of the knowledge of the behavioral sciences, to provide a much richer
description of the human-system interactions. By taking account more realistically of the cognitive processes of the operators, it is possible to be much more explicit about the kinds of conditions, or contexts, that are necessary to lead to high likelihood of failure, or may induce unsafe actions by operators. This came about because when analysts more closely examined
real-world event data especially in light of the advances in behavioral science and theories about how humans function, it was found that the more serious accidents involved:
the plant operating outside normal or expected conditions,
the resulting physical regime not being well understood by the operators,
operators refusing to believe or otherwise recognize evidence contrary to their belief as to what was happening in the plant, and
prepared plans were not always helpful or even applicable.
Hence, it was recognized that HRA methods needed to holistically account for both plant conditions and a widening range of operator influences (i.e., performance shaping factors) in order to be able to address the characteristics of the more serious accidents. While not perfect, some of the more recent methods provide considerable guidance on understanding, as much as possible, about the whole context of a situation faced by the operators in order to estimate the likelihood of operator unsafe acts. In some cases, this expanded understanding also includes more explicit treatment of errors of commission (EOCs). Even so, advancements are still needed to be able to address the potential impact of management and organizational influences, and the role of crew characteristics and team dynamics on crew performance. Additionally, more analysis of operational and simulator experience is needed in order to add credibility to the methods and particularly the likelihoods of operator errors as estimated using these methods.
With respect to the basis for quantification, there are generally three different bases (see Table 2 for examples of each):
the method provides a numerical basis, such as HEP values that are tabulated or expressed in a time/reliability relationship (this may, in turn, be based on actual experience, simulator
observations, judgment, etc.)
the method provides ways to elicit or manipulate expert judgment
the method provides ways to obtain data from plant-specific data sources (such as simulators).
While in some cases, the method may use somewhat of a mixture of the bases shown, the predominant mechanism by which human error probabilities are quantified is shown in Table 2.
It is noticeable that most of the more recent HRA methods rely on some form of expert judgment, whereas the majority of the earlier methods rely on tabulated or time reliability based failure estimates. One reason for this difference is that the effects of contexts considered in more recent methods like CREAM [5], ATHEANA[3] and MERMOS[50] is much more complex and not readily reducible to simple tables or correlations using a few performance shaping factors.
Hence, while the use of simple tables, correlations, and related multiplicative factors make the earlier methods somewhat easier to use, the simplicity of the methods that allowed the use of such approaches was in fact one of the primary criticisms of them that led to evolution of HRA models.
35 4.2.3 Summary of Evolution of HRA
The evolution of HRA technology and the methods for evaluating human performance and estimating human error probabilities associated with nuclear power plant applications has
occurred consistently over the past thirty years. In large part, this evolution has been in response to our needs to understand the drivers of human performance in increasing detail, as well as in response to changes in the industry and due to the ability to incorporate our knowledge from the behavioral sciences. Simple modeling and quantitative techniques were and continue to be useful for simpler types of human errors (and, hence, are still used today). However, as we improve the man-machine interfaces in our nuclear plants, it has become increasingly important to understand and model the more cognitive aspects of human performance within the context of situations that operators may experience during abnormal events and in accidents. This has required more complex and sophisticated modeling as well as more reliance on expert elicitation quantitative techniques. These advances reflect our improving ability to understand and predict human behavior in challenging situations. Nevertheless, not all known factors are yet routinely and completely addressed in the current HRA methods (e.g., organizational influences). Also, while analysts generally believe the quantitative estimates are reasonable, HRA is still struggling to obtain and use sufficient real world experience to “gauge” the accuracy of our HEPs. Since serious challenges to operator performance tend to be rare (which is fortunate), such data is slow in coming and it will take time to be able to validate our quantitative estimation techniques.
5. CAN WE PREDICT UNSAFE HUMAN ACTIONS AND THEIR LIKELIHOODS?
The short answer is yes, we can, but some discussion is needed. Our current human reliability modeling techniques have become far more sophisticated and, generally, can account for many more influences on human performance than was available with the earliest methods.
Taking into account the advances in the behavioral sciences in our current models, we believe it is possible to identify those situations that tend to make human error more likely. This allows us to define potential vulnerabilities and make improvements in plant design and operational practices, as well as in procedures and operator training that collectively, can lessen the chances of unsafe human actions.
There are many HRA methods giving human reliability analysts an arsenal of tools for identifying conditions prone to operators making unsafe acts. Some methods and their tools treat human performance relatively simply and account for only a few influencing factors. Such treatment may be adequate for situations that are not complex and when the most likely influencing factors are within the capabilities of the method. Other methods involve more complex modeling of human performance, and are most useful and probably necessary for conditions requiring consideration of many influencing factors. Although the USNRC has recently provided some guidance on the appropriate use of various types of HRA methods [47],
knowing when to use as well as how to use a method is part of the “art” of HRA and requires a sufficiently trained analyst to make the appropriate judgments required by any of the methods.
The methods also have their associated means for quantifying the likelihood of operators performing unsafe actions. These range from the simple use of data tables (that are based on experience and judgment) to more complex expert elicitation processes (that, preferably, use personnel knowledgeable in the tasks being examined). While there is the belief that these
quantification techniques generally provide reasonable probabilities if applied correctly and to the right situations, all of HRA still suffers from having too little relevant experience to “calibrate” or otherwise validate these quantification techniques. Efforts continue to make such data (and analysis of the data) available along with the need to improve our modeling of human
performance so as to handle yet additional performance shaping factors such as organizational influences.
Thus, the state-of-the-art in HRA is such that we believe we can identify conditions that tend to make errors more likely and estimate “reasonable” probabilities for the errors. This should not be confused, however, with being able to predict the next critical human error. Just as we know that the probability of getting a “head” when flipping a coin is 0.5, that does not mean we can predict whether or not the next flip of a coin will produce a “head”, or even the flip after that or after that. We can say that given a sufficient number of flips, you will see a “head” 50% of the time. In a more complex example, we cannot predict the specific paths of neutrons during nuclear fission and whether specific neutrons will cause other fissions of uranium nuclei. But, we know “on average” what will happen and this is sufficient for us to be able to design and build operating reactors.
It is the same with our human reliability predictions. For a case where we estimate a high probability of failure by the operators, that probability is a reflection of various influencing factors that given the situation, we believe tend to make human error likely. In such cases, the most negative influences can be defined and improvements made to lessen the chance of an unsafe action. Conversely, a low probability is a reflection of all the positive influences that exist for the situation that should make a human error unlikely. Such results are useful, even if we cannot predict that given a particular circumstance and the related influences, that an error will or will not occur.
Clearly, further advances in the field of HRA are needed. It is not clear that the behavioral sciences will be able to produce an adequate integrated model of human performance to support direct quantification of HFEs. Therefore, the systematic collection of a “database” (or information source) of operational events and simulator experience to support HRA quantification would seem to be a very high priority. Such a database should be made up of national and international data, collected on actual events across the different industries, and from investigations using simulators. Such data will continue to strengthen our ability to understand the characteristics of situations that can lead to unsafe human actions and provide an additional basis for estimating the likelihood of those unsafe actions. The USNRC is currently supporting several national and
37
international efforts along these lines, including work at the Halden Research Project in Norway using state-of-the-art nuclear power plant simulators, the international Organization for Economic Cooperation and Development Nuclear Energy Agency (OECD/NEA) efforts to develop an international database of nuclear power plants events, and work by Idaho National Laboratory to build a structured database for collecting information associated with unsafe human actions that could be used to support quantification (e, g., Hallbert et al. [67]).
In the meantime, HRA provides us with useful insights and allows us to make meaningful improvements to lessen the likelihood that unsafe actions will occur.
6. REFERENCES
[1] Gitus, J. H. The Chernobyl accident and its consequences. London, United KingdomAtomic Energy Authority, 1988.
[2] Air Florida, Inc., Boeing 737-222, N62AF, Collision with 14th Street Bridge, near Washington Nat'l Airport, Washington, DC, January 13, 1982. National Transportation Safety Board Report Number: AAR-82-08, Washington DC, USA, 1982.
[3] Technical basis and implementation guidelines for A Technique for Human Event Analysis (ATHEANA), NUREG-1624, Rev. 1, US Nuclear Regulatory Commission, Washington, D.C., May 2000.
[4] Reason, J. Human error. Cambridge, Cambridge University Press, 1990.
[5] Hollnagel, E. Cognitive reliability and error analysis method (CREAM). York: Elsevier Science, New York, 1998.
[6] An analysis of root cause failures in 1983 significant event reports. INPO 84-027, Atlanta, GA: Institute of Nuclear Power Operations, 1984.
[7] A maintenance analysis of 1983 significant events. Atlanta, GA: Institute of Nuclear Power Operations, 1985.
[8] Gertman, D.I. and Blackman, H.S., Human reliability and safety analysis data handbook, John Wiley & Sons, 1994.
[9] Reason, J. Managing the risks of organizational accidents. Ashgate Publishing Company, Brookfield, Vermont, 1997.
[10] Kletz, T.A. What went wrong? Case histories of process plant disasters, Gulf Publishing Co., London,1985.
[11] Dekker, S. The field guide to human error investigations, 2002 Ashgate Publishing Co., 2002.
[12] Harré, R., and Lamb, R. (Eds.). The Encyclopedic Dictionary of Psychology. Cambridge, MA: MIT Press; 1983.
[13] Swain, A.D., and Guttmann, H.E. Handbook of human reliability analysis with emphasis on nuclear power plant applications - final report, NUREG/CR-1278, SAND80-0200, Sandia National Laboratories, August 1983.
39
[14] Rasmussen, J., and Rouse, W.B. Human Detection and Diagnosis of System Failures, New York: Plenum Press, 1981.
[15] Hall, R.E., Fragola, J. and Wreathall, J. Post event human decision errors: Operator action tree/time reliability correlation, NUREG/CR-3010, U.S. Nuclear Regulatory Commission, Washington, 1982.
[16] Woods, D.D., Roth, E.M., and Hanes, L.F. Models of Cognitive Behavior in Nuclear Power Plant Personnel, Westinghouse Science & Technology Center, Pittsburgh, PA NUREG/CR-4532, July 1986.
[17] Dougherty, E.M., and Fragola, J.R. Human reliability analysis. A systems engineering approach with nuclear power plant applications, New York, John Wiley and Sons, 1988.
[18] Reason, J. The Review of Mistakes: A Brief View of Planning Failures. In: Rasmussen J, Duncan K, Leplat J, eds. New Technology and Human Error. New York: John Wiley &
Sons; 1987.
[19] Minsky, M. A framework for representing knowledge, In P. Winston (Ed.), The Psychology of Computer Vision. New York:McGraw-Hill, 1975.
[20] Rumelhart, D.E. Notes on a schema for stories. In D. Bobrow and A. Collins (Eds.) Representation and Understanding: Studies in Cognitive Science. New York: Academic Press, 1975.
[21] Tversky, A., and Kahneman, D. Judgment under uncertainty: Heuristics and biases.
Science, Vol. 185, pp 1124-1131, 1974.
[21] Woods, D.D., Johannesen, L.J, Cook, R.I., and Sarter, N.B. Behind Human Error:
Cognitive Systems, Computers, and Hindsight. Wright-Patterson Air Force Base, OH:
Crew System Ergonomics Information Analysis Center; 1994.
[23] Hollnagel, E. Human Reliability Analysis: Context and Control. San Diego, CA:
Academic Press, Inc., 1993.
[24] Hollnagel, E. Reliability of Cognition: Foundations of Human Reliability Analysis. New York: Basic Books, 1994.
[25] Roth, E.M., Mumaw, R.J., and Lewis, P.M. An Empirical Investigation of Operator Performance in Cognitively Demanding Simulated Emergencies. Report No.
NUREG/CR-6208. Pittsburgh, PA: Westinghouse Science & Technology Center; July 1994.
[26] Sträter, O. Evaluation of human reliability on the basis of operational experience. GRS-170, Koln Germany: 2000.
40
[27] Sträter, O, and Bubb, H. “Design of Systems in Settings with Remote Access to Cognitive Performance,” In Hollnagel, E. and Suparamaniam, N. (Eds.) Handbook of Cognitive Task Design, Lawrence Erlbaum, Hillsdale, 2003.
[28] Norman, D.A. Categorization of action slips. Psychological Review 1981; 88:1-15.
[29] Kemeny, J. The Need for Change: Report of the President's Commission on the Accident at Three Mile Island. New York: Pergamon Press; 1979
[30] Rasmussen, J. “Skills, rules, knowledge: signals, signs and symbols and other distinctions in human performance models,” IEEE Transactions: Systems, Man, and Cybernetics, 1983, SMC-13,257 -267.
[31] Hutchins, E. Cognition in the Wild. Cambridge: MIT Press; 1995.
[32] Turner, B.A, and Pidgeon, N.F. Man-made Disasters. Second ed. Boston: Butterworth-Heinemann; 1997.
[33] Klein, G.A., Sources of Power: How People Make Decisions, Cambridge, MA: MIT Press; 1999.
[34] Klein, G.A., and Salas, E. Linking Expertise and Natural Decision Making. Mahwah, NJ:
Lawrence Erlbaum Associates; 2001.
[35] Gladwell, M., Blink: The Power of Thinking Without Thinking. New York: Little, Brown
& Co., 2005.
[36] Kauffman, J.V. Engineering Evaluation: Operating Events with Inappropriate Bypass or Defeat of Engineered Safety Features. Washington, DC: Office for Analysis and
Evaluation of Operational Data, U.S. Nuclear Regulatory Commission, 1995.
[37] Barriere, M.T., Wreathall, J., Cooper, S.E., Bley, D.C., Luckas, W.J., Ramey-Smith, A.
Multidisciplinary Framework for Human Reliability Analysis with an Application to Errors of Commission and Dependencies. NUREG/CR-6265, BNL-NUREG-52431.
Upton, NY: Brookhaven National Laboratory, 1995.
[38] DiNunno, J.J., Anderson, F., Baker, R., and Waterfield, R. Calculation of distance factors for power and test reactor sites, U.S. Atomic Energy Commission report TID-14844, March 1962.
[39] Theoretical possibilities and consequences of major accidents in large nuclear power plants, WASH-740, U.S. Atomic Energy Commission, 1957.
[40] Rasmussen, N.C., et al., The reactor safety study, WASH-1400 (NUREG-75-014), U.S.
Nuclear Regulatory Commission, Washington, D.C., 1975.
41
[41] Congressional Testimony. Reactor safety study (Rasmussen report), oversight hearings before the subcommittee on energy and the environment of the committee on interior and insular affairs, House of Representatives, Ninety Fourth Congress, Second Session, Serial No. 94-61, Washington D.C., June 11, 1976.
[42] Lewis, H.W., et al. Risk assessment review group report to the U.S. Nuclear Regulatory Commission, NUREG/CR-0400, U.S. Nuclear Regulatory Commission, Washington, 1978.
[43] Standard for probabilistic risk assessment for nuclear power plant applications, ASME RA-Sa-2003, Addenda A to ASME-RA-S-2002, American Society of Mechanical Engineers, December 5, 2003.
[44] Guide for incorporating human action reliability analysis for nuclear power generating stations, IEEE Standard 1082, Institute of Electronic and Electrical Engineers,
(1997/reaffirmed, 2001).
[45] Good practices for implementing human reliability analysis, NUREG-1792, US Nuclear Regulatory Commission, Washington, D.C., 2005.
[46] Swain, A.D. Comparative Evaluation of Methods for Human Reliability Analysis.
Gesellschaft fur Reaktorsicherheit (GRS), GRS-71, Koln Germany, ISBN 3-923875-21-5, 1989.
[47] Evaluation of human reliability analysis methods against good practices, NUREG-1842, Draft for Public Comment, US Nuclear Regulatory Commission, Washington, D.C., April 2006.
[48] Dougherty, E.M. Guest editorial: human reliability analysis–where shouldst thou turn?
Reliability Engineering & System Safety, 29: 281-299, 1990.
[49] Probabilistic Risk Assessment Reference Document, NUREG-1050, U.S. Nuclear Regulatory Commission, Washington, DC, 1984.
[50] Bieder, C., Le-Bot, P., Desmares, E., Bonnet, J-L., Cara, F. “MERMOS: EDF's new advanced HRA method,” in Probabilistic Safety Assessment and Management (PSAM 4), A. Mosleh and R.A. Bari (Eds), Springer-Verlag, New York, 1998.
[51] Potash, L.M. et al. Experience in integrating the operator contributions in the PRA of actual operating plants. ANS/ENS Topical Meeting on PRA, Port Chester, NY.
LaGrange, IL: American Nuclear Society, 1981.
[52] Phillips, L.D, Humphreys, P.C., and Embrey, D.E. A Sociological Approach to assessing Human Reliability, Oak Ridge, TN, Oak Ridge National Laboratory, 83-4, 1983.
42
[53] Embrey, D.E. The use of performance shaping factors and quantified expert judgment in the evaluation of human reliability: an initial appraisal, NUREG/CR-2986, Brookhaven National Laboratory, Upton, NY, 1983.
[54] Hannaman, G.W., Spurgin, A.J., and Lukic, Y.D. A model for assessing human cognitive reliability in PRA studies. In: Proceedings of 1985 IEEE third conference on human factors and power plants, Monterey, California, 85CH22350, IEEE, New York,1985.
[55] Embrey, D.E., Humphreys, P., Rosa, E.A., Kirwan, B., and Rea, K. SLIM-MAUD: An approach to assessing human error probabilities using structured expert judgment (Vols.
I & II), NUREG/CR-3518, Brookhaven National Laboratory, Upton, NY, 1984.
[56] Hannaman, G.W., and Spurgin, A.J. Systematic human action reliability procedure.
Electric Power Research Institute, EPRI NP-3583, 1984.
[57] Wakefield, D.J., Parry, G.W., Hannaman, G.W., and Spurgin, A.J. SHARP1 - A revised systematic human action reliability procedure, EPRI TR-101711, Tier 2, Electric Power Research Institute, December 1992.
[58] Swain, A.D. Accident sequence evaluation program human reliability analysis
procedure, NUREG/CR-4772, SAND86-1996, Sandia National Laboratories, February 1987.
[59] Whitehead, D. W. Recovery actions in PRA for the Risk Methods Integration and
[59] Whitehead, D. W. Recovery actions in PRA for the Risk Methods Integration and