We take as this chapter’s goal, then, to formalize the definition of hazard from Leveson’s Engineering a Safer World in order to develop a vocabulary and reference model that relate an entity to its hierarchical environment. It is our hope that in doing so we can provide a) a clearer statement of hazard-related concepts in a hierarchical systems context, b) a basis for greater automation (which may come as a bridge between the system safety and formal methods communities), and c) allow reuse of risk management assets in order to enable compositional approaches to safety assessment like the one described in Chapter 4. Leveson’s definition is “[a] system state. . . that, together with a particular set of worst- case environmental conditions, will lead to an accident.” [30] It is not within the scope of this work to defend the validity of this definition, but as it has been proven through Leveson’s work and similar notions appear in some safety standards, we take it as axiomatic. This definition gives us a clear partition between the system and its environment, and by
making explicit the environment’s role, it associates a notion of harm with some event that crosses the system boundary. This definition works well for components that exist at the penultimate level of a system’s hierarchy—the top-level sensors, controllers, and actuators that directly sense or modify the state of the environment. But it is unsatisfactory for lower levels: it is likely that, for example, our system’s controller component is itself composed of sub-sensors, -actuators, and -controllers that all interact with the component’s local environment, but not the top-level system environment (i.e., all these sub-sensors may do is “sense” messages arriving on the network, rather than directly monitoring some aspect of the controlled process).
We cannot consider these subcomponents in the context of the top-level environment or we will lose the ability to characterize a component’s properties using only local concepts— and reasoning locally about a component is the key to global compositional reasoning. What, then, can we know about a component without considering the full system of which it is a part? Rather than marry our components tightly to a specific controlled process, we recognize that components view the state of the system through the lens of their role: an actuator’s view of the system state is simply “should actuate” or “should not actuate1.” As part of this work, we have focused on five such roles. First, a special role, top, that includes the system and its environment, and four that the top-level component decomposes into (which come from STPA) [30]: sensor, actuator, controller, and controlled process. Consider the train speed sensor from Leveson’s example: if it fails to correctly inform the door controller that the train is moving, that will clearly contribute to the previously discussed hazard. But this same sensor might be used in a different application that requires knowledge of a train’s motion, and—if certain conditions are met—it stands to reason that some of the original safety analysis may be reusable. We formalize this intuition by introducing a generalization of Leveson’s hazard, which we term undesirability.
Definition 1: Undesirability —A component state that, together with a particular set
1In [30], Leveson gives a number of hazard identification guidewords; ultimately, though, they all condense
of worst-case conditions of its environment, will produce an unwanted, observable effect. The foundational issues addressed by this definition are that undesirability manifests as a) a pair of one component state and one state of its environment that is b) observable (i.e., the component is somehow affecting its environment) and c) unwanted. In fact, a hazard can be seen as a special case of this definition, where our component is the conventional system, the accident (i.e., harm) is the unwanted effect, and the observer is the controlled process. In this work, we have developed formalisms that result from the natural progression of this thinking. First, we allow reasoning about what observable, abstract system states a component can be in. Second, we allow the underlying notions of undesirability to be linked to a component’s interaction points (i.e., the ways in which the component can affect its environment via communication, energy dissipation, etc.). Third, we discuss how these links can drive analysis of a) component- and system-level safety using formalisms from the formal methods community, and b) impacts of internal faults or externally-produced errors. Then, to the extent that these abstractions are generic (and subject to underlying assumptions), the analysis is re-usable: an actuator which responds to external commands need not have its safety-related aspects re-examined if only the source of the commands change. That is, while we do not claim to have a fully compositional process for system safety (nor do we claim that such a concept is even possible), we believe that the work in this chapter is an important step toward supporting re-use of component-level safety-related reasoning.