Momento 1 Recuperación del proceso vivido

When assessing the execution of a task, an evaluation can be obtained comparing the recorded outcome against an ideal result. Shooting a target, could provide a clear example. The perfect shot is the one that hits the centre of the target. The more a shot departs from the centre, the lower the points that the shooter receives. Using this analogy, an expert hits the centre of the target more frequently than someone with less expertise. In paragraph 2.1.1 it was highlighted how expertise is domain specific. To understand the essence of expert performance in a specific domain, researchers have created standardized representative tasks that could be presented to a group and then identified those skills and results that best discriminated experts (K. Anders Ericsson, 2006a). To our knowledge this standardized representative task does not exist for handling large ships.

The first step in this study was then to create suitable tests or manoeuvres. Referring back to the example of the shooter, the first question then was: would it be possible to create an analogous “target”, suitable to highlight elements of shiphandling expertise? Fortunately, as detailed in paragraph 3.2, to answer to this first question, it was possible to rely on the use of a full mission bridge simulator. Similarly to a laboratory, a full mission bridge simulator offered the possibility to replicate exactly the same experimental conditions. More than a laboratory, though, the simulator provided such a high level of ecological validity, that the experimental setting could be considered similar to that of a field study. The next question then was: in order to clearly identify components of expertise, would it possible to define the perfect execution of a manoeuvre (if “the best manoeuvre” could be defined, would it be the fastest or perhaps the shortest..?)? In reality, shiphandlers know that there is nothing more evanescent than the definition of a “perfect manoeuvre”. In the real world every manoeuvre is different, even though two manoeuvres might be conducted with the same ship, in the same port and to the same berth. The reasons are many: every ship has her own manoeuvrability that depends on hull shape, propulsion, steering, loading conditions etc... The environment plays an extremely important role. Different wind, tidal and current conditions will deeply influence the way the same vessel will respond. In a real port, all these “parameters” constantly change, making shiphandling every time a different exercise. In addition, personal preferences of the shiphandler and choices to adopt different manoeuvring techniques play their part. So, how would it be possible to tame such variability into a standardised evaluation?

As detailed in the introduction, in this research participants that were selected were considered “experts” in shiphandling, based on their years of experience and current employment as marine pilots. As experts it was reasonable to assume that they understood the implications of the manoeuvring conditions provided to them. It was

105

assumed that, taking into account those implications, they were able to provide a manoeuvring plan, a strategy, capable to bring safely and successfully their ship alongside the assigned berth. As detailed in paragraph 3.4.1 the first stage of the research was to specifically ask the participants to verbalise their intentions, once they were made aware of what the manoeuvring conditions were. At this stage, through a direct exchange between the researcher and the participants, manoeuvring intentions were translated into numerical quantities (see Appendix 6) that could be compared later on with the execution. Using the example of the shooter, this “Detailed Manoeuvring Plan” served as the “target”, as the “ideal manoeuvre” against which it was possible to make the evaluation. In the context of this study, the more the execution matched the plan the more the shiphandler was able to prove his/her competence. The novelty of this approach was then to evaluate the participants on two of the most important aspects of expertise: capability to plan and capability to execute. Matching plan with execution may not encompass all the possible manifestations of shiphandling expertise. Nevertheless, those two activities both imply in-depth knowledge, to forecast what to expect in the manoeuvre and to timely and accurately interact with available resources and means. Lacking in any of the two would have proved succeeding in the simulator very difficult. As already encountered in paragraph 2.1.8, where theories of expert Decision Making were summarised (see Table 4 (G Klein et al., 2006; Salas et al., 2010)), one of the most important characteristics of experts was that they own accurate internal representations of how things work in their domain of practice (Rouse & Morris, 1986). Experts’ intuition utilises situation assessment and problem representation, which includes maintaining an understanding of the entire picture (Mica R Endsley, 1995; Flin et al., 1996; Mosier, 1991; Randel et al., 1996). Experts engage in problem detection, identification, anticipatory thinking, forming of explanations, identifying explanations, discovering inadequacies in initial explanations, and projecting the future (Gary Klein et al., 2007; Gary A Klein, 1993; Weick, 1993, 1995). The detailed manoeuvring plan (see paragraph 3.4.1) obtained from the pilots before the manoeuvres were conducted in the simulator, captured and quantified all those elements. Referring back to section 2.2, the DMP (Detailed Manoeuvring Plan) acted as the practical translation of pilots’ mental models, meaning the “mechanisms whereby humans are able to generate descriptions of system purpose and form, explanations of system functioning and observed system states, and predictions of future states” (Rouse & Morris, 1986). This was the first component of expertise that was considered in paper II.

As it can be noticed in Figure 18, the mental models were formed during the planning stage, and were then transferred to the execution stage. In the execution stage those mental models, were compared to reality, developed and maintained as pilots’ situation awareness. Mismatches between mental models (expectations) and situation awareness

(perceived reality) fed Decision Making. Communications and orders translated into action decisions taken by pilots aiming to rectify perceived those mismatches.

At this point, moving into the conduction of the manoeuvre, it was considered the second fundamental component of expertise considered in paper II: accurate performance when experts are acting in their domain (G Klein et al., 2006). This was the aspect that was specifically explored with the detailed analysis of the execution of the manoeuvres. Smooth and efficient actions were expected, while achieving the desired outcomes (as expressed in the DMP) (Posner & Snyder, 2004).

Those two fundamental aspects of expertise, planning and executing capabilities, were compared against each other to obtain an overall evaluation. If pilots were not capable to foresee the implications of the proposed manoeuvring conditions, they would have encountered serious difficulties to put their intentions into practise. On the other hand, given the accuracy of their plan, to complete their manoeuvres they had also to follow up in the simulator, demonstrating competent vessel conduction. The dependent variables that were developed in this research, were specifically designed to target the discrepancy between the plan and the actual execution. Those variables were comparing a “forecasted quantity” as expressed in the DMP, against an “actual quantity”, as recorded during the execution in the simulator. These elements are shown at the bottom of Figure 18 (analytical comparison of the “plan” against the “simulator data”).

As outlined in paragraph 4.1.2 several results were obtained. Among those results, the XTD (the dependent variable measuring the distance between the expected position against the actual vessel position during the manoeuvred) clearly indicated how higher scores were recorded during the swing. The swing was the phase when the vessel had to be rotated 180 degrees. In this phase, pilots were statistically able to remain within 100 meters of their intended position 80% of the time. This information becomes crucial when swinging vessels in constrained waters, where 100 meters could make the difference between a safe manoeuvre or an incident. The contribution of this study, though, was not simply in the specific quantification of the 100 meters (which was dependant, of course, on the experimental conditions). The contribution was also in how the quantification was carried out, providing an example of how, when intentions differ from reality, this can be accounted for, especially when conducting manoeuvres in a real port. Another example was obtained from the analysis of the difference between the intended use of the main engine and its actual use (independent variable EngEst). Pilots expected to use the propulsion much less than what experienced in the simulation. In the swing the difference between the planned and the effective use, reached values of 50%, when the engine was already working up to 80% of its maximum power. Pilots’ plans did not consider to use the main engine that much, nor so close to its maximum availability. These results showed how it is possible to identify and quantify discrepancies

107

that may depend either on lack of initial understanding or limitations in execution skills (or both). Either ways, similar outcomes would help to identify were the manoeuvres, in particular conditions, could require a different approach or more training (or both) to increase safety margins.

This approach provided the advantage of detaching the assessment from port or environment specificity. The manoeuvres chosen in this study were but a few of the many that could be developed to replicate and test specific conditions of interest. Nevertheless, the way to assess the outcome would be exactly the same: asking in advance what is expected in the manoeuvre according to the specific conditions, and then compare and assess the actual execution against such prediction. The practical achievement of this analytical comparison, as detailed in paper II (see section 6.2), can be applied for any intended or adopted manoeuvre in any specific port. In a more realistic environment, where manoeuvring context would dynamically develop due to, for example, changes in environmental conditions or the occurrence of unexpected events, this approach would still be practicable. Should an unexpected event happen (i.e. a mechanical failure), pilots would still need to assess the situation and consequently take action, keeping aware the bridge team about their understanding of the implications. This is exactly the process that should occur in emergencies and the application of contingency plans. It naturally follows then that the assessment of Pilot’s reactions in emergencies and their application of contingency plans would further support the assessment of their expertise.

The capability to predict the expected outcome and translate it into practise is fundamental for expert shiphandling. Every day Pilots commit to take vessels in and out of ports, based on the assumption that they will be able to maintain control of the vessel within specific safety boundaries. Their timely appreciation of significant changes in shiphandling conditions (vessel characteristics, wind, current, tide etc..) and their understanding of the consequent implications is what (or at least partially) can define their expertise and keep ships and ports safe.

The results provided and discussed in paper II (See section 4.1.2), illustrated how theoretical elements of expertise were unpacked, measured and quantified. But how does this impact the reality of the shipping industry? Based on the manoeuvres adopted, this research was able to define differences and probabilities within which the execution was able to match the planning in the group of participants. Similar and more tailored work can be carried out for specific manoeuvres, in specific ports with different groups of pilots. Achieving such empirical quantification becomes extremely important in the conduction of real world activities. Such evaluation is the main and most difficult exercise in the conduction of risk assessments that would decide the suitability (or not) of certain operations. Deciding to carry out certain port operations (ship manoeuvres) without a

proper and thorough appreciations of the risks involved, is likely to increase the probability of facing the sorts of low probability, high severity incidents that must be avoided. This research also has direct implications on broader safety management issues, and although not the central issues in this thesis, and not an exhaustive list, these are briefly mentioned below:

Achieving High Reliability (highly reliable organisation theory (Roberts, 1990; Weick & Roberts, 1993)) - As Beyea describes in her work, Highly Reliable Organizations (HROs) are those that are known to be complex and risky, but still safe and effective, even at high levels of operational performance/demand. High reliability organizations, committing to safety, value teamwork and nourish a culture of continuous learning and improvement and redundancy in safety measures and personnel. One of the primary drivers of those organizations is that errors are important opportunities to learn, and when they occur, knowledge gained from them helps preventing similar events from occurring in the future. This is achieved by identifying how mistakes were made, examining errors to determine their root cause instead of blaming individuals (Beyea, 2005). Planning is instrumental to high reliability organizations, since it condenses, incorporates and applies all the lessons learnt from previous experiences. Not only, planning provides the inclusive framework within which individuals can share their contribution and coordinate their efforts with other members of the team. The contribution this thesis makes in the context of planning can therefore support higher levels of reliability.

Engineering Organisational Resilience - Engineering resilience in organizations aims to enhance the ability to create processes that are robust yet flexible, proactively using resources when disruptions may occur. In Resilience Engineering, failures are not interpreted as a breakdown or malfunctioning of normal functions, but rather shortcomings in the adaptations needed to face real world complexity. It is assumed that individuals and organisations must always adjust their performance to the current conditions. Since time and resources are finite it is inevitable that those adjustments are approximate. The goal then becomes to anticipate the changing risk before damage occurs, with failure simply considered as the temporary or permanent lack of that anticipation. The way to build such capacity is understanding how to create adaptive margins into systems, able to anticipate and absorb pressures, variations and disruptions. Here are some fundamental traits of organizations and individuals applying resilient engineering. Present success does not guarantee future safety. Past results are not enough to rely on their adaptive strategies for future success. Risk is always consciously considered, even when everything looks safe, since the idea of what is risky may have become old or wrong. Doubt is welcome as well as opinions from minorities, to maintain an open-mind and remain sensitive to changes (Sidney Dekker & Cook, 2008). In this context, planning can offer the opportunity to raise concerns and consider alternative

109

options. Plans would include contingencies, accounting for foreseeable disruptions, increasing the probability of finding a way out. By providing a more detailed understanding of the nature of marine pilotage expertise, the knowledge developed in this thesis can contribute to more resilient shiphandling in several ways:

 Obtaining more precise measurement of performance in the simulator,

 Identifying mismatches between mental models and executed tasks, and also;

 Identifying issues in how pilots maintain their perceptual cycle over the course of a manoeuvre.

Managing Automation Challenges - As documented in the literature (Billings, 1991; Neville Moray, 1986; C.D. Wickens, 1984) adoption of automation may expose operators and systems to what has been referred as the “out-of-the-loop” performance problem (Mica R Endsley & Kiris, 1995). This problem refers to operators of automated systems and their limitations in the ability to take over manual operations in the event of automation failure. There are many contributing factors to this phenomenon. Some of them are: a possible loss of skills arising from complacency, the shift from active to passive information processing and the change in feedback modalities made available to the operator. Endsley and Kiris (1995) point out how a lower involvement of operator control when interacting with automation is a major contributor to the loss of SA, although this concept of SA “loss” is acknowledged as being based on a somewhat circular argument. So how would planning be involved in the mitigation of such risk? The results of this thesis suggest that proper planning would include when automation would be allowed (and at what level) or in which context manual conduction should be resumed. An example could be provided referring to the use of autopilots on board of vessels. Integrated navigation systems or dynamic positioning systems are technologies that allow operators to set a track on a charting system and have the vessel automatically following that track, without operator intervention. The thorough understanding by operators of how such technology works, with its strengths and its weaknesses, becomes fundamental to understand the limits of applicability and use of those features. Safe manoeuvres in ports, depending on circumstances, may or may not suggest the use of certain technology. Planning would be the stage at which to consider the suitability of automation options. During planning would be also the time when to state how changes in levels of automation should occur, in order to smoothen otherwise traumatic transitions from automatic to manual control. Further, the measurement of workload and gaze has implications for the understanding of this type of automation-related problem – how the pilot adjusts their attention once they are required to take manual control, and how their workload changes during that time.

Port team resource management - In a recent report, Goodfellow (2008) argues about the eroding competence of crews. Risks rising from reduced crew competencies, require

proper mitigation using all available means that a port can deploy. These means include and are not limited to VTS (Vessel traffic services), tugs, linelaunches, linesmen etc.. The master-pilot exchange briefing (2007; Wild & Constable, 2013) is meant to offer to the captain (bridge team) and the pilots an opportunity to exchange all the information relevant to the upcoming manoeuvring operations. The sharing of that information with the port team, to whatever extent is reasonable, supports more effective and resilient operations, identifying and assigning clear goals and tasks to those involved in the process. This thesis makes a contribution to this issue by identifying measures that can profitably be used to optimise port team performance. For example, measuring gaze (as a proxy for attention) and workload across a team could be used to optimise workload and the perceptual cycle of all participants – rather than the current situation which has tug masters and VTS Operators playing much more passive roles.