While the work previously discussed has focused on considerations that need to be made during design, attention also needs to be paid to the possible issues during deployment of the new system. At this stage, previously acquired knowledge and experience can be invaluable as developers often find that, if they knew at the outset what they knew at the end of the project, they may have done things differently[72].
People routinely tell ‘war stories’ to one another as a ‘natural’ way to share their knowledge and communicate their experiences. Facilitating this knowledge sharing can be beneficial to companies, allowing the detail of situations often lost in documentation to also be shared. Knowledge of procedures and practices which have worked well in the past, and may still be relevant, can help improve work in the future[73]. The complexity of technology and the increasing demands for efficiency and higher performance make it essential for people to learn from each other as much as possible [74], allowing common solutions to problems to be developed which can be transferred to others.
Mackie et al. developed a resource to store information regarding possible ‘risks’ in deployment using a series of organised web pages containing war stories in their natural narrative form [72]. Each story includes a description, accompanied by what was learnt from it. The stories are organised by the stage of deployment at which they occurred, the stage it is thought necessary to be aware of them, and the type of story, giving the user various options for finding the stories relevant to them. The stages of
deployment include; procurement, award and signing of contract, data collection, database build and configuration, integration, testing, transition management, domestication and evolution and maintenance. Types include access, bespoke or off the peg, communication, configuration, incomplete data sets, integration, local versus global, outside commitments, participation, relationships, schedules, security, suppliers, and support and training. Identifying which type or stage is applicable may not be easy when you approach the site, unless you have a particular focus in your project. Many may read the exhaustive list of stories provided at the procurement phase or browse through each category until they find something relevant.
The initial site was populated with stories identified from ethnographic material regarding the deployment of healthcare systems, where something caused a problem in the deployment process. Although this is not ideal as the narratives for the stories had to be developed from other types of ethnographic material, a submission facility has been provided allowing managers to contribute their own stories to the site. Stories can also be elicited by an ethnographer in interviews or they may witness their use during observations and these can then be added to the site.
Although the initial site focused on medical settings, the idea could be taken further and applied to other complex settings, where designing and deploying systems can be problematic. The usefulness of past experience is not restricted to the healthcare domain. The use of the resource was initially aimed at designers but, due to its simplicity, it could potentially be used by other stakeholders, helping to share knowledge and aid decision making.
Deployed systems are often viewed as finished products, however new technologies are almost never perfect upon initial introduction[31] and much innovation has been witnessed at the point of implementation, as well as during the design phase of the system life cycle. Once a new system has been deployed it becomes ‘domesticated’, embedded in the systems of culture and information practices of the organisation[32]. Although it is common for designers to try and constrain the ways in which users can interact with a system, they will often find novel ways of working around this. Users can always choose not to utilise the technology or modify the ways they interact with it and without this interaction the technology remains inanimate and hence
ineffectual[75]. Through using the system, for example, users gain familiarity with it and with this develop more efficient ways of using it. When a new patient administration system was introduced in New South Wales, users initially found it to be cumbersome and inflexible. However, over time they adapted and came to like it[74]. It is important to gain a better understanding of how the use of computer artefacts develops in practice and how models are created and adapted in unusual situations[76]. Developers could learn from this and gain ideas for future improvements they could make to the system. Taking advantage of this process of ‘innofusion’, where the user’s needs and requirements are discovered and incorporated in to the system while they struggle to get it to work in useful ways[33], can lead to systems being developed that are more suitable for their purpose. This however often relies on the users telling developers about the workarounds they have created and this feedback can often be missing[77].
Stewart and Williams suggest taking a social learning perspective, viewing the system deployed as unfinished in relation to the complex and evolving user requirements, to gain a greater understanding of the innovation process and ease the domestication process in the hope of creating more acceptable systems [32].
In an attempt to modify the process of system development to accommodate system adaptation through use, Hartswood et al. suggest a process of co-realisation. This involves the IT professional entering the organisation after the system is deployed and attending to the specifics of the workplace and the unnoticed ways in which work goes on[34]. The idea is based on the fact that it is only through use that a user fully attends to the system and recognises deficiencies in what it provides with respect to completing the work. Once these are identified, improvements can be made and further editions of the system released. As the evolution progresses the users become more competent with the system and are able to help generate new ideas for future developments. This process of co-realisation however relies on it being possible to deploy early, possibly incomplete, versions of a system which can be built on. This is not always possible as in some settings a complete system is necessary for work to be carried out. Co-realisation pushes the control of design and development into the workplace, a feature that may not be suitable for large scale projects with large
teams[34]; however it can be adapted to the needs of the project without losing its benefits.
Whilst it is recognised that innovation can occur after a system has been deployed, it is also noted that this is a contradictory and uncertain process where there is often a struggle between solving technical problems and the articulation of the varying needs of the users[78]. Orlikowski recognised that the effort people put into adapting a new system is often focussed during the immediate aftermath of its introduction[79]. This effort dies off dramatically when other work pressures start to take affect and habitual patterns of use develop. This is not to say that innovation cannot occur after this initial ‘window of opportunity’ but rather these windows are brief and should be taken advantage of when they occur. Further adaptation may occur if there is a change to the work setting that interrupts the daily routine of the users. As the technology becomes part of organisational routines, users rapidly form interpretations of the technology which can become particularly influential in their acceptance of it and these early interpretations may be difficult to change[79].
Work has been carried out to investigate the process of domestication in healthcare systems, where each region is integrated with a larger, national computer system. In the past COTS systems have been used and these need to be configured to fit the specific needs of the domain, where the requirements are continually changing and developing. A mismatch between the new system and the social system it is supposed to support can potentially result in the failure of the project[74]. The use of COTS systems is becoming more common as an alternative to developing a system in house, whilst still providing the flexibility to configure it to their needs. The configuration needs to support the actual work practices of the setting and the system needs to be integrated into these practices. To some extent it is impossible to know in advance what the impacts will be on work practices as these evolve in response to the system[38]. Designers however, must try to take into account these possible results when making decisions regarding the system’s configuration, as there is a high risk of failure if they fail to do so. In a safety critical domain like healthcare this can be particularly dangerous. After a system has been deployed users often make them aware of errors in their judgment and of the severity of the impacts these have. Discussing design decisions with the users before deployment can be a useful exercise
in gaining a more detailed understanding of their work practices and may help reduce these errors in judgment.
In large-scale systems, such as those in the healthcare domain, carrying out a full ethnographic study to gain an insight into the social aspects of the setting is impossible. Some form of ethnography however, can and should be carried out. Martin recommends targeting studies to those questions that appear crucial to the system design [38] and the results of these studies would give the design team a more detailed insight into the complexities of the work that the system needs to support. A small scale, targeted ethnography could also be carried out during procurement to gain an understanding of current work practices which would help ensure that the chosen system can be suitably configured to support these.
Users are often only consulted late in the design process, when it is too late to make major changes to the system, but they are the only ones who really know what it needs to do. If they are not consulted and are only then told why something can’t be done this can lead to problems in their acceptance of the system. Getting users to actively participate in the development process can help ensure their requirements are met and avoid the problem of user resistance. Many researchers have pointed to the strong link between user participation in system development and system success, with system use and user satisfaction being commonly used as a measure of success. While participation can vary in scope during the different phases of the development process, it is particularly appropriate during the early stages relating to problem definition and requirements identification, and also during the testing and installation of the system[80]. Although users may not necessarily want to or have the time to get involved, if they perceive the system as important to their jobs they may be more willing to take part. Baronas and Louis propose that the process of implementing a new system represents a threat to users’ sense of control over their work[81]. By getting them involved any fear and uncertainty they may have had about the new system is reduced and they feel they are regaining some control and this may result in them developing more positive feelings towards the system.
Whilst it is recognised that design is a process full of compromises, in the system studied, Martin found that these would often appear to users as being at their expense
[38]. Presenting information to users throughout the design process can help prepare them for the new system and aid the integration of the system into their working lives. They are more likely to accept compromises and make changes if they understand the reasons why they are being made. Hartwick and Barki recommend having users participate in the system development as participation and conflict are likely to play a constructive role in the development process [82]. Like with any interdisciplinary work the different backgrounds and knowledge levels of the users and developers can make interactions difficult. It can sometimes be difficult for them to envisage a design and understand its implications before a prototype is produced that they can interact with[38]. Lin and Shao however propose that user participation should be promoted as a process of interaction between the two parties through which they can learn about each other’s expectations and requirements and, hence resolve their conflicts[83]. How successful this process is can determine the effectiveness of the participation, so effective communication is critical. A huge cost is incurred when users reject systems that organisations install[84]. Although directive strategies imposed by management can help when dealing with system resistance these are often viewed negatively by the users.
The ETHICS (Effective Technical and Human Implementation of Computer based Systems) methodology was developed to assist and encourage user involvement throughout the development process to help create systems that are both technologically efficient and provide job satisfaction for the users. It aims to help a design group diagnose and formulate the problem, set objectives, develop alternatives, and take other appropriate actions right through to implementing and evaluating the new system [85]. The process of considering a large number of alternatives can however be time consuming as different groups may prefer different, sometimes conflicting solutions. Hirschheim, Tait and Vessey found that time constraints lead to a substantial decrease in the participation of users in the development process[80]. The ETHICS approach also fails to consider organisational constraints and the sometimes overriding objectives of management.
Although Martin found that integrating a system into working practices was a complex process, the study focused on how this process could be improved prior to roll-out and did not demonstrate how both the system and the users changed to create
a good fit [38]. Anderson et al. studied the deployment of PiMS, also in the healthcare domain and aimed at integrating and standardising patient administration [86]. Each department had its own ways of working that they had developed to suit their needs and these needed to be generalised and aligned so they could be considered in the system design[87]. Local meanings can also be attributed to data and these may not even be consistent within a group of similar users. Mutual trust between users from various departments of the same organisation is often missing and introducing a new system can bring conflicts to the surface, which then need to be dealt with [85].
Developing a standardised classification scheme removes local meanings and variations and everyone must then adapt to use it. Achieving shared meanings in a heterogeneous organisational setting requires the integration of often incompatible meaning structures and in this case, the introduction of the software heightened inconsistencies in the existing routines of different departments[88]. The new menu system that was introduced confused users and, due to the significance of several aspects of work practice being misjudged by the design team, did not provide a good fit with everyday work practice. This ‘misfit’ often led to decisions being made that reduced the quality of the data being stored which is especially important in healthcare systems. The users were initially blamed for making poor decisions until the design team discovered the difficulties that they were encountering.
Users may also have different interpretations of the technology and its purpose. They may form varying expectations towards it and these may shape their future use of the system. Discussing these differences in their technological frames early in the design process can be important in removing misunderstandings between the different groups of users and stakeholders and help manage the change process [79].
The PiMS project also pointed to communication difficulties between users and the design team and the need for a clear understanding of their requirements. Although a user group was in place, people didn’t know how to get involved in it and this caused them to be under-represented. Effective feedback is essential if the project team are to make informed decisions regarding the future of the project. Of those who did bring feedback on the system it was often a case of listening to ‘who shouts the
loudest’ when deciding whose views were relevant [86]. The study also supports Martin’s view that involving users in the design process and making them aware of the benefits of the new system is important as they are the experts in their work and the ones any changes will affect [38]. Determining what these benefits are and which will be most important to the users can however be difficult as Southon et al. found in their study of the New South Wales Health System [74].
The study of PiMS helped to show that systems evolve over time and through use issues will arise that can aid in developing it further to become more usable, a view supported by Anderson et al. [86]. It is important to recognise that if a system configuration needs to be changed this is not a failure. Stewart et al. [32] and Tyre et al. [31] are in agreement that, as with PiMS, systems will change and this period of ‘domestication’ is important in the production of more useful and dependable systems[73].
When the New South Wales Health Authority decided to introduce a new system for finance, pathology and clinical services they recognised that commitment to the project at all levels would be necessary for it to succeed. When the order communication part of the system was introduced however it failed to meet the users’ expectations. Only those who used the system regularly were able use it to meet their needs. Others experienced a great deal of frustration due to problems with the user friendliness of the interface. The project pointed to problems in sectors such as healthcare, where it is often difficult to see where decision making authority is actually held. Both the software vendor and the department coordinating the project lacked the ability to control the doctors and get them to engage with the project and the new processes it would introduce. This is a similar problem to that faced in the GEOMETER project when attempting to engage with academics. The dispersed nature of the decision making power caused decisions to be made that might not have