Once the IOIS development is completed the challenges that firms face in implementation change form from systems development to more of a change management issue (that is, the diffusion and infusion of the technology across the organisation). Indeed, many IS projects fail to deliver the key business benefits for the organisation because of unused or underused systems (Charette 2005; Dwivedi et al. 2015). This puts a significant emphasis on post-implementation evaluation which not only helps to identify the key issues that impact the systems uptake, but also the factors which play a vital role in implementation success. In this regard, the Task 5 can be viewed as the continuation of Task 4 where the aim is to analyse the IOIS adoption to determine the key issues that hinder the diffusion and infusion of the IOIS across the case study organisation. Although the IOIS in the case studied was operational with seven suppliers,
following the piloting phase the contractor organisation decided to adopt the new system with only three suppliers. Historical purchasing figures indicate that the three suppliers on-board represent just over 10% of the case study organisations expenditure which is an evidence of the significance of the suppliers in the IOIS project.
4.3.2.1 Method
Task 5 was primarily concerned with two research questions: (i) what is the extent of the IOIS adoption within the first year of going live with the implementation, and (ii) what are the challenges that the end-users face during its on-going use. The variables used to measure the IOIS adoption were borrowed from the IS literature which provide the foundational basis to research exploring the IS adoption, acceptance and continuance use (Dwivedi et al. 2015). Paper 5, which is attached Appendix F, provides a summary of the additional literature studied and the framework adopted. In brief, the framework (the DM model) adopted for the evaluation of the IOIS posits that the level of adoption and, ultimately the success of IOIS implementation, can be determined by looking at the IS quality (that is, system, information and service quality) and the use (user satisfaction, and intended or actual usage). Rather than investigating the extent to which the DM model variables moderate the IS adoption—or non-adoption (which is the most common application of the DM model in the literature), end-users’ perspectives were taken as the primary source for understanding and investigating of the specific issues associated with IS adoption.
Similar to the previous task, a longitudinal, multi-method data collection and analysis strategy was adopted which is summarised in Table 4:4. The two questionnaire surveys were conducted with the help from e-Procurement managers in the case study organisation. The first Table 4:3 The challenges and barriers experienced during the supplier on-boarding phase
of case study IOIS implementation project.
Category Challenges Barriers
Technical Capability and adaptation of existing systems
Availability of human and IT resources Impact of integration on existing commercial
processes and EDIs Lack of clear specification Late design changes Inadequate documentation
Lack of suppliers’ flexibility to align their systems or accommodate the necessary changes required in implementation
Cost of development for suppliers
Coordination Lack of adequate project coordination
Inadequate change management Large number of people involved
Clear identification of roles/responsibilities
Dispersed virtual teams Differences in time-zones Social/cultural diversity of teams
Integration Single-hub connections: impact of any new changes
on existing connections/processes
Inter-hub connections: duplicating data conversion efforts
Competition amongst hub-providers
Organisational Poor implementation strategy
Lack of commitment from suppliers
Lack of strategic support from senior management High staff turnover in project
management/coordination teams
Lack of implementation teams’ skills, knowledge and experience in EDI projects.
Supplier relationship discontinuity Supplier business uncertainty
survey was issued in the first 6 months of go-live to capture the end-users’ perceptions on information quality, system quality, service quality and overall satisfaction with the system. The second survey, which was issued towards the end of first year, sought to capture the perceived utility of the IOIS from supplier integration perspective. More specifically, the survey explored the importance of the suppliers in the eyes of end-users.
On the other hand, the issue logs and usage statistics were obtained from the sponsoring company. The usage statistics consisted of three data sets: attraction (user logins), interaction (blanket orders raised for each project and business unit), and transaction (actual transaction data of each user, project and business unit). In terms of analysis, the qualitative data (issue logs and responses provided in the surveys) were analysed through thematic analysis, whereas sums and averages were used for the analysis of the quantitative data. Bar and pie charts were also used to display and compare the frequency and averages of various categories of data. It must be acknowledged that the four data sets were not cross-linked (as surveys were anonymised in order to encourage objective commentary and encourage participation from the end-users). Hence, the causality in the findings are not absolute but rather implicative of the relationship between the end-users’ perception and the IS adoption challenges.
4.3.2.2 Findings
The research findings in Task 5 point to number of issues that had serious impact on the adoption of the IOIS. The following issues were identified as a key factor in system quality.
• User-friendliness of the system and Graphical User Interface (GUI) design issues. • Errors and performance related issues with the PunchOut interface.
• Internet connectivity issues experienced from the projects’ site office. • Inconsistencies in IOIS functionality in different web-browsers. With respect to the information quality, following issues were prominent.
• Lack of information provided after a transaction is completed (for example end-users were unable to trace the orders placed and their status)
• Accuracy of the information provided on the supplier catalogues (for example information on stock levels).
• Lack of IOIS’ ability to process information attached to the purchase orders.
• Lack of interaction functionality (for example, functionality to help with regular/repeat orders) other than creating purchase orders.
The key concerns that were raised by the end-users in relation to the service quality dimension are as follows.
• End-users training and support should incorporate variety of methods within its scope, including user guidance documents, demonstration videos and one-on-one session.
Table 4:4 The data collection approach adopted in Task 5.
Data Sources: Survey 1
(30 responses) Survey 2 (31 responses) Usage Data (12 months) Issue Log (48 issues) DM Model variables: System Quality
Information Quality
Service Quality
Use
User Satisfaction
Net Benefits
• Technical guidance documents must cover all the use case scenarios in the purchasing process.
• Issue resolution process must adopt collaborative processes, procedures and protocols between the contractor firm, supplier, and intermediary hub provider’s IT team for management of the technical and non-technical (e-commerce related) issues.
In terms of the user satisfaction, the responses to questionnaire survey indicate there were high levels of satisfaction with the IOIS amongst the majority (76%) of the end-users. Those who were dissatisfied with the system echoed their concerns in relation to the usability of the IOIS (e.g. difficult to use) and the e-commerce process which lacks extensive functionality to cater for the post-purchasing order management.
The level of IOIS use amongst the users/projects and business units reveal significant differences at each level of analysis. End-users in Administrator role were the key users of the system whom, compared with the rest of the user groups, were much more regular and consistent in terms of IOIS utilisation. At the project level, the findings showed that the IOIS usage has not been equally spread across different industries and sectors (see Figure 4 in Paper 5). Although the projects in Facilities Management industry utilised the IOIS more often, in terms of the value of the transactions, the projects within the Construction industry accounted for more than half of the total spend. The projects in the Transport sector in particular (where the duration of the projects span over a relatively longer period of time with budgets usually over several hundred million) were the most prominent users within the Construction industry. The analysis also revealed that the depth of adoption at the business unit level also differs significantly amongst the different businesses of the case study organisation (see Figure 5 in Paper 5). However, the usage data revealed a positive trend in the number and value of orders made at each business unit which is presumed as an indication of the gradual acceptance and continuance use of the IOIS across the organisation.
One way of measuring the IOIS success is studying the value of suppliers as perceived by the buyers (end-users) at the contractor firm. Findings in Task 5 outline several important advantages being obtained in relation to the actor, process and technology dimension of supplier integration (see Paper 5 for more information). However, findings from the survey responses indicate that the perceived significance of one of the three suppliers on-board (Supplier 3, which is a construction tools and equipment supplier) is comparatively low which indicates a lack of significant advantage from the IOIS implementation with that particular supplier. This finding was further supported with the supplier spend data which showed that the total value of transactions conducted through the IOIS for Supplier 3 was quite low (around 10%). In contrast, the level of perceived significance and the amount of supplier spend data for Suppliers 1 and 2 (which are 49% and 41%, respectively), were comparably substantial (see Figure 6 in Paper 5), which implies higher levels of net benefits being realised from the use of IOIS with these two suppliers.
Evident from the findings, long-standing issues which commonly plague the IS development projects also appear in the IOIS projects however there are a number of IOIS- specific issues (including the need for inter-firm issue resolution, interfacing across different systems, and functionality and performance of the system in different web-browsers) that were experienced in the case study. Paper 5 provides a discussion around the implications of the research findings for future IOIS implementation projects. Overall, the findings drawn from the case study have serious implications for the planning, design, development and implementation of the IOIS projects. In essence, the suggestions in Paper 5 frame around an IOIS development strategy which incorporates (i) requirements engineering, (ii) business process reengineering and (iii) change management activities within its framework. Since IOIS implementation involves joint effort by multiple parties (contractor firm and its suppliers) to integrate systems and processes, contractor organisations have to be selective in their supply
chain integration strategy. As well as carrying out a commercial assessment of the construction projects and business units (which could benefit from a more efficient and effective supplier engagement strategy), contractor organisations should take into consideration the end-user perspectives in integrating with particular suppliers.
4.3.3 Summary
The research carried out in Objective 2 was focused in the study of IOIS integration, implementation and delivery, and drew on case study research strategy for an extensive evaluation of the key constraints in IOIS implementation projects. The RE took up the role of participant-as-observer to study an IOIS implementation project in its natural setting. This enabled detailed inferences to be made from multiple stages of the IOIS implementation lifecycle. In Task 4, the RE focused on the IOIS project, especially the supplier on-boarding process to identify the key challenges and barriers involved with its delivery. Findings suggest that the challenges and constraints regarding the supplier on-boarding are largely non-technical, coordination related issues which are borne due to poor planning and execution of the IOIS implementation process. Although the scope of the implementation in Task 4 is not a complex integration activity, the magnitude of transformation in the procurement process (i.e. from paper-based to digital approach) was quite significant for the supply chain operations. In Task 5 the RE turned his attention to adoption and acceptance of IOIS where the research was specifically concerned with the challenges and barriers that contractor organisations face when implementing IOIS. The results of the research carried out in Task 5 highlight a range of factors related to the system, service and information quality dimension of IOIS.