VARIABLES

procedures for conducting such reviews.

A PIR’s success will largely depend on the credibility and competence of the individuals conducting the review. They should be familiar with established policies and procedures and be objective, well trained, and experienced in conducting PIRs. Training might include a class on

assessing projects based on some developed criteria. Also, the team leader should have past experience conducting similar investment reviews.

Activities Activity 1: The investment board identifies which projects will have

a PIR conducted.

While it may not be necessary to conduct PIRs on all investments, an organization should review a mix of successful and less successful

Section 5: Critical Processes for the ITIM Stages •Stage 3: Developing a Complete Investment Portfolio ••Conducting Postimplementation Reviews

in the investment process, the organization can enhance the success of the portfolio as a whole.

In accordance with organizational policy, an IT investment board will identify and designate the projects for which a PIR will be conducted. Projects need to be selected based on some type of selection criteria. Then one or more examining teams will conduct the PIR(s) on the selected projects. The standard PIR process may be tailored to the specific investment being reviewed.

Often a centralized group (e.g., quality assurance, audit committee, etc.) will conduct PIRs under the direction of the enterprisewide IT investment board. Owners or users of the investment under review can participate in its PIR. This approach enhances the consistency of the data collected and ensures coverage of the appropriate projects. However, there are other acceptable approaches that may be used. The organization should employ the approach that best meets its needs as constrained by its resources. Activity 2: Quantitative and qualitative investment data are collected, evaluated for reliability, and analyzed during the PIRs. As part of the objective analysis of the investment, quantitative PIR data are collected. These data should largely arise from the selection and control process activities that have been conducted previously. Specific types of quantitative data can include

• performance expectations and actual outcomes;

• updated performance data and explanations for changes;

• measures of business or mission objectives, such as operating costs, schedule, and product cycle time;

• measurements of improved technical capability; and

• contribution of the investments toward achieving both the strategy and the objectives of the organization’s IT strategic plan.

In addition to quantitative investment data, qualitative information, such as the perspectives and insights of project participants and end users, may serve to validate or raise questions about the quantitative information and

Section 5: Critical Processes for the ITIM Stages •Stage 3: Developing a Complete Investment Portfolio ••Conducting Postimplementation Reviews

the existing investment management processes used by the organization. Qualitative data can include

• surveys and interviews of end users, customers, project management, project staff, contractors, and developers;

• project management and staff interviews; and

• interviews with senior decision makers involved in investment oversight.

Some common techniques for performing analyses during a PIR can include

• conducting trend analysis using historical investment data, • conducting a review of how benefits are realized,

• conducting means-end analysis to compare results with known causal factors, and

• performing force field analysis to understand the effects on the investment of major decisions that were made.

Activity 3: Lessons learned and recommendations for improving the investment process are developed during the PIR, documented, and then distributed to all stakeholders.

The investment board, with the assistance of the project team, conducts a review of the investment selection and control process in order to extract “lessons learned.” Based on these lessons learned, recommendations can then be developed to (1) improve the investment process (i.e., select, control, and evaluate) and (2) improve the management of individual investments. Additional recommendations could include improving the development of the business case, refining the project/portfolio selection criteria, or determining whether or not the objectives of the project are being met.

Once the PIR has been documented, it should be made available to all stakeholders and then archived for future reference. A positive outcome of identifying lessons learned and developing recommendations is having the

Section 5: Critical Processes for the ITIM Stages •Stage 3: Developing a Complete Investment Portfolio ••Conducting Postimplementation Reviews

information available for future reference. Sharing experiences, both good and bad, promotes improvements to the overall investment process.

Using the collective results of PIRs from IT projects during different stages of their life cycles, organizations can learn valuable lessons and gain insights. The results from one project will often not provide enough information to allow significant modifications to be made to the investment decision-making process. PIRs help organizations focus their evaluation efforts on areas they deem important and avoid pitfalls and problems that others have experienced. Furthermore, PIRs help organizations address specific issues that may be impeding progress toward the effective management of an IT investment or the development of an assessment framework to fine-tune the IT investment process.

Section 5: Critical Processes for the ITIM Stages