TÉCNICAS E INSTRUMENTOS

Ultimately, an organization needs to know how well its collected pool of investments in information management and technology is contributing to improvements in mission performance. Improving the Portfolio’s

Performance is, at the level of the investment portfolio, the equivalent of Stage 3’s PIR for an investment. This critical process seeks to determine how well a portfolio of IT investments is (1) helping to achieve the strategic needs of the enterprise, (2) satisfying the needs of individual units and users with IT products and services, and (3) improving IT business performance for users and for the enterprise as a whole. To determine these things, performance information for an organization’s entire portfolio of investments should be compiled and analyzed and trends examined. Key input for these reviews includes PIRs, the IT investment board’s experiences, and results to date for major investments, extracted from control process activities. These data are generally project or investment- specific and often are not aggregated for general trend analysis.

Section 5: Critical Processes for the ITIM Stages •Stage 4: Improving the Investment Process ••Improving the Portfolio’s Performance

Figure 19: Improving the Portfolio’s Performance

Purpose To assess and improve the performance of the IT investment

portfolio and the investment management process.

Organizational Commitments Commitment 1: The organization has documented policies and

procedures for evaluating and improving the performance of its portfolio(s).

These policies and procedures typically specify

• that each IT investment board be responsible for managing a comprehensive portfolio evaluation and improvement process,

• that access to portfolio data be provided and confidential/sensitive data be appropriately controlled,

• that each portfolio be evaluated at least annually to assess its Source: GAO.

Prerequisites:

1. Adequate resources are provided for evaluating and improving the portfolio's performance.

2. Board members who are responsible for evaluating and improving the investment processes and investment portfolio(s) exhibit core competencies in evaluating and improving portfolios.

Activities:

1. IT portfolio performance measurement data are defined and collected using agreed-upon methods.

2. Aggregate performance data and trends are analyzed.

3. Recommendations for improving the investment process and portfolio are developed and implemented.

Purpose: To assess and improve the performance of the IT investment portfolio and the investment management process.

Organizational commitment:

The organization has documented policies and procedures for evaluating and improving the performance of its portfolio(s).

Section 5: Critical Processes for the ITIM Stages •Stage 4: Improving the Investment Process ••Improving the Portfolio’s Performance

• a mechanism for assembling and aggregating the investment performance data,

• the key measures and methods used to assess portfolio performance (e.g., a “balanced scorecard” approach),

• methods for analyzing the performance data,

• methods for comparing portfolio performance and portfolio expectations, and

• a mechanism for reporting the results of the analysis.

Prerequisites Prerequisite 1: Adequate resources are provided for evaluating and

improving the portfolio’s performance. These resources can include

• support staff for executing the activities in this critical process, • methods and tools to aid the teams conducting the PIRs, and • current and historical portfolio data.

Prerequisite 2: Board members who are responsible for evaluating and improving the investment processes and investment

portfolio(s) exhibit core competencies in evaluating and improving portfolios.

These board members should be familiar with the IT investment

management approach. Training for this critical process may also include familiarizing executives with techniques for economic and process management analysis. Training in quality management analysis and tools may also be helpful.

Knowledge building and/or training may be provided, ranging from • in-depth courses for new members to

• an annual overview, for all board members, of the investment process, current process modifications, and operational procedures for

Section 5: Critical Processes for the ITIM Stages •Stage 4: Improving the Investment Process ••Improving the Portfolio’s Performance

Activities Activity 1: Comprehensive performance measurement data for IT

portfolios are defined and collected using agreed-upon methods. The portfolio of investments should be evaluated on its ability to meet the strategic needs of the organization, provide general user satisfaction with product and service delivery and management, and deliver effective and efficient IT business functions (e.g., applications development,

infrastructure availability, project performance). A combination of quantitative data and supporting qualitative information can be used to construct a picture of the organization’s IT portfolio performance. This can be analogous to developing a balanced scorecard for IT investments. (For more information, see GAO’s Executive Guide: Measuring Performance

and Delivering Results of Information Technology Investments,

GAO/AIMD-97-163, September 1997.)

Data collection and information synthesis should focus on answering key portfolio performance questions, such as the following:

• Is IT spending in line with expectations?

• Are we consistently producing cost-effective results? • How well is the portfolio being managed?

• Are users satisfied with the products and services that are being delivered?

• Are IT projects delivering their expected share of process improvements?

• How well are integrated project teams being used on major investment projects?

• Are quality IT products and services being delivered within general industry standards?

• Are accepted methods and tools being used on major systems investment projects?

• Is the IT infrastructure providing reliable and needed support for the organization?

Section 5: Critical Processes for the ITIM Stages •Stage 4: Improving the Investment Process ••Managing the Succession of Information Systems

Measures should be constructed to help objectively determine

performance outcomes in these areas. In addition, the results of individual PIRs, as well as internal and external audits or reviews, should be

examined. Other types of analyses, such as total cost of ownership, can also provide useful performance data on specific IT portfolio categories, such as infrastructure O&M.

Activity 2: Aggregate performance data and trends are analyzed. Trend analysis and reports can help provide evidence that the investments in the IT portfolio have helped to achieve expected improvements in the effectiveness and efficiency of operations or service delivery. To make this a meaningful exercise, it is critical to develop baseline performance data. Activity 3: Recommendations for improving the investment process and portfolio are developed and implemented.

Based on its analysis of the data, the organization develops

recommendations for improving the investment process and the portfolio. After the board approves the recommendations, a plan should be put in place for implementing them. There are always problems in implementing change, but there are opportunities as well. Addressing problems or opportunities usually involves

• developing recommendations for the IT investment board; • documenting the decision criteria, justification, and rationale for

investment decisions;

• defining the expected benefits of the action recommended; • making a decision on implementing each recommendation; and • tracking the recommended action as it is implemented.

Managing the Succession of