AH group

Option selection is the process through which strategy recommendations are determined, and so many experts see it as the be-all-and-end-all of a strategy. As can be seen in Table 2, six of the nine considered case studies utilised a multi criteria assessment (MCA) process to select recommended options. None of the considered case studies utilised cost benefit analysis.

Findings show that: (i) MCA weightings should reflect strategy objectives; (ii) option selection processes should not preclude conventional options; (iii) sometimes multiple stages of option assessment is appropriate as this allows more options to be initially considered in lower detail, but primary criteria should remain consistent; (iv) care needs to be taken when allocating scores in relation to a baseline; (v) dividing MCA scores by an appropriate unit can calculate efficiency and allow comparison of small and large options.

Reflecting objectives in option selection

IUWM strategies generally identify a small number of key objectives, but do not always place high emphasis on these objectives during option selection.

The overarching objective of the case studies was “best community outcomes”, but they also each identified some more specific key objectives such as: water supply security, increased engagement with the community, and protection of a particular ecosystem. However these key objectives were poorly reflected in the weightings of the MCA.

In SA Water’s Long Term Plan for Eyre Region (A5), an engaged community was highlighted as a key objective of the strategy. However, the MCA that was used gave “community acceptance” a weighting of only 1%. The recommended option of this strategy received a poor score for community

acceptance, but this had little bearing on the total score. The result was a recommendation that could be seen to contradict a key objective of the strategy.

Strategy recommendation 13: It is important to apportion strategy implementation

responsibilities between stakeholders during the strategy. This is especially important when private companies, like developers, are involved.

Strategy recommendation 14: Stakeholder responsibility mapping should be conducted early in the strategy process to avoid later difficulties with approvals and implementation.

Strategy recommendation 15: Future MCA weightings should be strongly focused on reflecting the goals, values, and objectives of a strategy.

IMPROVING PLANNING PROCESSES FOR IUWMINFRASTRUCTURE

Water Research Australia Ltd – PhD Project 4504 19

Fair consideration of traditional options

Traditional options can often be the best way to meet the objectives of an IUWM strategy. If a strategy aims to achieve best community outcomes, then all options for achieving this should be fairly

considered, “integrated” or otherwise.

Following the MCA in the Footscray IWM Investigation (A4) the “non-integrated” base case received a significantly higher score than other option but was not selected as the preferred option. This was justified by City West Water by saying that the non-financial benefits of an integrated option could not be fully incorporated into the MCA.

The authors believe that in principle, once a decision making process is agreed upon, the outcomes of that process should be accepted.

Multiple phases of assessment

MCAs can be used for more than just option selection; they can also be used at earlier stages of the planning process for shortlisting options. During the Footscray IWM Investigation (A4) City West Water were able to assess a greater number of options by using a brief “high-level” MCA during the option shortlisting phase and a more detailed MCA during the option selection phase. A two-tiered MCA been used as part of the two assessment phases in this strategy, with both phases using the same primary criteria with the same weighting. The researchers believe this is preferable method as the MCA can largely remain the same, with more detail added in for later stages through the

secondary criteria.

It is conceivable that if planners use different primary criteria weighting at different stages, an option which would be the preferred option as assessed by the final assessment could be eliminated at an earlier stage.

Using baselines in Multi Criteria Assessment scoring

When scoring an option in an MCA it is very important to have a baseline from which all the options can be judged. Baselines should be consistently applied across both criteria, and also between options. Baselines can be set at upper and lower limits, or they can be set at a midpoint with positive and negative scores on either side. Either way the baseline should be set entirely independently of option specifications.

In the Footscray IWM Investigation (A4), City West Water gave each option a score of 0 – 1 relative to the highest and lowest score for each quantitative criteria. Four options were assessed, the best was scored 1, the worst 0, and the other two were interpolated between the best and worst. This form of base lining gives no indication as to how the options compare with best practice, and can

inappropriately over or understate the differences between options, i.e. options could be very similar or different in a criteria, but in each case one option will be scored 0 and another scored 1.

SA Water’s Long Term Plan for Eyre Region (A5) inconsistently applied a baseline scoring method across options. One option was given a 2/5 (low being good) score because, although it did provide a benefit, it was not entirely within their control. Another option which had no impact on that criteria was given a 1/5 (perfect score) for having no negative impact. Therefore an option that would have had a net positive impact was scored worse than an option with a neutral impact.

Strategy recommendation 16: The highest scoring option in an option selection process should form the basis of the strategy recommendations, whether it is “integrated” or not.

Strategy recommendation 17: When using two phases of multi criteria assessment, it is

important to have two tiers of criteria (primary and secondary). The primary criteria should remain the same for both phases, while the secondary criteria are adapted.

INDUSTRY FINDINGS REPORT

20 Water Research Australia Ltd – PhD Project 4504

The Northern Growth Area IWCM Plan (A9) provides a good example of using baselines in MCA scoring. Every option was not assessed in relation to one another but rather against best practice.

Each option was scored 50/100 if it met current best practice for those particular criteria, and could score above or below this point.

Comparing large and small options

It is difficult to use the same MCA when comparing large and small scale options. Small scale options will always have a less significant total impact (positive or negative) than large scale options. To account for this SA Water’s Long Term Plan for Eyre Region and the Northern Growth Area IWCM Plan (A5 & A9) divided their total MCA scores by water yield and financial cost respectively, the key objectives of each strategy. Using this method SA Water and Yarra Valley Water were able to see that while smaller options might have a smaller net impact they could provide greater efficiency.

2.1.6 Implementation planning

After specific options are recommended, a plan needs to be developed for how these options will be implemented. Currently only four of the nine strategy case studies have been implemented to some degree, but it is expected that over time this should increase to seven. The case studies have revealed that: (i) using implementation plans with specified trigger points can help to reduce risk associated with future uncertainties; and (ii) formalising an implementation plan without commitment to action steps can result in reputational damage and disgruntled stakeholders.

Implementation trigger points

As strategies take a long term view, up to 50 years, during the strategy it is often very difficult to schedule approximately when options should be constructed. Scheduling effectively many years into the future is reliant on growth, and climate modelling, which is often inaccurate. The result is that when future projections turn out to be inaccurate, the actual projects that are constructed may end up being substantially different to what is scheduled.

Instead of scheduling when certain options should be implemented, Water Forever South West (A7) put forward trigger points that would instigate detailed project planning. These trigger points were aligned with the water supply and demand balance of the system. The result was less risk of being locked into building inappropriately sized infrastructure based on inaccurate climate or growth estimates.

While this is the only case study to use implementation trigger points, the authors believe that this practice could be beneficial in a wide variety of situations. In particular it is considered that growth could be used as a trigger point in future implementation plans.

Strategy recommendation 18: MCAs require baselines to rate options logically and consistently across criteria.

Strategy recommendation 19: Dividing the MCA score by a value that represents a key objective gives planners the ability to compare small and large scale options.

Strategy recommendation 20: Using trigger points in an implementation plan mitigates the risks associated with being locked into a documented schedule which is based on potentially

inaccurate growth and climate estimates.

IMPROVING PLANNING PROCESSES FOR IUWMINFRASTRUCTURE

Water Research Australia Ltd – PhD Project 4504 21

Implementation planning in high-level objective setting strategies

High-level objective setting strategies align the visions of all stakeholders to help them work together and do not always require an implementation plan. Total Watermark (A3) was an update of an earlier strategy of the same name and was completed using a very similar process to the earlier one, except with one key difference. The newer version did not include an implementation plan. The City of Melbourne noted that much of the earlier version’s implementation plan had not been followed through on, largely because the implementation plan itself was not a key focus of the strategy and therefore did not have a lot of resources invested in it. They decided that there was a reputational risk in recommending projects without confidence that they will eventuate.

Strategy recommendation 21: An implementation plan should only be included in a strategy following thorough deliberation and if it has been determined that there is a strong likelihood of implementation.

INDUSTRY FINDINGS REPORT

22 Water Research Australia Ltd – PhD Project 4504