SOBRE INNOVACIÓN DIDÁCTICA
EN RELACIÓN AL ALUMNO Y LA DIDÁCTICA PREGUNTA a
steps to be completed within the NES comparative evaluation procedure in the framework of the KIND project, which may be considered as a possible template for the representation and interpretation of calculation results that may be implemented in the comparison of real systems.
In devising Table 5.7, it was assumed that if the evaluation of a certain indicator for two alternatives in the relevant scale is the same, this indicator will be assigned a value of 0. According to the assumptions made about the form of the single-attribute value functions, this will not lead to a contribution of the corresponding indicators to the values of the multi-attribute value functions for the considered alternatives. Note that a 10 point scoring scale provides more subtle opportunities for evaluating the alternatives. For example, on a two point scoring scale, alternatives may have the same evaluation, while on a broader scale they may be awarded various scores.
5.2.3.3. Weighting factors
In order to evaluate the weights, it is assumed that at each level of the objectives tree, the importance of all high level objectives, the areas of evaluation and the KIs are identical. This option suggests that the experts and the decision maker need to agree that high level objectives characterized by the cost, performance and acceptability of the aggregated goals are equally important. Since evaluation of the weights is to be done based on consideration of the ranges over which the indicators may vary, it may be possible to achieve this goal of equal weights by adjusting the ranges over which the single-attribute value functions are defined. Expanding a range will typically increase the corresponding weight of an indicator, and vice versa.
TABLE 5.7. PERFORMANCE TABLE High level
objectives Areas KIs abbr. Qualitative evaluation Two point
scoring scale 10 point scoring scale
NES-1 NES-2 NES-1 NES-2 NES-1 NES-2
Cost Economics E.1 xa 1 0 9 1
E.2 ~b ~ 0 0 4 2
Performance
Waste management
WM.1 x 0 1 2 9
WM.2 x 0 1 1 10
WM.3 x 0 1 2 10
Proliferation resistance
PR.1 x 1 0 10 2
PR.2 x 0 1 1 10
PR.3 ~ ~ 0 0 2 3
PR.4 ~ ~ 0 0 4 3
Environment ENV.1 x 0 1 1 9
Country specifics
S.1 ~ ~ 0 0 3 1
S.2 ~ ~ 0 0 2 2
S.3 x 1 0 10 1
S.4 x 0 1 1 10
S.5 x 0 1 2 9
Acceptability Maturity of technology
M.1 ~ ~ 0 0 4 1
M.2 ~ ~ 0 0 1 1
M.3 ~ ~ 0 0 2 3
M.4 x 1 0 9 2
a x indicates that the NES provides the best performance for a corresponding KI.
b ~ indicates a KI for which both NESs show comparable performance, and which may be differentiated by using different scoring scales.
At the evaluation area level, in accordance with the high level objectives (cost (economics), performance (waste management, proliferation resistance, environment, country specifics), and acceptability (maturity of technology)), equal weighting factors were assigned to each evaluation area. These factors were determined based on the requirement that the sum of all the weighting factors for each area be equal to unity. As the table shows, the sum of weighting factors in one branch of the objectives tree is equal to unity.
At the final level (the level of the KIs), the weighting factors for each indicator included in the corresponding evaluation area were assumed to be equal as well. The weighting factors calculated in accordance with the above mentioned assumptions for each indicator are shown in Table 5.8. The table illustrates the procedure for hierarchical weighting. It can be seen that the final weighting factors obtained satisfy the condition that their sum be equal to unity. These values were used for a comparative evaluation in both the 2 and 10 point scoring scales.
5.2.3.4. Ranking results and their interpretation
The ranking results based on the above mentioned assumptions are presented in Fig. 5.8 for a two point scoring scale. The results from utilizing a 10 point scoring scale are discussed in the sensitivity analysis section. The alternative NES-1 is the preferred one, while alternative NES-2 is less attractive. The values of the multi-attribute value functions are 0.288 (NES-1) and 0.221 (NES-2); therefore, these alternatives can be considered with a certain degree of confidence as being distinguishable.
The results for decomposition of the analysis into individual components in accordance with the objectives tree structure are shown in Table 5.9, which makes it possible to interpret the ranking results. As shown in Table 5.9, the score for the aggregated high level objective performance is greater for the NES-2 than for the NES-1, but the scores for the aggregated high level objectives cost and acceptability are higher for the first alternative than for the second alternative. These advantages ensure that the first alternative is the preferred one.
A comparison of aggregated scores for all the evaluation areas provides an opportunity to examine the contribution of these areas to the values for the high level objectives. The corresponding evaluations are shown in Table 5.10 and Fig. 5.9.
The high level objectives cost and acceptability only include one evaluation area (economics and maturity of technology, respectively), while the corresponding values of the aggregated scores for the high and intermediate levels of the objectives tree are the same. In terms of the economics and maturity of technology areas, NES-1 is preferable to NES-2.
Regarding the corresponding area contribution to the high level objective performance, it should be noted that the first alternative is a less attractive option in terms of waste management, environment and country specifics, while these alternatives are identical in terms of proliferation resistance.
FIG. 5.7. Value path.
TABLE 5.8. WEIGHTING FACTORS High level
objectives High level
objective weights Areas Area weights Indicators Indicator weights Final weighting factors
Cost 0.333 Economics 1 E.1 0.5 0.167
E.2 0.5 0.167
Performance 0.333
Waste
management 0.25
WM.1 0.333 0.028
WM.2 0.333 0.028
WM.3 0.333 0.028
Proliferation
resistance 0.25
PR.1 0.25 0.021
PR.2 0.25 0.021
PR.3 0.25 0.021
PR.4 0.25 0.021
Environment 0.25 ENV.1 1 0.083
Country specifics 0.25
S.1 0.2 0.017
S.2 0.2 0.017
S.3 0.2 0.017
S.4 0.2 0.017
S.5 0.2 0.017
Acceptability 0.333 Maturity of
technology 1
M.1 0.25 0.083
M.2 0.25 0.083
M.3 0.25 0.083
M.4 0.25 0.083
FIG. 5.8. Ranking results.
FIG. 5.7. Value path.
TABLE 5.8. WEIGHTING FACTORS High level
objectives High level
objective weights Areas Area weights Indicators Indicator weights Final weighting factors
Cost 0.333 Economics 1 E.1 0.5 0.167
E.2 0.5 0.167
Performance 0.333
Waste
management 0.25
WM.1 0.333 0.028
WM.2 0.333 0.028
WM.3 0.333 0.028
Proliferation
resistance 0.25
PR.1 0.25 0.021
PR.2 0.25 0.021
PR.3 0.25 0.021
PR.4 0.25 0.021
Environment 0.25 ENV.1 1 0.083
Country specifics 0.25
S.1 0.2 0.017
S.2 0.2 0.017
S.3 0.2 0.017
S.4 0.2 0.017
S.5 0.2 0.017
Acceptability 0.333 Maturity of
technology 1
M.1 0.25 0.083
M.2 0.25 0.083
M.3 0.25 0.083
M.4 0.25 0.083
FIG. 5.8. Ranking results.
5.2.3.5. Sensitivity analysis
(a) Impact of single-attribute value function forms
The overall scores for different assumptions regarding the scoring scales and domains are given in Table 5.11.
For a two point scoring scale, both the global and local domains provide the same ranks for the alternatives.
TABLE 5.9. HIGH LEVEL AGGREGATED OBJECTIVE SCORES
Cost score Performance score Acceptability score Overall score
NES-1 0.167 0.038 0.083 0.288
NES-2 0 0.221 0 0.221
TABLE 5.10. SECOND LEVEL SCORES
Economics Waste management Proliferation
resistance Environment Country specifics Maturity of technology
NES-1 0.167 0 0.021 0 0.017 0.083
NES-2 0 0.083 0.021 0.083 0.033 0
FIG. 5.9. Evaluation area scores.
TABLE 5.11. OVERALL SCORES FOR SCORING SCALES AND DOMAINS
Two point scoring scale 10 point scoring scale
Local domain Global domain Local domain Global domain
NES-1 0.288 0.288 0.575 0.375
NES-2 0.221 0.221 0.325 0.266
Utilization of a 10 point scoring scale provides more significant differences between the overall scores because this scale offers more precise indicator evaluation (for the local domain, these differences are 0.250 for a 10 point scoring scale and 0.067 for a 2 point scoring scale). When comparing two alternatives with a local domain of single-attribute value functions, the ranking results are not sensitive to the form of the single-single-attribute value functions:
both alternatives have the same ranks for any form of single-attribute value function with a local domain. The same is true for global domains of single-attribute value functions within a two point scoring scale.
If global domains have been chosen for all single-attribute value functions and if a 10 point scoring scale is used, the probability that the first alternative will have the first and second ranks will be equal to 93% and 7%, respectively. This also leads to a decrease in the difference between the values of the multi-attribute value functions: the overall scores for the NESs are equal to 0.375 and 0.266 for the first and second alternatives, respectively. Obviously, this situation makes the alternatives less distinguishable. This example demonstrates how the distinguishability of the alternatives is improved as a result of the utilization of local domains for single-attribute value functions and a wider scoring scale for indicator evaluations.
(b) Weight sensitivity
The results of the sensitivity analysis carried out based on the linear weights method are presented in Fig. 5.10.
These results correspond to the weighting factors for the high level objectives of the objectives tree for a two point scoring scale. The values of the weighting factors, at which the lines corresponding to different alternatives are crossed, divide the range of the weighting factors into areas where the alternatives being considered have different ranks. NES-1 looks more attractive than NES-2 owing to the lower values (less than 0.4) of the weighting factors for the performance objective. Such an analysis can be performed for both the intermediate level of the objectives tree (evaluation area level) and the KI level.
The analysis of the impact of the indicators’ weighting factor values on the ranking results makes it possible to identify indicators that could ultimately provide a change in the ranks of the alternatives. These are the indicators numbered 3, 4, 5, 7, 10, 14 and 15. However, in order to realize this rank reversal, it is necessary to increase the weighting factor values for the corresponding indicators at least twice.
In conclusion, the results of an expanded sensitivity analysis regarding the weighting factors for the high level objectives are presented. The analysis implied simultaneous variation of weighting factors for the high level objectives over a wide range and identification of the most preferred NES for corresponding weight combinations.
The NES attractiveness for different weight combinations is shown in Fig. 5.11. Based on this information, the preference probabilities for NES-1 and NES-2 (the relative square of corresponding areas) may be evaluated, and these are 63% and 37%, respectively.
The case studies presented, describing the comparison of five or two hypothetical NESs, have demonstrated the basic steps to be completed for a multi-criteria comparative evaluation of NESs based on the MAVT method within the framework of the KIND approach and represent a possible template for presenting and interpreting calculation results that may be implemented in the comparison of real systems.
5.3. INNOVATIVE VERSUS INNOVATIVE NUCLEAR ENERGY SYSTEMS