4.3.1 Finding Forecast and Verification Information
Participants had some difficulties in finding the probabilistic forecasts and verification information. Only three out of twelve groups were able to find the information. This was the China Meteorological Administration (CMA)/Bejing Climate Centre (BCC), Met Office (United Kingdom) and APEC (Asia-Pacific Economic Cooperation) Climate Centre (APCC), Republic of Korea. The main barrier was the lack of user friendliness the web pages offered. Most of them are difficult to browse and not intuitive when it comes to finding information. Also the language was a problem: Meteo-France is in French and the Centre for Weather Forecasts and Climate Studies/National Institute for Space Research (CPTEC/INPE) and KMA are only partly in English. Another hindrance to access the information was that some GPCs required the user to log in (e.g. Meteo-France and APCC). In general the participants were surprised about the degree of difficulty in accessing the forecast and verification information, which they shared in the discussion round. The outcomes reflect the difficulties to find the forecast and verification information on the GPCs websites. It is of great importance to optimise the search options to facilitate to access this data, for instance, by creating a standard outlay website for all GPCs to help the user navigate through the information. GPCs that require a small number of clicks to find the forecast graphics are generally easier to handle than the ones with more than five clicks.
4.3.2 Forecast and Verification Information in Decision-Making Processes
For the second exercise, participants had to interpret the forecast and verification graphics in order to take decisions regarding the management of water reserves for energy generation. The reference period of DJF 2009/2010 was used for a precipitation forecast with one month lead time. This task was interesting for participants because they were asked to integrate probabilistic forecasts and verification information in their decision-making process.
4.2). One group decided that the forecast shows a big uncertainty and that they would not use the forecast information for this year. The other group stated that they were going to use the water now because the most probable tercile is above-normal and the skill score lies between 0.7 and 0.8, which means that the skill is relatively high.
One group analysed the IRI GPC (see Appendix 4.3). They concluded that the data was clear to understand. The forecast indicates a 40-45% below-normal precipitation for the season. Although the verification does not show forecast skill, the group decided to use the hydropower water resources now.
The next two groups had the graphics from the ECMWF centre (see Appendix 4.5 and 4.4). The first one found out that the forecast indicates a 40-80% probability above normal precipitation in the whole region. The verification does not forecast skill – it is not better from using the long-term average distribution know as climatology. Therefore, this group would not use the forecast in their decision-making process. Instead, they would make use of climatology data to make decisions.
The second group could not answer the questions. There were too many unfamiliar terms in the graphics in order to identify the forecast and verification information and to make a decision. The participants did not know what a tercile, System 3 and DJF meant. Also they did not know what the percentages mean and the abbreviation “prob.” stood for.
The next group analysed the APCC Centre (see Appendix 4.6). There was no verification data available, only the forecast information. They pointed out that the forecast shows a 60-70% decrease in precipitation in DJF 2009/2010. For hydropower planning the available data especially the lack of verification is not enough for decision-making, but still it shows some decrease in rainfall. They decided that the reservoirs should be kept filled for the coming season for normal operation of power plant reducing the downstream flow in case there is a below normal precipitation season.
The last two groups were given the CPTEC forecast and verification graphics (see Appendix 4.7 and 4.8). The first group identified that the forecast indicates a severe decrease of precipitation. The verification did not support the forecast very much.
This group would therefore not use the forecast information for their decision-making process and chose to reserve the reservoir water to insure sufficient power over winter for energy needs.
The second group answered that the forecast and the verification predicted, that there will be a likely below-normal rainfall. They concluded that it is best to use the hydropower in the next season and not at present.
Note that there is a full range of decision taken based on the different GPC data. This also highlights the human aspect on the use of probabilistic information versus the amount if risk an individual is willing to take.
4.3.3 Feedback from Participants regarding Visualisation Techniques
The participants were given the final section of the questionnaire (section3.2), which deals with the visualisation of forecast and verification graphics. There are the four maps, two forecasts and two corresponding verification maps, which the interviewees were given. Each graphic has five questions concerning the graphic’s comprehensibility, the legend, the lettering, the choice of colours and the limitations.
The first graphic shows the probability wind speed forecast (IC3) (see Figure 6). Participants answered that the comprehensibility was “ok” and that there is not too much information in the graphic. One added that the explanation for the title is missing.
The second question regarding their evaluation of the legend showing below/above/normal forecast categories had similar results. Overall it was considered as “good”. Two answered that the percentage (%) unit is missing for most likely category and one included that the white part of the legend has to be divided into percentages as the other colours do.
The lettering was also seen as “ok” but it can be specified what MAM means. As for the colours, the opinions were different. Half of the participants answered that the use of the colours is “clear”. The rest indicated that in their opinion the colours are not adequate. Suggestions were made to include units, explain terminologies such as MAM, tercile and add a little explanation.
The next graphic shows the ensemble-mean correlation of the probability wind forecast (IC3) (see Figure 7). All participants answered that there is enough information in the graphic concerning the comprehensibility. One added that the map scale is too big for this kind of information. The legend was criticised because there are no units, also are the labels that indicate the latitude and longitude missing.
The lettering was evaluated as “good” and “ok” but it was also mentioned that it is necessary to explain what is being is correlated.
The colours were considered “fine” and “ok” but also as “bad” because they do not depict exact numbers in the map.
There are several deficiencies identified: An explanation about the fields being correlated and the significance of correlation should be included. There should be latitude and longitude labels so that everybody can read the map and locate the countries and refer to them. Also there is a lack of description. The colours could be more discrete to make the scale visible and more understandable.
On the following graphic the probability precipitation forecast from the GPC IRI can be seen (see Figure 8). Most of the participants said that the map is comprehensible. One answered that it is difficult to understand because it is not clear what the colour white stands for. Another one added that it remains unclear why in the normal range the percentage is 40%. Also it was said that the scale is too big.
The map’s legend, which illustrates the below-normal, normal and above-normal categories, was evaluated from “good” and “excellent” to “very confusing”. One was not sure if all categories of the scale are present on the map.
The title /description was overall considered as “good”.
The colours were evaluated from “ok” to “poor” and really confusing because you cannot distinguish between normal and above-normal and they are not intuitive. As deficiencies, the colours were named. They should be changed and be more discrete. In particular it was suggested that there should be more colours in the normal category. The white colour and the dry season D, should be marked bigger and explained more. The description was also said to be too brief.
The related verification graphic represents the generalized ROC (GROC) of the precipitation forecast (IRI) (see Figure 9). The comprehensibility and the legend of this verification graphic were “fine” and “clear”. The lettering is “good” but the
meaning of ROC is missing and it is necessary to specify what fields are being correlated.
The colours are seen as “good” but maybe not intuitive (red would be best).
Suggestions were, to include an explanation of GROC and ROC. One participant especially liked this graphic because of the detail it provides and how the colours are presented. He suggested that non-scientists would be able to use this information.
The next chapter links the results to the existing scientific debate, and gives answers and possible conclusions as well as recommendations to the key questions of this thesis.