The FEH (Vol. 3, Section 17.3.2) tested the goodness of fit of various candidate families of distributions, which suggested that the GLO distribution would be a
generally applicable distribution for flood estimation in the UK. This test of fit was based on the work by Hosking and Wallis (1997: Section 5.2). A later report (Morris, 2003) raised the concern that the test of fit, as used in the FEH, was structured in such a way that the estimates of L-moment ratios used as the “pooling-group estimates” were calculated using a simple weighting scheme that was not the same as that put forward as the weighting scheme suggested to users of the FEH methodology, and concerns were raised that the results might be somewhat affected, or that at least there was some inconsistency.
7.1
The Hosking and Wallis test
It should first be noted that, while Hosking and Wallis (1997) proposed their suggested test in a pooling-group context (“regionalisation” in their terminology), the test is
applicable even to records for individual catchments. It is therefore of interest to consider in general terms the effect of the number of catchments in the pooling-group on this test, as this gives some guidance regarding the importance of the weighting scheme used within the test. The test is a comparison of the raw sample-based estimate of the L-Kurtosis with the value of the L-Kurtosis predicted by a fitted model. This difference is scaled by a value for the standard deviation which essentially measures how well the difference is estimated from the data contributing to the estimate of the difference. In the present circumstances, one may think of the
difference in the L-Kurtosis values as being relatively fixed (if there really is a lack of fit) as more catchments are added to the pooling-group, while the variability of the
difference decreases (because a better estimate of the difference is obtained by using data from extra catchments). Thus the standard deviation used for the devisor would go down and larger values of the test statistic would result, leading to more rejections of the hypothesis of an adequate fit, since the test-statistic is judged against a fixed critical value. The use of a larger pooling-group effectively increases the power of the test. However, the size of the pooling-group needs to be restricted to a size such that the assumption used within the test remains appropriate. Specifically, that it is
reasonable to use a single common distribution to represent the standardised flood distribution for all catchments in the pooling-group.
The above considerations can be extended to consider the effects of spatial
dependence on the test results. The values of the standard deviation used in the test are obtained by simulation of independently distributed flood-values for the catchments before these are combined, via weighted averages, into estimates of the 3rd and 4th L- moment ratios for the pooling-group. However, the presence of spatial dependence in the real data, and its absence in the simulated data, means that the simulations will under-estimate the variability of these pooled L-moment ratios. The standard deviation used as the divisor in the test will therefore be too small compared to the quantity that should ideally be used in the test. Thus (positive) spatial dependence will tend to lead to higher (more extreme) values of the test statistic, and this will lead to the null hypothesis that a given family of distribution fits being rejected too often compared to the target frequency for false rejections.
The role of the specific weighting scheme used to estimate the pooled L-moment ratios can also be considered. Firstly, it is important that exactly the same weighting scheme is used in calculating both the L-moment ratios used to calculate the difference of the L-Kurtosis values for the actual data, and for the equivalent steps when applied to the
simulated data. This has always been the case. Secondly, given this assumption, the effect of changing the weighting scheme for a given number of catchments will be similar to changing the number of catchments used in a fixed weighting scheme. Thus some weighting schemes may give more precise estimates of the difference of the two L-Kurtoses and lead to more power for the test. Using a weighting scheme within the test that is not “optimal” does not invalidate the test.
It should also be recalled that the test statistic suggested by Hosking and Wallis (1997) has been used not only for formal tests for whether there is enough evidence to reject the choice of a given 3-parameter family of distributions, but also as a way of indicating which of a number of families is “best”. In this instance, for a given pooling-group, equivalent test-statistics are calculated for a number of candidate families and the family for which the test-statistic is smallest (or indicates least lack-of-fit). This usage should not be badly affected by the problem relating to the inadequate representation of spatial dependence of the annual maximum values, since the statistics for each of the families should be affected roughly equally.
7.2
Revision of the Hosking and Wallis test
On examining the principles behind the test of lack-of-fit as set-out by Hosking & Wallis (1997), a number of points arise. Some of these points are treated in more detail here. These considerations have led to the formulation of an alternative test-statistic which looks superficially similar to that of Hosking and Wallis, but the details of the
calculations are rather different. A simulation-based study similar to that reported by Hosking and Wallis (1997; Table 5.2) has shown that the version of the statistic adopted here has properties which are superior to those of the original, in terms of having a much better match to the target acceptance rate of 90 per cent when the test is applied to cases where the distribution being tested is the same as the distribution from which the simulated data were generated
As discussed above, it seems likely that the effect of spatial dependence would mean that the variance estimated from independent samples would be too small and thus that more “rejections” of the individual tests would occur than the notional frequencies of 90 per cent acceptances and 10 per cent rejections for a critical value of Z of 1.64. The relative acceptability of the candidate distributions should be unaffected. In
contrast, the effect of heterogeneity should be broadly neutral, provided that the distributions associated with each site are treated as fixed in the simulations.
Hosking and Wallis (1997) define the basis of their test-statistic in their Equation (5.3) in the following way, although a modified notation is used here. Firstly, the test is based on the idea that, for the 3-parameter distributions being treated, the theoretical value (according to the fitted distribution) of the L-Kurtosis can be evaluated and compared with the sample estimate of the L-Kurtosis obtained directly from the data. The existing methods of fitting the 3-parameter distributions that are being considered do not make any use of the sample L-Kurtosis and the basis of the test is to compare the sample L- Kurtosis with the model-derived L-Kurtosis for the fitted model. In practice, these model-derived values for the 4th L-moment ratio, t4DIST , are obtained as a fixed
(distribution-dependent) function of the L-Skewness (3rd L-moment ratio):
)
(
34
h
t
t
DIST DIST=
where hDIST is the function that gives the theoretical L-Kurtosis in terms of the
theoretical L-Skewness
)
(
3 4τ
τ
DIST DISTh
=
,and where t3 is the sample L-Skewness. The basic form of the test statistic is defined as 4 4 4
σ
DIST DISTt
t
Z
=
−
where σ4 represents a standard deviation to be discussed later. The sample L- Kurtosis, t4, and the sample L-Skewness, t3 (from which t4DIST is derived) are both
derived by a pooling-group scheme if more than one catchment is being considered, otherwise the usual single-catchment estimates would be used.
Note that Hosking and Wallis present a revised formulation (their equation (5.6)) which, with a reversal of sign to accord with the above, gives the final version of the test statistics as 4 4 4 4
σ
B
t
t
Z
DIST DIST=
−
−
where B4 is a bias correction term. In the revised version used here, the bias correction
term is much smaller than in the original and can be omitted without much effect. Hosking and Wallis (1997) gave a complicated expression for
σ
4, involving B4, but thiscan be simplified to being identical to the sample variance of certain simulated quantities. In addition, Hosking and Wallis’s equation (5.6) is given with
τ
4DIST instead oft
4DIST, presumably to indicate that the value is treated as fixed (see below).According to the approach of Hosking and Wallis, σ4 should be the standard deviation of t . However, it is arguable that 4 σ4DIST should be the standard deviation of
DIST t t4− 4 ,
which might well be a rather smaller quantity. An alternative is that
σ
4DIST should be the conditional variance of t given 4 t4DIST, but this would be rather more complicated to turn into a practical procedure. The question here is what should be treated as being the test statistic. The choices are t ,4(
t4−t4DIST)
or(
t4t4DIST)
. One of the revisions to theprocedure that has been adopted here is to treat
(
t tDIST)
44− as the test statistic.
The Hosking and Wallis procedure is to test several families of distributions
simultaneously for their lack of fit and to do so using a single base set of simulations from a Kappa distribution (which is a 4-parameter family of distributions). Thus the simulations are for a distribution which does not have theoretical L-moment ratios that correspond to
(
t t tDIST)
4 3,
, , but rather has L-moment ratios
(
t,t3,t4)
. While somearguments can be made that support this, it seems better to perform separate sets of simulations using whichever distribution is being tested to generate the simulated data. This eliminates several approximations and correction-terms that are required in the argument needed to support the use of a single common set of simulations.
7.3
The test procedure
The procedure for testing the goodness of fit of a given family of distributions is as follows.
• Calculate the observed test statistic
(
DIST)
obs t t
T = 4− 4 .
• Calculate a number, N, of simulated versions of the test statistic
{
T( )i i N}
sim; =1,K,using Monte Carlo simulations. Each of these simulated test statistics is calculated by constructing a set of data of the same size as the observed data (in terms of the number of sites in the pooling-group and the record lengths) independently between years and sites, from the distribution in the given family
which has the observed L-moment ratios
(
t3, t4)
and a unit mean. In particular,this means calculating simulated versions of t and 3 t and then using the former 4 to calculate t4DIST =hDIST(t3) from the simulated value of
t
3. Finally, the simulated values of the test statistic is calculated as ( )i(
DIST)
sim t t
T = 4 − 4 .
• Calculate the sample mean B , and the sample variance, 4 σ42, from the set of
simulated test statistics
{
T( )i i N}
sim; =1,K, .
• Calculate the test statistic
4 4 4 4 σ B t t Z DIST DIST = − − .
• Compare the absolute value ZDIST with 1.64, and count the fit as acceptable if
64 . 1 ≤
DIST
Z . Otherwise reject the particular family of distributions for the particular pooling-group.
As noted earlier, the bias correction B4 is small and can be omitted. It is important that the test statistics carried over from the individual simulated data sets are the
differences ( )i
(
DIST)
sim t tT = 4 − 4 and not just the L-kurtosis t as used by Hosking & Wallis 4 (1997).
7.4 Results
This section summarises the results obtained by applying the test procedure outlined above in Section 7.2 to pooling-groups formed as outlined in Chapter 6 (considering the catchment to be ungauged) for each of the 602 catchments used in this study. The following five 3-parameter distributions were considered as possible candidate
distributions:
• Generalised Logistic (GLO).
• Generalised Extreme Value (GEV).
• Generalised Normal (GNO), also known as the 3-parameter Log-Normal. • Person type 3 (PE3).
• Generalised Pareto (GPA).
For a further description of each of these distributions, please refer to the FEH (Vol. 3, Chapter 15) or Hosking and Wallis (1997).
The results of the analysis are summarised in Table 7.1, where the first row (labelled “Chosen”) contains the number of times (out of 602) that a particular distribution was chosen as the preferred option (smallest value of ZDIST ). The second row (labelled
“Accepted”) contains the number of times a particular distribution gave a value of the test statistic satisfying ZDIST ≤1.64. Finally, the last row (labelled “Rejected”) counts the number of times a particular distribution was rejected.
Table 7.1 Results of the goodness of fit test applied to pooling-groups formed for each of the 602 catchments.
Test GLO GEV GNO PE3 GPA
Chosen 283 167 106 46 0
Accepted 364 358 339 209 0
From the results in Table 7.1, it is clear that the GLO distribution remains the best choice for a default UK distribution as it is both chosen and accepted more often than any of the other candidate distributions. However, the results for the two distributions are somewhat closer than reported for the FEH study. The numbers of catchments for which the GLO and GEV distributions are “accepted” are almost equal, but the
comparison favours the GLO marginally. While it appears that the main distinction in the results between the GLO and GEV distributions lies in the number of times that the distribution is chosen as having the “best” fit, it should be recalled that this comparison is affected by which other distributions have been included in the competing set. It is not clear how many of the catchments which have the GNO, PE3 and GPA
distributions as their “chosen” distributions would select the GLO if the only options were GLO and GEV.
8
Summary of new flood
estimation procedures
This chapter provides a short summary of the new procedures introduced in this project. While maintaining the conceptual basis of the index-flood method, as
implemented in the FEH, the work undertaken in the current project has improved the estimation of QMED and the growth curve at both gauged and ungauged catchments.
8.1
Estimation of QMED
The recommended method for estimating the index flood (QMED) depends on whether the subject site is a gauged or an ungauged catchment.
8.1.1
Estimating QMED at a gauged catchment
Detailed guidelines for estimation of QMED from flood data were provided as part of the FEH (Vol.3 Ch. 2). No further investigation into this aspect of the method has been undertaken as part of this study. Note that for the development of the regression model linking QMED to catchment descriptors (the QMED model) in the current study, all sample values of QMED were estimated as the median of the AMAX series regardless of record length. Also, the QMED values were not subjected to adjustment for climatic variation as in the FEH.
8.1.2 Estimating QMED at an ungauged catchment
When no flood data are available at the site of interest, QMED has to be estimated either from catchment descriptors (possibly including data transfer from a nearby gauged donor catchment) or using some other method.