The success of LTCM’s model-based strategy meant that its capital base grew rapidly. Pressure grew to invest that capital, but there were only limited numbers of convergence trades available for the fund to undertake. This led LTCM to move into more aggressive trading strategies which were different from the convergence trades on which its reputation and success had been built. By late 1997 the firm was taking extremely large positions in areas such as merger arbitrage in which bets are taken on whether or not corporate mergers would be completed.
In 1998 LTCM lost $4.6 billion in less than four months following the Russian financial crisis, and the fund closed in early 2000. In its annual report, one of its corporate investors, Merrill Lynch, observed that mathematical risk models ‘may provide a greater sense of security than warranted; therefore,
reliance on these models should be limited’.
LTCM’s experience shows how misuse or over-reliance on a model, especially attempts to use its output in ways that are inappropriate, can lead to highly sub-optimal outcomes.
In any firm, the simplifying or limiting assumptions that were made when building its models need to be well understood by those using them.
These limiting assumptions include:
• The shape of any underlying distribution used by the model – for example, if market returns are assumed to be normally distributed, then the model will not be suitable for instruments whose returns are not normally distributed.
• The relationship between the past and the future – although investors are constantly told that past performance does not provide a guide to future returns, many models do assume that the future will be similar to the past. This assumption is especially important in the context of model risk because it may be hidden from the end users of the outputs.
• The state of the business environment at the point when the model was designed – unless models are updated in line with changes to the business environment, their usefulness over time will be limited.
• For example, a firm might use Value-at-Risk (VaR) to monitor market risk. When the VaR measure was implemented, perhaps the firm’s traders took little or no basis risk (see Chapter 5 on Market Risk) so it was coded with a fixed spread assumption. If traders have since started taking significant basis risk, they may not realise that the model is failing to capture it.
• Another example, might be a brokerage firm that is expanding its derivatives operation into emerging markets. If the firm fails to modify its pricing models to reflect the lack of liquidity in these markets, it will underestimate the cost of hedging its positions.
Another limitation when using models is the tendency to focus on the maximum loss at less than the 100% confidence level. For example, knowing that the loss at the 95% confidence level is £5 million tells the model’s users nothing about how much could be lost in the five days out of 100 on which that value is exceeded – except that it will be upwards of £5 million. For this reason, stress testing is extremely important as a means of understanding extreme outcomes which the model may fail to capture.
Decision-makers need to understand the limitations of a model to avoid using it in ways that are not consistent with the original intent. Limitations come in part from weaknesses in the model due to its various shortcomings, approximations and uncertainties. Limitations are also a consequence of assumptions underlying a model that may restrict the scope to a limited set of specific circumstances and situations.
1.2
Examples of Some Commonly Used Models
Learning Objective
8.1.2 Know the major models utilised in operational, credit, market and liquidity risks
The following common risk models will be described in this section:
• An operational risk scenario model. • Credit Value-at-Risk.
• Market Value-at-Risk. • Liquidity-at-Risk.
Model Risk
185
8
Risk workshops are typically held with senior risk and business personnel to ensure good quality model inputs, and the participants are asked to consider plausible but unlikely scenarios that could have a significant impact were they to occur.
The workshop outputs capture, amongst other things, estimates for each scenario of:
• the frequency with which such a scenario might occur • the typical loss were it to occur
• the severity of the loss that could occur in an extreme case.
In order to arrive at the level of capital necessary to cover the potential scenarios, the loss impacts and frequencies are subjected to a modelled stress. Key to this stress is the conversion of the impacts for each scenario into a non-normal distribution to reflect the fact that, should it occur, (a) the actual impact could be greater than the extreme impact estimate, and (b) it could occur more than once in a given year.
Operational risk loss events are not ‘normally distributed’, but instead follow a distribution that exhibits
fat tail characteristics. This is intuitively the case in so far as firms have far more low level losses than
catastrophic ones. There are various commonly used distributions which exhibit ‘fat tail’ characteristics, and scenario impact distributions are often simulated with a lognormal distribution.
Having established the estimates from the business areas, it is important to account for the fact that the scenario participants would typically be able to draw on only about 20 years of experience when assessing the impacts. In other words, their extreme impact estimates represent only a one in 20-year view (ie, a 95% confidence level).