La Protección Civil

Scorecards have something in common with people and the products they make . . . Even if they are maintained in ideal conditions, they can still get old and tattered. Companies may schedule regular redevelopments, even though the extent of economic, market, operational, and other changes has been minimal. This reduces emphasis on intensive scorecard monitoring, and ensures that the scorecards are aligned with the current business. This would not have been possible, even in the 1990s. As the costs and hassles of scorecard developments have reduced, especially as companies are moving the function in-house, they are being done more frequently, often every 18 to 24 months. Even so, some lenders will still use the same scorecard for five years or more. By this time, the business should be asking whether scorecards are applicable at all within that environment, or whether some other option could serve its needs better.

3.5 Summary

This section has focused upon some of the mechanics of credit scoring, in order to provide an overview of several topics presented in subsequent modules. The first issue was the scorecard appearance. Most lenders use traditional scoring models that: (i) break various characteristics (like ‘applicant age’) into attributes (like ‘less than 30’); and (ii) assign points to each attribute, such that the total score for each case provides a measure of risk relative to other cases.

Desirable customers will get high scores, while those less palatable get low scores.

There are a number of different statistical techniques that can be used to develop scoring models. Traditional models are associated with parametric techniques, which unfortunately make certain assumptions about the underlying data that are not always true. In the early days, DA and LPM were the primary choices, largely because of computational speed.

Unfortunately, they are considered ill-suited to situations with binary outcomes (good/bad, default/not-default), and logistic regression is now favoured. Even so, most results compar-isons show there is no clear winner. The differences in ranking ability are small, as the manner in which DA and LPM are applied tends to address the assumption violations. It is also pos-sible to use non-parametric techniques that require no data assumptions. Machine learning techniques, such as NNs, genetic algorithms, and K-nearest neighbours, suffer from a lack of transparency, and are prone to overfitting. Of these, neural networks are the most commonly used, primarily for fraud scoring. Decision trees are powerful analysis tools, but provide poor results, because they require huge amounts of data.

While a lot of time and effort is spent on obtaining data, developing scorecards, and design-ing strategies, gremlins can creep into the system. Biases can arise, no matter whether humans or machines make the decisions. There will always be a flat maximum, in terms of quality, that no model can exceed, and it can only be hoped that it passes a less stringent ‘reasonable-model’

test. Bias exists to the extent that a model’s quality falls short of this level. While some of it results from assumptions made during the scorecard development process (sample selection, transformation), it is more likely to arise because of data issues (poor data quality, lack of access to key data sources), or misapplication of the final model.

90 Module A : Setting the scene

Where the bias is greater than what is considered acceptable, other options are available.

First, cases can be scored for guidance only. This applies especially where the potential loss or profit is high, and key information cannot be captured in the score. Second, there may be generic scores available to supplement internal data. For many small lenders, the cost of developing bespoke scorecards is not worthwhile, and they may rely solely upon bureaux’ generics. And finally, underwriter experience can be used to develop an expert model. This can provide a viable alternative in instances where no generic exists, or where there are one or more bespoke or generic models—and possibly other information—that can be integrated into a hybrid.

When measuring the results, there are a number of different aspects. Given that credit scor-ing is used to drive business processes, lenders need to know what value it is providscor-ing in those processes, both in terms of selection (accept/reject rates), and performance (good/bad/default rates). At the same time, they may also wish to use the scores further in risk-based pricing and finance calculations. Credit scores can be used to derive default probabilities, and other meas-ures can be derived for loss severity (a function of EAD, LGD, and remaining maturity). To ensure that they are working, assessments can be done of scorecards’: (i) power, their ability to discriminate according to risk; (ii) accuracy, how close the estimate is to the actual result;

and (iii) drift, the extent to which the power and accuracy change over time. Power and drift can be measured using measures of separation/divergence, the primary ones being the Gini coefficient, Kullback divergence measure, chi-square statistic, and K-S statistic. Accuracy is usually assessed at the portfolio level, such as changes to overall default rates, but it is also possible to use binomial probabilities, the Hosmer–Lemeshow statistic, or the log-likelihood measure.

The scorecard development process is quite a long one, especially for greenfield develop-ments. Lenders must first decide upon their objectives, which may include process efficiencies, increased market share, and/or reduced bad debts. A feasibility study helps to determine whether or not this is possible, and key players need to identified. A critical component of the process is data preparation, which includes: data acquisition, good/bad definition, observation and outcome windows, and sampling. Thereafter, scorecard modelling requires: data transform-ation, variable selection, reject inference, segmenttransform-ation, and training. Once completed, the scorecards can be finalised, which requires: scorecard validation and sign-off, calibration, strategy setting, loading, testing, and post-implementation monitoring. Decision making and strategy issues include: level of automation, change management, overrides, policy rules, refer-rals, and strategy enhancement. Finally, there are also security issues, relating to: documenta-tion, confidentiality agreements, and change control.

Credit scoring relies on a base assumption that the future will be like the past. It is usually sufficiently true, but drift may occur that affects scorecard and system performance, including changes to: (i) the economy; (ii) the market being serviced; (iii) lenders’ infrastructure; and (iv) borrowers’ attitude towards credit. Changes in available technology will also play a role, and lenders may wish to replace scorecards because new or improved data is available.

Likewise, new systems may be implemented that improve operational performance. In any case, lenders may nonetheless opt to replace scorecards for no specific reason, other than to ensure they are current, and maintain a competitive edge.

This page intentionally left blank