𝐵𝑎𝑑 𝑅𝑎𝑡𝑒 = 𝛽0+ β1Credit − score + C
This SLRM model helps lenders observe bad rate - credit score relationship over time thereby influencing the change of credit acceptance policy by either tightening or loosening the cut-off or threshold levels. Credit policy change in turn has bearing to the probability of missingness, a framework crucial for the eventual application of the proposed Bayesian reject-inference method: Bound and Collapse.
Since the sample was small to guarantee sample split into development and testing samples, we used 100% of the original sample as development sample (refer 4.3.2.6). To simulate credit granting policies of ZimSME bank, the preliminary CRM model developed on the KGB sample was applied to the development sample. To generate missing data, we simulated two (2) rejection policies: weak selection when the cut-off point was high and strong selection when cut-off point was low. By strong selection we meant that the bank was risk averse by being strict in offering loans to its credit applicants and by setting a lower threshold meant rejecting many applicants. On the other hand, by weak selection, the bank loosened its lending policy by setting a higher threshold, that is, the bank is a risk-taker. Using the weak selection model, more applicants are selected for loaning. After defining the missingness function, we therefore applied a Bayesian theoretically based BC reject inference technique to impute the credit quality of the rejected applicants
4.4.8.1 Estimation of the missingness probabilities
The issue of how missing data mechanisms should be estimated needed thorough discussion for that had some bearing on the application of the BC methodology. In the presence of the MNAR missingness, there are two (2) approaches to determine the missing data pattern, which are selection models and pattern mixture models. For this work, pattern mixture modelling was adopted because the BC reject inference model was seen closely related to it (Little, 1993). Probabilistically, the pattern mixture model is defined as:
𝑃(𝑌𝑚𝑖𝑠𝑠, 𝑌𝑜𝑏𝑠) = 𝑃(𝑌𝑜𝑏𝑠|𝑌𝑚𝑖𝑠𝑠)𝑃(𝑌𝑚𝑖𝑠𝑠).
The implication of this model is that missing data are classified by their respective missingness and describe the observed data within each missing group. Therefore, without the knowledge of the missingness of the missing data the pattern mixture model would be undefined. This pointed to the fact that for a pattern mixture model some identifying constraints were required from information on missing data.
Using the pattern mixture model to BC reject inference, observation with missing values were grouped in different missing patterns through an underlying CRM score, S. From loan process, it is logical to suggest the probability of having a bad credit quality as a good proxy for the missing data mechanism (MDM) (Chen & Astebro, 2012). This implied that the original credit score provides the much-needed information that paved way for the estimation of the missingness mechanism. For estimating the missing data mechanism, we used linear extrapolation of bad rates versus original score. The estimated probability for the rejected loan application’s being bad is considered as the probability of missingness.
This general linear model helped lenders to observe bad rate - credit score relationship over time thereby influencing the change of credit acceptance policy by either tightening or loosening the cut-off or threshold levels. Credit policy change in turn has bearing to the probability of missingness, a framework crucial for the eventual application of the proposed Bayesian reject- inference method: BC. Once estimated missingness was computed, the simulated value of the missing datum was calculated using equation (83). Iteratively, we imputed all the values of the missing credit quality of rejected applicants thereby generating a complete AGB sample, a good representation of the TTD population to be scored in future.
4.4.8.2 Selection of priors
Since the BC model uses a multivariate generalisation of the Beta distribution, the Dirichlet distribution, as the conjugate prior distribution, there was great need to select appropriate priors for its application. Several non-informative Dirichlet priors offered strong candidature for the selection. For simplicity, it was settled to choose between 𝛼𝑖𝑗 = 0 and 𝛼𝑖𝑗 = 1 priors. If we desired to achieve a uniform density such that the Dirichlet density function assigns equal weight to any vector 𝜃 compliant to the constraint that ∑ 𝜃𝑖𝑗 𝑖𝑗 = 1, it would be proper to set for 𝛼𝑖𝑗 = 1 for all 𝑖 and 𝑗. On the other hand, if it were to set for 𝛼𝑖𝑗 = 0 for all j that would get an improper prior distribution, that would be uniform in the log (𝜃𝑖𝑗)’s but the resultant posterior is proper if there is at least one observation in each score range. Fortunately, if the concerned sample were relatively large, then the difference in results between these two (2) prior densities would not be that big. On that basis we, for this work, selected the non-informative Dirichlet prior 𝛼𝑖𝑗 = 0 and based on simplicity in the application of the Bound and Collapse reject inference technology.
4.4.8.3 Verification
As soon as reject inference (Refer to 4.3.2.11) was successfully done, some simple verifications procedures were done. Firstly, some comparison of bad rates or odds of the post-inferred, “all- good-bad” and the “known-good-bad” samples were carried out to find out whether lending industry rules were not vilified. Reject inference techniques could be used to the satisfaction of industry norms which are usually based on the approval rate and the level of confidence of the preliminary model used for credit granting decisions. In this instance, if the preliminary model were good and the approval rate subsequently could be high, resulting consequentially in that the inferred rejects should have a bad rate at least three (3) times that of the approved.
There was also the need to carry out some comparisons of the bad rates of the grouped attributes for the KGB and the AGB samples. Some grouped attributes which are characterized by low acceptance rates and high WoE should display through the distributions of their WoE consistently with business considerations or in a way explainable by business experiences. It is also through this verification that the reject inference employed, and the corresponding estimated parameters are tested by means of what are called “fake rejects”. To do this test, the accepted/approved subsamples was split into arbitrary accepts and rejects in the ratio 70% to 30% and the final model
developed from the AGB sample. The classification of the 30% split was already known, therefore any observable misclassification due to the application of the final model was used to gauge the performance of the model developed.
Once thorough verification has been done, the combined sample of the approved and the inferred rejects was created to form an AGB sample, a random sample on which the final model would be built. This was the sample on which selectivity bias has been resolved, restoring the expected randomness character of a sample, a basis for any statistical inference of reality phenomenon. The resultant sample was assumed a better representation of the through-the-door population of the SME loan applicants. Using the same procedure as for the preliminary modelling, some characterization of the AGB sample was done prior to final model development.
4.4.8.4 Initial characterisation analysis on all-known-good-bad (AGB) sample
After the missing credit quality for the rejected SME loan applicants were imputed, initial characteristic analysis and statistical modelling procedures were carried out to generate final set of characteristics for the final CRM model. Post-inferred dataset constituted the AGB sample representative of the through-the-door SME loan application population on which the desired and final model was constructed. There was no limit to characteristics selected in the preliminary characteristics’ analysis, as some characteristics became weaker and some stronger after imputation of missing credit quality. This implied that variable selection was repeated exploring the post-inferred development dataset. Unlike the preliminary model development, which was constructed on the KGB development sample, the final model was derived after performing initial characteristic analysis and running the logistic regression onto the AGB sample. The resultant logistic regression model, parameter estimates, and model performance statistics were the major outcomes from the post-inferred sample. The scaling of the scores, validation of point allocation, misclassification and strength of the model were addressed as soon as final model was derived. 4.4.9 Validation
When the final model was built, the next procedure was validation. The validation process was carried out to confirm whether the developed model was serving the purpose it was built for. Was the final model applicable to the through-the-door population TTD of SME loan applicants? The
As in the model development stage, the 100% of the working sample constituted the development sample whilst the arbitrary 80% constituted the holdout sample. The holdout sample was used for validation process (Siddiqui, 2005). This was a process whereby the distributions of measured goods and bads across the development and holdout samples were compared. To carry out the comparisons, some goodness of fit statistical measures were employed. For this work, the Receiver Operating Curve (ROC) was used. It is a plot of the true positive rate against false positive rate at different cut-off points. The area under the ROC is used to measure the CRM model’s classification power.
The misclassification statistics were also used to assess the predictive prowess of the final model. For operational use of this statistic, a minimum level of acceptable bad rate was chosen as a “cut- off.” Loan applicants whose CRM scores were below the set “cut-off” point were declined loan and tagged as potential defaulters. This process could lead to the commission of type I and type II errors, where an actual good is wrongly classified as bad and consequentially declined loan services and vice versa. To ensure suitability of a final model, it was measured in such a way that both errors were minimized, that is, such that the level of misclassification was at minimal. There were several measures for misclassification which were based on the confusion matrix. These include accuracy, error rate, sensitivity and specificity.
Table 4.5: Confusion matrix
Actual Predicted
Good Bad
Good Bad
True positive (good decision) False Negative (Type I error) False Positive (type II error) True Negative (good decision) A good final credit risk measurement model would be one where the “true” cases are maximised, and “false” cases are minimised. The measures are defined as follows: