Can I still use my models when everything keeps changing? Part 2.

Previously, we've discussed various internal factors that may influence models’ effectiveness and how organisations should track and react to the changes they observe. 

This month, I want to move on to external factors, including economic and business challenges and seasonality. External factors add another dimension to portfolio tracking and have become a major focus area for anyone who uses data to predict behaviour. 

Challenges, Seasonality & Economic Cycles - Image Credit: Carolyn Röhm & MidJourney

• Business challenges: In a stable economy the goals of the organisation may be driven by growth, making certain strategies more aggressive. Keep an eye on inflation and how it will change the features in your models or credit limits. 

  A developing economy brings additional challenges, and you may want to ensure the model is more robust than usual. You can do so by avoiding financial characteristics such as income or ensuring the coarse classing of the attributes is extremely tight to ensure that population shifts do not affect the scorecard.

  You may also be driven by what the competition is doing, and this may result in population shifts. How aggressive the shift is will depend on how aggressive their strategies are, how attractive their product is compared to yours, and whether there are any entrants in the marketplace. 

  You may be faced with having to become as aggressive to maintain your acceptance rates. This may result in higher bad debt coming from the new strategies. Make sure you don’t counteract what you are doing in your strategy by a subsequent change in cut-off. You may have to decide between keeping volumes and maintaining bad rates. 

  As with everything else, make sure you keep a log so that you future model developers can take these into account.

  
  

Business challenges do alter the portfolio, but they rarely affect the models in an extreme way. 

• Seasonality: During the development, both the application and delinquency seasonal effects should have been considered. Some of the tracking reports are generated monthly or quarterly, so before reacting in any way to one month/quarter data, a full understanding of how the portfolio works on a monthly basis is necessary. 

  No changes to strategy or models should be made based on monthly or quarterly data only, so comparative analysis should be done. This may include yearly reports or comparisons between this year’s and last year’s data. The vintage analysis report is also very helpful at determining the effect of seasonality and differentiating it from real shifts in the portfolio. 

  
  

• Economy: Changes in the economy are very often difficult to quantify and detect early enough to make changes to strategies. They do have an effect on the reports, but by the time they are observed it may be too late to minimise the effect of a trend on your portfolio. 

  Tracking prime rates, inflation and other macro-economic factors and plotting them with some of the internal statistics may provide some insight into the relationships between the economic factors and the portfolio changes. 

  Some organisations will turn their backs on scoring during recessions. Some even blame the models for the lack of quality of the portfolio. The truth is that going back to the manual approach is not going to help. If the model was not developed with robust groupings, it may deteriorate and need redevelopment. Once again, careful monitoring will give you the answer on how to react to each effect you observe. 

  Some solutions may include redevelopment or perhaps adding tighter policies or making the initial product offering more conservative to counteract some of that bad debt. 

In my previous blog, I touched on the concept of patterns. During a recent model development, I had to deal with selecting a window and the COVID pandemic was in the middle of my optimum maturity period. One of the questions that came up was how COVID had affected the model’s performance and whether the patterns remained the same. 

I was curious, so I tested my theory and selected three different windows, one before COVID, one during and just after COVID, and one post-COVID. The results showed that, even though the predictive power of the features was diminished during COVID, the patterns remained the same. Married applicants were still a better risk than single people; higher debt ratios were still riskier… 

The results showed that, even though the predictive power of the features was diminished during COVID, the patterns remained the same

Most of the usual features that appear in application models were still holding. The number of bad payment profiles had just increased, making the variables slightly less predictive. The client could have continued to use a robust model, but became less aggressive rather than regress to manual underwriting. 

Over the last couple of years, I have been seeing the same result in most models. In account management and collections, once again the features and their patterns persist. Ye, they lose predictive power and, in some cases, not even that significantly. 

Whether it is the economic crisis faced in recent years, recession or pandemics, models have been shown to provide an effective tool managing risk. It is the building, understanding and use of the models that need to be adapted when faced with a major event driven by external factors. 

So, what conclusion can be drawn from all this? Clearly, I am not advocating that businesses should not close the doors to new applicants during major events like COVID. What is clear is that, when your organisation decides to open the doors, effective models have a place in it and will still work. As long as they are built to sustain some level of change, they should still provide benefit to the organisation. 

Careful monitoring of both models and strategies will allow the organisation to counteract any events, whether internal or external, and provide a framework to ensure the profitability of portfolios. Redevelopment using the new status quo is advisable as soon as possible, whilst continuing to use the models with adapted decision strategies. 

It is the building, understanding and use of models that need to be adapted when faced with a major event driven by external factors. 

As always, if you have any queries on model building and performance or you just want to discuss your specific needs, please email us or book a 30 min online meeting with us.

Until next time…