Monday, June 13, 2016

From model literacy to model risk management

While regulators and banking industry re-debate use of models to determine capital needs, hardly anyone debates model literacy. What is model literacy? I would define it as out ability to use models correctly.

What is a model?

Many think of models as complex mathematical beasts created and appreciated by mathematicians and alike. Few come to realize that we use them on a daily basis.

Model is nothing more than a operationalized metaphor. Every time we say that "X is like Y", we use Y as a model for X. It is true for the complex financial models used in the banking industry, it also true in for simple models we use in our daily lives.

For the sake of an example let's consider a person who has a domestic cat , but does not know what the tiger is. It is easy to see how a simple model "tiger is like a big cat" could substitute a detailed description of a tiger. At the same time, it is obvious that the model has limitations - it does not capture the full essence of a tiger.

How not to manage model risk

The most common ways people deal with model risk are - ignoring the risk or rejecting the model.

Those who ignoring model risk tend to embrace them as an ultimately knowable truth. While they tend to accept that model is not a perfect representation of reality, they also tend to consider that it is the best knowledge we have. One can easily see how model "tiger is like a big cat" could backfire during safari for someone used to playing with his pet.

Rejection of models is the other extreme especially pronounced in individuals who used models unsuccessfully. They consider models to be useless and judgment a superior way of making decisions. And they were shown to be wrong so many times.

How to manage model risk?

"Essentially, all models are wrong, but some are useful" by G.E.P. Box is probably the most used and abused quotes about modeling. However, it summarizes two key aspects of any modeling effort - knowing how the model is useful and where it is wrong.

Going back to cat example, "tiger is like a big cat" is perfectly reasonable and usable model to recognize tigers in the wild. Using the same model to set safety restrictions could lead to undesirable outcomes, however.

But knowing limitations of the model allows to substitute it for "tiger is like a bear", in this case, what would yield a lot batter outcome. Using multiple models that compliment each other is often the best model risk management approach. Doing it correctly, however, requires high level of model literacy.

Back to finance

It is probably time to go back to model risk financial risk management.

While one intuitively would expect people who do model for living not to be that simplistic, my personal experience suggests that at least in finance this is not a case. Even PhDs in Statistics tend to forget key ideas behind models, not to mention executives who have to use mode outputs. Many chief risk officers in their decision are ignoring model risk, rejecting the models or, most often, both depending on the issue at hand.

Even post-crisis debate on VaR is dominated by these simplistic positions. One side is saying that VaR is useless, the other insists that it still should be the key element of the decision-making process. Few question overall use models in the financial industry and need to make us better skilled with model outputs. No matter how good the model is, no good decisions can be made by users who do not understand what the output really means.


Share to Facebook Share to Twitter Share on Google Plus Share to LinkedIn Pin This

0 comments:

Post a Comment