Published on 00/00/0000
Last updated on 00/00/0000
Published on 00/00/0000
Last updated on 00/00/0000
Share
4 min read
Published on 06/21/2021
Last updated on 02/05/2024
Explaining Explainable AI in FinTech World
Share
In recent years, the usage of machine learning techniques has grown substantially, both in industry and research. New types of models based on deep neural networks (DNNs) are now becoming ubiquitous in the industry. These models are extremely powerful and have substantially improved the state of the art in many domains. This improvement, however, is achieved at a cost. Compared to classical machine learning algorithms (e.g., Logistic Regression, SVM), these models are more complex, and typically use many orders of magnitudes more parameters. This increased level of complexity and subsequent opacity, makes it very difficult to understand the inner mechanism of these models, resulting in their limited adaptation in many domains. Specifically, in areas that involve human centric decision, explainability of decisions is critical without which the broad applicability of these models will remain questionable despite substantially improving prediction accuracy. As a result, the explainability and transparency of AI models has become a pressing issue in recent years and explainable AI (xAI) as a framework has gained a lot of attention. Moreover, many AI libraries such as PyTorch and Tensorflow have offered a specialized xAI extensions (i.e., Pytorch Captum and Tensorflow tf-explain) in response to this new important trend.
Let’s look at an AI-powered loan approval process is an example of domains that require transparent and explainable AI. Consider a financial institution that aims at maximizing its profit by approving loan applications for the applicants with a low likelihood of default. Given the abundance of historical loan application records, they can train a complex model to accurately estimate the likelihood of default based on the factors such as income, loan amount, credit score, etc. Even though learning of such a model using DNNs or Gradient BOOST algorithms is quite easy, it’s likely that they resort to simpler models such as decision trees or linear models because these models are easier to understand and to explain, both to customers and to regulatory bodies. Banks are usually expected to explain the rationale for the rejection of an application to the customer. Additionally, laws and regulations would require certain level of explanation and transparency to ensure the institution is using a fair and impartial system and does not discriminate on the basis of factors such as race, sex or ethnicity.
Unlike DNNs, classical machine learning models such as linear models, logistic regression and decision trees are simple and usually easy to understand and interpret. Decision trees for example, offer an intuitive and easy to comprehend set of inference rules that can be expressed as a series of Boolean conditions. For instance, the inference rule for disapproving a rule obtained by decision tree algorithm may look like:
Reject loan if: Customer Income < 100K & Requested Loan > 10K & Credit score < 600 (1)
Alternatively, models such as logistic regression express the inference decision as a simple mathematical (linear) function of input features. This makes it easy to evaluate the most important factors in the decision via evaluating the feature importance. For instance, this analysis may reveal that, the customer income, loan amount and the credit scores are the main deciding factors. In recent years, techniques such as SHAP values, LIME, and InterpretML have enabled us to evaluate the feature importance for more complex models, such as ensemble models and neural networks. Even though the new xAI techniques offer more transparency and insights into the decision-making process compared to the opaque Blackbox model, it is not as intuitive as the rules created by decision trees. From the customer’s perspective, a decision rule as in (1) provides a clear understanding of the decision process. For instance, based on the stated rule, customer understands that a certain credit score is required to obtain the approval. On the other hand, this level of intuitive understanding is not immediately available for the case of the most machine learning techniques.
These shortcomings highlight the need for future research in xAI to find better algorithms that could offer comparative performance without compromising the explainability and transparency.
Subscribe to
the Shift!
Get emerging insights on emerging technology straight to your inbox.
