Introduction
As more and more businesses are facing credit card fraud and identity theft, the popularity of “fraud detection” is rising in Google Trends.
Companies are looking for credit card fraud detection software that will help to eliminate this problem or at least reduce the possible dangers. Before looking at the SPD Group credit card fraud detection project, let’s answer the most common questions:
What is a Fraud Detection System?
It is a set of activities undertaken to prevent money or property from being obtained through false pretenses.
What are Fraud Detection Predictive Models?
Models make predictions based on information about a transaction and some context (historical) information. To make the model more robust, we used only the most important features which were selected based on χ² (a chi-square is a test that measures how expectations compare to actual observed data) and recursive feature elimination techniques.
How can Neural Networks be Used to Detect Fraud?
Neural networks are highly effective when the data scientist has access to a large dataset (say 100,000 or more data samples). They are able to seek patterns and smartly detect new behavior that seems too distinct from the normal flow. However, in our case, we decided to rely on other Machine Learning models such as Classification trees because the RNN performance did not show the accuracy we expected, most likely because of a dataset that was not large enough.
Anomaly Detection Solution for E-Commerce Credit Card Transactions from SPD Group
Development time – 3 months
Team size – 6 experts
Platform – Web
Overview of the Credit Card Fraud Detection Project
SPD Group was contacted by an E-commerce and Financial Service company that offered products and services that can be paid for using Mobile Money or a bank card (e.g., Visa and MasterCard) to make their platform a safer online transaction place for their customers. Along with the increase in the number of customers who faced issues with their money suddenly disappearing or being transferred to another unknown account, our client thought of implementing a modern fraud prevention method for his platform. Therefore, he contacted us and decided to rely on what Machine Learning can do here.
Challenges
To dive into the challenges and obstacles of this project, we got a quote from a Machine Learning Engineer from our development team:
“The most complicated part of the solution was to achieve good metrics for users who have made only a few transactions. We could apply the regular model, which is good for users with a rich transaction history, but it would give worse scores if there is a lack of historical data (for example, a new user). Another obvious solution is to treat such users as empty accounts that have only identity information without any transaction history. In this case, we lose the advantage of having at least some data about the users, but the results that such a model provides are quite stable (underfitting). After making a weekly stand up on the matter, we decided to look into ‘few-shot learning’ techniques, which could help us improve our metrics. We have prepared a PoC, but it didn’t give us the drastic improvement we had expected. Nevertheless, we proceeded with experimenting and diving into our client business domain; it allowed us to develop features which have made a huge impact on our model that is based on ‘few-shot learning’ techniques. Because of the domain features, our main score improved by more than 15% and it became the production solution.”
Solution
Our R&D team worked on the project for 3 months, using Classification rather than classical Anomaly Detection methods. After an intense feature generation phase (about 700 features in total) they went to feature selection to choose only the most relevant ones. Finally, it was a blend of Classification methods such as GXBoost, Catboost, and LightGBM that got us close to the desired score.
Credit Card Fraud Detection Dataset
The platform is an e-commerce and financial service app serving 12,000+ customers daily. This dataset included a sample of approximately 140,000 transactions that occurred between October 2018 and April 2019. One of the fraud detection challenges is that the data is highly imbalanced. There were around 130,000 normal transactions and only 6% of them were fraudulent. We addressed the problem of an imbalanced dataset with various techniques such as data oversampling (augmenting the existing data samples) and data sample generation.
Credit Card Fraud Detection Algorithm
Once the Machine Learning-driven fraud protection module was integrated into the e-commerce platform, it started tracking the transactions. Whenever a user requests a transaction, it is being processed for some time. Depending on the level of predicted fraud probability, there are 3 kinds of possible output:
- If the probability is less than 10%, the transaction is allowed
- If the probability is between 10% and 80%, an additional authentication factor (e.g. a one-time SMS code, a fingerprint, or a secret question) should be applied.
- If the probability is more than 80%, the transaction is frozen, so it should be processed manually.
The model estimates the probability of a fraudulent transaction based on the following transaction information: Date and Time, Product Category, Amount, Provider (Seller), Client Information, Agent Information, Location, and Client’s Behavioral Patterns. Contextual and aggregated data is produced by a Machine Learning engineer based on the previously mentioned data.
Technology for Credit Card Fraud Detection Software
Classification models:
- Decision Trees
- Isolation Trees
- Random Forest
- Bootstrap
Anomaly detection:
- PCA
- Mahalanobis Distance
- Local Outlier Factor
Results
After this solution was implemented, the entire e-commerce platform received tangible benefits. In only 6 months after production, we can highlight the following areas:
- Cost Reduction for solving problems with fraud: communication with Customer Support, filling in forms for refund, refund verification by a manager, etc.
- Customer Satisfaction because of simplified authentication for a transaction with a low probability of fraudulence (usually, most transactions have low fraudulence probability).
What we achieved with the solution: More than 140,000 transactions were analyzed with 6% of fraudulent data points a year. Fewer customers claimed to have fraudulent transactions. Our client’s online card transaction platform became a safer service and gained more loyalty from their customers. We continued to support the project after release because it is very important to continuously train the Fraud Detection model whenever new data arrives, so new fraud schemas/patterns can be learned and detected as early as possible.
Originally posted here.
