Machine learning

Payment Fraud Detection

hard4–5 hours2 datasets

Company: Razorpay
Job positions: Data ScientistML EngineerData Analyst
Topics: ClassificationImbalanced dataPrecision and recallData leakageThreshold tuning

The scenario

Razorpay: Build a classifier that flags fraudulent payments before they're approved, and pick a review threshold based on what fraud actually costs.

Razorpay processes online payments for businesses across India, from food delivery apps to airlines. Fraudsters use stolen cards and brand-new accounts, and every fraudulent payment that goes through comes back as a chargeback that costs the merchant the full amount plus a fee. You're a data scientist on the risk team, asked to build a model that sends suspicious payments to manual review before they're approved.

Your task

Train a classifier that scores each payment's fraud risk using only information available at payment time, choose a review threshold based on business cost, and score the most recent three weeks of payments.

Instructions

1Explore the training data: how rare fraud is, and how the fraud rate changes with payment method, merchant category, card type and time of day.
2Check every column for leakage. Leave out anything that is only known after a payment is approved, and explain why.
3Build a rules baseline, such as flagging international cards used from a different country, and measure its precision, recall and average precision.
4Engineer features that make sense at payment time, such as how unusual an amount is for its merchant category, and train at least two models. Handle the class imbalance, for example with class weights.
5Validate on the most recent weeks of the training data instead of a random split, since the model will score future payments.
6Choose a threshold for sending payments to review. Assume each missed fraud costs its full amount_inr plus a ₹500 chargeback fee, and each legitimate payment sent to review costs ₹40 of analyst time. Compare the total cost at several thresholds.
7Score every payment in transactions_test.csv and save predictions.csv with the columns transaction_id, fraud_score and flag_for_review (1 or 0).

Datasets

All files come in one download.

transactions_train.csv

About 20,000 payments from June 1 to August 10, 2025, with confirmed fraud labels.

20,026 rows · 15 columns · 1.7 MB

Column	Type	Description
transaction_id	text	Unique ID of the payment.
created_at	datetime	When the payment was attempted, in UTC.
merchant_id	integer	The business receiving the payment.
merchant_category	text	The merchant's business type.
amount_inr	decimal	Payment amount, in Indian rupees.
payment_method	text	upi, card, netbanking or wallet.
card_network	text	visa, mastercard or rupay. Empty for payments not made by card.
is_international_card	text	Whether the card was issued outside India: yes or no. Empty for payments not made by card.
customer_account_age_days	integer	Days since the customer's account was created.
device_type	text	android, ios or web.
customer_txns_last_24h	integer	Payments the same customer attempted in the previous 24 hours.
failed_attempts_last_hour	integer	Failed payment attempts by the same customer in the previous hour.
ip_country_matches_card	text	Whether the customer's IP address is in the card's country: yes or no.
chargeback_received	text	Whether the customer's bank later reversed the payment: yes or no. This arrives weeks after the payment.
is_fraud	integer	1 if the payment was confirmed as fraud, otherwise 0. This is what you predict.

Preview the first 5 rows

transaction_id	created_at	merchant_id	merchant_category	amount_inr	payment_method	card_network	is_international_card	customer_account_age_days	device_type	customer_txns_last_24h	failed_attempts_last_hour	ip_country_matches_card	chargeback_received
pay_700001	2025-06-01 00:01:51	1171	utilities	841.3	card	visa	no	384	android	3	1	yes	no
pay_700002	2025-06-01 00:13:36	1275	ecommerce	271.24	upi	empty	empty	587	android	2	1	yes	no
pay_700003	2025-06-01 00:18:39	1125	education	4247.66	upi	empty	empty	34	web	0	0	yes	no
pay_700004	2025-06-01 00:20:08	1128	ecommerce	684.11	upi	empty	empty	230	android	1	0	yes	no
pay_700005	2025-06-01 00:25:47	1188	education	7694.18	netbanking	empty	empty	306	android	1	2	yes	no

transactions_test.csv

About 6,000 payments from August 11 to 31, 2025, without labels.

5,974 rows · 13 columns · 476 KB

Column	Type	Description
transaction_id	text	Unique ID of the payment.
created_at	datetime	When the payment was attempted, in UTC.
merchant_id	integer	The business receiving the payment.
merchant_category	text	The merchant's business type.
amount_inr	decimal	Payment amount, in Indian rupees.
payment_method	text	upi, card, netbanking or wallet.
card_network	text	visa, mastercard or rupay. Empty for payments not made by card.
is_international_card	text	Whether the card was issued outside India: yes or no. Empty for payments not made by card.
customer_account_age_days	integer	Days since the customer's account was created.
device_type	text	android, ios or web.
customer_txns_last_24h	integer	Payments the same customer attempted in the previous 24 hours.
failed_attempts_last_hour	integer	Failed payment attempts by the same customer in the previous hour.
ip_country_matches_card	text	Whether the customer's IP address is in the card's country: yes or no.

Preview the first 5 rows

transaction_id	created_at	merchant_id	merchant_category	amount_inr	payment_method	card_network	is_international_card	customer_account_age_days	device_type	customer_txns_last_24h	failed_attempts_last_hour	ip_country_matches_card
pay_720027	2025-08-11 00:01:21	1220	utilities	2736.32	upi	empty	empty	12	android	2	1	yes
pay_720028	2025-08-11 00:01:30	1167	utilities	848.56	card	visa	no	426	android	3	0	no
pay_720029	2025-08-11 00:03:22	1158	utilities	490.76	upi	empty	empty	747	android	0	0	yes
pay_720030	2025-08-11 00:05:36	1220	utilities	967.39	wallet	empty	empty	114	android	0	1	yes
pay_720031	2025-08-11 00:21:19	1006	gaming	139.18	upi	empty	empty	399	android	1	0	yes

Hints

Accuracy is misleading when only about 1 in 75 payments is fraud: a model that flags nothing is almost 99% accurate. Use precision, recall and average precision instead.

from sklearn.metrics import average_precision_score
average_precision_score(y_valid, valid_scores)

Sort by time and hold out the last few weeks, so your validation looks like the real task of scoring future payments.

train = df[df["created_at"] < "2025-07-21"]
valid = df[df["created_at"] >= "2025-07-21"]

Deliverable

A public GitHub repo with your notebook or scripts, predictions.csv for the test payments, and a README covering your leakage checks, validation approach, model comparison, cost-based threshold and the strongest fraud signals.

When you're done, post your repo in the Solutions tab to share it with other learners.

What grading checks

Use this checklist to review your own work before you post and share it.

Submitted GitHub repo is public and reachable.
Repo contains at least one notebook or script file.
predictions.csv has a fraud_score and flag_for_review for every test payment.
fraud_score reaches an average precision of at least 0.35 on the test payments.
Under the cost assumptions in the brief, the flagged payments cost less than reviewing none.
The column only known after approval is left out of the model, with the reason explained.
Models are compared on a time-based validation set with precision, recall and average precision.