Predictive Modeling and Risk Scoring for Bank Customer Churn

An end-to-end machine learning and MLOps platform designed to predict customer churn, segment customers, quantify financial risk, explain model decisions, and prioritize retention actions.

Executive Summary

This project evolved from a traditional customer churn prediction model into an end-to-end customer intelligence platform for banking. The solution combines supervised machine learning, customer segmentation, quantitative risk scoring, explainable AI, model calibration, monitoring, REST APIs, cloud data storage, experiment tracking, and automated CI/CD.

Five classification models were evaluated using multiple performance metrics. Gradient Boosting is currently used within the risk-scoring and monitoring workflow, while Logistic Regression is retained for interpretable quantitative analysis. The platform also uses K-Means clustering to identify customer segments and combines churn probability with customer value to support retention prioritization.

Live Demonstration:

Streamlit App

Business Problem

Customer attrition can significantly affect revenue and long-term customer value in the banking sector. A useful churn system therefore needs to do more than simply predict whether a customer will leave.

The objective of this project is to identify customers with elevated churn probability, understand the factors associated with that risk, estimate potential financial exposure, and provide a framework for prioritizing retention activity.

Project Objectives

Predict customer churn probability using supervised machine learning.
Compare multiple classification algorithms using comprehensive evaluation metrics.
Engineer business-oriented customer features.
Segment customers using unsupervised learning.
Estimate expected financial loss associated with churn.
Estimate retention cost, expected saved value, and ROI.
Explain individual and global model predictions.
Monitor data drift, prediction drift, and model performance.
Expose predictions through a production-style REST API.
Integrate persistent customer risk data using Supabase.

Dataset Overview

The project uses a European bank customer churn dataset containing 10,000 customer records with demographic, financial, product, engagement, and churn information.

10,000

Customer Records

14

Original Columns

20.37%

Observed Churn Rate

Binary

Classification Target

The original dataset includes customer identifiers, demographic attributes, credit information, banking products, account balance, estimated salary, engagement, and the churn target.

Data Validation & Quality

A dedicated validation layer was introduced before model training to verify the structure and quality of incoming data.

Required-column validation
Missing-value checks
Duplicate Customer ID detection
Binary target validation
Numerical range validation
Categorical-value validation

Raw Dataset → Validation → Feature Engineering → Modeling

Feature Engineering

The modeling pipeline generates additional customer-level features designed to capture financial value, engagement, product usage, and customer tenure relationships.

BalanceSalaryRatio — relationship between account balance and estimated salary.
CustomerValue — balance combined with estimated salary.
ProductDensity — product usage relative to tenure.
EngagementScore — combined customer engagement indicators.
AgeTenureInteraction — interaction between age and customer tenure.
HasBalance — indicator of whether the customer maintains a balance.

Machine Learning Pipeline

Raw Data → Validation → Feature Engineering → Preprocessing → SMOTE → Model Training → Evaluation → Risk Scoring

The supervised learning workflow uses Scikit-Learn and Imbalanced-Learn pipelines. Numerical variables are standardized, categorical variables are encoded, and SMOTE is applied within the training pipeline to address class imbalance.

Models Evaluated

Model	Role
Logistic Regression	Interpretable baseline and quantitative probability analysis
Decision Tree	Rule-based classification and nonlinear splits
Random Forest	Ensemble-based classification
Gradient Boosting	Boosted-tree prediction and risk-scoring workflow
XGBoost	Gradient-boosted tree benchmark

Models are evaluated using multiple complementary metrics rather than relying on accuracy alone.

Model Evaluation

Model performance is compared using classification, ranking, and probability-quality metrics.

Accuracy

Overall classification performance

Precision

Positive prediction quality

Recall

Churn detection sensitivity

ROC-AUC

Ranking discrimination

Additional evaluation includes F1-score, PR-AUC, log loss, Brier score, confusion-matrix metrics, and threshold optimization.

The complete comparison is stored in the project's artifacts/metrics/model_comparison.csv artifact.

Customer Segmentation

The platform also incorporates unsupervised learning to identify groups of customers with similar demographic, financial, product, and engagement characteristics.

K-Means clustering was evaluated across multiple values of K. Cluster quality was assessed using Silhouette Score, Calinski-Harabasz Score, Davies-Bouldin Score, and inertia.

Customer Features → Standardization → K-Means → Cluster Profiles → Business Segmentation

PCA-based visualization is also included to provide a two-dimensional representation of customer segments.

Quantitative Customer Risk Scoring

The platform converts model predictions into a structured customer-risk framework by combining churn probability with customer value.

Expected Loss = Churn Probability × Customer Value

Customers are categorized into Low, Medium, High, and Critical risk groups based on predicted churn probability.

Customer Value & Retention Economics

The risk engine estimates the potential financial exposure associated with customer churn.

Customer Value = Balance + Estimated Salary

Retention Cost = Customer Value × Retention Cost Rate

Expected Saved Value = Expected Loss × Treatment Effectiveness

Treatment effectiveness and retention cost assumptions are explicitly configurable business assumptions rather than parameters learned directly from the churn dataset.

Retention ROI & Prioritization

Risk scoring is combined with customer value to determine which customers may warrant different levels of retention attention.

ROI = (Expected Saved Value − Retention Cost) / Retention Cost

P1 — Immediate Retention
P2 — High Priority
P3 — Targeted Retention
P3 — Relationship Management
P4 — Monitor

Explainable AI

Explainability was incorporated to make model predictions more transparent to analysts and business stakeholders.

Global SHAP feature importance
Customer-level SHAP explanations
Logistic Regression coefficients
Odds-ratio interpretation
Partial Dependence Analysis
Feature contribution analysis

These tools help analysts investigate which characteristics contribute to a customer's predicted churn risk.

Probability Calibration

Because churn probabilities are subsequently used in financial risk calculations, the project also evaluates probability quality rather than treating classification accuracy as sufficient.

Brier Score
Log Loss
Calibration Metrics
Reliability / Calibration Curve

Calibration analysis helps assess whether predicted probabilities correspond reasonably to observed outcomes.

MLOps & Model Monitoring

The project includes a dedicated monitoring layer designed to simulate production machine-learning operations.

Feature Drift Detection
Data Distribution Monitoring
Prediction Drift Monitoring
Prediction Probability Monitoring
Risk Category Monitoring
Model Performance Monitoring
Champion Model Validation
Evidently Monitoring Reports

Production-Style REST API

The trained model is exposed through a FastAPI application that provides structured prediction endpoints.

Customer Input → FastAPI → Feature Engineering → Model → Churn Probability → Risk Category

The API includes validation, health monitoring, prediction logging, model information, and monitoring-related functionality.

Supabase Integration

Supabase is used as a persistent data layer for customer risk and prediction information.

The application can retrieve customer risk-scoring records containing customer attributes, churn probability, risk category, customer value, expected loss, retention economics, and retention priority.

Prediction → Risk Scoring → Supabase → Dashboard / Analytics

MLflow Experiment & Model Tracking

MLflow is incorporated into the project for experiment tracking and model lifecycle management.

Experiment tracking
Model registration
Model aliases
Champion-model workflow
Prediction tracing

CI/CD & Automated Testing

GitHub Actions is used to automate validation of the application whenever changes are pushed to the repository.

Git Push → GitHub Actions → Install Dependencies → Pytest → API Tests → Feature Tests

The automated test suite covers API endpoints, validation behavior, feature engineering, and customer-value logic.

End-to-End Architecture

Data → Validation → Feature Engineering → Supervised ML + Unsupervised ML → Explainability → Risk Scoring → API → Supabase → Monitoring

Technology Stack

Python Pandas NumPy Scikit-Learn XGBoost Imbalanced-Learn SHAP Plotly Streamlit FastAPI Pydantic MLflow Evidently Supabase PostgreSQL Docker GitHub Actions Git

Key Project Artifacts

model_comparison.csv — supervised model evaluation
customer_segments.csv — customer segmentation results
customer_risk_scoring.csv — customer-level risk and financial analysis
shap_global_importance.csv — global explainability results
shap_local_explanations.csv — customer-level explanations
calibration_metrics.csv — probability calibration analysis
feature_drift_report.csv — feature drift analysis
performance_comparison.csv — model performance monitoring
prediction_drift_report.json — prediction monitoring

Business Impact

The platform connects predictive modeling with business decision support. Instead of stopping at a churn prediction, the system provides a framework for understanding customer risk, customer value, potential financial exposure, and retention economics.

This enables analysts and business teams to move from descriptive customer analysis toward proactive, data-driven retention planning.