PRAGMA: Revolut Foundation Model
Revolut's in-house foundation model learns from raw banking event sequences and beats task-specific models on fraud, credit, and customer value prediction.
The Thesis
PRAGMA is a family of large pretrained models built specifically for banking transaction data — the streams of deposits, payments, and account events that financial institutions collect at massive scale. Instead of training a separate model for each task, Revolut pretrained one general model on all that raw event data, then showed it transfers cheaply to credit scoring, fraud detection, and lifetime value prediction. This matters because financial AI today is dominated by hand-engineered features and narrow task-specific models, which are expensive to maintain and brittle when customer behavior shifts. The catch is that this paper is an internal system report from Revolut: the data, model weights, and full benchmarks are not publicly available, so independent replication is impossible for now.
Catalyst
Transformer architectures — the same class of models behind large language models — have only recently been proven to generalize well to non-language sequential data, including tabular event logs. Revolut also operates at a scale (tens of millions of active users generating dense transaction histories) that makes self-supervised pretraining practical in a way it simply was not for smaller institutions five years ago. The combination of scale, architecture maturity, and internal data moats makes this timing credible.
What's New
Prior financial ML relied on hand-crafted feature engineering: aggregating raw transactions into summary statistics (average spend, transaction count by category) and feeding those into gradient-boosted tree models. More recent academic work applied Transformers to tabular data row by row, but didn't pretrain on heterogeneous sequences across products. PRAGMA treats a customer's entire banking history — across cards, transfers, loans, and app events — as a single variable-length sequence and applies masked modelling (predicting hidden events from context), borrowing the pretraining recipe from large language models but adapting it to discrete financial tokens.
The Counter
This paper is an internal whitepaper from Revolut, and the data, code, and model weights are not publicly released — which means every performance claim is unverifiable by outside researchers. The baselines compared against are not described in sufficient detail to judge whether PRAGMA is beating well-tuned gradient-boosted models or strawman alternatives. Financial foundation models have been proposed before (by JPMorgan, Mastercard, and academic groups) with similarly strong internal claims, and none have become industry standard, partly because data is siloed and regulatory constraints make cross-institution transfer nearly impossible. PRAGMA's generalization story also depends on Revolut's specific data mix, which is unusually dense for a mobile-first neobank — it may not transfer to traditional banks whose customers have sparser interaction histories. Finally, the business moat here is the data, not the architecture: any sufficiently large bank could replicate the approach, which means the research contribution is more about demonstrating feasibility than introducing a defensible technical barrier.
Longs
- V (Visa) — transaction data infrastructure benefits from better fraud models across the network
- PAYX (Paychex) — payroll and SMB banking players face competitive pressure from embedded AI credit scoring
- AFRM (Affirm) — buy-now-pay-later firms with dense event data could deploy similar architectures
- SOFI (SoFi Technologies) — digital bank with transaction scale to benefit from or replicate this approach
- FICO — incumbent credit scoring vendor whose moat erodes if raw-sequence models outperform engineered scores
Shorts
- FICO — their FICO Score depends on a narrow, engineered feature set; a pretrained sequence model trained on richer behavioral data could consistently outperform it for lenders with sufficient transaction history
- Established fraud analytics vendors (e.g., Featurespace, NICE Actimize) — their value proposition is custom feature engineering and rule tuning, which becomes less defensible if a single pretrained model transfers across fraud tasks out of the box
- Consulting firms selling bespoke credit model builds — the 'train a linear model on top of embeddings' result implies dramatically reduced implementation cost per task
Enablers (Picks & Shovels)
- Hugging Face Transformers — open-source library underpinning the masked modelling pretraining approach
- Apache Kafka / event streaming infrastructure — required to serve dense real-time banking event sequences at training scale
- dbt (data build tool) — SQL-based transformation layer commonly used to prepare financial event logs for ML pipelines
- NVIDIA H100 / A100 GPUs — hardware class required to pretrain Transformer models at this sequence length and corpus size
Private Watchlist
- Sardine — real-time fraud and compliance platform for fintechs
- Inscribe — document and transaction fraud detection for lenders
- Zest AI — alternative credit underwriting using machine learning
- Taktile — ML model deployment infrastructure for financial decision-making
Resources
The Paper
Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for multi-source banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.