Phase 2 · Ground-truth check
Synthetic credit-limit RCT
Synthetic credit-limit RCT with known segments and true CATE. I built Sleeping Dogs around the contact-saturation idea from Phase 1. Outcome = good_standing at 3 months.
Simulated n
40,000
Treatment share 49.5%
Test n
8000
Safer on contamination
CausalForestDML
0.00% Sleeping Dogs in top decile
Built-in answer key
Latent segments (DGP)
Persuadables
true CATE +0.175 · share 19.6% · obs gap +0.191
Moderate utilization, decent payment history, more App / Urban. True CATE about +0.15 to +0.20.
Sure Things
true CATE -0.000 · share 30.0% · obs gap +0.001
High payment consistency, low utilization. True CATE near 0 (basically noise).
Lost Causes
true CATE -0.000 · share 25.0% · obs gap +0.007
Low payment consistency, high utilization. True CATE near 0 (basically noise).
Sleeping Dogs
true CATE -0.125 · share 25.3% · obs gap -0.121
High utilization, lots of active lines, more Rural / CallCenter (too much contact). True CATE about -0.10 to -0.15.
Main result
True vs recovered mean CATE
Grouped bars compare true mean CATE to T-learner and CausalForestDML on the test set. This is the check that the models recover the simulated segments, not just overall good_standing risk.
Swipe or drag sideways to see more columns
| Model | PEHE | corr(pred, true) |
|---|---|---|
| T-learner | 0.0037 | 0.847 |
| CausalForestDML | 0.0015 | 0.927 |
Policy ranking
Qini curves
Population fraction targeted
Swipe or drag sideways to see more columns
| Model | Qini | Uplift@10% | Uplift@30% | Uplift@50% |
|---|---|---|---|---|
| T-learner | 91.34 | 0.185 | 0.105 | 0.065 |
| CausalForestDML | 87.66 | 0.202 | 0.106 | 0.060 |
Who would get treated
Top-decile true-segment mix
If you treat the top 10% by predicted CATE, which true segments show up?
Swipe or drag sideways to see more columns
| Model | True segment | % of top | Pop share | Enrichment |
|---|---|---|---|---|
| T-learner | Persuadables | 95.5% | 19.7% | 4.85× |
| T-learner | Sure Things | 3.8% | 29.9% | 0.13× |
| T-learner | Lost Causes | 0.6% | 25.3% | 0.02× |
| T-learner | Sleeping Dogs | 0.1% | 25.1% | 0.00× |
| CausalForestDML | Persuadables | 100.0% | 19.7% | 5.08× |
| CausalForestDML | Sure Things | 0.0% | 29.9% | 0.00× |
| CausalForestDML | Lost Causes | 0.0% | 25.3% | 0.00× |
| CausalForestDML | Sleeping Dogs | 0.0% | 25.1% | 0.00× |
Takeaway