Phase 1 · Methodology check
Hillstrom MineThatData
I used a real randomized email experiment first to check that the uplift methods work before building synthetic credit data. Mens E-Mail vs No E-Mail, outcome = visit.
Test n
8523
Top-decile overlap
58.1%
Jaccard 0.409
Neg. CATE share
1.4%
123 of 8523 test rows
Policy ranking
Qini curves
Population fraction targeted
Swipe or drag sideways to see more columns
| Model | Qini | Uplift@10% | Uplift@30% | Uplift@50% |
|---|---|---|---|---|
| T-learner | 7.74 | 0.116 | 0.094 | 0.080 |
| CausalForestDML | 6.26 | 0.153 | 0.086 | 0.081 |
Who gets ranked high
Persuadables vs low-uplift profile
Recency (mean months)
Top decile
4.3
Bottom decile
5.1
History (mean $)
Top decile
$459
Bottom decile
$302
Mens purchase rate
Top decile
82.3%
Bottom decile
53.4%
Womens purchase rate
Top decile
77.6%
Bottom decile
52.5%
Newbie rate
Top decile
55.9%
Bottom decile
45.7%
Modal history segment
Top decile
3) $200 - $350
Bottom decile
1) $0 - $100
Modal zip
Top decile
Urban
Bottom decile
Suburban
Modal channel
Top decile
Web
Bottom decile
Web
Observed uplift in slice
Top decile
0.116
Bottom decile
0.098
Profiles shown for T-learner (Phase 1 top vs bottom decile).
Swipe or drag sideways to see more columns
| Feature | Top decile | Bottom decile |
|---|---|---|
| Recency (mean months) | 4.3 | 5.1 |
| History (mean $) | $459 | $302 |
| Mens purchase rate | 82.3% | 53.4% |
| Womens purchase rate | 77.6% | 52.5% |
| Newbie rate | 55.9% | 45.7% |
| Modal history segment | 3) $200 - $350 | 1) $0 - $100 |
| Modal zip | Urban | Suburban |
| Modal channel | Web | Web |
| Observed uplift in slice | 0.116 | 0.098 |
Profiles shown for T-learner (Phase 1 top vs bottom decile).
Model disagreement
They do not pick the same people
Sleeping Dogs candidate
Concentrated negative CATE
Swipe or drag sideways to see more columns
| Feature | Level | Neg share | Test share | Gap |
|---|---|---|---|---|
| history_segment | 1) $0 - $100 | 13.0% | 36.1% | -23.1 pp |
| history_segment | 6) $750 - $1,000 | 21.1% | 3.0% | +18.2 pp |
| zip_code | Urban | 23.6% | 40.7% | -17.2 pp |
| zip_code | Rural | 31.7% | 15.1% | +16.6 pp |
| channel | Multichannel | 27.6% | 12.2% | +15.5 pp |
| history_segment | 3) $200 - $350 | 6.5% | 19.6% | -13.1 pp |
History mean in negative group $555 vs test $240 (2.32× ratio).