Investigation 01 · Zocket · 2024
The $22K wallet leak nobody could see.
The tip-off
Customers fund ad wallets through Stripe. Spend flows out through the Facebook Ads API. Everyone assumed the two ledgers reconciled — nobody had actually checked. Then small gaps started surfacing in finance reviews.
The audit
No sampling. Every wallet, every movement, both directions — a transaction-level join across all three ledgers.
-- full-population reconciliation: no sampling WITH stripe AS (SELECT wallet_id, txn_id, amount FROM stripe_charges), spend AS (SELECT wallet_id, ad_id, spend FROM fb_ads_spend), ledger AS (SELECT wallet_id, entry_id, delta FROM wallet_ledger) SELECT w.wallet_id, SUM(s.amount) AS funded, SUM(sp.spend) AS spent, SUM(l.delta) AS ledger_balance, funded - spent - ledger_balance AS drift -- should be 0. always. FROM wallets w LEFT JOIN stripe s USING(wallet_id) LEFT JOIN spend sp USING(wallet_id) LEFT JOIN ledger l USING(wallet_id) GROUP BY 1 HAVING drift != 0;
The find — exhibit A: duplicate entries
| entry_id | wallet | type | amount | created_at |
|---|---|---|---|---|
| e_88213 | w_1042 | top-up | $500.00 | 02:14:07.113 |
| e_88214 | w_1042 | top-up | $500.00 | 02:14:07.891 |
| e_88215 | w_2210 | top-up | $1,200.00 | 02:16:33.402 |
| e_88216 | w_2210 | top-up | $1,200.00 | 02:16:34.008 |
| e_88217 | w_0977 | top-up | $350.00 | 02:18:01.554 |
Retried webhooks written twice, ~800ms apart. Balances inflated; spending limits computed off the inflated number — the platform was quietly overcharging itself. Two bugs, individually invisible.
The fix
Documented the exact failure paths, handed engineering a reproducible case list, and added standing reconciliation checks — so the next drift shows up on a dashboard, not in a quarter-end surprise.