The push-ingest pattern: product metrics when a provider has no API

Every monitoring setup eventually hits the same wall. Traffic is easy — GA4 and Amplitude have APIs. Search is easy — Google and Bing publish position data. Then you try to put revenue on the same screen and discover that your payment provider offers a web dashboard, a CSV export button, and nothing you would want to poll in production. Smaller merchant-of-record providers, regional payment rails, app-store-adjacent marketplaces — many of them simply have no reporting API, or one so rate-limited and lagged it is worse than nothing.

The reflex is to scrape or to scheduled-export your way around it. Both are traps: scrapers break on every dashboard redesign, and nightly CSV imports mean your "operations room" shows yesterday. The durable fix is to stop trying to pull data out of a system that will not give it, and instead push it from the one system that always knows — your own product.

Your backend learns about every purchase the moment it happens, because it has to: the provider calls your webhook so you can grant access to whatever was bought. That webhook handler is the single most reliable source of revenue truth you have. The push-ingest pattern is simply: at the moment you process that event, also emit it to your metrics endpoint.

if a provider won't let you pull, push. your webhook handler already knows every sale before any dashboard does.

The receiving side is deliberately small. One HTTPS endpoint, one bearer token, one JSON shape:

POST /ingest with a body like {"metric": "revenue", "value": 1.00, "currency": "USD", "ts": "2026-06-27T09:14:03Z", "idempotency_key": "creem_tx_8f3a91", "source": "creem"}. The server validates the token, writes the raw event, and returns 202. That is the entire protocol. Resist the urge to make it richer — every field you add is a migration you will run later.

Five rules make it production-grade rather than a demo. First, idempotency keys are not optional. Use the provider's transaction id. Webhooks retry, deploys replay queues, and one Tuesday you will double-send an hour of events; a unique key on the raw-events table turns all of that into harmless no-ops. Second, send the provider's timestamp, not the time your code ran. Webhooks arrive late — sometimes hours late after an outage — and if you stamp events on receipt, every provider incident becomes a fake revenue spike on the wrong day. Third, never block the purchase path. Emit the metric after the business-critical work, fire-and-forget with a short timeout, and if the ingest endpoint is down, drop the event into a local retry queue. A metrics outage must never cost you a sale. Fourth, store raw events and aggregate at read time. Storing daily totals feels efficient until you need to reclassify refunds, split by product, or fix a currency bug retroactively; raw events make every one of those a query instead of a data-loss apology. Fifth, verify the webhook signature before you emit anything. An ingest pipeline that trusts unverified webhooks is a machine for turning a forged request into a corrupted revenue chart.

The pattern generalizes past revenue immediately, which is where it gets quietly powerful. Signups, activations, subscription upgrades, churn events, video render completions — anything your code witnesses can go through the same endpoint with a different metric name. You end up with one write path for every number your dashboards will ever draw, instead of a new integration per metric. When we added purchase alerts to a product, the alert and the metric were the same event with two consumers.

Backfill deserves a paragraph, because day one you will have history trapped in the provider's dashboard. The same endpoint handles it: export whatever the provider gives you — even a manual CSV — transform rows into the same JSON events with their original timestamps and transaction ids as idempotency keys, and replay them through /ingest. Because the store is idempotent and timestamped, backfill is safe to re-run and the charts render history and live data identically. One-off script, permanent history.

Be honest about the pattern's weakness: you are now the source of truth for your own revenue numbers, and a bug in your webhook handler is a bug in your analytics. The mitigation is reconciliation, not perfection — once a month, pull the provider's statement (the CSV button finally earns its keep) and diff the total against your ingested sum. If they match, your pipeline is proven. If they drift, you have a bounded window to search and, in our experience, a refund path you forgot to instrument.

Pull integrations are still better when they exist — they are independent of your code and audit you for free. But "this provider has no API" stopped being a reason for a blind spot on the revenue chart the day we inverted the arrow. The data was always there. It was just flowing in the other direction.