OLDGABE — research log

Does the CAUTION-recovery re-ranker actually beat the production tier scorer? — head-to-head + de-confounding

The framing. OLDGABE's fusion stage sorts every candidate fire into tiers; the great majority land in a low-confidence CAUTION tier and are never surfaced. The question that had never been measured: among those CAUTION events, can a learned recovery model rank the genuine hidden fires better than the production tier/confidence scorer does?

BLUF. On the operationally meaningful metrics the recovery model decisively beats production. At an 80%-precision operating point it recovers 60.5% of confirmed real fires vs production's 9.5% (Truth1), and 27.2% vs 0% under a second, HMS-independent truth (Truth2). Production holds a small global-AUC edge on Truth1 (0.951 vs 0.936) — but that edge is a confound: production uses the NOAA HMS smoke channel, and Truth1 is 91.5% HMS-confirmed, so it is partly scoring HMS against HMS. When the truth is switched to an independent polar-satellite corroboration, production's AUC edge reverses (recovery 0.845 vs production 0.786).

Populations & truths (reproducible). Candidate set = every canonical event with event_corroboration_tier.tier='CAUTION' that has an event_ground_truth row (n = 197,325). Truth1 (multi-source) = event_ground_truth.confirmed (6,867 positive, 3.48%; for CAUTION this channel is 91.5% NOAA-HMS-confirmed). Truth2 (neutral polar) = a VIIRS/MODIS active-fire detection from a different canonical event within 5 km and −1..+14 days of the event's first detection (7,791 positive, 3.95%). The two truths overlap only partially (both-positive 1,753; Truth1-only 5,114; Truth2-only 6,038), so Truth2 is a genuine cross-check, not a relabelling.

Features (non-leaky, detection-time only). Evidence is joined excluding the confirmation channels IRWIN, HMS, SAR and S2. Per event: n_det, n_fam, max/mean_frp, max/mean_conf, span_h, n_days, max_ti4, ti4−ti5 contrast, frac_night, mean_pixel_area, max_ti5, abi_temp_k, abi_area, modis_brightness, mean_hour, fp_dist_km, frp_per_pixel. Model: gradient-boosted trees (HistGradientBoostingClassifier, class-weight balanced), scored out-of-fold under 5-fold GroupKFold grouped on 0.2° spatial cells (a held-out cell is never in training). The production scorer is the fixed tier_and_conf() confidence; its value is pinned at 0.02 for ~94% of CAUTION events (the GOES/ABI-only majority), which is why it cannot rank within the tier.

Result.

De-confounding (does the win survive removing the reporting channel?). Feature-group AUCs, per truth — month-only / geography-only (lat,lon) / detection-only / all:

Reads: (1) Season is not a confound — the candidate window is summer-only, so the summer-share of positives and of negatives is identical (gap = 0) and month-only ranking is near-chance (~0.57). (2) Geography is a large prior under both truths (0.903 vs 0.877), which already argues it is real fire geography rather than pure reporting bias — a reporting-only signal would predict the HMS truth far better than the polar truth. (3) Production's Truth1 AUC lead does not survive the neutral truth. The honest operating-point story is unchanged: production cannot deliver a high-precision set within CAUTION; the recovery model can.

Adding geography to the recovery re-ranker — a defensible lift that grows under the neutral truth

The framing. Entry 0001 showed geography alone ranks CAUTION fires surprisingly well. Wildfires cluster by fuel, terrain and climate, so a location prior should carry real signal — but a location prior can also just memorise where fires get reported. This entry adds geography to the recovery model and tests, honestly, whether the lift is real fire behaviour or a reporting/coverage artifact.

BLUF. Adding raw lat/lon to the detection feature set (recovery+geo) is a defensible improvement over both the detection-only recovery model and production, on both truths and every metric. The decisive de-confounding result: the geo lift is larger under the HMS-independent polar truth (+0.081 AUC) than under the reporting-heavy truth (+0.036 AUC). If geography were merely encoding reporting bias, its benefit would shrink when the truth stops depending on reporting — instead it grows. So the geographic signal is real fire geography.

What "geo" is (hard data bound). OLDGABE currently has no populated land-cover / fuel / terrain / climatology layer — the fire-weather cell table is empty, weather stations are a current snapshot (temporally invalid for historical events), and the raw detection JSON carries no geographic fields. So the only non-leaky geographic priors available are raw lat/lon + distance-to-known-false-source. A Truth1-only supplementary that adds an out-of-neighbourhood polar-thermal climatology density (a fuel/fire-proneness proxy; circular against Truth2, hence Truth1-only) lifts a little further (AUC 0.978, recall@80% 0.771), indicating headroom if a real independent fuel/climatology layer were ingested.

Method delta from entry 0001. Same populations (n = 202,005; the DB ingested more June/July events between runs), same non-leaky features, same 5-fold GroupKFold on 0.2° cells, same two truths. New models: recovery+geo = detection features + lat + lon; geo-only = lat,lon; all = detection + lat + lon + month. A June↔July temporal-transport split is added as the only distribution shift the data permits.

Result — head-to-head.

The geo lift over detection-only is +0.036 AUC on Truth1 and +0.081 AUC on Truth2 — it grows under the neutral truth. recovery+geo also closes and reverses the one metric where production had led (Truth1 AUC, now 0.973 vs 0.950). Detection and geography are complementary: recovery+geo (0.941) beats both detection-only (0.860) and geo-only (0.886), so neither is redundant. Month adds nothing (all ≈ recovery+geo).

Result — temporal transport (train one month, test the other).

recovery+geo beats detection-only in all four cells and production in three of four (it loses only Jun→Jul on the HMS-confounded Truth1, by 0.007). Under the neutral truth it wins both directions by wide margins, and it stabilises the hard direction (Jun→Jul detection-only falls to 0.888; recovery+geo holds 0.934). Within the available window, the geographic prior transports across time and does not hurt.

The cross-sensor fusion safety-net: two independent satellites agreeing is 4.5× better than one

The framing. Each satellite makes its own noisy fire hunches. OLDGABE's fusion daemon is the “two-witnesses” rule: it links detections of the same fire across sensors and only trusts a candidate strongly when independent instruments agree. This entry measures how much that agreement is worth.

BLUF. OLDGABE ingests ~1.80 million raw per-sensor fire detections. The fusion daemon cross-links them into 11,441 multi-sensor clusters and grades them. Requiring ≥2 independent CORE thermal sensors to agree (the “confirmed” tier, n=620) raises the IRWIN/perimeter match rate to 37.7%, versus 8.4% for a random raw single detection — a 4.5× precision lift — while GOES-only single-frame candidates (the “early” tier) match just 5.5%. Separately, the false-positive catalog + status logic demote 620,160 raw candidates (34.4%) to masked-fp so they never reach the map.

Result.

Dozier subpixel strict-gate: a clean precision-for-recall trade, and why we keep it out of fusion

The framing. The Dozier subpixel method estimates what fraction of a coarse GOES pixel is actually burning (p_subpixel). Tightening a threshold on that fraction is a dial between “catch everything, mostly noise” and “only the unambiguous, but almost nothing.”

BLUF. Gating on p_subpixel trades recall for precision cleanly: the IRWIN match rate rises from 2.8% (ungated) to ~18–27% at p_subpixel ≥ 0.005–0.01, but keeps only ~10–19% of Dozier's own positives. Because the high-precision operating point yields so few detections — and essentially none the polar sensors don't already provide — OLDGABE's fusion daemon deliberately excludes Dozier. A measured “keep it, but don't fuse it” decision, not an oversight.

Result (60,000 sampled Dozier detections; 1,681 IRWIN-positive = 2.8% base):

The deployed strict gate independently scores 26.3% precision (G19) / 48.3% (G18) at ~1.3% whole-fire recall in the daily-scorer (entry 0007) — consistent with this sweep.

Negative result: GOES temporal persistence does not add defensible recall beyond polar

The framing. A geostationary satellite stares at the same spot every few minutes, so a tempting idea is: “if GOES keeps seeing a hotspot in one place for hours, that persistence is extra evidence of a real fire the polar satellites missed between overpasses.” We tested it. It does not hold up.

BLUF. GOES near-match “recall” is saturated by coverage: 92.4% of IRWIN fire cells have some GOES detection within ~5 km, simply because GOES-ABI emits candidate hotspots almost everywhere over CONUS (its measured precision is only ~15% — 85% are not fire). Requiring temporal persistence (a cell seen in ≥3 distinct hours) only drops this coverage-inflated number (92.4% → 66.1%); it never adds genuine recall. The apparent 43.8% “marginal over polar” is the same coverage/false-positive floor, not independent detections. Conclusion: GOES temporal persistence is not a validated recall lever — OLDGABE's defensible cross-sensor recall comes from polar corroboration and fusion (entry 0003).

Result (45-day window; 2,592 IRWIN truth cells at 0.05°):

Wildfire-camera smoke detection: highest precision of any source, but coverage/visibility-bound

The framing. OLDGABE runs a smoke/flame classifier on public wildfire-camera frames and, when two cameras see the same plume, triangulates the bearings into a map location. Cameras see smoke a satellite can't — but only where a camera is pointed, in daylight, through clear air, within range.

BLUF. When a camera fires on a fire IRWIN also records, it is usually right — daily-scorer precision 66.7%, the highest of any single OLDGABE source. But it matches only 1.2% of IRWIN fires: coverage and visibility (line-of-sight, daylight, haze, range) are the binding constraint, not the classifier. Honest status: a high-precision corroborator where cameras can see, with no proven standalone field value yet at network scale.

Result (daily-scorer, vs IRWIN 10 nm / 24 h):

Per-sensor scorecard vs NIFC IRWIN — what each US datalink actually contributes

The framing. Rather than argue which feed matters, OLDGABE runs a daily replay scorer that matches every sensor's detections to NIFC IRWIN ground truth and records precision, recall and count per source. This entry is that scorecard, unedited.

BLUF. Polar VIIRS is the best single recall+precision workhorse; the fused OLDGABE output leads recall (72.4%); cameras lead precision (66.7%); GOES-ABI is high-volume/low-precision; SAR and Sentinel-2 SWIR contributed nothing in this window. This is the empirical basis for weighting the fusion daemon (entry 0003) toward polar + agreement.

Result (daily-scorer run 18, 24 h window ending 2026-07-08, match 10 nm / 24 h vs IRWIN):