# ADR-118 Courier Tracking Backbone + Human-in-the-Loop CAPTCHA Relay

## Status

Proposed, 2026-07-27 (amended 2026-07-27 twice — DTDC screenshot-relay retired in favor of a manual
worklist; both-channels reconciliation + "Delivered (Unconfirmed)" state adopted).

## Status History

```yaml
status_history:
  - date: 2026-07-27
    status: Proposed
    changed_by: hkl
    reason: Initial proposal — tracking + verification strategy for DispatchWithAutomation.
    changed_via: adr-kit (360lm) — authored from DL CLI via SSH
  - date: 2026-07-27
    status: Proposed (amended)
    changed_by: hkl
    reason: |
      VCC built the DTDC screenshot-relay for real (dtdc-relay-proxy, Playwright/Chromium,
      real verified selectors from DTDC's live page, thread-safe session manager, TTL +
      concurrency cap, fail-closed auth) and tested it against a real historical AWB
      (7D125054944). It never reached the CAPTCHA: DTDC's own site returned its bot-
      protection "URL requested has been blocked" page before the tracking form even
      rendered — a WAF block independent of and prior to the CAPTCHA challenge this ADR
      was designed around. No evasion was attempted (rotating fingerprints, stealth
      plugins, etc.) — out of scope for this house. Given DTDC's CAPTCHA already required
      a human per-shipment even if the WAF weren't there, a server relay was never going to
      save that human a step over just deep-linking — so the fallback costs nothing lost.
      Decision: DTDC moves to the SAME manual-worklist pattern as Xpressbees (deep-link +
      rep reads the result and records it). The screenshot-relay design for DTDC is
      retired; dtdc-relay-proxy (container, ~2GB Playwright/Chromium image, compose entry,
      /opt/dtdc-relay-proxy source) was torn down completely, not left orphaned.
    changed_via: adr-kit (360lm) — VCC build session, dev-first
  - date: 2026-07-27
    status: Proposed (amended)
    changed_by: hkl
    reason: |
      Owner reported a same-day TIMEX incident: Shiprocket's aggregated status can falsely
      LEAD the courier's own portal, and two concrete bugs resulted — (1) a live-recheck
      skip-gate keyed on "Shiprocket says Delivered" froze the courier-direct status, so a
      report kept showing Delivered while the courier portal still said in-transit; (2) a
      `"DELIVERED" in status` substring check matched "UNDELIVERED-1ST ATTEMPT". Owner asked
      for an audit of this build's ported status logic against both bugs before adopting the
      TIMEX toolchain's fixes as-is (the staged copy at /opt/shiprocket-proxy-src/ predates
      them). Audit found bug (2) IS present here: `status_bucket` (in shiprocket_proxy.py)
      already had the UNDELIVERED-first check, but `classify_courier_stage` — which feeds
      `reconcile_status`'s direct-courier comparison — did not, so a courier status of
      "UNDELIVERED-1ST ATTEMPT" would be misread as courier_rank=Delivered and could
      override a correct Shiprocket "In Transit" read. A second, related bug was found in
      the same audit: `_write_shipment_events` tagged every mock-mode /track result
      `source='manual'` regardless of which channel (Shiprocket-aggregated vs a direct
      courier check) it actually represented — this would have silently corrupted the very
      "independent confirmation" signal this amendment depends on. Bug (1) (the skip-gate)
      does not currently exist as live code in this build — no "stuck/near-SLA" exception
      trigger has been implemented yet — but the principle is recorded here so it is not
      introduced incorrectly when that trigger IS built.

      Decision: adopt the same both-channels policy TIMEX adopted. A shipment shows
      "Delivered" as soon as any source (including Shiprocket alone) reports it — that part
      is unchanged and still useful. But it is flagged **"Delivered (Unconfirmed)"** until an
      INDEPENDENT source (anything with `shipment_events.source <> 'shiprocket'`) also
      observes it delivered. In this architecture, "independent confirmation" concretely
      means: the exception-only direct-courier tier (Blue Dart/Delhivery auto, this ADR's own
      §Implementation) OR the DTDC/Xpressbees manual worklists (a human reading the
      courier's own portal, per this ADR's first amendment) — Shiprocket's own aggregated
      feed is the backbone by default and can never itself supply the confirmation. Any
      future "which shipments get an exception-tier recheck" trigger MUST gate on
      `delivered_confirmed_by_courier = false`, never on "not yet Shiprocket-delivered" —
      gating on the latter is exactly TIMEX's bug (1).
    changed_via: adr-kit (360lm) — VCC build session, dev-first, reconcile-correctness audit
```

## Context

Shipment status/EDD must be tracked across a campaign's life. Two independent sources exist:
Shiprocket's aggregated tracking API (`courier/track/awb/{awb}` — no CAPTCHA, uniform, all
couriers) and each courier's own portal (an independent cross-check). Portal access varies:
**Blue Dart** = plain HTTP (server-automatable); **Delhivery** = headless browser (server-
automatable); **DTDC** = distorted-text image CAPTCHA; **Xpressbees** = Google reCAPTCHA v2
checkbox. A user-proposed design was to screenshot the CAPTCHA into the PWA, have the user answer,
and relay it back. Fable's review (DL 2026-07-27) confirmed this works for DTDC's image CAPTCHA but
is **architecturally impossible for Xpressbees**: a reCAPTCHA token is minted inside the courier
page's own browser context and bound to site-key + origin, so a click in the PWA produces nothing
relayable, and a datacenter-IP headless click usually escalates to an image-grid challenge.
Uncapped persistent browser sessions also threaten memory on the shared ~7.8 GB VPS.

## Decision

Make **Shiprocket's aggregated `courier/track/awb` the default server-side tracking backbone**
(polled by `shiprocket-proxy`, feeding `dispatch.shipment_events`). Treat **direct-courier-portal
verification as an exception-only tier** (run only for stuck / stale / near-SLA-breach shipments):
Blue Dart + Delhivery auto in the proxy; **DTDC and Xpressbees both via a queued manual worklist**
(PWA deep-links the AWB to the real courier tracking page; a human reads the status and pastes it
back — NOT a relay for either, as of the 2026-07-27 amendment above). The originally-designed DTDC
screenshot-relay (server-held persistent Playwright session, TTL + concurrency cap) was built,
tested against a real historical AWB, and retired after DTDC's own site blocked headless access
before the CAPTCHA even loaded — see the amendment entry for the full finding.

**Both-channels reconciliation (2026-07-27 amendment):** Shiprocket's aggregated read is never,
by itself, sufficient to call a shipment definitively Delivered — its own sync lags the courier's
real status by some amount, and treating it as terminal truth is exactly the failure mode a
same-day TIMEX incident hit. A shipment is shown **"Delivered (Unconfirmed)"** until an
independent source (a direct-courier check or a manual-worklist entry — anything with
`source <> 'shiprocket'`) also observes it delivered, at which point it becomes plain "Delivered".
This is computed in `dispatch.shipment_current`, not inside any single status-reconciliation call.

**Decision Maker:** hkl

## Implementation Notes

- **Backbone:** `shiprocket-proxy` polls `courier/track/awb`, diffs vs last-known, appends only changed observations to `dispatch.shipment_events` with `source='shiprocket'`.
- **DTDC (amended — manual, not relay):** worklist row + deep-link to `https://www.dtdc.com/track-your-shipment/`; human-entered status → `source='dtdc_manual'` (renamed from the original `dtdc_relay` enum value via `migrate_dispatch_v4.sql` — 0 rows existed under the old value, pure rename). Same UI pattern as Xpressbees, implemented as one shared `loadManualCourierWorklist(courierKey)` function in `dispatch/index.html`, not duplicated per courier.
- **Xpressbees:** worklist row + deep-link; human-entered status → `source='xpressbees_manual'`.
- **Exception trigger:** only cross-check where Shiprocket status is stale/stuck or courier EDD is within N days of SLA — not every shipment every poll. **Must gate on `dispatch.shipment_current.delivered_confirmed_by_courier = false`, never on "Shiprocket hasn't said Delivered yet"** — the latter is the exact skip-gate bug TIMEX hit (2026-07-27 amendment). No such trigger is implemented yet in this build (BD/Delhivery direct polling is still mock-only), so this is a constraint on the *future* implementation, not a fix to existing code.
- **Delivered (Unconfirmed) — how it's computed:** `dispatch.shipment_current` (`migrate_dispatch_v7.sql`) joins the latest `shipment_events` row per shipment against a check for any `source <> 'shiprocket'` row that also observed a delivered-shaped status (excluding `UNDELIVERED`-prefixed text, same substring trap as below). Exposes `delivered_confirmed_by_courier boolean` + a `display_status` that reads "Delivered (Unconfirmed)" until confirmed.
- **Substring bug (found + fixed 2026-07-27, auditing this exact function against the same-day TIMEX fix):** `status_bucket` already checked `"UNDELIVERED" in s` before `"DELIVERED" in s`; `classify_courier_stage` (feeding `reconcile_status`'s courier-side comparison) did not, so "UNDELIVERED-1ST ATTEMPT" could be misread as a Delivered-stage courier confirmation. Fixed to check UNDELIVERED first, same as `status_bucket`.
- **Source-tagging bug (found in the same audit):** `_write_shipment_events` tagged every mock-mode `/track` result `source='manual'` regardless of which channel it simulated — since mock-vs-live is orthogonal to which channel a result represents, this would have silently broken `delivered_confirmed_by_courier` (which trusts `source` to distinguish Shiprocket's aggregated feed from an independent check). Fixed via `_track_source_for(booking_point_id)`, derived from the booking-point's `kind`, not from the mock flag.
- **Retired:** the `dtdc-relay-proxy` container (Playwright/Chromium, ~2GB image), its compose service block, and `/opt/dtdc-relay-proxy` source were built, verified working up to the CAPTCHA step, then torn down completely after the WAF-block finding — nothing left orphaned on the shared VPS.
- **Find sites:** `grep -rn "dtdc_manual\|loadManualCourierWorklist\|delivered_confirmed_by_courier" /var/www/360lm/dispatch /opt/shiprocket-proxy`

## Alternatives Considered

- **Single-click PWA relay for Xpressbees reCAPTCHA.** Rejected: token is bound to the courier page's browser/origin; a PWA click yields no valid token, and checkbox clicks escalate to image grids.
- **Paid CAPTCHA-solving service (2Captcha-class).** Rejected for v1: ToS-grey, adds a vendor + failure mode; revisit only if Xpressbees volume justifies it.
- **Stream/remote-control the server browser into the PWA (VNC/CDP).** Rejected for v1: effectively a mini remote-desktop product, disproportionate for one courier; a Phase-3 option.
- **Rely on direct-portal checks as the primary source for all couriers.** Rejected: two need human CAPTCHAs; unscalable per-refresh.

## Consequences

**Positive:**
- Fully automated backbone for all couriers; human CAPTCHA effort collapses to a handful per campaign; DTDC still gets an independent cross-check; honest about Xpressbees' hard limit.

**Negative / Trade-offs:**
- Xpressbees direct verification stays manual; Shiprocket-aggregated is "good enough truth" for most shipments (not an independent portal reading).

**Risks and mitigations:**
- *Orphaned Playwright contexts leak memory on the shared VPS* → TTL reap + hard concurrency cap from day one (the load-bearing mitigation).
- *DTDC CAPTCHA expiry mid-relay* → short TTL + retry-with-fresh-screenshot on wrong answer.

## Related Decisions

- ADR-117 (the proxy that hosts this), ADR-120 (the event store this feeds), ADR-004 (maps/geocode for zones). Complements ADR-059.

## References

- `/var/www/360lm/docs/MDD_dispatch_automation.md` §9.
- Fable architecture review (CAPTCHA feasibility verdict), DL 2026-07-27.
- `…/shiprocket-timex/HANDOFF.md` (courier-status verification section; TIMEX CAPTCHA experience).