# Module Design Document — Dispatch (with Automation) / "DispatchWithAutomation"

**Status:** PLANNED (design) — 2026-07-27. Not yet built. Supersedes `MDD_dispatch.md` (the manual-only dispatch module); reuses and extends the existing `dispatch` DB schema.
**Origin:** Productization of the proven TIMEX Shiprocket bulk-dispatch toolchain (authored/operated on DL — see `C:\Users\Lenovo\Documents\shiprocket-timex\HANDOFF.md`), merged with the existing manual dispatch PWA. Design reviewed by Fable (architecture feasibility) and a web/GitHub research sweep (courier-selection rule frameworks). Source plan: `DispatchWithAutomation_PLAN.md` (DL).
**Module:** `dispatch` (evolution — schema reuse), dir target `/var/www/360lm/dispatch/` (see §2 dir-anomaly resolution).
**ADRs complied-with (existing):** see §17. **New ADRs to author + Accept before/at build:** ADR-117…ADR-122 (proposed, §17).
**Governance:** VPS-side build under 360LM rules — parallel-session locks, dev-first migrations (ADR-015/DB_MIGRATIONS), VCC checklist (ADR-068), ADR process (ADR-107), cross-PWA safety gate (ADR-067), `/new-pwa` Q0 for build mechanics.

---

## 1. Why this exists

360DLM runs bulk B2B dispatch campaigns — send marketing material (e.g. 1 standee + 200 leaflets)
to dozens/hundreds of dealer stores per campaign, pan-India, prepaid, heavy-ish (~3 kg). The TIMEX
campaign proved a full laptop-bound Python toolchain for this (Shiprocket booking → tracking →
dual-status client reports → performance analysis), but it is single-operator, single-campaign,
and depends on human-in-the-loop steps that only run on one laptop. Separately, an earlier **manual
dispatch PWA** exists (manual booking + manual carrier tracking, no API; dev-ready, prod-broken —
0 rows ever). This module **merges** the two: the manual capability plus Shiprocket (and future
booking-point) automation, as a multi-user 360LM PWA that runs a campaign end-to-end:

**intake → store consolidation → rule-driven courier booking → lifecycle tracking (EDD commitment → actual) → NDR handling → proof-of-delivery → client & MIS reports → performance analysis.**

A new campaign is configuration, not code.

## 2. Relationship to the existing `dispatch` module (the merger)

The existing module (`MDD_dispatch.md`, schema `dispatch` live on **dev** `lm360`, absent on prod)
already provides a strong base to **reuse and extend, not replace**:

| Existing asset | Reuse in this module |
|---|---|
| `dispatch.dispatches` (client, job_ref, courier, dates, status, counts, remarks) | Becomes/links **campaign + order** headers |
| `dispatch.consignees` (name, city, address, phone, pincode, material, qty, docket, status, last_event, last_scan_location, last_tracked_at, **manual_status, manual_delivered, manual_called_by, manual_remarks, pod_uploaded**) | Becomes the **shipment** row; the `manual_*` + `pod_uploaded` columns already model the **call-log + proof-of-delivery** requirements |
| `dispatch.tracking_events` (append-only: docket, status, event_text, scan_location, fetched_at) | Foundation for the **append-only lifecycle event store** (§6/§9) — extend with EDD + source |
| `dispatch-ai` proxy (llava OCR, gemma3 address parsing) | Foundation for **AI Excel field-mapping / address parsing** on intake |
| 5 panes (new / dashboard / track / history / config-master) | Kept + extended |

**Two inherited compliance gaps to fix in this build** (from `MDD_dispatch.md` §6a): ADR-001 `?next=`
redirect (was NOT implemented) and ADR-081/002 safe-bottom (was NOT implemented).

**Directory-anomaly resolution (was an open decision):** fold `/var/www/dispatch/` into
`/var/www/360lm/dispatch/` and retire the standalone `dispatch-pwa` nginx container + its Traefik
router, so the module gets git tracking + `deploy-prod.sh` parity + dev-host serving. Touches
TRAEFIK + DOCKER_SHARED zones → a locked change window during the build (ADR-067 safety gate).

## 3. Scope & inspiration

**Booking-model inspiration (research-backed — mirror, don't reinvent):** the engine mirrors
**Shiprocket's own allocation model** (Courier Recommendation Engine modes: Rating / Price /
Delivery-perf / Pickup-perf; Courier Rules with 10 conditions; Courier Priority list) so nothing is
lost by booking through it, and adds 360DLM's policy rules on top. Structural template = **Pickrr's
rule shape** `{Rule Name, Order-Matching conditions, Courier-Selection mode, Courier Blacklist, Rule
Priority}`. Zone model = Shiprocket **A–E**. Wider patterns studied: ShipStation Rate Shopper,
Sendcloud rules, Cahoot (cheapest-within-SLA), EasyPost SmartRate (percentile transit time),
ClickPost, carrier-scorecard practice. OSS studied (rate-normalize layer, not a ready rules DSL):
Karrio, Purplship, ShipperHQ. (Full source list in the DL research task output; summarized in
`DispatchWithAutomation_PLAN.md` §18.)

**Out of scope v1:** consumer-facing tracking; multi-language; SMS/WhatsApp from the PWA (Shiprocket
already notifies consignees); automated pickup scheduling; automated NDR *action* (panel-side, kept
manual); streamed-browser CAPTCHA for Xpressbees; carrier-scorecard allocation (needs accumulated
history first).

## 4. Build phases

- **Phase 1 (MVP):** merged intake (Excel AI-map + manual) → store consolidation into counter-master
  → booking rules engine + replay-test harness → rule-driven auto-book via Shiprocket → tracking
  backbone (Shiprocket aggregated `courier/track/awb` → event store) → MIS + Client reports →
  retention/deletion + CSV-export-before-delete → auth/roles. Basic manual (non-API) booking retained.
- **Phase 2:** direct-courier verification tier (Blue Dart HTTP + Delhivery headless auto; DTDC
  CAPTCHA screenshot-relay; Xpressbees manual worklist); NDR surface + deep-link + log;
  proof-of-delivery (call logs + photo → PPTX, Recce/ADR-095 pattern).
- **Phase 3:** additional booking points (360DL Shiprocket account, Blue Dart direct B2B API);
  carrier scorecard / performance-by-lane allocation; cheapest-within-SLA; streamed-browser Xpressbees
  and/or panel automation for pickup/NDR *if* proven.

(Phase-1 inclusions of proof-of-delivery and/or verification are an open decision — §19.)

## 5. Architecture — the Python / PostgREST boundary (Fable-reviewed)

Idiomatic 360LM: **vanilla-JS PWA (ADR-013) + PostgREST/Postgres (ADR-014) + a containerized
`shiprocket-proxy` sidecar** (precedent: ADR-059 tour-ai sidecar; ADR-105 JWT auth; ADR-010
cross-schema-via-proxy). **Rule (Fable): "Python touches the outside world and normalizes it;
Postgres never sees a raw courier string."**

- **`shiprocket-proxy` (Python) owns:** Shiprocket API orchestration (booking, wallet checks, AWB
  assign incl. the serviceability-vs-assign mismatch + silent-AWB-revert detection); fastest/cheapest
  selection (inseparable from the live rate fetch); ALL browser automation + CAPTCHA relay; the
  multi-format date parser + the DELIVERED/UNDELIVERED-substring fix (status normalization at
  ingestion); AI field-mapping; Excel + PPTX generation. Auth: ADR-105 Bearer-JWT, fail-closed.
- **PostgREST/Postgres owns:** campaign / order / store-master CRUD; the event store + derived views;
  MIS & courier-performance analytics (SQL's strength); retention/deletion rules; booking-approval
  state; call-log & proof-photo metadata.
- Proxy writes **normalized** events into `dispatch.*`; the PWA reads only via PostgREST.
- **Watch (Fable):** boundary creep — proxy writes events, SQL computes everything derived.

## 6. Data model — schema `dispatch` (extend; dev-first per ADR-015, own-schema per ADR-009)

Reuse existing tables (§2); add:
- `campaigns` (client, name, status active|closed, dates, `theme_preset` for client view).
- `stores` — the **counter-master** (§7): normalized name/address/city/state/PIN/phone(s)/geo;
  `store_aliases(alias_normalized, store_id)` for learned matches. (Consolidate with the existing
  `counters` module master where appropriate — cross-schema read via ADR-010/106 pattern; final
  matching key set is an open item, §19.)
- `orders` / `shipments` — extend `consignees`: campaign + store link; package dims/weight;
  `booking_point` (§15); chosen courier; AWB; **`booked_edd` (immutable commitment)**.
- `shipment_events` — **append-only** (extend `tracking_events`): `(shipment_id, event_time,
  observed_at, status_normalized, status_raw, edd, source, raw_payload jsonb)`, one row per *observed
  change* (proxy diffs vs last-known). `source ∈ {shiprocket, bluedart, delhivery, xpressbees_manual,
  dtdc_relay, booking, manual}`.
- `shipment_current` — view (or trigger-maintained): latest status + latest EDD + booked_edd + delta.
- `shipment_final` — **physically separate, DELETE-protected** table (REVOKE DELETE / BEFORE-DELETE
  trigger): booking details, cost, courier, AWB, booked EDD, final delivery date, final status.
- `call_logs` — rep confirmation calls (reuse/extend `manual_called_by`/`manual_remarks`).
- `proof_photos` — shipment, file ref, caption, uploaded_by, ts (feeds PPTX; reuse `pod_uploaded`).
- `booking_rules` — the rules-engine config (§8). `booking_points` — the extensibility abstraction (§15).

**Fable's two modeling pitfalls to respect:** (1) store BOTH `observed_at` and courier `event_time`
— couriers backfill/reorder; define "EDD changed" as a change between consecutive *observations*.
(2) AWB can go backwards (assigned→unassigned→new) — model AWB as a changeable attribute, never part
of shipment identity.

## 7. Store consolidation → counter-master (Fable's tiered matching)

On intake, consolidate each target into `stores`:
- **Tier 1 (auto-link + enrich):** exact on normalized composite key — `normalize(name)` (upper,
  strip punctuation/&/-, drop noise tokens M/S, THE) **+ PIN** (+ phone if present); phone alone
  (10-digit) is also near-certain. Enrich blanks only; never silently overwrite (conflicts → review note).
- **Tier 2 (suggest, human confirms):** same PIN + `pg_trgm` name trigram (~0.45–0.55), or same
  phone + different PIN → one-click confirm/reject worklist.
- **Tier 3 (no match):** hold for review with a default-accept "insert as new" one-click.
- Write a `store_aliases` row on every confirmed match (next upload of that spelling → Tier-1).
- **Biggest risk (Fable): false-merge ≫ duplication** — a false-merge mails one dealer's material to
  another and pollutes that store's analytics forever. Bias every threshold to "ask the human"; log
  which upload+rule created each link (traceable/reversible).

## 8. Booking rules engine (research-backed)

Books via Shiprocket, so the engine mirrors Shiprocket's allocation model + 360DLM policy on top.
**v1 rule dimensions (must-have):** (1) **Zone A–E** from pickup+delivery PIN (backbone); (2)
**carrier priority list + restrict/blacklist** per condition (first serviceable wins — Shiprocket's
model); (3) **zone→mode→carrier** mapping to encode policy ("Zone-D out-of-state → Blue Dart",
"far zones → Air-only") as explicit editable rules; (4) **weight-slab**; (5) **serviceability gate**
(drop couriers not covering the destination PIN); (6) **selection-mode tie-break** (fastest /
cheapest / best-rated, reusing Shiprocket's own rate/ETD/rating fields). **Later:** cheapest-within-SLA;
carrier scorecard / performance-by-lane (from own history); order-value/SKU conditions.

**Booking mode:** rule-driven **auto-book** (user decision — a wrong courier is recoverable via panel
reassign or cancel). **Mandatory safety before trusting it:** a **replay test harness** that runs the
rules against real past campaigns (TIMEX etc.) and confirms they'd pick what was actually picked.

## 9. Tracking & status backbone (Fable-reviewed)

- **Default server-side truth = Shiprocket's aggregated `courier/track/awb`** (no CAPTCHA, all
  couriers) → feeds `shipment_events`.
- **Direct-courier verification = exception-only tier** (only stuck/stale/near-SLA-breach — not all
  every refresh): Blue Dart (plain HTTP) + Delhivery (headless) auto in the proxy; **DTDC via the
  screenshot-relay** (server-held **persistent Playwright session**: show CAPTCHA in PWA → user types
  → proxy submits into the same live session; session registry, ~2–3 min TTL, concurrency cap);
  **Xpressbees (reCAPTCHA checkbox) = queued manual worklist** — single-click relay is architecturally
  impossible (token bound to the courier page's own browser+domain), so deep-link the AWB and have a
  human paste the observed status back. Streamed-browser remote-control is a Phase-3 option.
- **Key risk (Fable):** orphaned Playwright contexts leak memory on the shared VPS — TTL-reap
  aggressively, cap concurrency from day one.

## 10. Pickup scheduling — fully manual (v1)

Shiprocket's pickup API 403'd for this account; pickups mostly auto-schedule anyway. v1: the PWA
records pickup status from tracking; scheduling is a human/panel action. No panel automation until
explicitly proven and re-approved.

## 11. NDR handling — surface + deep-link + log

PWA shows the live NDR list (`ndr/all` API), one-click deep-links to act in the Shiprocket panel (the
reliable path — the `ndr/reattempt` API dead-ends once Shiprocket auto-actions), and logs the
resolution + buyer-call notes (ties into §12). No fragile panel automation for the action in v1.

## 12. Proof of delivery — call logs + photo → PPTX (reuse Recce / ADR-095)

Reps log delivery-confirmation calls (outcome + notes); attach delivery photos via the PWA
(ADR-072 capture/annotation, ADR-085 upload); generate the client proof deck **inside the PWA reusing
the Recce PWA's photo→PPTX pipeline** (ADR-095 client-deck contract, ADR-024 GAS/slides path or
proxy-side python-pptx — pick one, don't build new). Files served via signed proxy (ADR-049).

## 13. Reports

- **MIS report — login-only, private, FULL detail** (phone, address, cost, everything). Internal.
- **Client report — share link** (external-client access via ADR-048 magic-link, not hub-PIN):
  drill-down overview + Delivery Status page **with address + phone** (user's confirmed stance).
  Link-security is an open decision (§19.2). Screenshot-deterrence per ADR-035.
- Both are PostgREST-native consumers (ADR-074 accept-profile) of the same data; Excel export retained
  (legacy channel). Generalizes the current standalone `timex-dashboard` (retire it post-transition).

## 14. Retention & deletion

Deletable: the transactional timeline (`shipment_events` etc.). Never deletable: the **final record**
(`shipment_final` — booking incl. cost + final delivery date; enforced by REVOKE DELETE / BEFORE-DELETE
trigger — extends ADR-041 supersede-not-delete). **Who/when:** admin/ops-lead only, only after a
campaign is marked **closed** (ADR-045 edit/delete governance). **CSV export before delete is
mandatory** — the PWA must let the user download the data (phone/desktop) before the Postgres delete runs.

## 15. Auth, roles, tenancy / booking-points

- **Auth:** hub SSO (ADR-012) for internal staff, `?next=` (ADR-001), PIN (ADR-011). Client report =
  external magic-link (ADR-048), RPC-only client surface (ADR-050).
- **Roles (proposed, §19):** admin/ops-lead (campaign close, deletion, rule config), rep/operator
  (intake, booking, call logs, photos, NDR), viewer (read MIS).
- **Booking-points (extensibility):** one active Shiprocket account (YEMO) now, modeled as a
  first-class `booking_point` so future ones drop in without a rewrite: 360DL Shiprocket account
  (business@360degreelogicalmktg.com), Blue Dart direct B2B API, plus the manual (non-API) path. Each
  shipment records its booking-point. (May warrant ADR-122.)

## 16. Data sensitivity & access model

- **MIS:** full PII, login-gated, internal only.
- **Client report:** carries address + phone (client's own dealers) on an external link — the
  security of that link is an explicit open decision (§19.2: per-client password/login vs unlisted vs
  expiring). This *reverses* the TIMEX web-dashboard's PII-trim, so it must be a conscious, recorded call.
- **Creds:** Shiprocket credentials live only in the `shiprocket-proxy` (env/secret), never in the DB
  or client. No consignee PII in any public/unauthenticated JSON unless the §19.2 decision permits it.

## 17. ADR compliance checklist (per ADR-107 discipline)

**Complied-with (existing ADRs):** ADR-001 (?next=, FIX inherited gap), ADR-002/081 (safe-bottom, FIX
inherited gap), ADR-003 (scroll), ADR-005 (SW cache-ver), ADR-009 (own schema — reuse `dispatch`),
ADR-010 (cross-schema via proxy — read counters/stores), ADR-011/012 (PIN + hub SSO), ADR-013
(single-file — see risk §18), ADR-014 (PostgREST), ADR-015 (dev-first), ADR-016 (hub registry),
ADR-017/073 (Traefik), ADR-035 (screenshot deterrence — client view), ADR-041 (supersede-not-delete),
ADR-045 (edit/delete governance), ADR-048 (external client magic-link — client report), ADR-049 (HMAC
signed file proxy — proof photos/PPTX), ADR-050 (client auth RPC-only), ADR-053/054 (counter/store
master), ADR-059 (dedicated sidecar proxy — precedent for shiprocket-proxy), ADR-060 (prod deploy
rsync), ADR-062 (live AI pipeline — field mapping), ADR-066 (parallel session), ADR-067 (cross-PWA
safety gate), ADR-068 (VCC), ADR-069 (hotfix), ADR-070 (IST), ADR-071 (Indian amounts — cost),
ADR-072 (proof capture/annotation), ADR-074 (accept-profile), ADR-075 (RPC errors), ADR-076
(viewport), ADR-083 (form validation), ADR-085 (file upload), ADR-095 (client-deck/PPTX contract),
ADR-100 (theme-v2 presets), ADR-105 (JWT for proxy), ADR-106 (cross-schema grant-gated), ADR-113
(email/tel validation), ADR-022 (Playwright e2e).

**New ADRs to author + Accept before/at build (proposed):**
- **ADR-117** — Courier dispatch via external-aggregator proxy service (`shiprocket-proxy`): creds
  isolation, booking orchestration, serviceability/assign reconciliation (extends ADR-059 + ADR-105 + ADR-010).
- **ADR-118** — Human-in-the-loop CAPTCHA relay via persistent proxy browser session (DTDC), with
  Xpressbees manual-worklist fallback and Shiprocket-aggregated as default.
- **ADR-119** — Rule-driven courier auto-selection mirroring Shiprocket allocation, gated by a
  replay-test harness before trust.
- **ADR-120** — Append-only shipment lifecycle event store + delete-protected immutable final record
  + admin-gated post-close retention purge with mandatory CSV export.
- **ADR-121** — Dispatch-target consolidation into the counter/store master via tiered exact+fuzzy
  matching with human review (extends ADR-053/054).
- **ADR-122** (maybe) — Booking-point abstraction for multi-carrier-account/direct-API extensibility.

## 18. Red-team / failure modes (Fable + carried TIMEX lessons)

- **Rule-driven auto-book = highest risk** (real money, live AWBs, silent-AWB-revert) → the replay
  harness is mandatory; keep panel reassign/cancel as the recovery net.
- **False-merge in store consolidation** (§7) → bias to human review, log every link.
- **Playwright/CAPTCHA session leaks** on the shared 7.8 GB VPS → TTL-reap + concurrency cap.
- **`"DELIVERED" ⊂ "UNDELIVERED"` substring trap** (real bug hit + fixed in TIMEX) → normalize status
  in Python at ingestion; never naive-substring a negative-form status; keep the fix in the proxy.
- **Serviceability list ≠ what assign-AWB accepts**, and **AWB can silently revert** → keep the
  empirically-tuned Python retry/verify logic; re-verify AWB presence after every booking run.
- **Loose multi-format dates** (ISO / "Mon,27th Jul'26" / Delhivery ranges) → keep `parse_loose_date`
  + 90-day sanity clamp in the proxy.
- **Panel-dependent actions** (pickup, NDR action) can't be reliably server-automated → kept manual/worklist.
- **Client PII on an external link** → §19.2 must be decided before any client link is issued.
- **ADR-013 single-file vs app size** — this is a large multi-function app; if a single HTML file
  becomes unmaintainable, author an ADR to permit a controlled split rather than silently violating ADR-013.

## 19. Open decisions (need user sign-off before/at build)

1. **Phasing** — is §4 right, or pull proof-of-delivery and/or direct-courier verification into Phase 1?
2. **Client share-link security** — client report carries address+phone: per-client password/login vs
   unlisted URL vs expiring link? (Reverses the TIMEX PII-trim — must be recorded.)
3. **Role model** — admin / rep / viewer (§15) or different?
4. **Existing `timex-dashboard`** — retire once the PWA client report covers it, or keep during transition?
5. **Manual (non-API) booking** — basic in Phase 1 (honoring the merger) or Phase 3 with other booking points?
6. **Counter-master matching key set** — confirm the exact Tier-1 composite key vs the existing `counters` module master.
7. **Directory anomaly** (§2) — confirm folding `/var/www/dispatch/` into `/var/www/360lm/dispatch/` + retiring the standalone container.

## 20. Lessons carried forward from the TIMEX toolchain (do not re-encounter)

- The `"DELIVERED"⊂"UNDELIVERED"` status miscount (silently counted NDRs as delivered) — §18.
- `ndr/reattempt` API dead-ends once Shiprocket auto-actions → panel is the reliable NDR path (§11).
- Xpressbees/DTDC CAPTCHA can't be server-automated unattended (§9); Shiprocket-aggregated is the backbone.
- Scoped courier-status re-checks are cheaper than full-batch (feed the exception-only tier, §9).
- ETD buckets must be granular (per-day, not a vague "this week") for client legibility.
- Full stage-wise EDD/status history (commitment → actual) is the basis of performance analysis (§6/§9).
- (Full detail: `HANDOFF.md` in the DL shiprocket-timex folder.)

## 21. References

- DL plan: `C:\Users\Lenovo\Documents\shiprocket-timex\DispatchWithAutomation_PLAN.md`.
- TIMEX toolchain handoff + lessons: `…\shiprocket-timex\HANDOFF.md`.
- Fable architecture review (CAPTCHA relay, Python/JS split, event model, store matching) — DL session, 2026-07-27.
- Courier-rule research (Shiprocket CORE + Courier Rules, zones A–E, ShipStation/Sendcloud/Cahoot/
  EasyPost/ClickPost/Pickrr, Karrio/Purplship OSS, carrier scorecards) — DL research task, 2026-07-27.
- Supersedes: `/var/www/dispatch/MDD_dispatch.md` (manual-only dispatch, dev-ready/prod-broken).

---
**END. Status: PLANNED. Next: user sign-off on §19 open decisions → author ADR-117…122 (`/adr`) →
`/new-pwa` Q0 for build mechanics → Phase-1 build (dev-first).**

---
## 22. §19 decisions RESOLVED + ADRs authored (2026-07-27 12:12)
All 7 §19 open decisions resolved (see `dispatch_automation_build_spec.md` §"§19 decisions — RESOLVED"):
complete phased build; client report = unlisted URL with PII (ADR-123); Admin+Rep+Viewer; keep-then-retire
timex-dashboard; manual booking in Phase 1; tiered store-match WITH user approval (ADR-121); fold the dir-anomaly.
ADR-117…123 authored (Proposed) in `docs/adr/` + indexed. /new-pwa Q0 answers + phased build plan in
`docs/dispatch_automation_build_spec.md`. Ready for VPS-side dev-first build (Phase 0 → 3).