# ADR-019: OCR Strategy — Client-Side Tesseract for Identity Docs, Server-Side Proxy for Bill Photos

## Status

Accepted, 2026-06-25.

## Status History

```yaml
status_history:
  - date: 2026-06-25
    status: Proposed
    changed_by: hkl
    reason: Formalising split OCR strategy across HR and Expense PWAs
    changed_via: adr-kit (360lm)
  - date: 2026-06-25
    status: Accepted
    changed_by: hkl
    reason: Both OCR paths live and stable across HR and Expense PWAs
    changed_via: adr-kit (360lm)
```

## Context

Two PWAs need OCR: (1) HR needs to extract data from identity documents (Aadhar, PAN, Voter ID) uploaded during employee onboarding — structured text on clean printed cards; (2) Expense needs to extract amounts and merchant names from bill photos taken in the field — variable lighting, angles, handwriting, and languages. These are fundamentally different OCR problems with different accuracy requirements and data sensitivity profiles. Identity documents contain PII that should not leave the device if avoidable.

## Decision

**HR PWA — Tesseract.js (client-side, in-browser):** Identity document images are processed entirely in the browser using Tesseract.js. No image is sent to a server. Suitable because: documents are clean printed text, Tesseract accuracy is acceptable for structured cards, PII stays on-device.

**Expense PWA — Server-side OCR proxy (`/ocr-proxy/`, port 8766):** Bill photos are sent to the server-side OCR proxy which uses a fallback chain: Gemini Flash 2.0 (primary, best accuracy for handwritten/multilingual bills) → Ollama llava:7B (local fallback, slower but free, ~30-50s cold start). Suitable because: bill photos are noisy and variable, vision LLMs outperform traditional OCR for this use case, cost is acceptable per-bill.

New PWAs needing OCR must choose the appropriate path based on: (a) whether input is clean structured text (→ Tesseract client-side) or noisy/variable photos (→ server proxy), and (b) whether the data is PII that should not leave the device (→ client-side only).

**Decision Maker:** hkl

## Alternatives Considered

- **Tesseract.js for everything.** Rejected: Tesseract accuracy on real-world bill photos (varied angles, lighting, handwritten amounts, mixed Hindi/English) is too low — expense extraction became unreliable in testing; vision LLMs are significantly more accurate for this task.
- **Server-side OCR for everything (including identity docs).** Rejected: Aadhar/PAN cards contain PII; sending them to a server (even self-hosted) creates a data collection point; client-side processing eliminates this risk at no functional cost since Tesseract handles structured printed text well.
- **Google Cloud Vision API for bills.** Rejected: cost scales with volume; requires exposing an API key; Gemini Flash 2.0 via the same AI proxy provides comparable accuracy; Ollama local fallback provides resilience at zero per-call cost.
- **Single cloud vision API for both document types.** Rejected: same PII concern for identity documents; over-engineered for HR onboarding frequency (< 5 new employees/month) where Tesseract is sufficient.

## Consequences

**Positive:**
- Identity doc PII never leaves the user's device.
- Bill OCR uses best-available model (Gemini Flash 2.0) with local fallback (Ollama) — no single point of failure.
- Client-side Tesseract adds no server cost regardless of HR onboarding volume.

**Negative / Trade-offs:**
- Two OCR systems to maintain and understand.
- Tesseract.js is a large WASM download (~10 MB) on first HR onboarding use.
- Ollama llava cold start is ~50s — expense OCR can feel slow when Gemini is unavailable.
- Server proxy requires `GEMINI_API_KEY` to be configured; missing key degrades to Ollama only.

**Risks and mitigations:**
- Tesseract accuracy fails on a damaged/dirty identity doc: user must manually correct extracted fields — HR form has editable fields for this.
- Both Gemini and Ollama unavailable: OCR proxy returns error; expense form allows manual amount entry as fallback.
- New developer uses Tesseract for bill photos: poor extraction accuracy in field conditions. Mitigated: this ADR documents the reason for the split.

## Related Decisions

- ADR-013 (single HTML file) — Tesseract.js is loaded as a script tag in hr/index.html.
- ADR-010 (cross-schema via proxy) — OCR proxy follows the same proxy pattern for server-side processing.

## References

- `memory/infra_vps.md` — OCR proxy container (port 8766), Ollama container
- `memory/dbt_infra.md` — OCR proxy dual-backend details
- `hr/index.html` — Tesseract.js usage for identity doc OCR
- `finance/custodian/` — expense bill OCR via /ocr-proxy/