# ADR-064 Video Tutorial Production Pipeline — scripts.json → edge-tts → Playwright → ffmpeg

## Status

Accepted, 2026-06-26.

## Status History

```yaml
status_history:
  - date: 2026-06-26
    status: Accepted
    changed_by: hkl
    reason: Reference implementation in place (Fund Custodian Phase 4.10/4.11); pipeline documented and ready for reuse by future PWAs
    changed_via: adr-kit (360lm)
```

## Context

The Learning Hub (`/learn/` PWA, ADR-051) serves two content types: **Scene Guides** (interactive scenes) and **Screencasts** (video tutorials). Screencasts are produced via a manual five-stage pipeline currently documented only in the reference implementation (Fund Custodian, `finance/custodian/tutorial/`).

**Problem:** Developers building a new PWA tutorial would re-derive the entire pipeline from scratch — toolchain, script format, audio TTS generation, recording parameters, video encoding, HTML template — with high risk of inconsistency, repeated mistakes (wrong edge-tts API usage, missing ffmpeg flags, broken Playwright recordings), and lost institutional knowledge.

**Evidence:**
- Fund Custodian tutorial (Phase 4.10/4.11): scripts.json → generate_tts.py → Playwright recording → build_tutorial.py → tutorial.html. Works end-to-end but rationale is embedded in code comments and verbal knowledge.
- No ADR documents toolchain choices: why edge-tts over Google Cloud TTS, why Playwright over OBS, why H.264 CRF 18 over H.265, why 2.5× slowdown constant.
- Video DevGuide (`/var/www/360lm/video_tutorial_style.md`, alias: "video devGuide") captures implementation details but lacks architectural decision context for supersession or evolution.

**Constraints:**
- Bilingual requirement: field staff primarily Hindi-speaking; English for supervisors and reporting (ADR-051 context).
- No external SaaS: all tools must be free, self-hosted, or already licensed (edge-tts is free; Playwright already installed for ADR-022 E2E tests).
- Hub authentication required: tutorial.html must redirect unauthenticated users to hub with `?next=` pattern (ADR-001, ADR-012).
- Single HTML file, no framework (ADR-013): tutorial.html must be standalone, embeddable in PWA header.
- Learning Hub view tracking: tutorials trigger start_view / end_view beacon (ADR-052).

**Affected PWAs:**
- Fund Custodian (reference; 8 sections, 19 clips, ~7 min hi / ~6.5 min en, ~8 MB each output).
- Future PWAs with multi-step flows not self-evident from UI (Vehicle, Tour Planner, HR, Sales, others per Phase planning).

## Decision

**A standardised five-stage pipeline governs all Screencast tutorial production. The pipeline is deterministic, reproducible, language-paired (hi + en), and self-hosted. All code, scripts, and outputs live under `[pwa]/tutorial/` with intermediate artifacts in `production/` excluded from web serve and deployment.**

### Stage 1 — Script Authoring (`scripts.json`)

Single JSON file per PWA, defines all narrations in both languages (hi + en):

```json
{
  "pwa": "custodian",
  "title": "Fund Custodian Tutorial",
  "intro": {
    "hi": "फंड कस्टोडियन में प्रवेश करते हैं।",
    "en": "Let's explore Fund Custodian."
  },
  "sections": [
    { "id": "s1", "hi": "खाता खोलना", "en": "Opening an Account" },
    { "id": "s2", "hi": "स्थानांतरण भेजना", "en": "Sending Transfers" }
  ],
  "clips": [
    {
      "id": "open_account",
      "section": "s1",
      "file": "open_account.webm",
      "narration": {
        "hi": "खाता खोलने के लिए, सबसे पहले डैशबोर्ड खुलता है।",
        "en": "To open an account, first the dashboard loads."
      }
    }
  ],
  "outro": {
    "hi": "अधिक जानने के लिए आइकन दबाएं।",
    "en": "Press the help icon to learn more."
  }
}
```

**Segment types in segments order:**
- `intro` (lavfi color card): 5–10s, title slide
- `sections[]` (lavfi color card): 2–3s each, chapter marker
- `clips[]` (Playwright WebM + narration): 15–30s real time (~37–75s slowed)
- `outro` (lavfi color card): 3–5s, end card

**Narration style:** conversational, second-person, under 20s per clip, natural pauses for action on screen.

### Stage 2 — TTS Generation (`generate_tts.py`)

Library: **edge-tts v7** (free, high-quality Indian voices, no API key management)

**Voices:**
- Hindi: `hi-IN-MadhurNeural` (Hinglish, natural for field staff)
- English: `en-IN-PrabhatNeural` (Indian English, natural for supervisors)

**API usage (v7 critical gotcha):**
```python
import asyncio
from edge_tts import Communicate

async def generate_narration(text, lang_voice):
    """v7: use Communicate.stream() with explicit async for, NOT asyncio.run() or old async-iterator form."""
    communicate = Communicate(text=text, voice=lang_voice, rate="-10%")
    audio_data = b""
    async for chunk in communicate.stream():
        if chunk["type"] == "audio":
            audio_data += chunk["data"]
    return audio_data
```

**Outputs:**
- MP3 per segment: `production/audio/hi/intro.mp3`, `production/audio/en/intro.mp3`, etc.
- SRT captions: `production/srt/hi/`, `production/srt/en/` (one .srt per clip)

**SRT timing formula:** `offset ÷ 10_000_000` (edge-tts returns 100-nanosecond units). Target 6 words per cue for readability.

### Stage 3 — Screen Recording (Playwright)

**Test spec with video recording:**
```javascript
const config = {
  use: {
    launchOptions: {
      slowMo: 800  // CRITICAL: inside launchOptions, NOT top-level
    },
    video: 'on'  // Top-level (outside launchOptions)
  },
  webServer: { ... }
};

test.describe('Custodian Tutorial Recording', () => {
  test.beforeAll(async ({ browser }) => {
    // One clip = one user flow
  });
  
  test('record: open account flow', async ({ page }) => {
    await page.goto('/custodian/');
    await page.click('[data-test=open-account]');
    // ... user flow steps ...
    // Playwright records to test-results/videos/[clip-id].webm at slowMo=800
  });
});
```

**Critical parameters:**
- `slowMo: 800` **inside `launchOptions`** (top-level slowMo is silently ignored)
- `video: 'on'` (also records failed runs; clean up before build)
- Viewport: `800×450` (matches final video resolution)
- Clip duration: <30s real time (~75s slowed) — ensures narration fits without gaps

**Output:** WebM files at test-results/videos/[clip-id].webm, copied to `[pwa]/tutorial/videos/` before Stage 4.

### Stage 4 — Video Assembly (`build_tutorial.py`)

**Key constants:**
- `CRF = 18` (H.264, near-lossless for screencast material — low motion, high text clarity)
- `SLOWDOWN = 2.5` (all clips played at 0.4× speed; narration duration drives final clip length, not action speed)
- Resolution: `800×450` (matches Playwright viewport; scales 2× on mobile via CSS)

**Per-clip ffmpeg pipeline:**

```bash
# Input: slowed WebM + MP3 narration
# Stage 1: speed up WebM by 2.5× (reverse the 0.4× playback)
ffmpeg -i video.webm -vf "setpts=2.5*PTS" \
  -c:v libx264 -crf 18 -pix_fmt yuv420p \
  tmp_video.mp4

# Stage 2: pad last frame when audio outlasts video
ffmpeg -i tmp_video.mp4 -i narration.mp3 \
  -vf "tpad=stop_mode=clone:stop_duration=$(audio_duration)" \
  -c:v libx264 -crf 18 -pix_fmt yuv420p \
  tmp_padded.mp4

# Stage 3: burn in SRT subtitles (commas in force_style MUST be escaped as \,)
ffmpeg -i tmp_padded.mp4 \
  -vf "subtitles=captions.srt:force_style='FontName=Arial,FontSize=14,BorderStyle=1'" \
  -c:v libx264 -crf 18 -pix_fmt yuv420p \
  tmp_subtitled.mp4

# Stage 4: mux video + audio
ffmpeg -i tmp_subtitled.mp4 -i narration.mp3 \
  -c:v copy -c:a aac -shortest \
  -movflags +faststart \
  segment.mp4
```

**Lavfi color source for title cards:**
```bash
ffmpeg -f lavfi -i "color=c=0x0f172a:size=800x450:rate=25:duration=5" \
  -vf "drawtext=text='Fund Custodian': fontsize=48" \
  intro.mp4
```

**Critical ffmpeg flags:**
- `-movflags +faststart`: moves moov atom to front for browser streaming without full download. **Mandatory** for tutorial.html playback to not stall.
- Commas in `force_style=` **must be escaped** as `\,` in simple filtergraph mode.
- `-pix_fmt yuv420p`: ensures H.264 output is compatible with all browsers and devices.

**Final concat (demuxer method):**
```
concat:intro.mp4|s1_card.mp4|open_account.mp4|s2_card.mp4|...|outro.mp4 → master.mp4 (H.264 CRF 18)
```

**Output:** `[pwa]-tutorial-hi-h264.mp4` and `[pwa]-tutorial-en-h264.mp4`, auto-deployed to `[pwa]/tutorial/` on build success via deploy-prod.sh.

### Stage 5 — Tutorial HTML Page (`tutorial.html`)

**Template:** `finance/custodian/tutorial/tutorial.html`

**Features:**
- **Auth:** redirects unauthenticated users to hub with `?next=<encoded-path>` (ADR-001)
- **Language switcher:** top button `EN | हिं` preserves `currentTime` and play state across language swap
- **Chapter index:** section markers with timestamps (e.g., "Opening an Account 0:45")
- **Full-screen button:** native HTML5 video fullscreen
- **View tracking:** triggers start_view on play, end_view on 80%+ watch time or unload (ADR-052, two-RPC beacon)
- **Responsive:** CSS media queries, safe-bottom padding (ADR-002)

**Critical code pattern (view tracking):**
```javascript
let viewed_pct = 0;
let view_started = false;

video.addEventListener('play', async () => {
  if (!view_started) {
    await rpc('start_view', { tutorial_id: 'custodian_tutorial', lang });
    view_started = true;
  }
});

video.addEventListener('timeupdate', () => {
  viewed_pct = (video.currentTime / video.duration) * 100;
  if (viewed_pct >= 80 && !end_view_sent) {
    rpc('end_view', { tutorial_id: 'custodian_tutorial', lang, watched_pct: viewed_pct });
    end_view_sent = true;
  }
});
```

**Video naming convention:**
- `[pwa]-tutorial-hi-h264.mp4` (Hindi version)
- `[pwa]-tutorial-en-h264.mp4` (English version)

Both served from `[pwa]/tutorial/` root; tutorial.html switches src on language button click.

## Directory Structure

```
[pwa]/tutorial/
├── tutorial.html                    ← user-facing (served)
├── [pwa]-tutorial-hi-h264.mp4      ← user-facing (served)
├── [pwa]-tutorial-en-h264.mp4      ← user-facing (served)
├── videos/                          ← source WebMs (NOT served, optional after build)
│   ├── open_account.webm
│   ├── send_transfer.webm
│   └── ...
└── production/                      ← build artifacts (EXCLUDED from web serve + rsync deploy)
    ├── scripts.json
    ├── generate_tts.py
    ├── build_tutorial.py
    ├── audio/
    │   ├── hi/
    │   │   ├── intro.mp3
    │   │   ├── s1_card.mp3
    │   │   └── ...
    │   └── en/
    │       └── ...
    ├── srt/
    │   ├── hi/
    │   │   ├── open_account.srt
    │   │   └── ...
    │   └── en/
    │       └── ...
    ├── processed/                   ← temp intermediates (hi/, en/)
    └── output/                      ← final MP4s before auto-deploy
        ├── custodian-tutorial-hi-h264.mp4
        └── custodian-tutorial-en-h264.mp4
```

**Deploy exclusion:** `deploy-prod.sh` uses `rsync --exclude='production/'` so intermediate build artifacts never reach production web server.

## When to Build / Rebuild

**Build a tutorial:**
- Multi-step workflows not self-evident from UI (e.g., account opening, transfer flow)
- Role-based access patterns (admin mode vs. field staff view)
- Complex approval queue states
- No in-app guided tour exists

**Rebuild:**
- Major feature additions that change visible flow (new form fields, new approval stages)
- **Do NOT rebuild** for minor UI tweaks (button text, color, icon changes) — edit tutorial.html chapter index instead

## Reference Implementation

**PWA:** Fund Custodian (`finance/custodian/tutorial/`)
- Phases: 4.10 (scripts + build scripts), 4.11 (video updates)
- Sections: 8 · Clips: 19 · Output: ~7 min (hi), ~6.5 min (en), ~8 MB each
- Full pipeline: `/var/www/360lm/video_tutorial_style.md` (alias: "video devGuide")

**Run the pipeline:**
```bash
cd /var/www/360lm/finance/custodian/tutorial/production
python3 generate_tts.py && python3 build_tutorial.py both
# Outputs: ../custodian-tutorial-hi-h264.mp4, ../custodian-tutorial-en-h264.mp4
```

**Copy to next PWA:**
1. Clone `finance/custodian/tutorial/production/` to `[pwa]/tutorial/production/`
2. Edit `scripts.json` with new PWA's narration
3. Record Playwright clips to `[pwa]/tutorial/videos/`
4. Run `python3 generate_tts.py && python3 build_tutorial.py both`
5. Add PWA header button: `<a href="/pwa/tutorial/tutorial.html" target="_blank">▶ Tutorial</a>`
6. Update PWA's `sw.js` CACHE_VER (ADR-005)

## Implementation Notes

**Files:**
- Template HTML: `/var/www/360lm/finance/custodian/tutorial/tutorial.html`
- Reference scripts: `/var/www/360lm/finance/custodian/tutorial/production/generate_tts.py`, `build_tutorial.py`, `scripts.json`
- Playwright spec: `/var/www/360lm/tests/custodian_tutorial_screenshots.spec.js` (recording test example)
- Full DevGuide: `/var/www/360lm/video_tutorial_style.md`

**Key identifiers:**
- edge-tts voices: `hi-IN-MadhurNeural`, `en-IN-PrabhatNeural`
- ffmpeg CRF: 18 (near-lossless screencast)
- slowdown factor: 2.5 (0.4× playback speed)
- Playwright slowMo: 800ms (inside `launchOptions`)
- SRT timing: `offset ÷ 10_000_000` (100-nanosecond units to seconds)
- viewport: 800×450

**Find reference implementations:**
```bash
grep -r "setpts=2.5\*PTS" /var/www/360lm --include="*.py"  # ffmpeg slowdown
grep -r "MadhurNeural\|PrabhatNeural" /var/www/360lm --include="*.py"  # TTS voices
grep -r "force_style=" /var/www/360lm --include="*.py"  # subtitle burning
find /var/www/360lm -name "tutorial.html" -type f  # all tutorial pages
```

**Critical gotchas:**

1. **edge-tts v7 API change:** use `.stream()` with `async for`, NOT the old async-iterator pattern or `asyncio.run()`. Old code will fail silently or hang.

2. **Playwright slowMo placement:** must be inside `launchOptions: { slowMo: 800 }`. Top-level `slowMo: 800` is silently ignored by Playwright config parser.

3. **ffmpeg force_style commas:** escape as `\,` in simple filtergraph. Unescaped commas break the filter chain.

4. **ffmpeg -movflags +faststart:** mandatory for browser streaming. Without it, video stalls while downloading full file.

5. **SRT timing from edge-tts:** offset is in 100-nanosecond units. Divide by 10,000,000 to get seconds. Off-by-one will cause captions to appear 1–2s early/late.

6. **No H.265 encoding:** H.265 transcoding from H.264 CRF 18 source creates generation loss and produces larger files. H.265 only beneficial when encoding from raw/lossless. Keep final output as H.264.

7. **production/ excluded from deploy:** rsync `--exclude='production/'` in deploy-prod.sh ensures build artifacts never reach production. Source .webm files and intermediates are safe to keep locally.

## Alternatives Considered

- **Paid TTS (Google Cloud Speech, Amazon Polly):** Rejected. free edge-tts Indian voices are high quality for field staff; avoids API key management in build scripts; no per-character cost accumulation.

- **OBS Studio for recording:** Rejected. Playwright is already installed and used for E2E testing (ADR-022). No additional tooling. Playwright recordings are deterministic and reproducible; OBS requires manual timing and scene switching per clip.

- **Single audio track (no narration):** Rejected. Field staff are primarily Hindi-speaking. Without narration, tutorials become inaccessible to non-English viewers. Narration also guides attention to UI changes.

- **H.265 codec for smaller files:** Rejected. Transcoding from H.264 CRF 18 source to H.265 introduces generation loss and produces larger output files (screencast material has low motion, so codec efficiency gain is marginal). H.265 only beneficial when encoding from raw/lossless source. Limited Safari support on older iOS devices. Keep final output as H.264.

- **Single-language (English only):** Rejected. Field staff requirement is Hindi-first; dual-language is mandatory for adoption and Legal compliance.

- **Vimeo / YouTube for hosting:** Rejected. Same no-framework stance as ADR-013. Self-hosted MP4 keeps content behind hub auth (ADR-012). No external SaaS dependency. Playback analytics via ADR-052 beacon (start_view / end_view RPCs).

- **React video player component:** Rejected. Single HTML file constraint (ADR-013). HTML5 `<video>` element is sufficient for language switching and chapter navigation. No build step required.

## Consequences

**Positive:**
- **Reusable pipeline:** new PWAs can copy `production/` template and scripts.json, record clips, run build. No re-derivation of toolchain.
- **Deterministic output:** same scripts.json + Playwright spec always produce identical videos (bitwise same with fixed seed). Reproducible builds.
- **Self-hosted:** no SaaS vendor lock-in, no API key management, no per-minute TTS costs. Free toolchain (edge-tts, Playwright, ffmpeg).
- **Bilingual by default:** all tutorials instantly dual-language (hi + en). Single pass through pipeline.
- **Learning Hub integration:** view tracking via ADR-052 beacon. Tutorials counted toward learning completion (ADR-051).
- **Hub auth gating:** ADR-001 `?next=` pattern ensures users cannot leak tutorial URLs; content stays behind hub login.
- **Performance:** H.264 CRF 18 near-lossless quality, ~8 MB per 6–7 min video. Universal browser support (Chrome, Firefox, Safari, Android, iOS).
- **Documentation:** this ADR captures all architectural choices for future extensions (e.g., intro animations, chapter thumbnails, accessibility features).

**Negative / Trade-offs:**
- **Manual recording effort:** Playwright recording requires writing per-PWA test spec with step-by-step user flows. Not automated; sensitive to UI changes (selectors must be kept in sync with PWA).
- **Build script maintenance:** generate_tts.py and build_tutorial.py are Python; requires Python 3.7+, edge-tts v7, ffmpeg. Small deviation (wrong edge-tts version, ffmpeg missing) breaks the build.
- **Narration quality tied to voice:** MadhurNeural and PrabhatNeural are fixed; cannot customize accent, pace per clip (edge-tts rate adjustment is global). No human voiceover option.
- **Storage cost:** each PWA tutorial = 2 MP4s (~8 MB each per language). Over 10 PWAs, ~160 MB storage. Not a major issue but factor into backup strategy.
- **No analytics dashboard:** view_event data is in learn.view_events table; no built-in dashboard. Learning Hub TM dashboard (ADR-051) displays aggregate stats but not per-clip engagement.
- **Rebuild not incremental:** if one clip changes, entire pipeline re-runs (all TTS, all video assembly, all concatenation). No granular rebuild for single clip. Acceptable for current scale (1–2 PWAs per phase).

**Risks and mitigations:**

| Risk | Mitigation |
|------|-----------|
| Playwright recording breaks on PWA UI change | Keep `[data-test]` selectors stable; version selectors in per-PWA test spec. If changed, update test spec before rebuild. |
| edge-tts API version drift | Pin edge-tts to v7 in build script requirements.txt. Monitor for v8 release; evaluate compatibility before upgrade. |
| ffmpeg flag incompatibility (e.g., Ubuntu vs. macOS) | Test build scripts on both platforms before shipping. Document OS-specific flags (e.g., `-loglevel` formats differ). |
| Video file corruption during deploy | rsync with checksums (`--checksum` flag). Verify output file exists and is playable before marking build as success. |
| Storage quota exceeded | Monitor `/var/www/360lm/[pwa]/tutorial/` disk usage. Implement cleanup policy: keep latest 3 versions, archive older versions offline. |
| SRT caption timing drifts from narration | Generate SRT programmatically from edge-tts offset data; do not hand-edit. Validate timing against final MP4 before deploy. |

## Related Decisions

- **ADR-001:** All PWAs Pass ?next= When Redirecting to Hub Login — tutorial.html auth gate uses this pattern.
- **ADR-002:** Every PWA Must Link shared/safe-bottom.css — tutorial.html includes this.
- **ADR-012:** Hub PWA Is the Single SSO Gateway for All Employee-Facing PWAs — tutorial.html requires hub session.
- **ADR-013:** Each PWA Is a Single Self-Contained HTML File — No Framework, No Build Pipeline — tutorial.html is static HTML.
- **ADR-022:** Playwright Is the E2E Test Framework — One Spec File Per PWA — same Playwright infra used for recording.
- **ADR-051:** Learning Completion Is Threshold-Based — 100% Scenes or 80% Watch Time (Scrub-Proof) — tutorials tracked toward completion.
- **ADR-052:** Learning View Tracking Uses Two-RPC Beacon (start_view / end_view) — Not Single Write — tutorial.html fires beacon.

## References

- Fund Custodian tutorial (reference implementation): `/var/www/360lm/finance/custodian/tutorial/`
- Video Tutorial DevGuide (full pipeline): `/var/www/360lm/video_tutorial_style.md` (memory alias: "video devGuide")
- Playwright recording spec: `/var/www/360lm/tests/custodian_tutorial_screenshots.spec.js`
- Incident (Phase 4.10 initial build): Knowledge base entry (unavailable in this context; referenced during Phase 4.10 planning)
- Learning Hub view tracking: `learn.view_events` table schema in `/var/www/360lm/docs/db_schema.md`
- edge-tts v7 docs: https://github.com/rany2/edge-tts (v7 changelog: removed old async-iterator, standardized on `.stream()`)
- ffmpeg subtitles filter: https://ffmpeg.org/ffmpeg-filters.html#subtitles-1