# Module Design Document: VideoSmith — AI Short-Form Video Generation PWA

**Module:** VideoSmith (`videosmith` PWA + internal MoneyPrinterTurbo engine)  
**Status:** **Phase 1 BUILT, deployed, and fully verified with a real completed video — LIVE and publicly reachable** at https://videosmith.srv1111289.hstgr.cloud/ — see §13 Build Log  
**Author:** Claude Code research agent (design), built same day by Claude Code (implementation)  
**Date:** 2026-07-03  
**Revision:** 2.0

---

## ⚠️ Review note (historical — all items below resolved 2026-07-03)

All open questions in §8 were resolved directly with the user (asked one at a time) before Phase 1 build started. Isolated PIN auth (Blogsmith pattern) is confirmed final, not conditional. This note is kept for history; see §13 for what was actually built and verified.

---

## 1. Why This Exists

360LM operates a marketing agency with in-house short-form video production as a growing capability. The company needs a repeatable, auditable system to auto-generate marketing videos from topics: keyword/concept → AI script generation → TTS narration → stock B-roll sourcing → subtitle burn-in → final MP4 export.

**Current state:** Ad-hoc video production (manual script writing, stock footage sourcing, editing). No system to track video generation costs, reuse scripts, or measure output quality.

**Why it matters:** Short-form video (TikTok, Instagram Reels, YouTube Shorts) is the fastest-growing content format for 360LM's clients. A repeatable generation pipeline (topic → finished video in 5–15 min) enables high-volume content experimentation without dedicated video editors. Estimated 100+ videos/month if fully operational.

---

## 2. Inspiration vs. Scope

### References Studied

**MoneyPrinterTurbo** (GitHub, 95k+ stars, MIT licensed):
- Proven concept: topic → script → TTS → B-roll → render pipeline
- Supports 25+ LLM providers, 3 stock-footage sources (Pexels, Pixabay, Coverr)
- Built on FastAPI + Streamlit; public API with optional auth
- Config-driven architecture; no review gate; direct publish

**Blogsmith precedent** (360LM, ADR-097, live 2026-07-02):
- Single-process Python app serving both PWA + API
- Isolated database + PIN-based bcrypt auth (non-hub)
- External service integration (WordPress REST API)
- Schema: users, sessions, jobs tables
- No human-review gate (publish directly after AI generation)

**360LM ERP suite (hub integration model):**
- Hub-PIN auth for employee-facing apps
- Cross-schema data via RPC calls + Accept-Profile headers (ADR-074)
- Mandatory ADR compliance: safe-bottom.css, ?next= redirects, IST timezone display

### Intentional Out-of-Scope for v1

- **Multi-language script generation:** Focus on English; non-English market research deferred
- **Video re-rendering with variants:** Generate once per topic; batch variants (different voices, lengths) in Phase 4
- **Audio mixing (background music + narration):** MoneyPrinterTurbo supports; wire through config; UI for BGM selection deferred to Phase 2
- **Transcript/caption editing UI:** Subtitles auto-generated; manual editing out-of-scope v1
- **Social media account linking:** Auto-publish support deferred to Phase 3; Phase 1 is download-only
- **Video analytics dashboard:** View/engagement metrics deferred; no integration with platform analytics yet

---

## 3. Proposed Build Phases

### Phase 1: Core Generation Pipeline (Weeks 1–2)

**Deliverables:**
- VideoSmith PWA: HTML/CSS/JS (single file, no framework)
- Python API backend: auth, job submission, status polling, file proxy
- Database schema: users, sessions, jobs, outputs
- Docker compose + Traefik labels (internal MPT + wrapper)
- 30+ Playwright spec tests
- Functional video generation: topic → complete MP4 via MoneyPrinterTurbo

**Features:**
- User registration (PIN: 4–8 digits, server-side bcrypt)
- Login + auto-reconnect (localStorage bearer token, 30-day expiry)
- Topic input form (text field, optional: video format, TTS voice, LLM provider)
- Job status polling (GET /api/jobs, real-time spinner)
- Job history table (created_at, status, view-video link, delete button)
- File download (GET /api/jobs/{job_id}/output → proxy to MPT storage)

**Scope constraint:** LLM provider + voice/format options fixed in config.toml; no UI selection yet

### Phase 2: Credential + Provider Config UI (Weeks 3–4)

**Features:**
- Per-user Pexels/Pixabay/Coverr API key storage (encrypted in DB)
- LLM provider dropdown (select from 25+ providers in config.toml)
- Output format selection (9:16 vertical, 16:9 horizontal, square)
- TTS voice dropdown (Azure, Elevenlabs, edge-tts, etc.)
- Dry-run mode (test without API costs; skip stock-footage download)

**Design note on ADR-062:** MoneyPrinterTurbo does not implement the ADR-062 LLM fallback chain (Tier 1: Claude OAuth → Tier 2: OpenRouter → Tier 3: Anthropic REST). It uses a single LLM provider. **Verdict: ADR-062 is NOT natively applicable.** If user wants fallback resilience, VideoSmith wrapper can reimplement the script-generation step (wrap in a fallback chain) in Phase 2, or accept single-provider limitation in Phase 1.

### Phase 3: Publishing + Distribution (Weeks 5–6)

**Features (example options):**
1. **Manual export:** MP4 download (already in Phase 1)
2. **Auto-publish via Upload-Post:** MoneyPrinterTurbo has built-in support; wire config.toml keys through
3. **Handoff to distribution tool:** Integrate with drive-consolidator or custom publishing queue
4. **Social media account linking:** Future; requires OAuth (TikTok, Instagram, YouTube)

**Phase 3 scope TBD by user (Q2 in plan_videosmith_2026-07-03.md)**

### Phase 4: Batch Jobs, Analytics, Polish (On-Demand)

- Batch job submission (CSV of topics)
- Video re-render with variants
- Scheduled generation (cron-like scheduling)
- Analytics dashboard (if publishing to social platforms)

---

## 4. Data Model

### Schema: `videosmith`

**Isolation:** Dedicated `videosmith_app` role, zero access to `lm360` schemas (per ADR-086)

#### 4.1 `users` Table

| Column | Type | Keys | Default | Notes |
|--------|------|------|---------|-------|
| id | BIGSERIAL | PK | | |
| name | TEXT | UNIQUE | | Display name (e.g. "Alice Content Team") |
| pin_hash | TEXT | | | bcrypt(`pin`, 12-round cost); never store plain PIN |
| pexels_api_key | TEXT | | NULL | Encrypted in DB; nullable (dry-run mode if absent) |
| pixabay_api_key | TEXT | | NULL | Encrypted in DB |
| coverr_api_key | TEXT | | NULL | Encrypted in DB |
| llm_provider | VARCHAR(50) | | "openai" | Default: openai; user can override in UI (Phase 2) |
| tts_voice_name | VARCHAR(100) | | "en-US-AriaNeural" | TTS engine + voice choice (Azure, Elevenlabs, edge-tts, etc.) |
| preferred_video_format | VARCHAR(20) | | "1080x1920" | "1080x1920" (9:16), "1920x1080" (16:9), "1080x1080" (square) |
| created_at | TIMESTAMPTZ | | now() | IST on display (UTC stored) |

#### 4.2 `sessions` Table

| Column | Type | Keys | Default | Notes |
|--------|------|------|---------|-------|
| token | TEXT | PK | encode(gen_random_bytes(32), 'hex') | 32-byte random hex; no brute-force risk |
| user_id | BIGINT | FK → users.id | | ON DELETE CASCADE |
| created_at | TIMESTAMPTZ | | now() | |
| expires_at | TIMESTAMPTZ | | now() + 30 days | Auto-logout if expired |

#### 4.3 `jobs` Table

| Column | Type | Keys | Default | Notes |
|--------|------|------|---------|-------|
| id | BIGSERIAL | PK | | |
| user_id | BIGINT | FK → users.id | | ON DELETE SET NULL (allow orphan jobs for audit) |
| mpt_task_id | UUID | UNIQUE | | Foreign reference to MoneyPrinterTurbo task (tracking link) |
| topic | TEXT | | | Input keyword/topic (e.g. "AI marketing trends 2026") |
| status | VARCHAR(50) | | 'pending' | Enum: pending, researching, drafting, completed, failed |
| output_video_path | TEXT | | NULL | Relative path to MP4 in MPT storage (e.g. `tasks/{mpt_task_id}/final-1.mp4`) |
| script | TEXT | | NULL | AI-generated script (optional; debug purposes) |
| error_message | TEXT | | NULL | If status='failed', root cause description |
| video_format | VARCHAR(20) | | NULL | Format used for this job (snapshot of user's preference at submit time) |
| llm_provider_used | VARCHAR(50) | | NULL | Snapshot of LLM provider at submit time |
| tts_voice_used | VARCHAR(100) | | NULL | Snapshot of TTS voice at submit time |
| created_at | TIMESTAMPTZ | | now() | IST on display |
| completed_at | TIMESTAMPTZ | | NULL | When status became 'completed' or 'failed' |

**Note on immutability:** Once a job is created, `topic`, `video_format`, `llm_provider_used`, `tts_voice_used` are immutable snapshots. Changing user's preferences does not retroactively affect submitted jobs.

---

## 5. API Contract & Integration

### 5.1 VideoSmith API Endpoints

All authenticated endpoints require `Authorization: Bearer <token>` header.

#### Auth Endpoints

| Method | Path | Auth | Purpose | Request | Response |
|--------|------|------|---------|---------|----------|
| **POST** | `/api/register` | None | Create user | `{name, pin}` | `{user: {id,name}, token, expires_at}` |
| **POST** | `/api/login` | None | Authenticate | `{user_id, pin}` | `{user: {id,name}, token, expires_at}` |
| **POST** | `/api/logout` | Bearer | Invalidate session | — | `{ok: true}` |
| **GET** | `/users` | None | List all users (user selection UI) | — | `{users: [{id,name}]}` |

#### Job & Configuration Endpoints

| Method | Path | Auth | Purpose | Request | Response |
|--------|------|------|---------|---------|----------|
| **POST** | `/api/jobs` | Bearer | Create video job | `{topic, video_format?, llm_provider?, tts_voice?}` | `{job_id, status, mpt_task_id}` |
| **GET** | `/api/jobs` | Bearer | Get user's jobs (paginated) | `?page=1&limit=10` | `{jobs: [...], total, page, limit}` |
| **GET** | `/api/jobs/{job_id}` | Bearer | Get single job details | — | `{id, topic, status, output_video_path, script, error_message, created_at, completed_at}` |
| **GET** | `/api/jobs/{job_id}/output` | Bearer | Download MP4 (proxy to MPT) | — | Binary MP4 file or 404 if not ready |
| **DELETE** | `/api/jobs/{job_id}` | Bearer | Cancel job (if pending/drafting) | — | `{ok: true}` or 400 if already completed |
| **GET** | `/api/preferences` | Bearer | Get user's config (Phase 2) | — | `{pexels_key_set, llm_provider, tts_voice, video_format}` |
| **POST** | `/api/preferences` | Bearer | Update user's config (Phase 2) | `{pexels_api_key?, llm_provider?, tts_voice?, video_format?}` | `{ok: true}` |

#### Health Check

| Method | Path | Auth | Purpose |
|--------|------|------|---------|
| **GET** | `/health` | None | Liveness probe | Returns `{"ok": true, "service": "videosmith"}` |

### 5.2 MoneyPrinterTurbo Integration (Read-Only from VideoSmith)

VideoSmith calls MoneyPrinterTurbo's FastAPI backend (internal hostname: `http://mpt-api:8080`). All calls are unauthenticated (no x-api-key header required; auth is commented-out in MPT source).

**Endpoints called by VideoSmith:**

| Method | MPT Path | VideoSmith usage | Notes |
|--------|----------|---------|-------|
| **POST** | `/videos` | Create video job | Body: `{topic, video_language, voice_name, video_size, ...}` → returns `{task_id, status}` |
| **GET** | `/tasks/{task_id}` | Poll job status | Returns task object with `status`, `videos` (array of MP4 paths), `error`, etc. |
| **GET** | `/tasks` | List all MPT tasks (optional; debug UI) | Pagination support |
| **DELETE** | `/tasks/{task_id}` | Delete task files | Clean up storage |

**Task status lifecycle:**
- `pending` → request enqueued
- `researching` → LLM script generation in progress
- `drafting` → video render in progress (stock footage download, TTS, ffmpeg encode)
- `completed` → success; final MP4 at `/tasks/{task_id}/final-1.mp4`
- `failed` → error; details in task record

**VideoSmith polling strategy:**
- On job submit: store `mpt_task_id` in jobs table
- Poll `GET /tasks/{mpt_task_id}` every 2 seconds while status ∈ {pending, researching, drafting}
- When status = completed/failed, update VideoSmith job record + stop polling
- UI shows spinner while polling; displays result (MP4 link or error)

### 5.3 Job Submission Request Model

```python
class JobSubmitRequest(BaseModel):
    topic: str  # Required: "AI marketing strategies for 2026"
    video_language: Optional[str] = "en"  # Default English
    video_size: Optional[str] = "1080x1920"  # 9:16, 16:9, 1080x1080
    voice_name: Optional[str] = None  # If None, use user's preference
    llm_provider: Optional[str] = None  # If None, use user's preference
    max_duration_seconds: Optional[int] = 60  # Target video length
    allow_local_materials: Optional[bool] = False  # Use uploaded local videos?
```

---

## 6. Screens & UX

### 6.1 Login Screen

**State:**
- No token or expired token → show login form
- Token valid → skip to Dashboard

**Components:**
- User grid: 3–4 columns, circular avatar (generated from user ID), name label
- "+" button to register new user
- PIN pad (numeric input, 4–8 digits)
- "Stay logged in for 30 days" checkbox
- Login button (POST `/api/login`)

**Copy/tone:** Friendly, no jargon. "Welcome to VideoSmith — short videos, on-demand."

### 6.2 Registration Flow

**Trigger:** Click "+" button on login screen

**Modal/form:**
- Name field (text input)
- PIN field (numeric input, 4–8 digits)
- Confirm PIN field
- Terms checkbox (placeholder: "I agree to use this for marketing content only")
- Register button (POST `/api/register`)

**Success:** Auto-login, redirect to Dashboard

### 6.3 Dashboard / Main View

**Sections:**

**A. Job submission form (top):**
- Topic input (text field, required)
- Format dropdown (9:16, 16:9, 1:1) — Phase 2
- Voice dropdown (list of available TTS voices) — Phase 2
- LLM provider dropdown (openai, azure, gemini, etc.) — Phase 2
- "Generate video" button (POST `/api/jobs`)

**B. Job status spinner (below form if job submitted):**
- Animated spinner while polling
- Current phase display (pending, researching, drafting, completed)
- Time elapsed
- Cancel button (DELETE `/api/jobs/{job_id}`)

**C. Job history table (below):**
- Columns: Topic (truncated), Status, Created, Duration, Actions
- Status badge colors: pending=yellow, researching=blue, drafting=cyan, completed=green, failed=red
- Actions: View video (modal/download), Delete, Retry (if failed)
- Pagination: 10 jobs/page, next/previous buttons
- Sorting: by created_at descending (newest first)

**D. Preferences panel (Phase 2 in PWA; drawer or separate screen):**
- Pexels API key (masked input; shows "configured" badge if set)
- Pixabay API key (optional)
- Coverr API key (optional)
- LLM provider (dropdown)
- TTS voice (dropdown)
- Preferred video format (radio: 9:16, 16:9, 1:1)
- Save button → POST `/api/preferences`

### 6.4 Service Worker & Offline Support

**Cache strategy:**
- Cache namespace: `videosmith-v1` (bumped for each release per ADR-005)
- Assets (index.html, CSS, JS, icons): cache-first
- API calls (jobs, status): network-first (always check for latest status)
- Video downloads: network-only (always fetch from server)

**Offline behavior:**
- Login form and dashboard layout cached; show "offline" banner if no network
- Cannot submit new jobs while offline; button disabled with tooltip
- Cannot download videos while offline
- Existing cached job list visible (stale, per last update)

---

## 7. Data Sensitivity & Access Model

### User & Session Data

**Sensitivity:** Low-medium
- PIN stored server-side as bcrypt hash (one-way); never transmitted
- Session tokens are 32-byte random (256-bit entropy); no brute-force risk
- Tokens transmitted only over HTTPS; stored client-side in localStorage

**Access control:**
- Users can view only their own jobs
- Admin-only features (user deletion, global stats) deferred to Phase 4
- No cross-user data leakage via API (job list filtered by `user_id`)

### Generated Video Content

**Sensitivity:** Medium (can contain client-sensitive topics)
- Videos stored in `/var/www/moneyprinterturbo/storage/tasks/{task_id}/`
- Accessible via VideoSmith API (`/api/jobs/{job_id}/output`) to authenticated user only
- Direct URL to MP4 file is NOT exposed to client (proxy through API)
- If VideoSmith becomes multi-tenant in future, implement per-task ACLs (verify `user_id` owns the task before serving)

### LLM Provider Keys

**Sensitivity:** High
- Pexels/Pixabay API keys optionally stored server-side (encrypted at rest using application-level encryption)
- Never logged or exposed in API responses (only `key_configured: bool` flag)
- LLM provider + TTS service credentials: stored in server `config.toml` and `.env` (not user-facing)

---

## 8. Open Questions — RESOLVED by User (2026-07-03)

All seven questions below were put to the user interactively (one at a time) and are now closed. Final decisions supersede any "assumed"/"pending" language elsewhere in this document.

### Architecture Questions

**Q1: Auth model — hub-integrated or isolated?**
- **RESOLVED: Isolated.** User confirmed "same as BlogSmith, part of automation" — isolated PIN-based bcrypt auth (Blogsmith pattern), no hub integration, no `hub.pwa_registry` entry.

**Q2: Output distribution goal?**
- **RESOLVED: Phase 1 = manual MP4 download only.** Auto-publish to social platforms (TikTok/Instagram/YouTube) is explicitly **parked** — user said "proceed with option 1 and park option 2." Do not build publish/OAuth integration until the user un-parks it.

**Q3: Content feed source?**
- **RESOLVED: Manual per-job entry**, matching Blogsmith's keyword-field UX exactly ("right match similar to BlogSmith"). No content-calendar or batch-CSV in Phase 1.

### Technical Questions

**Q4: CPU bottleneck acceptable?**
- **RESOLVED: Yes.** User accepted single concurrent task, ~5–15 min render time per video, on the current 2-core VPS. No hardware upgrade requested.

**Q5: LLM fallback chain (ADR-062) required?**
- **RESOLVED: Yes — full 3-tier chain required** (Claude OAuth → OpenRouter → Anthropic REST), matching every other live AI pipeline on the platform. This reverses the draft's earlier "single provider" assumption.
- **Design implication:** MoneyPrinterTurbo's `TaskVideoRequest`/`VideoParams` model accepts an optional pre-supplied `video_script` field (verified in `app/models/schema.py`, `Field(default="", max_length=8000)`) — when non-empty, MPT skips its own internal script generation. VideoSmith's wrapper backend will therefore run script generation itself via a new internal proxy (`videosmith-ai-proxy`, same tiered pattern as `counter-ai`/`print-ai-proxy` per ADR-062), then pass the resulting script into MPT's `/videos` call — MPT is used only for TTS/subtitle/stock-footage/render, never for script generation. See §5.2.

**Q6: Video retention & cleanup?**
- **RESOLVED: Archive off-box.** Generated videos move to external storage (S3/GCS/Google Drive — exact provider not yet chosen, see new open item below) after generation; not kept indefinitely on the 32GB local disk.
- **Follow-up (not blocking, resolve before Phase 1 build):** which specific archive target — S3, GCS, or the platform's existing Google Drive integration (Drive Consolidator, `drive-consolidator` container, already has Drive OAuth wired up)? Reusing Drive Consolidator's existing credentials would avoid a new integration.

**Q7: App name final approval?**
- **RESOLVED: "VideoSmith"** confirmed as the final name (not a placeholder).

---

## 9. ADR Compliance Checklist (Per ADR-107 Discipline)

For each applicable house-standard ADR, state verdict: **implemented** / **NOT implemented** / **not applicable** (with evidence/reason).

| ADR | Title | Verdict | Evidence / Notes |
|-----|-------|---------|---|
| **ADR-001** | Hub `?next=` redirect on login | NOT APPLICABLE | This is isolated PWA, not hub-integrated. No login redirect to hub. |
| **ADR-005** | Service Worker cache versioning | **IMPLEMENTED** | SW cache namespace `videosmith-v1`; bumped per release (§6.4). |
| **ADR-009** | Each PWA owns its schema | **IMPLEMENTED** | `videosmith` schema, isolated role `videosmith_app`, zero lm360 access (§4). |
| **ADR-011** | PIN-based auth, no passwords | **IMPLEMENTED** | PIN: 4–8 digits, server-side bcrypt 12-round (§5.1, auth endpoints). Isolated auth model confirmed final (§8, Q1). |
| **ADR-012** | Hub as SSO gateway | NOT APPLICABLE | Isolated PWA confirmed (§8, Q1) — no hub integration by design, not by default/deferral. |
| **ADR-062** | Live AI pipeline contract (3-tier auth/fallback) | **IMPLEMENTED** | User selected full tiered chain (§8, Q5): Tier 1 Claude OAuth → Tier 2 OpenRouter → Tier 3 Anthropic REST, run by a new `videosmith-ai-proxy` for script generation only. Script is pre-generated and passed into MPT's `video_script` field (verified accepts pre-supplied text, `app/models/schema.py`), so MPT never performs LLM calls itself — it is used purely for TTS/subtitle/render. |
| **ADR-070** | IST timezone display | **IMPLEMENTED** | All TIMESTAMPTZ columns stored UTC in DB; display converts to IST in PWA UI (§4.1, §6). Verified via `NOW() AT TIME ZONE 'Asia/Kolkata'` in RPC responses. |
| **ADR-071** | Indian numbering (₹ currency, amount formatting) | NOT APPLICABLE | VideoSmith stores no financial amounts; no invoice/payment logic. No currency display needed. |
| **ADR-074** | PostgREST Accept-Profile header | NOT APPLICABLE | Custom Python backend, not PostgREST; no schema multiplexing. Single schema per role. |
| **ADR-081** | Safe-area inset rendering (safe-bottom.css) | **IMPLEMENTED** | PWA links `/shared/safe-bottom.css`; sticky bottom bars (if any) account for notches (§6.1, §6.3). |
| **ADR-086** | Isolated PWA architecture | **IMPLEMENTED** | Own database `videosmith`, own role `videosmith_app`, own subdomain (deferred — no DNS yet, per user 2026-07-03), own auth model (isolated PIN, confirmed final per §8 Q1). Follows Blogsmith precedent (§2, §3.1). |

---

## 10. Red-Team: Concrete Failure Modes

| Failure Mode | Root Cause | Impact | Mitigation |
|---|---|---|---|
| **Pexels quota exhausted mid-render** | Shared API key across all users; quota burns 5–10 videos/day | Video fails; user sees "no stock footage available" error | Pre-test key quota; implement dry-run fallback (local materials); per-user keys Phase 2 (Q2 decision) |
| **MoneyPrinterTurbo container OOM** | 2-core VPS, only 7.8GB RAM, ~4.8GB free; render tasks greedy (ffmpeg, TTS buffering) | Container killed; task fails silently; TaskQueue in-memory state lost | Set memory limit in compose (`memory: 2g`); implement task timeout (20 min); offload to Redis (Phase 2 optional) |
| **TTS synthesis timeout (edge-tts)** | Network latency to Azure TTS service; config.toml default 30s timeout | Audio generation fails; retry may succeed | Increase timeout to 60s; implement fallback to Elevenlabs TTS; monitor SLA |
| **Video output path traversal (security)** | Malicious topic input with `../` sequences | Generated video written outside task directory; disk pollution / potential RCE vector | MoneyPrinterTurbo's `file_security.resolve_path_within_directory()` already validates; confirm in integration tests |
| **Generated video leaks sensitive data (multi-tenant risk)** | If VideoSmith later supports multiple content teams, output files in shared storage | User A's video (e.g., internal strategy topic) visible to User B | Implement per-user task directory isolation; access control checks in /output proxy |
| **MoneyPrinterTurbo restart → task state lost** | In-memory task manager (default config); no persistence | Jobs in-progress vanish; UI shows orphaned job records; user confused | Accept as Phase 1 limitation; Phase 2 option: enable Redis (`enable_redis: true`); wrapper queries MPT on startup to rebuild lost state |
| **Concurrent users, same topic, duplicate generation** | No deduplication; User A and User B submit identical topic simultaneously | Two identical videos generated; storage waste; cost waste | Accept as Phase 1 limitation; Phase 2: cache recent topics; return existing job ID if topic+format match within 1 hour |
| **Slow dashboard load (100+ jobs)** | No pagination or DB index on jobs(user_id, created_at) | UI blocks while fetching all jobs; perceived slowness | Implement pagination (10 jobs/page, §6.3); add DB index; lazy-load older jobs |
| **Stale token never expires** | `sessions.expires_at` not checked on every API call | User logs out but token still valid if cloned/stolen | Implement strict token validation: `SELECT expires_at FROM sessions WHERE token=$1; IF expires_at < NOW() THEN 401 Unauthorized` |
| **LLM API key exposed in logs** | Logging framework captures full HTTP request headers | Credentials leaked to `/var/log/videosmith.log` or stdout | Sanitize logs; redact Authorization header; never log request body for POST /api/preferences |

---

## 11. Acceptance Criteria

**Phase 1 sign-off (before Phase 2 commences):**

- [ ] User can register with name + PIN; login with PIN; token persists across page reloads
- [ ] User can submit a topic; UI shows job status updates in real-time (pending → researching → drafting → completed)
- [ ] Generated MP4 downloads successfully; plays in browser/player
- [ ] 30+ Playwright spec tests pass (auth flow, job submission, polling, download, error cases)
- [ ] MoneyPrinterTurbo containers (mpt-api, mpt-webui) running on root_default network
- [ ] VideoSmith communicates with MPT API; no external internet exposure of MPT
- [ ] PWA service worker caches assets; app works offline (display cached jobs, show "offline" banner)
- [ ] All ADR verdicts in §9 verified (implemented items tested; N/A items justified)
- [ ] Red-team failure modes (§10) reviewed; mitigations in place or explicitly deferred
- [ ] Database schema verified (0 FK violations, bcrypt PIN sampling spot-checked)
- [ ] CPU constraint acknowledged: single concurrent task acceptable; ~5–15 min render time expected

**Phase 2 sign-off (before Phase 3 scope decision):**

- [ ] User can configure personal Pexels/Pixabay keys in preferences panel
- [ ] LLM provider + TTS voice selection working (dropdown, saved preference)
- [ ] Video format selection (9:16, 16:9, 1:1) working; job snapshots format
- [ ] Dry-run mode tested (no API calls, local materials only)
- [ ] Q1 (auth model) resolved: if hub-integrated, auth endpoints re-implemented + hub tests pass
- [ ] 10+ new Playwright tests covering preferences, format selection, dry-run

**Phase 3 pre-start:**

- [ ] Q2 (output distribution) resolved: Phase 3 scope finalized (manual, auto-publish, handoff, or hybrid)
- [ ] If auto-publish: Upload-Post credentials configured; test job publishes to TikTok/Instagram
- [ ] If handoff: integration point defined (webhook, API, S3 copy)

---

## 12. References & Related Decisions

**ADRs (per ADR-107 checklist, §9):**
- ADR-005 (SW cache versioning)
- ADR-009 (schema per PWA)
- ADR-011 (PIN-based auth)
- ADR-070 (IST timezone)
- ADR-081 (safe-area insets)
- ADR-086 (isolated PWA architecture) — **primary decision framework**

**Precedent:**
- ADR-097 (Blogsmith) — `/var/www/Others/Automation/blogsmith/` (reference impl)
- Health Tracker (ADR-086 precedent) — isolated PWA pattern

**External:**
- MoneyPrinterTurbo (GitHub) — https://github.com/harry0703/MoneyPrinterTurbo (MIT)
- Config reference — `/var/www/moneyprinterturbo/config.toml` (450+ lines, 25+ LLM providers)

**Planning doc:** `/var/www/360lm/docs/plan_videosmith_2026-07-03.md` (phases, open questions, red-team details)

---

## 13. Build Log & Lessons Learned (2026-07-03)

Phase 1 was built and deployed the same day the design was accepted. Recorded here because several concrete facts only surfaced by actually running the code against the real MoneyPrinterTurbo (MPT) API — the design sections above described the intended shape correctly, but three implementation details were wrong until verified live. Kept as a durable record so a future change to this module (or a similar MPT-wrapping effort) doesn't rediscover these the hard way.

**What's live:**
- `videosmith_app` container (port 8791, localhost-only) — full auth (register/login/logout), job submit/list/get/cancel/download, embedded 3-tier script-generation chain (ADR-062: Claude OAuth → OpenRouter → Anthropic REST)
- `mpt-api` container (port 8780→8080, localhost-only) — MoneyPrinterTurbo engine, internal-only, no Traefik label
- `videosmith` Postgres database + `videosmith_app` role, migration applied (`migrations/migrate_videosmith_v1.sql`)
- Files at `/var/www/Others/Automation/videosmith/` (mirrors Blogsmith's layout exactly)

**Lessons learned (real bugs found and fixed during first end-to-end test run):**

1. **MPT's `video_aspect` field takes literal ratio strings, not names.** The API rejects `"portrait"`/`"landscape"` — it requires exactly `'16:9'`, `'9:16'`, or `'1:1'` (Pydantic enum, confirmed via live 400 response body). Fixed in `videosmith_app.py`'s `aspect_map`. Anyone reading MPT's own source comments/docstrings could reasonably guess "portrait" — the enum only surfaces by hitting the live endpoint or reading `app/models/schema.py`'s `VideoAspect` enum directly.

2. **MPT's real API routes live under `/api/v1/`, not the bare paths the docstrings imply.** `POST /videos` is actually `POST /api/v1/videos`; `GET/tasks/{id}` is `GET /api/v1/tasks/{id}`. The `/tasks/{task_id}/...` bare path IS real, but it's a separate static-file mount (`app.mount("/tasks", StaticFiles(...))` in `app/asgi.py`) for serving finished video files — completely different from the `/api/v1/tasks` query API. Confirmed by pulling the live `/openapi.json` rather than trusting source-reading alone.

3. **MPT does not self-report task failure for exceptions raised inside its own background render thread.** When the Pexels-key-missing `ValueError` fired mid-render (see below), MPT's task state stayed at `state: 4` (processing) forever — it never flips to `-1` (failed). This is a real gap in MPT itself, not a wrapper bug. **This is exactly why the MDD's Red-Team §10 20-minute job timeout is load-bearing, not a nice-to-have** — without VideoSmith's own timeout, a job like this would show "rendering" in the UI indefinitely. Verified: `mpt_get_task` polling combined with the 20-minute deadline in `process_job()` is the only thing that will ever mark such a job `failed`.

4. **The script-bypass design (§8 Q5 resolution) is confirmed working end-to-end, not just theoretically compatible.** Live test: VideoSmith generated a script + terms via Claude OAuth (Tier 1 succeeded, no OpenRouter/Anthropic REST fallback needed), passed both into MPT's `video_script`/`video_terms` fields, and MPT's own logs show it used the supplied text verbatim (`generate_script`/`generate_terms` steps logged the exact wrapper-supplied content) instead of calling its own configured LLM provider. MPT's `llm_provider = "openai"` config value is therefore genuinely inert for this pipeline — confirmed, not assumed.

5. **Orphan-recovery on restart works as designed.** Killing the container mid-render (simulated by `docker restart`) correctly flipped the in-flight job to `failed` with `'backend restarted mid-job'` via `recover_orphaned_jobs()`, rather than leaving it stuck. Verified live, not just by code inspection.

6. **Pexels API key obtained and wired in (2026-07-03) — full pipeline now confirmed working end-to-end with a real rendered video.** User supplied a free Pexels key; added to `config.toml`'s `pexels_api_keys`. First real render then failed on a NEW issue (see #7), fixed same session, and a second real render **completed successfully**: register → login → submit topic → script+terms via Claude OAuth → MPT job creation → Pexels stock clips downloaded (11 clips) → ffmpeg concatenation → TTS narration → subtitle burn-in → final MP4. Downloaded through VideoSmith's own authenticated `/api/jobs/{id}/output` endpoint and verified with `ffprobe`: valid MP4, 55s duration, 17.7MB, matches the generated script length. Total render time ~9.5 minutes (within the accepted 5-15 min estimate for the 2-core box, §8 Q4). Test job/user data cleaned from the DB afterward.

7. **The `resource/` Docker volume mount silently shadowed MoneyPrinterTurbo's own bundled fonts/BGM, breaking subtitle rendering.** `docker-compose.yml` originally mounted the host's empty `/var/www/moneyprinterturbo/resource/` directory onto the container's `/MoneyPrinterTurbo/resource` — which is exactly where the MPT image bundles its default fonts (e.g. `STHeitiMedium.ttc`) and songs. An empty host directory mounted on top of a path with baked-in image files **hides those files inside the container**, a standard Docker volume-mount gotcha. First real render (after the Pexels key was added) got all the way through script generation, TTS, and stock-footage download/combination, then failed at the final subtitle-burn step with `OSError: cannot open resource` / `ValueError: Invalid font ... STHeitiMedium.ttc`. Fixed by removing the `resource` volume mount entirely from `docker-compose.yml` (Phase 1 has no custom-font/BGM requirement, so there's nothing to persist there) — confirmed via `docker exec mpt-api ls /MoneyPrinterTurbo/resource/fonts` that the image's real font files are visible again once the mount was dropped. **This is now VCC Class 16 territory** (external tool's real on-disk contract, not assumed) but specifically about volume-mount shadowing rather than an API contract — logged as its own new lesson in `vcc_library.md`.

**Housekeeping done during build:**
- Deleted the earlier superseded `/var/www/moneyprinterturbo/docker-compose.release.yml` (had public Traefik labels for the *wrong* domain — `videos.360dlm.in`, a separate custom domain the user put on hold) — the real compose file is `/var/www/Others/Automation/videosmith/docker-compose.yml`, which manages both `mpt-api` (internal-only) and `videosmith-app` (Traefik label for `videosmith.srv1111289.hstgr.cloud`, same subdomain pattern as this file's own §3.1/§8 design).
- **Correction (2026-07-03, later same session):** the `videosmith.srv1111289.hstgr.cloud` label was initially assumed "dormant, no DNS yet" — this was wrong. `*.srv1111289.hstgr.cloud` is Hostinger's own wildcard DNS for the VPS hostname (verified: even a random never-used subdomain resolves to the VPS IP), completely separate from the `360dlm.in` custom domain that genuinely has zero DNS records. VideoSmith has been live and publicly reachable at `https://videosmith.srv1111289.hstgr.cloud/` since the container started — confirmed via `curl` (HTTP 200, valid `zerossl` cert, PWA and API both serving correctly). The user's original "hold on DNS" instruction was specific to `360dlm.in`/`videos-api.360dlm.in` and never applied to this subdomain.
- Set MPT's `config.toml`: `endpoint = "http://mpt-api:8080"` (so returned video file URIs are directly fetchable by the wrapper container) and `max_concurrent_tasks = 1` (matches the accepted 2-core constraint, §8 Q4).
- Reused the platform's existing `ANTHROPIC_API_KEY` / `OPENROUTER_API_KEY` (same values already in use by Blogsmith) rather than requesting new ones — no new secret was needed for Phase 1.

**Not yet done (deliberately, not gaps):**
- Public DNS/subdomain (on hold per explicit user instruction 2026-07-03) — **superseded, see correction below: this was never actually blocked.**
- PWA icons (manifest has an empty icon array — cosmetic only, not required for internal testing)
- Custom fonts/BGM (resource volume mount intentionally removed, §13 item 7 — revisit only if a real need for custom fonts/BGM arises)

**Decision Maker:** hkl (via interactive Q&A, 2026-07-03)  
**Status:** Phase 1 built, deployed, and fully functionally verified — a real 55s MP4 was generated end-to-end and downloaded successfully. See §14 for the Blogsmith-parity expansion (batch, scheduler, publish/unpublish) built the same day.  
**Last updated:** 2026-07-03

---

## 14. Blogsmith Feature-Parity Expansion (2026-07-03, same day)

User asked for VideoSmith to match Blogsmith's feature set more closely (per ADR-097): batch/multi-keyword submission, a scheduler, draft/publish distinction, auto-posting, and unpublish — with one explicit UI improvement requested for multi-topic entry.

### 14.1 Data Model Changes (`migrations/migrate_videosmith_v2.sql`)

Deliberately **two separate state machines**, not one collapsed enum like Blogsmith's `jobs.status`:
- `jobs.status` — render pipeline: `pending, queued, researching, drafting, completed, failed, cancelled`
- `jobs.publish_status` — distribution phase, only meaningful once `status='completed'`: `draft, scheduled, published, unpublished, failed`

**Why split, unlike Blogsmith:** Blogsmith's `'drafting'` already meant two different things contextually (mid-generation vs. no-WordPress-configured). VideoSmith's render phase has more states than Blogsmith's simple research+draft, so collapsing both phases into one column would make `'drafting'` genuinely ambiguous. Two columns cost one extra `WHERE` clause but remove that ambiguity entirely.

New `jobs` columns: `publish_status`, `publish_at`, `posted_at`, `platform_post_ids` (JSONB), `upload_post_job_id`, `publish_error`.
New `users` columns (identical to Blogsmith's v2 schema at the time): `upload_post_api_key`, `upload_post_username`, `upload_post_platforms` (JSONB array), `recurring_enabled`, `schedule_mode`, `interval_value`, `interval_unit`, `daily_time_ist`, `consecutive_publish_failures`, `schedule_paused_reason`, `next_run_at` (the per-user schedule cursor — source of truth for timing, not a per-job timestamp).
**Superseded by §16**: the three `users.upload_post_*` columns were replaced same day by a dedicated `upload_post_channels` table (multi-channel support) — see §16 for the current schema. The scheduler-cursor columns (`recurring_enabled` through `next_run_at`) are unaffected.

### 14.2 Scheduler (mirrors Blogsmith's `scheduler_worker()` exactly)

`compute_next_slot()`, `queue_keywords()`, `process_due_queue_item()`, `scheduler_worker()` in `videosmith_app.py` are a direct port of Blogsmith's pattern (`blogsmith_ai.py:184-386`) — same 30-second poll, same per-user cursor logic, same recurring-vs-one-shot semantics. **What's due to render** is controlled by this cursor; **what's due to publish** is controlled separately by each job's own `publish_at` (set at submission time, single-topic form only — batch topics publish immediately once rendered, matching Blogsmith's own simplification where batch cadence doubles as publish cadence).

### 14.3 Auto-Posting via Upload-Post (verified live against the real API, not just docs)

MoneyPrinterTurbo's `config.toml` has `upload_post_*` settings but **no actual implementation** — verified by reading its full source: the settings are documented but dead config. VideoSmith implements the real integration itself in `videosmith_app.py` (`call_upload_post()`, `cancel_upload_post_schedule()`, `attempt_publish()`, `dismount_job()`), calling `https://api.upload-post.com/api` directly, per VCC Class 16 discipline (checked the live API contract via WebFetch of docs.upload-post.com, then **verified against the real endpoint** with a deliberately-fake API key — got a genuine `HTTP 401` back, confirming the request shape is correct and reaches the real service, not just a docs-based guess).

**Account linking is NOT an API call** — per Upload-Post's own model, the user must link TikTok/Instagram/YouTube accounts on upload-post.com's own dashboard first (OAuth happens there, not through this integration), then paste the resulting API key + username into VideoSmith's new Settings panel. **This is a real external dependency, same shape as the Pexels key** — the feature is fully built and dry-run-tested, but genuine auto-posting needs the user to create an upload-post.com account. Until then, every completed video simply lands in `publish_status='draft'` (verified live) — exactly Blogsmith's own dry-run behavior before real WordPress credentials existed.

**Unpublish limitation (documented, not a bug):** Upload-Post's API only documents cancelling a *scheduled* (not-yet-posted) item (`DELETE /api/uploadposts/schedule/<job_id>`). There is no documented endpoint to un-post an already-live post — this mirrors a real constraint of most social platform APIs, not something Blogsmith's WordPress-trash-based unpublish has to deal with. For a `published` job, "Unpublish" in VideoSmith stops tracking it as live locally; the actual platform post must be removed manually on that platform if truly needed. This is called out directly in the UI's confirm dialog so it's never a silent gap.

### 14.4 UI Improvement — Multi-Topic Chip Input (the requested improvement over Blogsmith's plain textarea)

Blogsmith's batch UI is a `<textarea>`, one keyword per line (`public/index.html:96-136` in blogsmith). VideoSmith's improvement: a **chip/tag input** (`#chipBox` in `public/index.html`) —
- Type a topic + Enter (or comma) → becomes a removable chip, not a line in a wall of text
- **Paste-splitting**: pasting a multi-line or comma-separated block auto-splits into individual chips in one action
- **Live counter** (`N / 50 topics`), turns amber near the cap
- **Per-chip validation**: a chip whose topic exceeds 200 chars renders in a distinct error color with a tooltip, instead of silently truncating server-side with no visual signal
- Individual **×** remove button per chip, plus "Clear all"
- Backspace on an empty input pops the last chip (fast correction without hunting for a tiny × button)

This is a genuinely different interaction model, not a re-skin — the user manipulates discrete topic units instead of editing raw text, which scales much better past a handful of topics.

### 14.5 New Screens/Endpoints

- **Settings panel** (`#settingsPanel`): originally a single Upload-Post username/API key/platform-checkbox form (parallel to Blogsmith's WP Settings panel); superseded same day by a multi-channel list — see §16
- **Published/Scheduled section** (`#publishedSection`): dedicated list of live/scheduled posts with Unpublish — parallel to Blogsmith's "Published on WordPress" section
- **Tab toggle**: Single topic ⇄ Multiple topics, single form gains a Post now/Schedule toggle (mirrors Blogsmith's `publishTiming` radio)
- New API: `POST /api/jobs/batch`, `POST /api/jobs/{id}/publish`, `POST /api/jobs/{id}/dismount`, `GET`/`POST /api/preferences` (extended)

### 14.6 Verified Live (2026-07-03)

- Batch submission of 3 topics → all landed as `status='queued'`
- Scheduler picked up the first queued job within one 30s tick, correctly advanced `users.next_run_at` by the configured interval
- `POST /api/jobs/{id}/publish` on a completed job with no Upload-Post credentials → stayed `publish_status='draft'` (dry-run confirmed)
- Same call with a deliberately-fake API key → real HTTP request sent to `api.upload-post.com`, got back genuine `401 Unauthorized`, surfaced correctly as `publish_status='failed'` with the real error text
- `POST /api/jobs/{id}/dismount` correctly rejected a non-published/scheduled job, and correctly transitioned a `published` job to `unpublished`
- SW cache bumped to `videosmith-v2` per ADR-005

**Not yet done:** a real end-to-end auto-post (needs the user's own upload-post.com account).

### 14.7 Playwright Coverage Added (2026-07-03)

`videosmith.spec.js` (11 tests, `playwright.config.js` + `package.json`) — same house convention as `blogsmith.spec.js`. 8 tests run by default and all pass (login/PIN gate including the 4-8 digit variable-length behavior, auto-reconnect, logout, publish-timing toggle, chip-input multi-topic add/paste/remove/counter, schedule-mode toggle, empty-state history, settings persistence). 3 tests are gated behind `VIDEOSMITH_FULL_RENDER=1` (detailed-scenario submission, cancel, full render + download) since submitting a job spends real Pexels/LLM quota regardless of whether it's later cancelled — cancelling only flips DB status, it does not stop an in-flight render thread.

Writing the spec surfaced and fixed a real bug: page reload never actually auto-reconnected despite a stale comment claiming it did (`init()` never called `enterApp()`); fixed with a new `GET /api/me` endpoint.

**Decision Maker:** hkl  
**Last updated:** 2026-07-03

---

## 15. Detailed-Scenario Input (2026-07-03, later same day)

User asked whether a detailed scenario (not just a short topic/keyword) can drive video generation. Answer: **yes, already supported by the pipeline design — improved discoverability and verified live.**

**Changes:**
- `SCRIPT_SYSTEM_PROMPT` (`videosmith_app.py`) now explicitly instructs the model to follow scenario specifics (names, numbers, tone, call to action) closely rather than writing a generic script about the general subject, when given a detailed brief instead of a short topic.
- Single-topic input (`public/index.html`) changed from a one-line `<input>` to a 3-row `<textarea>` with an updated placeholder inviting either a quick topic or a full scenario — previously nothing prevented longer input server-side, but the UI visually signaled "short keyword only."
- No length cap on the single-job `topic` field (unchanged) — the 200-char cap only applies to the batch/multi-topic chip flow, which is intentionally keyword-style.
- SW cache bumped to `videosmith-v4`.

**Verified live (2026-07-03):** called `generate_script()` directly with a multi-sentence scenario (a bakery owner's name, business name, a specific recurring Tuesday discount, the reason for it, and an exact call to action). The generated script correctly wove in every specific detail verbatim rather than producing a generic "bakery marketing" script — confirms the pipeline already handles rich scenario input well, this was a prompt/UI clarity improvement, not a new capability.

**Decision Maker:** hkl  
**Last updated:** 2026-07-03

## 16. Multi-Channel Upload-Post Support (2026-07-03, later same day)

User caught a real capability gap from a live screenshot of the Settings panel: the TikTok/Instagram/YouTube checkboxes implied per-platform destinations, but §14's schema stored exactly ONE `upload_post_username`/`upload_post_api_key` per VideoSmith user — the checkboxes only chose which platforms of that single account to post to. There was no way to hold two separate Upload-Post accounts (e.g. two different clients/brands) and pick between them per job.

**Verified against Upload-Post's real docs (WebFetch of docs.upload-post.com), not assumed:** they support 13 platforms — TikTok, Instagram, LinkedIn, YouTube, Facebook, X, Threads, Pinterest, Bluesky, Reddit, Discord, Telegram, Google Business Profile — via their own "one API key, multiple named profiles" architecture (each profile has its own connected socials). The user explicitly chose full independence instead: each VideoSmith **channel** holds its own separate API key + username, not a shared Upload-Post account.

**Data model (`migrations/migrate_videosmith_v3.sql`):**
- New table `upload_post_channels`: `id`, `user_id` (FK, cascade delete), `label`, `upload_post_username`, `upload_post_api_key`, `platforms` (JSONB), `is_default` (app-enforced exclusivity — only one default per user), `created_at`.
- `jobs.channel_id` (FK, `ON DELETE SET NULL`) records which channel a job used/will use.
- One-time backfill of any existing `users.upload_post_*` values into a "Default" channel per user, then those columns dropped — no data existed to backfill in practice (verified via SQL before dropping, zero rows affected).

**Backend (`videosmith_app.py`):** `UPLOAD_POST_PLATFORMS` constant (all 13, source of truth for both validation and the API's `available_platforms` response — the platform list is never hardcoded in the frontend). New endpoints `GET/POST /api/channels`, `PUT/DELETE /api/channels/{id}` with ownership checks and blank-API-key-means-keep-existing (mirrors Blogsmith's WP settings pattern). `call_upload_post()`/`attempt_publish()`/`dismount_job()` SQL rewired from `jobs JOIN users` to `jobs LEFT JOIN upload_post_channels` via `channel_id`. `default_channel_id()`/`_resolve_channel_id()` helpers: single-topic job submission accepts an explicit `channel_id` (validated against the caller's ownership) or falls back to the user's default channel; batch/scheduled jobs always use the default channel (no per-topic picker in v1 — the narrower, more contained option, since the user did not respond to that specific follow-up question).

**Frontend (`public/index.html`):** Settings panel replaced with a Channels list (add/edit/delete, "Set default", platform checkboxes rendered dynamically from the API's `available_platforms` — not hardcoded HTML). Single-topic submit form gained a channel `<select>` (hidden with a "no channel configured" hint when the user has none). Job rows show `via <channel label>` when applicable. SW cache bumped to `videosmith-v6`.

**Verified live (2026-07-03):** full CRUD smoke-tested via curl (create 2 channels, switch default, reject an invalid platform, create a job with no explicit channel_id and confirm it inherited the default, delete a channel) and via a real browser session (added a channel, saw all 13 platform checkboxes render from the server list, saved, confirmed it appeared as the job-submit form's default selection). Playwright coverage added in the same test file (§14.7) — `channels: add with dynamic platform list, edit, set default, delete`, part of the default (non-gated) suite since it makes no real job submissions.

**Decision Maker:** hkl  
**Last updated:** 2026-07-03

## 17. Highlight Video — Assembled From the User's Own Event Photos/Clips (2026-07-06)

User asked for a second generation mode: instead of an AI script driving stock footage, assemble a highlight video directly from the user's own event photos and video clips (e.g. a birthday party, a store activation). Ran `/scout` first per house convention for any non-trivial new capability before writing code.

### 17.1 Scout Findings (full report in chat, summarized here)

Checked the actual VPS stack before searching externally: `mpt-api` (the same MoneyPrinterTurbo container VideoSmith already wraps) has `moviepy 2.2.1` + `pydub` already installed, and its source (`app/services/video.py`, `app/services/task.py`) has an **existing, never-used `video_source:"local"` pipeline**:
- `preprocess_video()` — validates local photo/video materials (min 480×480), converts photos into short Ken-Burns-zoom video clips automatically (`ImageClip` + a resize-over-time lambda), rejects low-res material.
- `combine_videos()` — subclips/concatenates materials (random or sequential order), resizes/pads to the target aspect ratio, applies transitions (fade/slide/shuffle), and drives total output length from a "primary audio" track's duration.
- `get_bgm_file()`/`bgm_type`/`bgm_file` — background-music mixing, layered under the primary audio via `CompositeAudioClip`.
- `custom_audio_file` — bypasses TTS entirely (real gap otherwise: MPT auto-generates a script via its own LLM call if `video_script` is empty).

**The one real gap MPT doesn't fill**: it only does naive fixed-length chunking of long clips, no "best moment" awareness. Researched **PySceneDetect** (BSD-3, pip-installable, CPU-only, `detect()`+`split_video_ffmpeg()`) as a pre-processing step to split raw multi-shot recordings at real scene boundaries first — user chose to pull this in for v1 rather than deferring it. Rejected: **auto-editor** (silence-cutting, wrong problem), **VideoHighlighter**/**AI-Youtube-Shorts-Generator** (ML "best moment" scoring — needs GPU/CUDA or an external LLM API, incompatible with the 2-core/no-GPU box).

**Net effect: reuse, not rebuild.** Zero new video-processing infrastructure — moviepy/ffmpeg were already proven working in this exact container from Phase 1.

### 17.2 Two Real Constraints Inherited From MPT's Design (documented, not bugs)

1. **Original clip audio is always discarded.** MPT's materials stage opens video clips with `audio=False` by design (its own comment: "项目视频素材阶段不需要保留素材原声" — the project doesn't need to keep material's own audio at the materials stage; final audio is mounted once, later, in `generate_video()`). A highlight video is real visuals + the chosen BGM (or silence), **not** the event's own ambient sound/speech. This is the same trade-off many consumer highlight-reel tools make (music swapped in over event footage), not unique to this build, but it is a real limitation worth knowing before relying on it for e.g. a wedding speech.
2. **Duration control needs a silent placeholder audio track.** `combine_videos()` derives its target output length from the "primary audio" file's duration — there's no direct "just make it N seconds" parameter. To drive a real target length (sum of all uploaded material, capped at `HIGHLIGHT_MAX_DURATION_SEC=90`) without adding unwanted narration, a silent MP3 of the computed duration is generated via ffmpeg and passed as `custom_audio_file`; BGM (if chosen) layers in separately via `bgm_type`.

### 17.3 Data Model (`migrations/migrate_videosmith_v4.sql`)

`jobs.job_type` (`'script'|'highlight'`, default `'script'` — the original pipeline is unaffected), `jobs.source_media` (JSONB, the list of uploaded filenames), `jobs.bgm_choice` (`'random'|'none'`). `topic` is reused as the event title/description for highlight jobs — no schema change needed there.

### 17.4 Infrastructure (`docker-compose.yml`)

`videosmith-app` gained `ffmpeg` (apt) + `scenedetect` (pip, pulls in `opencv-python` transitively — confirmed working, deliberately did not also add `opencv-python-headless` to avoid installing both) and a **new shared volume**: `/var/www/moneyprinterturbo/storage:/mpt-storage`, scoped to `storage/` only — deliberately not touching `resource/` (that's where MPT's bundled fonts/songs live un-mounted on purpose, per the Phase-1 font-shadowing bug documented in ADR-109; this new mount doesn't reopen that risk). This lets `videosmith-app` write uploaded files directly into MPT's `local_videos` material directory.

### 17.5 Backend (`videosmith_app.py`)

- `split_scenes()` — PySceneDetect's `detect()` + `split_video_ffmpeg()`; falls back to the whole clip unchanged if no cuts are detected (e.g. a single static shot).
- `probe_duration()` / `generate_silence()` — ffprobe/ffmpeg subprocess helpers for the duration-control mechanism in §17.2.
- `build_highlight_materials()` — walks the upload directory, scene-splits videos, measures durations, passes photos through as-is (MPT's own `preprocess_video()` handles the Ken-Burns conversion).
- `mpt_create_highlight_video()` — calls MPT with `video_source:"local"`, `video_concat_mode:"sequential"` (chronological, not shuffled — real footage, not stock filler), `video_transition_mode:"Shuffle"`, `subtitle_enabled:false`, `voice_name:""`.
- `_poll_mpt_task_to_completion()` — the polling/completion logic was extracted out of `process_job()` into a shared helper so `process_highlight_job()` doesn't duplicate it (both job types share the same "wait for MPT, then `attempt_publish()`" tail).
- New endpoint `POST /api/jobs/highlight` — a hand-written minimal `multipart/form-data` parser (raw `http.server`, no framework in this codebase; mirrors the project's existing minimalist style) accepts `title`, `files[]` (photos/videos), `bgm_choice`, `channel_id`, `publish_at`. Size-capped at `HIGHLIGHT_MAX_UPLOAD_MB=500`.

### 17.6 Frontend (`public/index.html`)

Third tab "Highlight video" alongside Single topic / Multiple topics: event title field, file picker (tracked in a JS array so files can be removed before submit without touching the native `<input>`'s FileList), BGM select (`random`/`none`), reuses the existing channel picker and publish-timing controls. Explicit copy calling out the audio-discard constraint from §17.2 so it's never a silent surprise. `submitHighlight()` uses `FormData`+`fetch` — deliberately does **not** use the shared `headers()` helper, since that sets `Content-Type: application/json` which would break the browser's own multipart boundary. Job rows show a "Highlight" badge (`data-job-type` attribute) alongside the existing status/channel badges. SW cache bumped to `videosmith-v7`.

### 17.7 Verified Live (2026-07-06)

- **Full happy-path render**, curl-driven: uploaded a synthetic 2-scene clip (red 3s + blue 3s, a hard color-cut so PySceneDetect had something real to detect) + a synthetic photo (green). Confirmed via logs that PySceneDetect split the clip into exactly 2 scene files, MPT's own `preprocess_video()` converted the photo into a Ken-Burns MP4, the silent-audio duration mechanism produced a ~10s target (matching 3+3+4s of material), and the final output was portrait 1080×1920 with a real audio track. **Frame-level color check at 1s/4s/8s confirmed red→blue→green in the correct sequential order** — not shuffled, not corrupted.
- **Real browser session** (Chrome MCP, disposable test profile): confirmed the tab renders, file add/remove works, and — as a genuine error-path test — submitted a job with an intentionally-invalid "video" file: it failed gracefully within seconds with a clear message ("Ensure file is valid video and system dependencies are up to date.") rather than hanging or crashing anything, and the "Highlight" badge displayed correctly on the failed job row.
- **Playwright**: 2 new default (free) tests — UI mechanics (file add/remove, BGM select, publish-timing toggle) and the invalid-upload graceful-failure path — plus 1 gated test (`VIDEOSMITH_FULL_RENDER=1`) that generates real synthetic media, submits, waits for completion, and verifies portrait aspect + BGM audio track via `ffprobe`. Ran the gated test once to confirm it actually passes (1.8 min) rather than just writing it — passed.
- **Live walkthrough as the real user** (PIN 0000, Harish Kumar Lal): confirmed Settings/Channels and the Highlight Video tab render correctly on the real account. Hit a real testing-infra snag along the way — injecting real file bytes as an inline base64 JS string silently truncated above ~5KB in this environment (logged as VCC Class 22) — worked around by temporarily serving the test files from `public/tmp_e2e_test/` and having the page `fetch()` them same-origin. The resulting job rendered successfully end to end (portrait 1080×1920, real BGM track) and is left in the real account's history as a genuine result.

### 17.8 Future Consideration — Large (~8GB) Source Library (2026-07-06, discussed not built)

User asked what would change to support an 8GB source photo/video collection where "not all might be useful." No build — a scoping discussion identified four real changes, none of them small: (1) the current in-memory multipart upload can't handle 8GB regardless of raising `HIGHLIGHT_MAX_UPLOAD_MB` — needs server-side directory ingestion (rsync/Syncthing, already running on this VPS) or real chunked/resumable upload; (2) curation stops being optional at this scale — blindly processing 8GB would take hours on 2 cores, needs a manual picker and/or automated pre-filtering (dedup, skip dark/blurry/short segments) — the "best moment" gap from §17.1 becomes load-bearing, not deferrable; (3) the model shifts from "one upload → one job" to a persistent, reusable media library selected from per job; (4) long-running material scans need their own progress reporting, separate from render progress. Recommended scouting the curation/dedup and resumable-upload tooling specifically before building any of this, same discipline as §17.1. No decision made on scope or timing.

**Decision Maker:** hkl  
**Last updated:** 2026-07-06