# ADR-115 Tailscale Mesh Is the Standing Admin-Access Fallback When ISP Routing Breaks the Public Path

## Status

Accepted, 2026-07-16.

## Status History

```yaml
status_history:
  - date: 2026-07-16
    status: Accepted
    changed_by: hkl
    reason: Operational fallback implemented and tested; decision ratified.
    changed_via: adr-kit (360lm)
```

## Context

The 360lm VPS is hosted on Hostinger at public IP 72.60.97.173 and is accessible via public hostname `srv1111289.hstgr.cloud` and Traefik-routed domains (`dev.srv1111289.hstgr.cloud`, `api.srv1111289.hstgr.cloud`, etc.). PWA end-users and admin tools (SSH, SFTP/FileZilla, Claude Desktop remote coding) rely on this public route.

**Incident (2026-07-15)**: Airtel broadband and mobile both broke the IPv4 route to the VPS public IP. Other ISPs (Jio) reported normal connectivity; external fetch tools confirmed the server remained reachable globally. Hostname DNS and IPv6 fallback attempts failed at the client (LAN did not support usable IPv6 for raw TCP). Public sshd port 22 and Traefik routes were unreachable from the owner's network for several hours.

**Problem**: ISP-level BGP or routing issues are uncontrollable; admin must be able to access the VPS for diagnostics and fixes when public routes fail. Using an alternate SSH port or waiting for Hostinger/ISP to restore service is unacceptable.

**Solution scope**: Admin-only fallback for SSH, SFTP, and remote coding tools. Not a replacement for public PWA routes (users should not be moved onto Tailscale). The underlying ISP issue still requires a Hostinger support ticket (pending user action).

## Decision

**Tailscale mesh VPN is the standing fallback for admin access when the public IPv4 route is broken.**

- `tailscaled` v1.98.9 runs as a system service on the VPS (device name: `srv1111289`, user: `haris.lal.1974@gmail.com`, tailnet: private).
- The owner's laptop and other authorized admin devices enroll in the same tailnet.
- Admin tooling (SSH, SFTP, Claude Desktop remote coding) targets the Tailscale mesh address `100.99.178.114` instead of the public IP when the public route is unavailable.
- Tailnet addresses are private to enrolled devices; reachable only via Tailscale DERP relays (automatically traverses ISP outages, firewalls, and NAT).
- Public sshd, Traefik routes, and PWA end-user access remain unchanged and on the public route.
- Tailscale is additive and does not replace the public path; once the ISP issue is resolved, admin can revert to public IP routes.

**Operational setup**:
- Tailscale device on VPS: `100.99.178.114` (via `tailscale ip -4`).
- Owner's laptop: `100.104.45.66` (via `tailscale ip -4`).
- DERP relays: Tailscale's global relay network (automatic; no manual configuration).
- Uninstall path: `sudo tailscale logout`, `sudo apt-get remove tailscale`, and delete the devices from the Tailscale admin console.

**Decision Maker:** hkl

## Implementation Notes

- **Files & setup**:
  - `/usr/bin/tailscale` — Binary (installed via `apt-get install tailscale`).
  - `/etc/tailscale/` — Config directory (key file stored at `/var/lib/tailscale/state.db`; encrypted locally).
  - `tailscaled` — Systemd service (enabled on boot via `systemctl enable tailscaled`).
  - Mesh address lookup: `tailscale ip -4` on each enrolled device.

- **First-time enrollment**: On each client, run `tailscale login --auth-key <key>` (auth key from Tailscale admin console) or `tailscale login` and follow the browser prompt. Claude Desktop **cannot display the trust prompt** during remote setup—pre-accept the host key via a manual SSH attempt in PowerShell: `ssh root@100.99.178.114` (first run accepts and caches the key).

- **Key identifiers**:
  - Tailnet ID: (private to user's Tailscale account).
  - Device tags: `tag:admin` (proposed for future ACL rules, not yet used).
  - ACL policy: Currently allows all traffic within the tailnet; can be locked down via Tailscale admin console.

- **Find all references**: Tailscale mesh addresses are typically documented in team wikis or deployment notes. Grep for `100.99.` or `100.104.` in local notes; these are the only mesh IPs in scope.

- **Gotchas**:
  - Tailscale does NOT disable the public sshd or Traefik; public routes remain unchanged.
  - If a client device is removed from the tailnet (via admin console), it loses access immediately; no grace period.
  - The tailnet is private; end-users and external services cannot reach the VPS via Tailscale addresses. This is intentional (admin-only fallback).
  - Tailscale DERP relays require outbound UDP port 41641 (or fallback via HTTPS). Check firewall rules if mesh connectivity fails.

## Alternatives Considered

- **Alternate SSH port (e.g., 2222).** Rejected: The ISP break was per-IP-route, not per-port. Alternate SSH ports would have been unreachable by the same mechanism.
- **Cloudflare fronting on the VPS's own domain.** Deferred: Cloudflare can proxy PWA traffic and bypass ISP breaks for end-users. This requires setup effort and DNS changes; revisit if Hostinger ticket fails and ISP issues recur.
- **Request a new VPS IP from Hostinger.** Complementary, not alternative: A fresh IP may help if the current IP is flagged or blacklisted. This is orthogonal to Tailscale and pending user's support ticket.

## Consequences

**Positive:**
- Admin access is restored within seconds, without waiting for Hostinger or ISP to fix routing.
- Tailscale DERP relays automatically traverse ISP breaks, firewalls, and NAT; no special firewall rules or port forwarding required.
- Access is isolated to enrolled admin devices; risk of unauthorized access is lower than opening an alternate SSH port to the public.
- Zero impact on PWA end-users or public Traefik routes.
- Operational simplicity: Tailscale is a single binary and system service; no custom orchestration.

**Negative / Trade-offs:**
- Requires one-time setup: install Tailscale on VPS and each admin device, and enroll in the tailnet.
- Adds a third-party VPN dependency (Tailscale's DERP relay network); outages at Tailscale would affect admin fallback.
- The tailnet is not encrypted at rest (Tailscale stores state locally); if a device is compromised, the tailnet is compromised. Mitigated by removing the device from the tailnet immediately if suspected.

**Risks and mitigations:**
- **Risk**: Tailscale DERP relay is unavailable or slow. **Mitigation**: Tailscale provides monitoring at tailscale.com/admin/; if relays are down, the fallback fails but public routes are still available (ISP permitting).
- **Risk**: Admin accidentally leaves Tailscale enabled on a shared or untrusted device, compromising the mesh. **Mitigation**: Tailscale admin console allows per-device revocation; remove compromised devices immediately.
- **Risk**: User forgets the tailnet address and mixes it up with the public IP. **Mitigation**: Document mesh addresses clearly in local notes or environment (e.g., `.ssh/config` with named hosts).

## Related Decisions

- None. This ADR is a standalone operational fallback and does not depend on or supersede other architectural decisions.

## References

- Incident: Airtel ISP break (2026-07-15) at owner's location; Jio and external routes remained functional.
- Tailscale documentation: https://tailscale.com/docs
- Tailscale DERP relays: https://tailscale.com/blog/how-tailscale-works
- Enrollment guide: https://tailscale.com/kb/1147/managing-devices
- Memory reference: (pending user's Hostinger support ticket for public IP restoration).