outofcontext — Engineering Spec

Drop into any video without the prerequisite context. A browser extension that overlays minimal, frosted-glass concept bubbles on videos (lectures, podcasts, vlogs, news). Bubbles appear quietly as concepts come up; clicking one expands it into a glass card, runs LLM inference on demand, and shows an in-context explanation plus a link to where the concept was first introduced in your library.

Design reference: Figma — outofcontext — "Extension Overlay" + "Lumen Home" frames (file wJS1H6oi3yn4AWTbkDN9Aa). The product name is outofcontext; swap the "Lumen" logo text in the mockup. Frame exports are checked in at design/extension-overlay.png and design/home-library.png.

Player overlay

Home — annotated library

---

1. Product surface (v1)

1. Player overlay (content script on YouTube):
   • Small numbered glass bubbles anchored to the player, appearing/disappearing in sync with video.currentTime.
   • A ● N concepts pill in the top-right of the player — the only always-visible footprint. Bubbles auto-surface on their own as each concept comes up (passive discovery is the whole point — you find concepts you didn't know to look for); the pill toggles the entire bubble layer on/off, and on re-enable the bubbles fan out from the pill onto the rail.
   • Click a bubble → it morphs in place into a glass annotation card that expands leftward from the bubble's rail position (not centered, so the video stays visible): concept title, 2–3 sentence explanation generated on click (streamed in), divider, "FIRST INTRODUCED" reference row linking to another video/timestamp in your library.
2. Home page (extension page, chrome-extension://…/home.html):
   • Library of every video you've annotated: thumbnail grid, bubble-count chips, filters (All / Lectures / Podcasts / News / Vlogs), search.
   • Clicking a card reopens the video at the source URL.

Out of scope for v1: native apps (Spotify/TikTok — no extension surface), non-YouTube sites (adapter interface exists, implementations come later), accounts/sync, social features.

---

2. Architecture

┌─────────────────────────────────────────────────────────────┐
│ Chrome (MV3)                                                │
│                                                             │
│  ┌────────────────────────┐    chrome.runtime messages      │
│  │ Content script (YT tab)│◄──────────────┐                 │
│  │  - site adapter        │               │                 │
│  │  - shadow-DOM overlay  │    ┌──────────▼─────────────┐   │
│  │  - React + motion UI   │    │ Background service     │   │
│  └────────────────────────┘    │ worker                 │   │
│                                │  - Anthropic API calls │   │
│  ┌────────────────────────┐    │  - transcript fetch    │   │
│  │ Home page (ext. page)  │◄──►│  - job queue/cache     │   │
│  │  - React + motion UI   │    └──────────┬─────────────┘   │
│  └────────────────────────┘               │                 │
│                  ▲                        │                 │
│                  └──────── IndexedDB (Dexie) ◄──────────────┘
└─────────────────────────────────────────────────────────────┘
                                            │ HTTPS
                                   api.anthropic.com

Key decisions:

• All Anthropic calls live in the background service worker. MV3 service workers make cross-origin fetches freely when api.anthropic.com is in host_permissions — no CORS, and the API key never touches page-world JS. Content script and home page talk to it via chrome.runtime.sendMessage / Port (ports for streaming).
• Overlay mounts inside a shadow root appended to YouTube's player container (#movie_player), not document.body. Two reasons: style isolation (YT's CSS can't bleed in, ours can't leak out), and fullscreen — the fullscreen element is the player container, so children of it stay visible in fullscreen while body-level overlays vanish.
• Local-first. All state in IndexedDB via Dexie. The user's own Anthropic API key (stored in chrome.storage.local, settable in options page). No backend in v1; a thin backend (Hono on Cloudflare Workers) is the v2 path if/when sync or key-proxying is wanted.

Repo layout (WXT)

outofcontext/
  wxt.config.ts
  package.json
  src/
    entrypoints/
      content.tsx          # YouTube content script (matches *://*.youtube.com/*)
      background.ts        # service worker
      home/                # library page (chrome-extension://…/home.html)
      options/             # API key + settings
    adapters/
      types.ts             # SiteAdapter interface
      youtube.ts
    llm/
      client.ts            # Anthropic SDK wrapper (background only)
      conceptPass.ts       # pass 1: transcript → concept map
      explain.ts           # pass 2: on-click explanation (streaming)
      prompts.ts
      schemas.ts           # JSON schemas for structured outputs
    db/
      index.ts             # Dexie schema + queries
    ui/
      overlay/             # Bubble, AnnotationCard, ConceptsPill
      home/                # VideoCard, FilterPills, SearchBar
      glass.css            # frosted-glass tokens
      motion.ts            # shared variants/springs
    transcript/
      youtube.ts           # caption track discovery + timedtext fetch

Stack: WXT (Vite-based MV3 framework, HMR for content scripts) · React 19 · TypeScript · motion (the framer-motion successor; import { motion, AnimatePresence } from "motion/react") · Dexie · @anthropic-ai/sdk · vanilla CSS for glass (it's ~40 lines; Tailwind optional later).

---

3. Site adapter interface

Everything YouTube-specific is quarantined behind this, so Vimeo/podcast-web-player adapters are additive:

interface SiteAdapter {
  /** Does this adapter handle the current URL? */
  matches(url: URL): boolean;
  /** Stable ID for the video (e.g. YT videoId) + metadata */
  getVideoInfo(): Promise<VideoInfo>;          // { id, title, channel, durationS, thumbnailUrl, url }
  /** The element to mount the overlay shadow root into (must be the fullscreen element or inside it) */
  getOverlayMount(): HTMLElement;
  /** The <video> element, for currentTime + seeking */
  getMediaElement(): HTMLVideoElement;
  /** Fetch transcript with per-cue timestamps, or null if unavailable */
  getTranscript(): Promise<TranscriptCue[] | null>;  // { startS, endS, text }[]
  /** SPA navigation hook — YouTube never full-reloads */
  onNavigate(cb: (url: URL) => void): () => void;    // yt: listen for "yt-navigate-finish"
}

YouTube specifics:

• Transcript: read ytInitialPlayerResponse.captions.playerCaptionsTracklistRenderer.captionTracks, prefer manual track over ASR, fetch baseUrl + "&fmt=json3", normalize to TranscriptCue[]. This is unofficial and the #1 breakage risk (see §10).
• No captions → show the pill in a disabled "no transcript" state. (Whisper-on-audio is explicitly out of scope for v1.)

---

4. LLM pipeline

Two passes. Pass 1 is automatic and cheap-ish (one call per video); pass 2 runs only when the user clicks — this is the "do inference on demand" requirement, and it keeps idle cost at ~zero.

Pass 1 — concept mapping (on first visit to a video, or on pill click)

One request: system prompt + full transcript (with timestamps) → structured output.

• Model: claude-opus-4-8 (default; configurable in options — claude-haiku-4-5 is ~5× cheaper input / ~5× cheaper output and likely fine for extraction, but that's a user choice, not a silent default).
• Structured outputs via output_config.format (json_schema) so parsing never fails:

// schemas.ts (shape, not full schema)
{
  concepts: [{
    name: string,           // "New Journalism"
    timestampS: number,     // when it first becomes load-bearing in THIS video
    anchorQuote: string,    // short transcript quote, for context windowing later
    importance: 1 | 2 | 3,  // 3 = headline concept; controls how many bubbles show
  }]
}

• Cap displayed bubbles (importance-weighted, ~1 per 2–3 minutes) — "not too disruptive" is a product requirement, enforced here, not in the prompt alone.
• Result stored in Dexie; never recomputed unless the user forces a refresh.

Pass 2 — explanation (on bubble click)

• Model: claude-opus-4-8, adaptive thinking (thinking: { type: "adaptive" }), streaming (client.messages.stream) so the card fills in token-by-token over a Port.
• Prompt: system + transcript window (±90s of cues around the concept's timestamp) + concept name + library context (titles + concept lists of the user's other annotated videos, compact) → asks for: 2–3 sentence explanation grounded in what the speaker is using the concept for, plus an optional cross-reference { videoId, timestampS } chosen ONLY from the provided library list (no hallucinated references — schema constrains it to known IDs, and we validate against Dexie before rendering the row).
• Prompt caching: system prompt and transcript are stable per-video — mark the transcript block with cache_control: { type: "ephemeral" }. Clicking 3 bubbles on the same video = 1 cache write + 2 cheap reads (~0.1× input price). Order: static system → transcript (breakpoint) → per-click suffix.
• Explanations are cached in Dexie keyed by (videoId, conceptName) — a second click is instant and free.

Cross-video "first introduced" linking

• v1: done inside pass 2 as above — the model picks from the user's actual library index. Cheap, no extra infra, can't fabricate links.
• v2: embeddings for semantic concept matching across a large library (Anthropic has no embeddings endpoint — use Voyage AI or a small local model). Not needed until the library is big.

Cost envelope (opus 4.8: $5/M in, $25/M out)

A heavy session (5 videos, 20 clicks) ≈ $0.65. Fine for personal use on own key.

---

5. Data model (Dexie)

// db/index.ts
class OocDB extends Dexie {
  videos!: Table<Video, string>;          // pk: videoId
  concepts!: Table<Concept, number>;      // ++id, idx: videoId, name
  explanations!: Table<Explanation, number>; // ++id, idx: [videoId+conceptName]
}

interface Video {
  videoId: string; url: string; title: string; channel: string;
  durationS: number; thumbnailUrl: string;
  kind: "lecture" | "podcast" | "news" | "vlog" | "other";  // pass 1 classifies
  transcriptStatus: "ok" | "none";
  annotatedAt: number; lastOpenedAt: number;
}

interface Concept {
  id?: number; videoId: string; name: string;
  timestampS: number; anchorQuote: string; importance: 1 | 2 | 3;
  dismissed?: boolean;   // user can ✕ a bubble permanently
}

interface Explanation {
  id?: number; videoId: string; conceptName: string;
  text: string;
  ref?: { videoId: string; timestampS: number } | null;
  model: string; createdAt: number;
}

chrome.storage.local: { apiKey, model, bubbleDensity, enabled }.

---

6. Overlay UX states & motion spec

The user-facing requirement: annotations pop up, not too disruptive, click to expand, infer, show result — with framer-motion animations. States:

hidden ──(currentTime enters window)──► bubble ──click──► card:loading ──stream──► card:ready
                                          │  ✕ on bubble = dismissed (persisted)
                                          ▲────────────── card ✕ / outside click / Esc

• Bubbles auto-surface by time window: a bubble is "live" from timestampS − 5s until timestampS + 45s, then fades out unless hovered. Max 2 live bubbles at once (importance wins). The pill toggles the whole layer; while the layer is on, individual bubbles still appear/retire by time window.
• Bubbles pause their timers while the card is open and never obscure YT's own controls (anchor zone excludes bottom 80px of player).

Motion (all springs, no duration-tweens; motion/react)

Glass tokens (from the mockup): dark fill rgb(13,15,20) at 30–45% opacity, backdrop-filter: blur(16–32px), 1px inside stroke rgba(255,255,255,.22–.45), radius 14–16, soft drop shadow. One cyan accent #8ad8ff reserved exclusively for "annotation" meaning.

6.1 Observed details (from the checked-in mockups)

Reconciling the SPEC against the actual frames — these are binding where they conflict with prose above:

• Bubbles are circular, ~28px, thin white stroke on dark glass fill, centered number (1/2/3), no accent color on the bubble itself.
• Bubbles sit on a fixed right-edge rail, stacked vertically below the concepts pill (~16px right margin, ~44px vertical pitch). The mockup was updated to this rail from an earlier scattered placement — v1 deliberately does no visual anchoring to screen regions (see §10), so a predictable rail beats bubbles that pretend to point at things.
• Concepts pill: top-right, ● N concepts, cyan dot + white text on a glass pill. Toggles the bubble layer; bubbles fan out from it on enable.
• Annotation card (~300×198): anchored to the rail — right edge ~24px left of the bubbles, top below the pill (≈x=580, y=60 in the 948×535 frame), so it expands leftward from the clicked bubble rather than centering. Header row = concept title (left) + × (right); body = 2–4 line explanation; 1px divider; reference row = a play-button thumbnail (~56px) on the left, then a stacked label block (FIRST INTRODUCED in cyan, uppercase, letter-spaced ~11px + Title · MM:SS ~15px), and a › chevron at the far right. The thumbnail + chevron afford "jump to source" — the whole row is the click target.
• Home page (1280×832): top bar = logo (cyan dot + outofcontext) left, search input right (Search your annotations…); filter pills All / Lectures / Podcasts / News / Vlogs (All filled/active, rest ghost); RECENTLY ANNOTATED section label; 3-column card grid, ~378px cards, 24px gutters.
• Library card: 16:9 thumbnail with centered play glyph, bubble-count chip bottom-left (● N bubbles), duration chip bottom-right (MM:SS); below: title (one line, truncates) + meta line (Channel/Kind · N bubbles · relative-time).

---

7. Messaging contracts

// content/home → background (request/response)
type Req =
  | { kind: "conceptPass"; video: VideoInfo; transcript: TranscriptCue[] }
  | { kind: "getVideoState"; videoId: string }
  | { kind: "library.list"; filter?: VideoKind }
  | { kind: "concept.dismiss"; videoId: string; name: string };

// content → background (Port "explain", streaming)
{ kind: "explain"; videoId: string; conceptName: string }
// background → content over the same port:
{ kind: "delta"; text: string } | { kind: "ref"; ref: ExplRef | null } | { kind: "done" } | { kind: "error"; message: string }

Background dedupes in-flight pass-1 jobs per videoId (user refreshes mid-call) and retries via the SDK's built-in backoff only.

---

8. Milestones

• M0 — skeleton (a weekend): WXT scaffold, YT adapter (video info + mount + currentTime), static bubbles at hardcoded timestamps in shadow DOM, glass CSS, bubble→card layoutId morph. No LLM. Proves the whole UX feel.
• M1 — transcript + pass 1: caption fetch, background worker + SDK + options page for API key, concept pass with structured output, bubbles driven by real concepts, Dexie persistence.
• M2 — pass 2: streaming explanation over Port, card loading/stream states, explanation cache, prompt caching.
• M3 — home page: library grid, filters, search, dismissals, "open at timestamp".
• M4 — cross-video refs: library index in pass-2 prompt, validated reference row, jump-to-source navigation.

Each milestone is shippable to yourself; M0 deliberately front-loads the animation work since the morph interaction is the product.

---

9. Security & privacy

• API key only in chrome.storage.local, only read by the background worker. Never injected into page world.
• Only transcript text + video metadata leave the machine (to Anthropic). No page content, no watch history beyond what you explicitly annotate.
• host_permissions: *://*.youtube.com/*, https://api.anthropic.com/* — nothing broader.
• Shadow DOM + no unsafe-eval; WXT's default MV3 CSP is sufficient.

10. Risks