Screen Recorder — part 2 (projects, the data model and the agent surface)

What it is

The second half of the Screen Recorder: multi-asset Projects, the stored data model, importing a video you did not record, and the surface that lets an agent read frames from a recording. Part 1, screen-recorder.md, covers recording, the editor, the library and sharing.

Where to find it

Projects sits inside the Screen Recording sidebar area, beside the recording library. The import and agent-frame reads are command-line only — they have no button in the app at all.

How it behaves

Projects (multi-asset compositions — in development)

In-development feature — gated. Projects are a second composition mode (alongside Quick Capture) that lets you combine clips and snips from many captures into a single timeline. Enable via Settings → Labs → Screen Capture Projects (screenCaptureProjectsEnabled, AMC_SHOW_SCREEN_CAPTURE_PROJECTS env), hidden by default.

What a Project is

A Project is a standalone multi-asset composition — a single timeline doc whose clips reference media from ANY captures (not just one recording). Unlike a recording's built-in edit doc (where projectId = the recording id), a Project is independent: each clip's source.ref points at a capture id (video clip → that recording's MP4, image clip → that snip's PNG), and the referenced screen_recordings rows stay untouched. The references live inside the timeline doc itself — there is no separate asset table.

Library: Projects tab

When the feature is enabled, a Projects segment appears in the library's filter bar alongside All / Recordings / Snips (with a live count). The Projects segment shows project cards with a distinct visual identity (Layers icon + tint) so they read differently from recordings and snips at a glance:

• Create — a New project button (Projects segment only) opens a dialog to name + create; the new project opens straight into the editor.
• Cards — name (inline rename, same commit-on-Enter pattern as capture cards), relative creation time, delete behind a ConfirmDialog naming the project.
• Open — clicking a card loads the project in the editor (the same compositor surface as recording mode, working over foreign-ref clips).
• The recording-specific filters (sort / date / shared / edited) hide in this segment; name search still applies.

Editor and asset import

The Project Mode Panel in the editor shows:

• Add media — opens a picker listing every ready recording and snip; the pick lands as a new clip at the playhead on the top video track (source.ref = that capture's id; snips get a 5-second default window). The same capture can appear in any number of projects.
• Timeline / compositing — the same unified z-stack as recording mode: multi-track chips, keyframes, zoom/mute ranges, annotations. Preview slots resolve each distinct ref to its own media element (recording://<ref> for video, screenshot://<ref> for images).
• Autosave / save — composition edits persist via SC_PROJECT_WRITE_DOC to the project's sidecar (a fresh project seeds an empty 1080p/30fps doc); the referenced capture rows are never written.

Storage

One table backs the project system: screen_capture_compositions — id, name, export state (export_status, exported_path, exported_at), share_token, soft delete (is_deleted) + created_at / updated_at. The DB row is the manifest; the composition doc lives on disk as a sidecar at <userData>/screen-recordings/projects/<id>/timeline.json, in the SAME v1|v2 wire format as a recording's edit doc (one editor, one serializer). Reopening rebuilds the scene from the sidecar without touching any capture rows.

Naming note: the table is deliberately screen_capture_compositions, not screen_capture_projects — a dormant, unrelated table of that name (plus a screen_capture_project_assets junction) exists from an older abandoned migration with zero code using it. Do not confuse the two.

Export and share

Single-source constraint (current): export works only when the composition reduces to a SINGLE video source (no multi-clip stacking, no multi-asset bake). Image-only projects and multi-source compositions return a clear "coming soon" message. The follow-up rework (baking multi-source compositions) is deferred.

When a single-source project is exported:

• Export calls SC_PROJECT_EXPORT with options (resolution, codec, bitrate, framerate). The offscreen export host composites + bakes the timeline, writing <userData>/screen-recordings/projects/<id>/final/export.<ext> and recording the lifecycle on the composition row (stranded 'exporting' rows reconcile at boot).
• Progress / completion reuse the RECORDING_EXPORT_* broadcasts (keyed by project id) and UPLOAD_PROGRESS push channel — no new push channels.
• Share calls SC_PROJECT_SHARE, uploads the baked MP4 to the Omniscio Shares Firebase project, and mints a streamed link (https://shares.omniscio.com/s/<shareToken>/run). The project's share token + expiry follow the recording share model (1 day / 7 days / 30 days / Never).
• Re-share after editing refreshes the SAME link in place: the share flow bakes if the doc is newer than the last export, overwrites the Storage objects under the existing token (content-hash drift detection), and the previously sent URL starts serving the edit.

IPC channels

Project operations live under the screen-capture-project:* namespace:

• Lifecycle: create, list, get, rename, delete
• Composition doc: read-doc, write-doc (same sidecar format as recordings)
• Export / share: export, cancel-export, share
• Push events: changed (any project CRUD — library refetches)

All channels validate via Zod and return the uniform { success } | { success, error } envelope. Gated by the screen-capture-projects unreleased feature — calls rejected if the feature is not visible.

Data model

The screen_recordings table holds one row per capture (recordings and snips):

• id, share_token, custom_slug (nullable)
• type — video (recording) | image (snip)
• status — recording | paused | transcoding | uploading | ready | published | discarded | recovered | error
• started_at, ended_at, duration_ms, file_size_bytes
• source_type (screen | window | app | webcam), source_label, monitor_id, source_count
• mic_device_id, cam_device_id, quality, frame_rate, webcam_shape
• recording_version (1 = single-source, 2 = multi-source/editor), cursor_tracking_enabled
• local_mp4_path (the PNG path for type='image' rows), temp_segments_dir, thumbnail_path, trimmed_path
• has_annotations, annotations_path, export_status, exported_path
• ai_title — the AI title, auto-generated after a recording finishes when Auto-titling is on (see Auto-titling); plus ai_tldr / ai_chapters_json from the manual Summarize action
• vimeo_url, view_count, expiry_at, folder_id, revoked_at
• watermark_text, watermark_image_path, watermark_position
• Soft delete: is_deleted, plus created_at / updated_at

Folders + tags live in adjacent screen_recording_folders / screen_recording_tags tables. Editor edit-models live on disk as sidecars under <id>/, not in SQL.

Importing a video the user did not record (CLI)

POST /capture/import brings an EXTERNAL video into the recordings library, so a walkthrough someone SENT the user — a Loom of a QA session, a bug repro — can be reviewed with an agent exactly like a capture they made. Before it existed the library's only front door was pressing record, even though every route above already works on any video file.

Body carries EXACTLY ONE source (both, or neither, is a 400):

• { "url": "https://www.loom.com/share/<id>" } — any public link the downloader handles (Loom, YouTube, Vimeo, a direct media URL).
• { "filePath": "C:/path/to/clip.mp4" } — a video already on this machine. It is COPIED into the library; the original is never moved or altered.
• { "transcribe": false } (optional) skips the local Whisper pass.

It resolves once the recording is ready: { id, title, durationMs, transcript, source }. transcript is processing | skipped | no-audio — it never claims a FINISHED transcript, because the Whisper job runs detached and takes minutes. Poll GET /capture/:id for the text.

cliTokenOnly, like its by-id library-mutation siblings: it writes into the recordings store and can reach the network, so a spawned session's scoped $AMC_CLI_TOKEN is refused (401). Use the GLOBAL ~/.amc/cli-token.

Two behaviours worth knowing before you build on it:

• A file with no video stream is refused, whatever it is named. That probe is the control that keeps filePath from being an arbitrary-file ingest, so expect a 400 on an mp3, a PDF renamed .mp4, or a private key.
• An imported recording tells the reviewing agent it was imported. Send one with Record for Agent and the opening message says someone ELSE recorded it and heads the transcript What they said:, instead of the first-person wording a real capture gets. Do not write copy that assumes the user is the speaker.

Agent frame access (CLI)

An agent working from a recording can pull its own stills, so it never has to describe a frame in prose:

• POST /capture/:id/frame — mint the frame at a given time into a session's attachment store, returning a ready-to-paste ![label](attachment://<session>/<file>) line.
• POST /capture/:id/crop — the same, cropped to a region and marked, for when the agent needs a different area or instant than the one it was handed.

Both are cliTokenOnly: authenticate with the GLOBAL ~/.amc/cli-token, never a spawned session's scoped $AMC_CLI_TOKEN (a flat 401 there). A recording has no owning session — the library is keyed by id alone — so a leaked session-scoped token could pull frames out of every recording the user has made. Both also require the Agent Screen Capture setting (agentScreenCaptureEnabled), returning 403 { disabled: true } when it is off.

Both go through the SAME frame-ops code path the pre-minted screenshots come from, so an agent-requested crop and a supplied one are indistinguishable at the point of use. Both write a capture_access_log row attributing the calling session (I14) and both consult the sensitive-window deny-list (I13). Scratch frames are deleted once their bytes are in the attachment store — minting copies rather than moves, so nothing else would ever clean them up (I20).

For agents

IPC channels

All channels live under the screen-recorder:* namespace in src/shared/ipc-channels/screen.ts; every handler validates via Zod through wrapHandler and returns the uniform { success } | { success, error } envelope:

• Lifecycle: get-state, quick-start, start (accepts a multi-source layout), stop, pause, resume, discard, discard-and-restart, toggle-mic, toggle-cam, panic-mute
• Source / device discovery: list-sources, list-audio-devices, list-video-devices
• Library: list-recordings, get-recording, update-recording, delete-recording
• Sharing: share, get-share-info, publish-to-vimeo, reveal-recording (publish is deprecated)
• Folders / tags: list-folders, create-folder, update-folder, delete-folder, list-tags
• Crash recovery: list-recovered, recover, discard-recovered
• Editor sidecars (all wired): write-/read-layout, write-/read-bookmarks, write-/read-click-events, write-/read-annotations, write-/read-cursor, write-/read-trim, write-/read-timeline, write-/read-webcam-position, plus write-trimmed
• Edited export: export, cancel-export (+ the offscreen export-host internal protocol)
• Push events: state-changed, warning, library-changed, transcode-progress, upload-progress, plus the export progress/complete/failed broadcasts

Host-renderer internal channels (host-ready, host-write-chunk, host-begin-capture, etc.) and the HUD heartbeat are user-invisible — they wire the hidden capture/export renderers to main.

Related

Part 1, screen-recorder.md, is the page to read first — it covers recording, the editor and sharing. screenshot-snip-tool.md is the image sibling that shares the same library and share surfaces.