← Docs

Scene Fetch Protocol (v1)

A small, git-shaped protocol for syncing a Daslab scene to a client. One wire format, two transports (HTTP pull + SSE push), one merge function.

If you can run git fetch, you can already think in this protocol: head is the cursor, since is where you were, complete: true is "fresh clone, here's everything."

Endpoints

GET /api/scenes/:id/fetch                       — full snapshot
GET /api/scenes/:id/fetch?since=<head>          — diff from <head> to current head
GET /api/scenes/:id/fetch/stream                — SSE: full snapshot + live pushes
GET /api/scenes/:id/fetch/stream?since=<head>   — SSE: catch-up diff + live pushes

All four return the same JSON shape (SSE wraps each push in an event: message frame). A third-party can prototype against the pull endpoint and swap to SSE without touching the decoder or the merge function.

Authentication: Authorization: Bearer <jwt> header. For SSE in browsers (where EventSource can't set custom headers) pass ?token=<jwt> as a query param instead.

Response shape

{
  "v": 1,
  "head":     "a8058ed876e9",
  "since":    "4fa2b99c80cd",
  "complete": false,

  "assets": {
    "changed": [ /* full Asset objects, the new state */ ],
    "removed": [ "ast_xyz", "ast_def" ]
  },

  "scene":    { /* always-latest scene metadata */ },
  "members":  [ /* always-latest members */ ],
  "jobs":     [ /* always-latest recent jobs, capped at 15 */ ],
  "children": [ /* root scenes only — non-archived direct children */ ],

  "assetGroupOrder": [ "ast_id1", "ast_id2", … ],
  "jobCounts":       { "open": 3, "done": 12, "archived": 1, "total": 16 }
}

Field	Meaning
`v`	Schema version. `1` today. New versions are additive whenever possible.
`head`	Server's current cursor. Advance your local `head` to this after merging.
`since`	Echoed back from your request. `null` on a fresh fetch.
`complete`	`true` means this is a full snapshot (cold client OR `since` not reachable).
`assets`	The only sub-tree with diff semantics. See below.
`scene`	Scene metadata (name, settings, etc.). Always replace your local copy.
`members`	Always replace.
`jobs`	Always replace (recent 15).
`children`	Always replace (root scenes only).
`assetGroupOrder`	Display order for the asset list. Always replace.
`jobCounts`	Open/done/archived/total job tallies for the scene. Always replace.

For streaming pushes _after_ the initial sync, scene / members / jobs / children may be omitted — they're "always-latest scalars" and your last-known copy is still current. Re-fetch via the pull endpoint if you need a forced refresh.

Three cases, one shape

1. Already at head. assets.changed and assets.removed are both empty.

{ "v": 1, "head": "abc", "since": "abc", "complete": false,
  "assets": { "changed": [], "removed": [] } }

Client noops. Advances cursor (it didn't change).

2. Diff available. Server walked the commit DAG and found your since as an ancestor of head. You get just what changed.

{ "v": 1, "head": "def", "since": "abc", "complete": false,
  "assets": { "changed": [ ... ], "removed": [ "ast_old" ] }, ... }

3. Full snapshot. Either you sent no since (fresh client) or your since isn't reachable (commit DAG was pruned, or never existed). assets.changed holds every current asset.

{ "v": 1, "head": "def", "since": null, "complete": true,
  "assets": { "changed": [ /* all of them */ ], "removed": [] }, ... }

When complete: true, clear your local assets first, then apply.

Merge contract

A correct client is this function:

function applyFetch(state, resp) {
  if (resp.complete) state.assets = new Map();

  for (const asset of resp.assets.changed) state.assets.set(asset.id, asset);
  for (const id    of resp.assets.removed) state.assets.delete(id);

  if (resp.scene    !== undefined) state.scene    = resp.scene;
  if (resp.members  !== undefined) state.members  = resp.members;
  if (resp.jobs     !== undefined) state.jobs     = resp.jobs;
  if (resp.children !== undefined) state.children = resp.children;

  state.head = resp.head;
}

Six lines plus a cursor bump. Same function used for pull and push.

Worked example

A brand-new client:

// 1. Fresh fetch.
const r1 = await fetch(`/api/scenes/${id}/fetch`, { headers: auth });
const p1 = await r1.json();
applyFetch(state, p1);
// state.head = p1.head = "abc"

// 2. Time passes. User flips back to this scene. Refresh:
const r2 = await fetch(`/api/scenes/${id}/fetch?since=${state.head}`, { headers: auth });
applyFetch(state, await r2.json());
// Either no-op (still "abc") or tiny delta. UI doesn't flicker.

// 3. Subscribe for live updates:
const es = new EventSource(`/api/scenes/${id}/fetch/stream?since=${state.head}&token=${tok}`);
es.onmessage = (m) => applyFetch(state, JSON.parse(m.data));
// Initial event: catch-up diff. Subsequent: live commits as they happen.

That's the whole client.

Why this shape

Five things informed the design:

One decoder. No "if assetsDelta present else assets" branch. No discriminator union with three variants. One struct, every field has a fixed type.
One merge function. Pull and push converge. The catch-up event from SSE on reconnect goes through the same code path as the manual refresh.
complete: true collapses the cold-start path. No special "first request" endpoint. No bootstrap. The same shape handles a brand-new client and a client missing one commit.
Git vocabulary. head, since, complete are words anyone shipping a sync API has already used. Nothing to learn.
v: 1 is the schema escape hatch. Adding v: 2 later doesn't break old clients — they keep asking for v: 1 (or read the v field and refuse). Removing fields takes a version bump.

Versioning & extension

Additive changes (new fields, new optional sub-trees) ship inside v: 1. Removing or changing the meaning of an existing field bumps to v: 2. New v values are advertised in capability discovery (GET /api/scenes/:id/capabilities, not yet shipped) so clients can negotiate.

Two extensions on the roadmap, both opt-in:

?patch=true — assets.changed[] entries may be { id, patch: [ ... ] } (RFC 6902 JSON Patch) instead of full Asset objects. Server falls back to full when the patch would be larger.
?include=scene,assets,jobs — narrow the always-latest scalars the client receives. Useful for "metadata-only refresh" or "assets-only diff during typing".

Both are off by default. The base protocol stays the six-line merge.

Errors & edge cases

Unauthenticated: 401. SSE: connection closes immediately.
Not a member of the scene: 403.
Scene doesn't exist: 404.
since is not a valid hash: server treats as unreachable → returns full snapshot (complete: true). Not an error.
SSE reconnect: pass your current head as ?since=; server sends a catch-up immediately, then resumes live pushes.
Server pruned old commits: same as unreachable since. Client gets a full snapshot once; subsequent fetches are diffs again.

Legacy endpoints

Two earlier endpoints still respond for backwards compatibility while iOS migrates to this protocol:

GET /api/scenes/:id/full[?since=<head>]   — old envelope shape
GET /api/scenes/:id/stream                — old SSE event shape

These will be removed after both clients ship on /fetch. New integrations should not use them.