Brain device sync protocol

Developer reference for the desktop endpoint used by HivemindOS Mobile. This is file replication, not memory extraction: Markdown notes, skill folders, and binary attachments keep their exact bytes. The server implementation is src/lib/services/obsidian/mobile-brain-sync.ts.

Basics

• Every request is POST /api/brain/sync with the dashboard session or paired-device authentication. Runtime-agent principals are rejected.
• The desktop uses its currently selected workspace vault. Requests cannot supply a vault path.
• Clients must stay on the original desktop connection and never fail over mutations to another machine.
• Paths are vault-relative, revisions are lowercase SHA-256 hex, and times are milliseconds since the Unix epoch.

Manifest

{"action":"manifest","limit":200}

Returns {ok:true,protocol:1,vaultId,entries,nextCursor?,complete,truncated,reasons,limits}. Each entry is {path,revision,size,modifiedAt,deleted?}. A deleted entry keeps its previous revision but has no bytes; recreate it with expectedRevision:null.

Paging. Pass nextCursor back as cursor. Pages are bounded, immutable, replayable, and survive server restarts. A cursor expires after ten minutes without progress, and the server keeps up to eight recent scans. An expired cursor returns 409 with code:"snapshot-expired": discard the partial traversal and start again. Files seen only on earlier pages are not assumed deleted.

Absence. Only a final page with complete:true proves a full, stable traversal. A final complete:false,truncated:true page still has usable entries and explicit tombstones, but the client must not infer deletion from paths it did not receive. reasons may include files-changed, file-too-large, directory-limit, unreadable-file, or unreadable-folder. A file that changed mid-scan is checked again next time.

The server keeps bounded history for detecting external deletions and may drop older entries, so clients keep their own per-desktop baseline and use a completed manifest to decide absence.

Read

{"action":"read","vaultId":"<returned-id>","files":[{"path":"Notes/Example.md","revision":"<sha256>"}]}

Returns {ok:true,vaultId,results}. Each result has path and a status of ok, not-found, conflict, too-large, or invalid. An ok result includes revision, size, modifiedAt, and contentBase64. A revision mismatch returns currentRevision; refresh before using the newer bytes as a baseline. Any file extension is allowed.

Apply

{"action":"apply","vaultId":"<returned-id>","changes":[{"path":"Notes/Example.md","expectedRevision":null,"contentBase64":"SGVsbG8K"}]}

To delete, omit contentBase64 and set deleted:true. Returns {ok:true,vaultId,results} with status of applied, unchanged, conflict, or invalid, plus optional revision, deleted, currentRevision, or conflictPath.

• Malformed batches are rejected before any write. Valid batches apply one file at a time; a later failure does not roll back earlier results.
• expectedRevision:null creates a file only if the path is absent (including after a tombstone). Updates and deletes need the current revision.
• Writing identical bytes succeeds idempotently, even if an earlier acknowledgment was lost.
• A conflicting update keeps the current file and saves the incoming bytes to a deterministic Sync Conflicts/ file. A conflicting delete keeps the current file. Clients must likewise preserve unsynced local bytes when accepting a desktop change.

Crash safety. Updates and deletes first move the old file into .trash/HivemindOS Sync/, with a durable preparation record for recovery. New content is written to a temporary file and installed with exclusive creation, so a competing editor’s file is never overwritten. Recovery restores an archived file only if its original path is still empty. Sync shares the existing cross-process lock, and a lock held by a dead process is recovered after its stale-lock interval.

Search freshness. Each committed change records a durable search invalidation. The canonical memory and full-vault search readers refresh their generated indexes before answering, and typed-memory reads rebuild under the memory write lock. Query and skill caches include the durable version, so no process serves stale results after a change. Failed invalidations stay in the sync journal and retry before the next operation. A burst of uploads therefore costs one refresh at read time, not a full rebuild per file.

Limits and exclusions

• 8 MiB per file, 12 MiB of decoded content per batch, 16 items per batch.
• Up to 500 manifest entries, 256 MiB of hashing, and 60,000 inspected directory entries per page. A single directory over the inspection limit makes that traversal partial.
• Symlinked files and folders are skipped. Traversal paths, hidden paths, path components longer than 240 characters, and trailing dots or spaces are rejected.
• Hidden configuration, build output, generated Brain indexes and replay generations, and runtime mirrors are not replicated. Ordinary Markdown under Operations/Brain Services/ is.
• Sync identities, scan pages, hashes, preparation records, and receipts live in private app state outside the vault. Neither they nor the recovery trash are replicated.
• Sync Conflicts/ and the recovery trash are created only when needed. No setup daemon or external sync service is installed.

Testing

Run pnpm test:brain-device-sync. It uses temporary vaults and the real route handler to cover conflicts, binary files, paging, external changes, symlink rejection, crash recovery, and the desktop’s memory, search, and skill readers after synced creates, edits, and deletes. It never touches your configured vault.