Design

How @nyte-ai/core is put together: content-addressed objects, atomic refs, fenced leases, and one ordered event stream

This is the design of @nyte-ai/core. It says what the runtime is, what shape each part has, and what a host or client can count on. When this page and any other document disagree about a contract, this page wins. must and must not are the load-bearing words. Whether a thing is built is a question for the tree and its tests, not for this page. What follows describes the kernel that exists today, checked against packages/core/src/kernel as of this writing.

The yardstick for every section is the same. What is the smallest set of mechanisms that keeps the promises below? If a feature can be a projection, it is a projection. If it can be a ref, it is a ref. If an operation can be one compare-and-swap, it is one compare-and-swap. I have tried to be ruthless about this, because every extra mechanism is another thing that can disagree with the others at 3am.

What core is

Nyte core is a durable runtime for agent conversations. Three promises drive everything else:

  • It runs anywhere. The same kernel runs inside an Electron main process, under a TUI, on a VM, and in request-scoped infrastructure. A runner reads all durable state at the start of each step. A long-lived process is an optimization, never a requirement.
  • Any number of participants may write at once. A phone, a web page, a desktop app, and a cloud job may all submit to one session. Participant writes never take the runner's lease. They linearize through compare-and-swap and retry the conflicts they are expected to meet.
  • Accepted input is not lost and published history is not rewritten. Objects are immutable. Refs say which objects are current. A failed publish leaves a loose object, not a half-applied transaction. Retention may collect an object only once no ref or retained ref event protects it and its grace period has passed.

Four scenes are the acceptance tests for the whole design. If a change to the kernel breaks one of these, the change is wrong, whatever else it improves.

  1. Three phones submit at the same moment into one lane of an idle head. Every submission succeeds. The tip retries form one change chain, the oldest change lands first, and the rest stay pending until later response boundaries. Nobody gets a contention error and nothing is lost.
  2. A run is killed after a tool intent but before a result. Another process acquires the expired lease. It runs a safe effect again, or settles a never effect as interrupted, then carries on from refs. The client watching sees a pause and nothing else.
  3. A tool parks in waiting. The asking host exits. An hour later a phone signals the effect. The first signal compare-and-swap wins, and whichever runner next acquires the head wakes and settles the call.
  4. A serverless handler submits a change, runs one durable step, streams events, and exits. The next request or alarm reads the run ref and takes the next step. No process-local run state is needed.

Facts we accept

Each mechanism on this page answers one fact I cannot change. That is what keeps the mechanism count down. One per fact, and a new mechanism needs a new fact behind it.

FactMechanismSection
The process can die at any moment, or never existedEvery uncertain tool call moves an effect ref from intent to resultRuns
Any process may be the one that continues a runA runner keeps nothing in memory across a stepMany runners
Writers race constantlyMulti-ref CAS, internal submission retry, and fenced runner publishesAdmission
Providers cache the context prefixBoundary lanes land at response boundaries; idle lanes wait for an idle headAdmission
The outside world will not join a database transactionPer-effect replay policy instead of impossible exactly-once executionRuns
A commit id is unknown while its message is streamingDeltas key on run, attempt, and content indexEvents and views
Clients may return after retained events have expiredThe event floor rejects the cursor and the client takes a snapshotEvents and views
A network client can fail between Enter and a receiptA local outbox retries one idempotency keyAdmission
Users change their mindsHistory is a tree and a head is a movable refHeads
A participant may never answer a parked callAn optional deadline lives on the waiting effect and wakes through the stepRuns

Roles

Six roles. One process can play several at once. The TUI plays all of them. A phone plays only client.

RoleIs
storeThe four durable authorities: objects, refs, leases, and events. A session is one repository.
runnerCode that holds a fenced lease over one head and advances its run one durable step at a time.
hostA process that opens the store, supplies models and plugins, and may volunteer runners.
clientCode that calls SDK operations, renders snapshots and events, and keeps a cursor plus a local submission outbox.
pluginHost-side code that contributes model context, tools, commands, settings, resources, and policy.
schemaThe shared message and JSON types used at the object boundary, by the agent loop, and by clients.

Protocol adds the shapes a client draws without knowing who produced them: Choice and Selection today, beside the setting, notification, and status shapes. They are data, never calls.

The model, in git terms

Core is git's object database with messages in place of files. If you know git, this table is the reading guide for the rest of the page. If you don't, it still tells you which ideas were borrowed and which were changed on purpose.

gitcore
object databaseobjects: immutable values addressed by the SHA-256 hash of canonical JSON
commit with parentconversation commit with exactly one context parent
extra parent or provenanceimports: provenance that never joins model context
branch refrefs/heads/<head>
HEADthe head name supplied by the client, never a stored checkout register
detached readreading at any commit without moving or checking out a head
update-ref compare-and-swapone atomic refs.update over any number of refs
lock filea fenced lease over refs/heads/<head>
reflogretained ref events in the session event stream
indexone pending change chain per lane, behind queue tip and base refs
stack metadatarefs/stacks/<head> pointing to { parent, base }
rebasenever replay generated text; the model writes a summary commit for carry-forward
worktreenone; a client reads any commit and a runner reads refs at every step
garbage collectionmark from refs and retained ref events, wait through a grace period, then sweep

The stricter rule, and the one people push back on first, is the one-parent rule. A commit has one parent, or null at a root. That chain alone becomes model context. imports may name commits that were summarized or carried across a branch, but imports are provenance only. They must not add a second context path. Git merges are fine for files because a three-way merge of text is a well-understood operation. There is no such operation for two conversations.

There is no worktree and no checkout. A client may read any commit. A runner advances one head without materializing files or holding conversation state outside the store.

Generated text must not be mechanically replayed onto a new base. Instead the object model has a summary commit body for a path carried forward by meaning. heads.move({ summary }) asks the model to summarize the abandoned path and writes that commit with the navigation target as its parent. A model failure returns failed and leaves the head where it was.

The reflog is retained, not eternal. The store exposes a floor. Events at or below it may be removed, and unreachable objects lose the protection those old ref events gave them. A grace period still protects loose objects and recently unreachable history before collection.

The four authorities

A session has four authorities. Everything else is a helper or a projection over them.

Objects

Objects are content-addressed and immutable. hashObject is a SHA-256 hex digest over the object's canonical JSON. Writing the same value twice returns the same oid and rewrites nothing. Any field change produces a different oid.

There are six object kinds:

ObjectMeaning
commitOne point in a conversation, with one parent, optional imports, provenance, and a body.
changeA commit body waiting for a parent, linked to the prior change in the same lane.
runOne head's current run, including its phase, attempts, folded configuration, and optional abort request.
effectOne tool call's durable state: intent, waiting, expired, signal, or result.
stackThe parent head and base commit for a stacked head.
blobA small JSON value behind a fact, cancellation marker, or deletion marker.

A commit's provenance is two optional oids. change names the submitted change it landed, and run names the run that wrote it. Neither is context. They exist so a client can ask which files one run touched, or which commit a receipt turned into, without guessing from timestamps.

A commit body is one of five kinds:

Commit bodyMeaning
messageOne schema Message: user, assistant, or tool result.
checkpointA context checkpoint with a summary, retained tail, token count, optional provider material, and optional usage.
summaryWhat a path being carried forward was about. It does not cut context.
configA model, thinking level, or agent declaration. The latest value of each field wins.
noteProduct-defined JSON, replayed by core and unseen by the model.

Objects must be written before the ref update that makes them reachable. A failed update may leave loose objects. That is safe, because no ref names them, and retention collects them after the grace period. I would rather have a few orphaned rows than a ref pointing at nothing.

One consequence takes people a moment. A commit oid cannot exist while an assistant message is still streaming, because the message is part of the bytes that get hashed. So streaming has its own identity. Provisional deltas name (runId, attempt, index), and the finished commit supplies its oid at settlement.

Refs

Refs are the only mutable domain state. They move only through one multi-ref compare-and-swap. Every update names the expected oid in from and the desired oid or absence in to. The store checks every expectation before writing any ref. All writes and their ref events commit together, or none do.

An update with to === from is an assertion. The store checks it, but writes no row and emits no event. Runner publishes use this to assert that refs/deleted is absent. Cancellation and redelivery use assertions to close races with landing.

The ref namespace is:

refs/heads/<head>              branch tip; absence means unborn
refs/stacks/<head>             Stack { parent, base }
refs/queues/<head>/<lane>/tip  newest submitted Change in the lane
refs/queues/<head>/<lane>/base last landed Change; pending is (base, tip]
refs/runs/<head>               current Run object and phase
refs/effects/<run>/<call>      Effect: intent -> waiting -> signal/expired -> result
refs/keys/<key>                idempotency receipt naming the Change
refs/facts/<key>               Blob holding a small session value
refs/cancelled/<change>        Blob tombstone for a withdrawn Change
refs/deleted                   Blob marking a session being deleted

Heads and lanes are names, and the kernel privileges none of them. Which lane lands when is the runner's landing policy, passed to every step. The default head (main) and the default lanes (steer, queue) belong to the SDK, which lists its default head before the head has a ref.

Leases

A lease is fenced execution authority over one name. Runners acquire the head ref name. A lease has name, owner, fence, and expiresAt. Leases are liveness, not history. They never enter the object graph or the reflog.

Acquire inserts a lease when none exists. After expiry, takeover changes the owner and increments the fence. Losing an acquire returns the current holder as a value, not an exception. Renew and release match both owner and fence, so an old process cannot extend or release its successor's lease. Renew ignores expiry for the matching holder. Expiry permits takeover. It is not an execution budget, and hosts should not treat it as one.

Every runner ref update and every runner event append must carry the lease. The store checks the stored owner and fence inside the transaction. A mismatch returns fenced before any ref or event changes.

Events

Events are both the reflog and the live feed. One increasing seq orders every event in a session, starting at 1. Four raw event bodies exist:

EventMeaning
refOne written ref move with name, from, to, reason, and optional actor.
deltaOne text or thinking fragment keyed by run id, attempt, and content index.
progressOne tool call's partial output keyed by run id and call id.
noticeAn informational, warning, or error notice owned by a component.

refs.update appends one ref event per actual ref write, followed by any extra events, in the same transaction. Assertions emit nothing. events.append adds deltas, progress, or notices without moving a ref, and may be fenced by a lease.

read and watch yield events after a cursor in sequence order. last returns the newest sequence, and floor returns the oldest valid cursor boundary. Trimming removes events at or below a chosen sequence and raises the floor, never past the newest event. Reading below the floor throws CursorExpired.

Heads are named refs

A head is refs/heads/<head> pointing to a commit oid, or absent for an unborn branch. The head name supplied to an operation defaults to main. It is client state, not a stored current-checkout register. Two clients may use different heads in one session and neither has to know.

Moves and live runs

A participant moves a head by compare-and-swap from the oid it read. It may also supply expect to reject a view that was already stale. A non-null target must name an existing commit. At the kernel level the outcome is moved, moved_since, or not_found.

The SDK's heads.move applies navigationTarget first, so selecting a user message restores it to the composer and targets its parent. With summary, the abandoned commits are summarized into one commit whose parent is that target, and one head CAS moves from the expected old tip to the summary. Nothing abandoned, or an empty summary, falls back to the plain move. Generated text is never replayed. A summarization failure returns failed without moving the head.

The low-level move does not take the runner's lease or wait for a live run. The SDK checks first and returns busy when one is live. A run that races the check still publishes with its expected head, expected run ref, lease, and an assertion that deletion has not started. If the participant moved the head in between, that publish fails. The runner re-reads and moves the same run to aborted in an update whose event reason is superseded. It leaves the participant's head alone and consumes no queued input. Any output objects prepared for the failed publish stay loose until retention collects them.

Every actual head ref write emits a retained ref event with its previous and next oid, reason, and optional actor. Undo is another move, and it works for as long as the needed ref event and object are still retained. The kernel must not promise undo beyond the configured event and object retention window.

Branching

Creating a head from another head copies its current tip and writes a stack object naming the parent head and that tip as the base. A parent is a name, so cutting from an unborn parent records no base. Creating from a commit writes only the head ref and no stack metadata. An unborn head therefore needs a stack parent. Moving an unborn head to a commit is what creates it. The SDK cuts from and moves only heads the session lists, or its default head, so a mistyped name creates nothing and answers unknown_parent or not_found.

Deleting a head removes its head ref, stack, every lane's tip and base refs, and run ref in one CAS. It refuses with busy while a live lease exists for that head. The kernel deletes any head. The SDK refuses its default head. The objects stay until retention proves them unreachable and old enough.

There is no stored HEAD and no forked worktree. Reading an older commit is a projection. Continuing from it means moving a head or creating a head there.

Stacks

refs/stacks/<head> points to a Stack { parent, base } object. A stacked head is stale exactly when base !== parent.tip. That one equality check is the whole staleness model.

The stack operations are deliberately narrow:

  • advanceBase returns unchanged when there is no stack or the base already matches. If the old base is still an ancestor of the parent's tip, it updates only the stack base and returns advanced. If the parent diverged, it returns diverged.
  • fastForward, exposed by the SDK as heads.merge, requires a current stack and a non-empty child. One CAS moves the parent head to the child tip and advances the child's stack base to the same oid. Its outcomes are merged, stale, empty, and no_stack.

Carry-forward without replay goes through heads.move({ summary }), which uses summary commits and imports. The stack code offers no general join of two heads, and I am not in a hurry to add one.

The turn

The kernel turn has two calls and one commit boundary between them:

interface Turn {
  respond(input: TurnInput): Promise<RespondOutcome>;
  tools(input: TurnInput & { assistant: AssistantMessage }): Promise<ToolBatchOutcome>;
}

respond receives the session, lease, run, attempt number, the commits since the newest checkpoint, an event emitter, and an abort signal. It projects model context and streams one assistant message. Its outcomes are checkpoint, complete, tools, retry, failed, and aborted. All but checkpoint carry the complete assistant message the step will commit. checkpoint carries a checkpoint body instead. The step commits it and calls respond again, so compaction is a commit like any other and never a special phase.

tools receives the same durable inputs plus the committed assistant message. It recovers or executes each tool call through its effect ref. Its outcomes are complete, waiting, and failed. Complete and failed outcomes carry tool-result messages in call order. Waiting names the parked call ids.

The turn emits text and thinking deltas with the supplied attempt number. Tool progress names the run and call. Emission is buffered through a fenced outbox and flushed before the call returns, so a fenced event append aborts the in-flight work rather than letting it finish into the void.

The tool batch

Tool calls use the existing agent loop. A length stop returns an error result for every tool call in the message, because truncated arguments cannot be trusted. Tool failures settle a structured error result, and the last partial progress is available to that settlement. Results come back in the assistant message's call order.

No turn state survives the call. On a resumed tools phase, the turn reads the assistant commit and every effect ref again and starts from what it finds.

Runs

A run is one head being advanced. refs/runs/<head> points to the current immutable Run object. Each phase change writes a new run object and moves that ref. The run id stays stable across all of them.

A run records id, head, phase, startedAt, attempts, and the branch configuration folded when the run began. The SDK runner resolves its model and thinking defaults into that configuration before publishing the run. Before responding, it also resolves legacy runs and records any fallback required when a successor host cannot supply the recorded model. Observers read these inputs from the run; their own host defaults say nothing about another host's execution. An idle head's snapshot combines declared configuration with the last recorded or observed response inputs. Legacy history can reveal a model without revealing its thinking level. A participant may set abortRequested. The phases are:

PhaseMeaning
respondThe next step may land a boundary-lane change, then ask the model for an answer.
toolsThe assistant message has tool calls to recover or execute.
waitingAt least one effect is parked until a signal, deadline, or abort arrives.
retryA provider attempt failed transiently and may resume at the stored time.
doneThe run completed.
abortedThe participant, host signal, deletion, or superseding head move stopped the run.
failedThe run ended with the stored error.

Leases and the step

step(session, turn, options) acquires a lease over the head unless the caller supplies one. It does one durable action and releases a lease it acquired itself. drive acquires once, renews before every step, loops over continue, waits through a retry deadline, and releases when it returns.

The step reads the head, run, and deletion refs. A session being deleted makes every step idle. Otherwise the step follows this table:

Run phasePending changeStep doesPublish CAS
none or terminalnonereturn idlenone
none or terminalsomeland from the first policy lane; start in respond with a message, otherwise donehead, queue base, run
respondboundary lane pendingland it before the next response, subject to the drain rule belowhead, lane base
respondnothing landablecall turn.respond over projected branch contextassistant commit at head, next run phase
toolsanycall turn.tools, commit settled results, or parkresult commits, effect cleanup, next run phase
waitinganywake on a signal, deadline, or abort; otherwise return waiting { until? }the same publish as tools when it wakes
retryanyreturn retry before at; after at, act as respondnone until responding

Two small rules sit inside the respond row. First, an abort request already on the run stores aborted without calling the model. Second, if the projected context ends in something other than a user message or a tool result, there is nothing to answer, and the run goes straight to done.

Every runner publish carries the lease and asserts refs/deleted absent. A participant head move, an abort request written through the run ref, another run ref change, or session deletion may make the CAS fail. The runner re-reads and ends or continues according to current refs. It must not force stale output.

A step ceiling is checked before turn.respond. The ceiling may be a fixed number or a function of the run, which is how an agent declares its own limit. Reaching it stores a failed run. Pending input is never consumed by failure, abort, or supersession. A later step starts a new run and lands it.

The effect sandwich

Each tool call owns refs/effects/<run>/<call>. The ref moves through immutable effect objects:

                   +-> signal --+
intent -> waiting -|            +-> result
   |               +-> expired -+
   +------------------------------>

The runner writes intent before executing the tool. The intent stores the run id, call id, tool, JSON arguments, replay policy, and time. Recovery depends only on the ref state:

StateRecovery
intentexecute again for safe; settle an interrupted error for never
waitingremain blocked unless the run is aborted or its deadline passes
expiredcall the tool's wake handler with expired: true
signalcall the tool's wake handler with the first reply
resultreuse the stored tool-result message and never execute again

A tool that declares no replay policy gets never. That is the conservative default on purpose. A tool has to opt into being run twice, because the kernel cannot know whether "send the email" is idempotent.

A tool parks by moving the effect ref to waiting. A participant replies by comparing that exact waiting oid and moving the ref to signal. At the deadline, a runner compares the same oid and moves the ref to expired. Those updates have one winner. Only then does the runner call the wake handler. A later reply returns not_waiting. The wake handler may settle a result or park again.

When a tool batch completes, one runner CAS advances the conversation head with result commits, moves the run back to respond, and deletes the run's effect refs. A failed batch stores a failed run. A waiting batch stores the waiting phase and keeps effect refs.

Exactly-once execution is not promised, and I want to be blunt about that. The outside world does not share the ref transaction. safe is at-least-once after an uncertain crash. never is at-most-once and reports an unknown outcome after interruption. Anything claiming better than that for an HTTP call is lying.

Wait and wake

A waiting run needs no live process and no held lease. Its run ref and effect refs contain everything needed to resume. runs.reply targets a call id and its exact waitId, optionally checks a run id, and uses the effect signal CAS. Its outcomes are signalled, not_waiting, and not_found.

On the next acquired step, a waiting run calls tools again if any effect has a signal or expiry, its nearest deadline has passed, or the run has an abort request. Otherwise the step returns waiting with that nearest deadline so a hot host can schedule another step. Ordinary submitted messages stay in their lane while the run waits. A tool reply is a signal, not a conversation message, and the two never get confused for each other.

A wait that needs a participant says so. The waiting effect may carry a selection: a Selection, which is a title, a non-empty list of Choices with ids and labels, optional multiple: true, and optionally other, the placeholder under which the participant's own words are accepted. A client answers with a SelectionReply that keeps picked ids and own text in separate fields. Text that looks like an id therefore stays text. A selection is data, not a call into a client. It lives on the ref, rides the effect event, and appears on the snapshot's parked list with a waitId that identifies that exact waiting effect, so the phone in scene 3 renders it an hour later without the tool's schema or the plugin that parked it. Re-parking the same call produces a new waitId; clients use it to discard local picks and reopen the new wait. A client renders a selection generically. A wait without one is background work, such as a delegated child, and a client must not present it as a request for input. Core and clients decide this by the selection's presence, never by the tool's name. What a plugin parks with is checked against the Selection schema before it is written; a malformed one fails that call as a tool error.

A waiting effect may also carry until, an epoch-ms deadline. The deadline is durable state, not a client timer. At or after it, a participant reply returns not_waiting. A runner first moves the exact waiting effect to expired, then calls the tool's wake handler with expired: true and no reply. A crash between those actions leaves an expired effect for the next runner to wake. The handler settles or parks again with a new waitId and deadline. Desktop and TUI may close their panels at the same time, but closing a panel never settles the effect. A host step does. Local timers are only wake-up hints and must re-check the epoch deadline, including when it exceeds the platform's maximum timer delay.

Choice, Selection, and SelectionReply are protocol's first UI shapes: vocabulary a plugin fills, core carries, and any client draws. They deliberately do not grow toward a widget language. A plugin that needs a richer rendering than a list gets it from a client-side presenter keyed by tool name, the same way tool results do, never from a bigger shape in core.

Retries

A retryable provider failure produces a complete assistant error message and a retry outcome. The step commits that assistant message, increments attempts, and writes the run phase as retry { at, error }. Before at, a step returns the durable deadline. At or after at, it calls respond again. drive waits in process. A step-at-a-time host schedules another entry.

The retry policy decides whether an assistant error is retryable and computes the delay, including any provider-requested minimum. Retries are a run phase. They are not timers in runner memory, because runner memory is exactly the thing this design refuses to depend on.

Checkpoints

A checkpoint commit is a context boundary. It stores a portable summary, retained tail, tokensBefore, optional usage, and optional provider checkpoint material. The context projection starts at the newest checkpoint. It uses provider material only when provider, API, and model all match. Transcript and usage projections keep the checkpoint as history.

Turn produces checkpoint outcomes for threshold and overflow compaction, and step commits them before asking for another response. runs.compact calls writeCheckpoint for a manual checkpoint under the head lease, summarizing with the model and thinking level the branch declares. Watchers receive the resulting head update as an ordinary commit event. There is no compaction event kind, because there is no compaction that is not a commit.

Two ways to run

  • Hot loop. A desktop, TUI, or VM calls drive, renews one lease, and executes until idle, waiting, finished, busy, or fenced.
  • Step at a time. A request, queue consumer, or alarm calls step, persists nothing locally, and schedules another entry when the outcome requires one.

Both have the same durable behavior, because a runner must not require in-memory state that outlives a step. The hot loop is just the step loop with the lease kept warm.

Admission is open

submit always creates a pending Change. It never appends directly to the conversation head and never inspects or acquires the runner lease. The lane is the submitter's choice. The SDK sends to the first lane of its landing policy when a client names none, and refuses a lane the policy does not list, since no runner would ever land it.

Submission reads the lane tip, builds a change whose previous names that tip, writes the object, then compares and moves the tip ref. A tip conflict retries internally, up to a bounded attempt limit that no real workload should reach. If the caller supplied a key, the same CAS writes refs/keys/<key> to the new change. A competing key write returns duplicate with the first change oid. The only successful outcomes are queued and duplicate.

Lanes are chains

Each head has one independent chain per lane, and a head has whichever lanes its refs name. A lane exists so a message can overtake older ones without rewriting their chain. A chain is content-addressed with one base pointer, so a runner takes only from its front, and each delivery timing needs a front of its own. The runner's landing policy lists the lanes it serves, in priority order, and says when each lands:

landsLands
boundaryBefore the next assistant response, and on an idle head
idleOnly when the run is absent or terminal

The SDK's default policy is steer at boundaries and queue when idle. A host may declare other lanes, such as one per user in a shared session.

For each lane, tip names the newest change and base names the last landed change. Pending is the half-open chain interval (base, tip], walked through Change.previous and presented oldest first. The store holds no pending array. Concurrent submissions form one chain because every tip move compares its predecessor.

The policy's drain says how much lands at once. "one" lands through the first message in the selected lane, including any config or note changes ahead of it, so the model answers one message at a time. "all" lands the whole lane. With "one", a boundary lane mid-run waits until the last landed message has its answer. Otherwise a steering message could land on top of an unanswered one and the model would see two questions for one reply. A response-phase step considers only boundary lanes. An idle or terminal head considers every lane in policy order.

pending walks every chain, drops tombstoned changes, and sorts the visible items by submission time and oid. All clients read that store projection and see the same list.

Cancel and redeliver

Cancel does not rewrite a change or splice a chain. It writes a blob behind refs/cancelled/<change> and asserts the source lane base in the same CAS. Pending walks skip the tombstone. Landing asserts that every change it selected has no tombstone. A cancel racing a land therefore settles as cancelled or landed, never both.

Redelivery moves a pending item between lanes in one atomic update. It writes the original change's tombstone, asserts the source base, writes a copy on the target chain, and advances the target tip. The copy keeps the body, author, and submission time, records supersedes, and gets a new oid because its predecessor and provenance changed. No intermediate state shows the item in both lanes or in neither.

A submitted message is never lost

The only ways a pending change stops being visible are landing and an explicit cancellation tombstone. A failed, aborted, or superseded run leaves pending changes in their chains. A reconnecting client asks the store for pending and sees the same set as every other client.

An abort interrupts a step, not the run: the runner cancels the call in flight, keeps whatever it produced, and at the next response boundary lands a message waiting in a boundary lane into the same run, clearing the flag. Only an empty boundary queue ends the run as aborted.

Each store transaction is atomic. If submit throws because the disk is full or the connection is lost, it may have written an unreachable loose object, but it has not half-applied a ref update. Core does not repair an unknown request. The client retries.

Core cannot cover the gap between a user pressing Enter and receiving a durable receipt. The client closes it with an outbox:

  1. Mint a key on Enter, keep the message locally as sending, and render it immediately.
  2. Submit with that key. On a network failure, keep the row and retry with backoff using the same key until it succeeds or the user cancels the local send.
  3. Treat queued and duplicate as durable. Replace the local row with the returned change oid.

The idempotency ref and queue tip move in one CAS. A retry after a lost response returns the first change and never enqueues the message twice.

Who runs?

Admission makes work durable but does not guarantee that this process will execute it. A host that called attach watches the session's event stream, wakes a driver for a head whenever its head, queue, run, or effect refs move, and races for the head lease. One runner wins. Thin clients do not call attach. Their submissions stay durable until a host volunteers.

Attribution is optional Actor data with clientId, userId, and device. A host sets it once in NyteOptions, and the SDK writes it on participant objects and ref events. Attribution is not authorization.

Many runners, one store

Heads and sessions are independent. Two runners may hold leases for different heads in one session. Any number of sessions may share one backend. Processes are interchangeable because a step reads objects and refs again and every runner write is fenced.

Takeover increments the lease fence. The old runner's next ref update or event append returns fenced before it changes durable state. A second participant may request abort without the lease. The runner honors the new run ref at its next publish.

The Store contract has four namespaces on every Session:

NamespaceRequired behavior
objectsIdempotent content-addressed put, get, list with write time, and collector delete.
refsRead, prefix list, and atomic multi-ref CAS with assertions, optional lease, actor, reason, and events.
leasesAcquire, renew, release, and read fenced execution rights.
eventsFenced append, cursor read, last, floor, trim, and gap-free ordered watch.

The transaction that matters spans a refs.update: check the lease, check every from, apply every non-assertion ref write, allocate sequences, append ref events and supplied events, then publish to watchers. A backend that cannot make that atomic must not host the kernel. Objects are put first and may stay loose if this transaction loses its race.

The SQLite backend uses five tables:

TableHolds
sessionsSession id, creation time, next event sequence, and event floor.
objectsSession, oid, object kind, canonical body, and write time.
refsSession and ref name to oid.
leasesSession and lease name to owner, fence, and expiry.
eventsSession and sequence to time and event body.

SQLite serializes writes with BEGIN IMMEDIATE. One session row allocates every event sequence. An assertion changes no ref row and emits no ref event. Trimming deletes event rows and raises the session floor in one transaction.

Deployment

Three placements use the same store and step contract:

  • Embedded. The host process is also runner and client. The TUI and desktop use a local store and in-process watch.
  • Server. A long-lived host owns a store, attaches runners, and exposes SDK operations over a wire.
  • Serverless. A request or alarm opens a session, calls step, and exits. A remote backend supplies the same four authorities.

Only the embedded placement ships today. The request protocol below is the shape a server would take, following the SDK rather than inventing a second runtime:

POST /sessions/{id}/messages          submit: queued | duplicate
GET  /sessions/{id}/snapshot          complete read model at seq
GET  /sessions/{id}/events?after=seq  retained events, then live events
POST /sessions/{id}/steps             idle | busy | waiting | retry | finished | fenced

Reconnect uses the last sequence. If the cursor is below the floor, the client takes a new snapshot and watches from the snapshot sequence. A protocol must keep the SDK's plain objects and discriminated outcomes, so another language can implement it without TypeScript object handles.

Events and views

The raw event stream is durable. Ref moves, deltas, tool progress, and notices all take sequence numbers in the same feed. That is what pays for cross-process streaming until retention advances the floor. A reconnect within the window replays the deltas it missed. A reconnect below the floor gets CursorExpired, throws away incremental state, takes sessions.snapshot, and resumes after the snapshot's seq.

The event stream is also the bounded reflog. The collector marks every current ref target and both ends of every retained ref event, then follows object links, including a commit's change provenance. It sweeps only unmarked objects older than the supplied grace period. A host that wants to reclaim history first calls trimStream, which advances the floor and drops old ref-event roots, then calls collect.

Retention is explicit. The kernel supplies floor, trim, mark, grace, and sweep. It does not pick a default event window, grace period, or schedule. Those are product decisions and I did not want to bake a guess into the kernel.

Projections in kernel/views are pure reads:

Projection areaExports
changesappendTurnChanges, changesFromTurns, diffStat, patchedPath, and readPatch
contextprojectContextStatus
directorysessionDirectoryEntry
presentationcreatePresenter, presentNote, presentTool, and projectToolView
transcriptappendTranscriptCommit, transcriptFromCommits, and turnPartId
treecollectAbandoned, navigationTarget, and projectTree
usageemptyUsageSummary, mergeUsageSummaries, and projectUsage

Projections must not write. The transcript must render complete settled messages without a plugin. Presentation refiners may improve a tool or note, but an unknown tool or note still has a base presentation. A client should never see a blank row because a plugin is missing.

Plugins

Plugins stay host composition. A plugin may contribute tools, commands, prompt sections, resources, settings, and agents, and may install hooks. The host resolves plugins only after workspace trust and passes the resolved list into core.

Hooks are the points where plugin code intercepts a run and returns a typed result. They are a separate list from event listeners on purpose. A listener has no return value the runner reads. A hook does, and keeping the two apart is what stops an observer from turning into an interceptor by accident. The hooks are transform_context, before_request, before_tool, after_tool, and before_compaction. before_tool is the policy hook. Its handlers run in registration order, modify decisions chain, and the first reject or error stops the chain. A handler that throws becomes error, so a broken policy plugin fails closed.

A command runs at once with the argument typed after it. It answers with text for the user, with a prompt the client sends as the user's next message, or with nothing. A plugin acts on its own session through api.session: info (the name, and whether the session is a child), rename, and context (the main branch as the model would see it next). A plugin that needs more asks for a new operation here. It does not reach for the store.

Hooks and events are two lists on purpose, the way a pre-commit hook and a post-receive hook are. A hook handler returns something the runner reads, so it can refuse or rewrite what is about to happen. api.events.subscribe delivers the same SessionEvent stream a client folds, after the fact, and nothing a listener returns or throws reaches the run. Observers built on it, such as notifications and the footer's status items, cannot become interceptors by accident.

Three more ways a plugin reaches the user without touching the conversation: diagnostics.notify asks for attention (a title, a message, and whether to make a sound), the status registry holds short items a client shows beside the model, and a tool that needs an answer parks with a Selection (throw new ToolWait({ selection, until }), or { kind: "wait", selection, until } from its wake handler). All three travel as data, notification and status_changed events and the waiting effect, so every client renders them the same way and none of them names a plugin. There is no ui namespace of calls a plugin makes into a client, because there is no attached client to call: a phone that connects tomorrow must see the same thing, and only data on a ref or an event can promise that. A tool may declare availability: "foreground" when it cannot run after its session moves to background. That is host placement metadata. It says nothing about the tool's name or what selection it may create.

Activation is per session and per host process. Handlers must be re-entrant. Cross-step state belongs behind session fact refs or in a host-owned service, never in a runner closure. Plugin failure must not weaken a policy decision into permission.

Facts use refs/facts/<key> pointing to blobs. The SDK keeps a session's name and parent there, and plugin storage lives under a per-plugin prefix. Ref updates make their changes visible in the same event stream as every other domain ref.

Agents

An agent is a host-resolved session preset, not a second runner type. A run using a preset stores its name in config. Delegation does not require a preset: the parent supplies a prompt and an exact provider/model through task. Its tool schema lists available models across authenticated providers, refreshed for each request. The parent honors the user's requested model or chooses one suited to the task. There is no global model override or automatic model substitution.

The child receives the shared session tools except task and stop_task, with its role and constraints supplied in the prompt. Its config records the exact model and the optional task.thinkingLevel override, or inherits the parent's thinking level when omitted. Availability is checked before child creation. A model lost from the catalog before execution fails the child rather than replacing the selected model with the host default.

Child work is a child session with durable parent coordinates stored as a parent fact: parent session, run, call, and depth. sessions.create({ parent }) writes them, sessions.list({ parent }) finds the children, and a plugin reads session.info().child. Durable child completion must travel through stored state, never through a process-local registry the model queries.

Delegation is a task tool that creates the child and parks through the effect sandwich. The child runs independently, so a tool batch may run several children at once and the parent holds no lease while it waits. Child completion signals the parent's effect; the wake handler reads the result from the child session rather than trusting process-local state. The SDK installs task only in root sessions, so delegation depth is 1. A client presents a parked task as background work rather than as a request for user input. Desktop exposes the live child transcript from the parent task or jobs panel, not as a separate sidebar thread.

The parent can call stop_task({ jobId }) with the ID returned by a background task. This cancels that child and its running work through the same durable cancellation used by clients. Only subagent jobs owned by the calling session can be stopped. Finished jobs stay unchanged; unknown job IDs return a tool error. Stopping a task does not abort the parent run.

The SDK

The kernel SDK is plain data and one options object per operation. SessionId is the only branded id. Run ids, head names, oids, and sequences use fields that name their meaning. main is the default head.

The root contract is:

Nyte
  landing    the landing policy in force: the lanes a client may send to, in priority order
  sessions   create · get · snapshot · list · rename · setPinned · setArchived · delete · configure
  messages   send · cancel · redeliver · list · pending
  runs       current · abort · wait · reply · compact · context · changes
  heads      list · create · move · delete · merge
  workspace  list · forget · vcs.status · vcs.diff
  provider   models.list · models.default
  plugins    list · commands.list · commands.run · settings.list · settings.apply · resources.list · status.list
  watch({ sessionId, afterSeq? } | { sessionId, live: true })
  attach({ sessions? })
  reactivate()
  setPlugins(plugins)
  close()

NyteOptions always supplies store, streamFn, models, and a fallback model. It may supply landing (the lanes this host serves; absent means steer and queue), actor, thinkingLevel, compaction, stream options, VCS, and a workspace registry. Static composition supplies plugins and env. Lazy composition supplies resolveActivation and must not also supply static plugins or env.

SessionInfo.activation is host-local state. activation_changed is replayed once on every watch with the latest durable sequence, but it is not ordered by the snapshot's seq and does not enter the durable event stream. reactivate() asks the host resolver again for blocked sessions.

runs.compact writes a manual checkpoint. heads.move({ summary }) carries abandoned history to the navigation target with a summary commit. runs.wait resolves idle only once the run is terminal, nothing is pending, and the runner has released its lease, because the last publish and the lease release are two writes and idle promises the head is free.

The outcome unions a client switches on are exactly:

ConfigureOutcome   queued { change } | unknown_model | unknown_agent
SendReceipt        queued { change } | duplicate { change }
CancelOutcome      cancelled | landed | not_found
RedeliverOutcome   redelivered { change } | unchanged | landed | not_found
AbortOutcome       requested { runId } | not_running
WaitOutcome        idle | waiting { runId }
ReplyOutcome       signalled | not_waiting | not_found
CompactOutcome     compacted { commit } | nothing_to_compact | busy { run } | failed { message }
CreateHeadOutcome  created { tip } | exists | unknown_parent
MoveOutcome        moved { from, restored?: { commit, content }, summary?: Oid } | busy { run } | moved_since { tip } | not_found | failed { message }
DeleteHeadOutcome  deleted | not_found | busy
MergeOutcome       merged { tip } | stale | empty | no_stack
CommandOutcome     ran { output? } | prompt { prompt } | not_found | failed { message }
ApplyOutcome       applied | not_found | invalid_choice

RunInfo is not a terminal-state union. It carries runId, head, the kernel RunPhase, startedAt, attempts, optional abortRequested, and the current lease owner and expiry when a lease exists.

Events

Every SessionEvent carries seq. The SDK projects raw ref events and objects into these exact kinds:

commit           { head, item: { oid, commit } }
head_moved       { head, from, to, reason, actor? }
run              { head, run }
queued           { head, item }
landed           { head, change }
queue_cancelled  { change }
effect           { runId, callId, state: intent | expired | signal | result, tool, args }
effect           { runId, callId, state: waiting, waitId, tool, args, selection?, until? }
stack            { head, parent, base }
fact             { key, value }
deleted
synced
text_delta       { runId, attempt, index, delta }
reasoning_delta  { runId, attempt, index, delta }
tool_progress    { runId, callId, progress }
diagnostic       { level: info | warn | error, owner, message }
plugins_changed  { plugins }
notification     { owner, title?, message, sound }
status_changed   { items }
activation_changed { activation }

The SDK event contract has no claim events, entry ids, retry overlay, compacting overlay, or compaction event. Deltas use the provisional run tuple because no commit oid exists until the assistant message is whole.

Invariants

This numbered list is the contract other documents and code comments cite by number. Numbers are never reused. A superseded number stays here with one line saying why.

Storage and session:

  1. Superseded. Entries and records no longer exist. All six object kinds are immutable and addressed by canonical-content hash.
  2. Superseded. seq orders events only. Objects, refs, and leases do not receive a shared log position.
  3. Superseded. Heads and facts are not the only mutable state. Every domain register is a ref, including queues, runs, effects, stacks, keys, cancellation, and deletion.
  4. Superseded. The object store canonicalizes typed objects directly. toJsonValue remains a boundary for unknown effect, note, and progress payloads, not every durable write.
  5. Superseded. Admission no longer chooses placed or queued by reading a claim. Every submit appends a change to a lane chain without taking a lease.
  6. Concurrent participant writes linearize through ref CAS. A head advances only from its expected oid, so concurrent writers do not create an implicit fork.
  7. Submission is idempotent under a caller-supplied key written with the queue tip in one CAS.

Leases and runs:

  1. At most one live lease exists per head name. Every runner ref write and event append is fenced, and a fenced transaction changes nothing.
  2. Any runner may acquire an expired head lease and resume the run named by refs. It recovers an effect from its stored state and replay policy and never executes a stored result again.
  3. Superseded. Tools no longer provision a result entry id. An effect ref names immutable intent, waiting, expired, signal, and result objects, and result messages enter the branch at tool-batch publish.
  4. Superseded. The kernel cannot promise eventual completion without a runner. A run remains durable in its current phase until some process advances it.
  5. A runner requires no in-memory state that outlives a step. The next step reads the head, run, queue, effect, and deletion refs.
  6. Superseded. Wake input is not ordinary message admission. A waiting run is woken by a signal, a durable expiry claim, or abort.

Turn and boundaries:

  1. Superseded. The kernel Turn receives the session and lease so it can maintain durable effect refs and emit fenced events.
  2. A length-stopped assistant message fails its whole tool batch.
  3. Tool results remain schema content parts end to end. Providers encode them at request time.

Events and views:

  1. Superseded. The event stream is not the object store. It is a retained stream of ref moves, deltas, progress, and notices, and replay below its floor is refused.
  2. Superseded. Streaming overlays cannot reference a provisioned entry. They reference run id, attempt, and content index until the completed message has a commit oid.
  3. Projections read objects, refs, and events and never write authoritative state.

Policy:

  1. Ported files change only within recorded divergence classes. New behavior composes around the port unless the divergence is named.
  2. Extension failures are contained, extension capabilities may degrade across hosts, and project extension code must not load before trust.
  3. Core has no global mutable defaults. The stream function and model catalog are injected.
  4. No public SDK operation encodes a wall-clock execution budget. Lease TTL is failover metadata; hosts own execution budgets.

Plugins:

  1. Nothing reaches the model except through host composition and plugin contribution. Built-ins are replaceable plugin contributions.
  2. Contribution tables are a function of the activation list. Replaying the same list over a fresh draft yields the same tables.
  3. Plugin session storage uses fact refs under the plugin's prefix. A plugin must not read or write another plugin's keys.

Agents:

  1. An agent is one plugin registry contribution. There is no separate subagent runner type, and its mode never grants a capability.
  2. A parent selects an agent by name. It must not set the child's model, system prompt, tools, or a weaker policy through model-supplied arguments.
  3. Superseded. There is no operation_started record. The run object stores configuration folded from config commits when the run starts.
  4. A model-facing background job settles through durable session state, never through a process-local registry the model queries.

Presentation:

  1. A settled tool result is self-contained. Core transcript and base presentation remain total for unknown tools and note types and do not depend on ephemeral progress.

Heads and pending input:

  1. Superseded. Head moves are logged as ref events, but the reflog is retained only above the event floor and may be trimmed.
  2. Parent, base, and staleness remain. Summary navigation carries an abandoned path forward with one summary commit at the new base and never replays generated text.
  3. Superseded. Pending input is not a record kind. It is the visible portion of the lane chains after tombstones are removed.

Kernel CAS, queues, retention, and streaming:

  1. Every ref CAS checks all from values before any write. to === from is an assertion that emits no event. Runner publishes must assert refs/deleted absent.
  2. Each lane is one Change.previous chain bounded by base and tip refs. Submitting must not replace it with an array or a client-owned queue.
  3. Cancellation is a tombstone ref. Landing and cancellation assert the state they race over, so a change is cancelled or landed, never both. Redelivery tombstones the original and appends its replacement in one CAS.
  4. Retention marks current refs and retained ref-event endpoints, follows all object links, and sweeps only unmarked objects older than grace. Raising the event floor must make older cursors fail with CursorExpired and take a snapshot.
  5. A streaming delta is keyed by (runId, attempt, index). It must not claim a commit oid before the complete message has been hashed.
  6. A client outbox must persist one key from Enter through receipt and retry that same key. queued and duplicate both prove that the submitted change is durable.
  7. A commit has exactly one context parent. imports record provenance and must not enter the model context chain.
  8. Objects must be put before a ref names them. A failed CAS may leave loose objects and must not expose a partial ref update.
  9. The kernel must not name a head or a lane. Which lanes exist and when each lands is the landing policy a runner passes to every step; the default head and default lanes are the SDK's. A lane a head has never used is created by the first change written to it.
  10. TypeBox is the one schema library. A TypeBox type is the JSON Schema providers and MCP exchange, so Tool.parameters crosses the wire without conversion, and hosts validate their own boundaries with the same library. Only Node-side code imports typebox/compile; a renderer under a no-eval CSP checks through typebox/value.
  11. A client must be able to render and answer any parked call that carries a Selection from the snapshot alone, and must not present one without as a request for input. Core, projections, and clients decide which is which by the selection's presence, never by the tool's name. A selection is made durable JSON and checked before the ref is written, so no plugin can leave an effect ref no reader can parse. Every parked call carries the waiting effect oid as waitId; re-parking the same call changes it so clients reset generation-local state.
  12. A waiting effect's optional until is the authority for timeout. At or after it, a runner must claim the exact waiting oid by moving it to expired before it wakes the tool with expired: true. A participant reply and that expiry claim race through the same ref CAS, so only one wins. A client may close its panel but must not manufacture a reply or settle the effect. Re-parking after expiry requires a new wait and, when it should expire again, a new deadline.

Open questions

  1. Long tools on stateless placements. A command that outlives a request needs a process that can finish it or an external service that later signals its effect. The wait mechanism exists. The first remote backend has to decide the host wiring.
  2. Joining heads. heads.merge fast-forwards a current stacked child into its parent. A general join of two named heads stays deferred until a product needs one.
  3. Identity and authorization. Actor is durable attribution. ACL-scoped callers are host and protocol territory.
  4. Host-scoped plugin storage. Fact refs belong to one session. Preferences shared across sessions need a host-owned store when a concrete plugin needs one.
  5. Retention defaults. The kernel exposes trim, floor, grace, mark, and sweep without choosing a window or schedule. Each host needs a policy before users can rely on a specific undo horizon.

Lineage

The kernel's git correspondence is deliberate. Object database to objects, refs and update-ref CAS to refs, lock files to leases, reflog to events. Nyte swaps the content for messages, enforces one context parent, has no worktree, and retains the reflog instead of keeping it forever.

The turn, schema message types, effect sandwich, and replay policies come from pi. Ported files keep their Based on credit, and THIRD-PARTY-NOTICES.md lists upstreams. The SDK's default steer and queue lanes follow OpenCode's admission vocabulary.

Stack metadata follows Graphite and its open-source fork charcoal: one parent branch and one base revision, with staleness as an equality check. The kernel does not copy their commit replay model. Generated conversation text is not a patch you can reapply safely, and pretending otherwise is how you end up with a transcript the model never said.

The plain-object, one-options-object-per-operation SDK form follows code.storage. Nyte keeps that wire-friendly shape while exposing kernel oids, run phases, lane names, and the event cursor directly.

On this page