07-24 What Should Not Live in Lyquor State?
Lyquor makes persistent state unusually convenient: network functions can update shared, versioned state, while instance functions can maintain node-local state. That convenience creates a design risk. If every order, observation, response, file, and audit record becomes state, both layers eventually inherit the cost of an unbounded database.
Our central judgment is: data belongs in state only when future execution, recovery, or validation of a future transition must read it as a current fact. Persistent data that fails this test needs a different home and a defined lifecycle; ephemeral data may not need persistence at all.
data
|
+-- needed as a current fact for execution, recovery, or transition validation?
|
+-- yes: must all hosting nodes agree?
| +-- yes -> network state
| +-- no -> instance state
|
+-- no: authoritative ordered evidence?
+-- yes -> event / history
+-- no: immutable source content that must be retained?
+-- yes -> object storage
+-- no: rebuildable consumer view?
+-- yes -> derived query / index
+-- no -> process-local or discard
Persistent Does Not Mean State
Lyquor's documented state model separates two kinds of persistent memory. Network state is shared, sequenced, and versioned by LyquidNumber. Instance state belongs to one node, persists locally, and may differ across nodes. This distinction answers who must agree on a value, but not whether the value should be state at all.
Consider a trading application:
- The current balance must affect whether the next withdrawal is valid.
- The current risk limit must affect whether the next order is accepted.
- A node's retry cursor may be required to resume its local worker.
- A raw market-data response may be useful for an audit but irrelevant to the next deterministic transition.
- A historical order body may be required for replay while no longer belonging in the live order book.
- A chart, search index, or daily report can be rebuilt from authoritative records.
All six items may need persistence. They do not need the same agreement, access path, retention period, or recovery guarantee.
The danger is especially subtle in Lyquor because Direct Memory Architecture makes state feel like ordinary Rust data. The Lyquor whitepaper describes network and instance state as persistent, byte-addressable memory with direct access to Rust data structures. This removes storage plumbing from application code. It does not remove replication, versioning, backup, migration, or retention costs from the system.
Growing network state increases the amount of versioned data that hosting nodes must preserve and recover. Growing instance state moves the problem rather than eliminating it: each operator must still store, back up, migrate, and possibly rebuild its own copy. A Vec that only grows is an append-only database, even when it is declared inside state!.
Five Data Destinations, Not Two
A more complete Lyquor data model needs at least five persistent destinations:
| Destination | What it represents | Examples | Required property |
|---|---|---|---|
| Network state | Current facts that all hosting nodes must apply consistently | balances, permissions, canonical order-book state, accepted risk parameters | deterministic, sequenced, versioned |
| Instance state | Current facts needed by one node's local execution or recovery | work cursor, bounded working set, local protocol checkpoint, source-health status | locally durable, isolated, recoverable |
| Event and history plane | Append-only evidence of what happened | submitted orders, cancels, fills, operation receipts, batch commitments and object references | ordered, queryable, replayable, prunable |
| External object plane | Large source content that should not sit in execution state | files, evidence bundles, raw responses, snapshots, model artifacts | addressable, durable, lifecycle-managed |
| Derived query plane | Views optimized for consumers rather than execution | search indexes, account history, charts, analytics | queryable, rebuildable, independently scalable |
The three non-state destinations are not interchangeable.
An event history preserves authoritative records in their committed order so they can support replay, audit, subscriptions, and downstream processing. It needs stable identifiers, completeness rules, and a defined relationship to committed execution.
An external object plane preserves large source content. A file referenced by a cryptographic digest cannot be silently replaced without changing that digest, but its bytes do not need to sit inside execution state.
A derived query plane reorganizes authoritative data for a particular consumer. A search index can be discarded and rebuilt, provided the underlying state, history, and source objects remain available.
This distinction is familiar in Ethereum. Solidity events produce searchable logs that applications can consume through JSON-RPC, while analytics systems transform blocks, transactions, logs, and traces into separate query models. Ethereum's data and analytics documentation makes that second layer visible: execution data and convenient application queries are different products.
Lyquor should not copy Ethereum's exact event encoding or storage economics. The useful lesson is the boundary:
state answers: what is true now?
history answers: what happened, in what order?
objects answer: where is the full body?
indexes answer: how can a consumer find it efficiently?
Five Questions for Placing Application Data
Before adding a field to either Lyquor state layer, an application designer can ask five questions.
1. Can this value change the validity of a future transition?
If removing the value would make the next deterministic call impossible to validate, it is a strong state candidate. Balances, nonces, permissions, open liabilities, and the currently active policy usually pass this test.
If the value only explains a past transition, it is more likely history. For example, the current position may belong in state; every intermediate calculation used to reach it usually does not.
2. Must all hosting nodes agree on it?
If yes, and the value affects shared execution, it belongs in network state. If only one node needs it to continue local work, instance state may be sufficient.
Local does not mean disposable. A retry cursor, local protocol checkpoint, or pending work item may require durable recovery. But it should not be promoted to network state merely because losing it would be inconvenient.
The same restraint applies to caches. A cache belongs in instance state only when it is bounded and persisting it provides a defined recovery benefit; otherwise, process memory is enough.
3. Can it be reconstructed?
Derived views should normally live outside state. Order-book depth charts, account activity pages, search results, aggregate volumes, and monitoring dashboards can be rebuilt from a canonical state plus complete history.
Reconstruction changes the reliability question. The system must define which source is authoritative, how a consumer detects missing ranges, and which checkpoint lets rebuilding start without replaying from genesis.
4. Is the body large while only its identity affects execution?
Store the commitment, not necessarily the body.
A shared state transition may need a digest, root, length, owner, status, or availability commitment. The corresponding batch body, file, evidence bundle, or model artifact can live in content-addressed external storage. This pattern keeps the execution boundary compact without making the underlying data unauditable.
It also creates a new obligation: the application must specify who retains the body, for how long, and what happens if the commitment remains available but the body does not.
5. When may the data be deleted?
State without a deletion rule tends to become history accidentally. Every collection should have an explicit lifecycle:
- overwritten when a newer current fact replaces it;
- removed after a terminal transition;
- retained for a dispute or recovery window;
- compacted into a snapshot;
- archived outside the hot path;
- or preserved permanently because the application explicitly requires it.
“Keep forever” can be valid, but it should be a product and cost decision rather than the default behavior of a container.
What This Means for Lyquor Applications
Applied to a DEX-like Lyquid, these principles yield the following initial allocation for discussion:
Network state
current balances, positions, reserves, permissions,
canonical order-book state, risk parameters, settlement checkpoints
Instance state
bounded working set, local source health, retry cursors,
pending local work and protocol checkpoints
Event/history plane
submitted orders, cancels, fills, liquidations,
certified decisions, state-transition receipts,
batch commitments and object references
External object plane
raw market-data captures, evidence bundles, batch bodies, snapshots, files
Derived query plane
account history indexes, analytics and monitoring views
The point of this allocation is that different types of data require different storage, ordering, and recovery guarantees. Each destination's interface, atomicity, retention policy, and failure-recovery mechanisms still need to be designed and validated. This does not imply that Lyquor already provides all of these storage services.
Instance functions run locally and sometimes need to persist data outside the state layers. For example, an application may need to write an evidence bundle, maintain a local database, append to a durable stream, or store a large artifact. Simply granting unrestricted filesystem access does not solve the problem, because access permissions, data migration, replication, and cleanup rules would remain undefined. A clearer interface should define its scope and policy—for example, content-addressed object storage, application-isolated namespaces, append-only streams, or operator-configured external storage targets.
The event/history plane also needs explicit rules. Developers need to know:
- whether a record is committed atomically with the state transition it describes;
- how records are ordered and identified;
- whether historical ranges can be queried and completeness can be checked;
- which records must be shared by all nodes and which remain local;
- how subscriptions recover after disconnection;
- how long records are retained and when they may be pruned or archived.
Without those answers, moving data out of state may reduce state growth but still create an unreliable second database.
The State Boundary Is a Product Boundary
The choice is not simply an embedded database versus a file system, or network versus instance. It is a choice about guarantees.
Network state carries the broadest operational burden because it represents a shared current truth. Instance state lets local execution retain working context without forcing global agreement. Event history serves users, auditors, and downstream systems that need more than the latest state. External object storage gives large source content an independent scaling and lifecycle path. Derived query systems let consumer-facing views scale and rebuild independently.
The practical rule is:
Keep the minimum facts required for execution, recovery, and verification in state. Put ordered evidence in history, large immutable bodies in object storage, and rebuildable views in indexes.
Our next research question is therefore concrete: what minimum guarantees should a Lyquor event and external-storage interface provide so that applications can remove data from state without losing auditability, recovery, or portability?