Sickle Sol

Proposal: generate the public-item tables from source

Most layer pages carry a table inventorying what a file exports: the item's name, its kind, its visibility, and what it means. Three of those four are facts the compiler already knows. This proposal is to derive them and keep only the fourth by hand.

"Most" is exact and load-bearing. Layer 02 has no such table and layer 06 has six rows, which is a fact about where this proposal pays and is measured below.

It is a proposal to size the work, not a request to start it.

The case is two defects, not a count

Both were found by a reviewer reading rows on PR #180. Neither is reachable by any check in scripts/dev/check-doc-citations.py, because an inventory row contains no citation to be in bounds, non-blank, unambiguous, or recomputable.

  • claim.rs's table had one row labelled "shared helpers" covering required_action and yield_fee_actions. Only the first is pub(crate). The row asserted a visibility the source contradicts.
  • owner_rebalance.rs's table omitted DrainEventGroups and map_drain_events, both pub(crate) and both imported by keeper_exit.rs. They were added by the same branch that wrote the table.

A further five stale rows were found on the same PR by searching for every symbol the branch changed: three listed as private to a file after they became pub(crate), two listed in files they had left, and one that arrived in policy.rs and was listed nowhere.

Seven defects. All seven are name, presence, or visibility — exactly the three columns a generator produces. None is in the Meaning column. That is the argument: not that the tables are large, but that the part of them going wrong is the part no human should be maintaining.

The shape

Generate Item, Kind and visibility from the source. Keep Meaning hand-written, keyed by item name, and merge it on regeneration. An item that appears with no Meaning is a new export nobody has described yet, which is a useful thing for a diff to say. An orphaned Meaning is an item that was removed or renamed, which is the defect above, reported instead of shipped.

What this generator owns

A row keyed by exactly one item, in a table whose remaining columns are properties of that item. 449 rows.

Six header shapes qualify: Item | Kind | Meaning and five variants adding a column the generator derives from the same item -- a line span, a constant's value, a definition line, or the file it lives in. One generator covers all six.

PageRows
03-shared-runners92
08-intent80
07-state-and-settings68
01-entry-points46
10-off-chain45
04-venue-adapters44
05-venue-bindings44
09-fees24
06-events-and-errors6
02-instruction-handlers0

404 are Rust and 45 are TypeScript. Layer 10's are statically derivable too but by a different tool, so scope this "Rust first, TypeScript decided separately" rather than as a class the approach cannot reach.

The largest page is one already written. Layer 03 is 92 rows, on the page contract 3 just finished, and layer 05 adds 44 from a page contract 1 passed over. The debt is already on disk and goes stale as each later contract lands, so generation retires existing debt rather than only future debt.

Layer 02's zero and layer 06's six are worth reading together: a generator scoped this way does almost nothing for the two pages contract 4 is closest to. That is sequencing information, not an argument against the proposal, and it is the strongest reason not to schedule the work purely as preparation for contract 4.

The 95 rows that key like an item and are not one

Within those six shapes, 95 rows do not name a single item. 87 group several symbols under one Meaning, such as YieldScope, AllYield and TradingFeesOnly sharing a row, and 8 carry a synthetic label such as private helpers. 30 are Rust and 65 TypeScript, so layer 10 holds more than half of them.

They are excluded, and the reason is this proposal's own design. Meaning is hand-written and keyed by item name. A row covering four symbols has one Meaning and no single key, so regeneration cannot decide which item keeps the prose; a synthetic label has no item at all. This proposal does not resolve them. The choice is between expanding each into per-item rows, which loses the shared sentence that is the point of grouping, and keeping them hand-written beside the generated table, which reintroduces by hand exactly the rows most likely to go stale. That is a decision for whoever builds it, and it should be made deliberately rather than discovered.

Three shapes that key like an inventory and are not one

70 further rows sit in tables keyed Item, Trait or Constant whose columns are not properties of the named item:

ShapeRowsWhy not
Trait | Account struct | Lines31A relation: trait to implementor
Constant | Preimage | Bytes25Columns come from recomputing sha256, not from a signature
Trait | Orca/Raydium/Meteora account struct14A relation across three venues

Twice now this document has claimed a set it had not defined: first 1,452 rows over every table under a ## Public items heading, then 614 selected by a first-column name rather than by what the table means. Each time the figure sized something other than the proposal, and each time a reviewer caught it. Selecting by a column's label is not selecting by shape, and selecting by shape is not selecting by whether a row has one key.

What it does not own

The ## Public items sections hold 1,391 rows in total. Beyond the 95 and the 70 above, another 777 are different schemas with different columns, and none is an export inventory:

ShapeRowsWhat it is
#193CPI account-meta order, index by index
File100Per-file summaries
Code100Error codes and messages
Instruction65Instruction surfaces
Parser50Parser layouts
Offset36Byte offsets
everything else23325 further header shapes across 19 first-column labels

Some are derivable in principle: error codes come from an enum, account-meta order from the #[derive(Accounts)] struct. Each needs its own generator with its own source of truth, and each should be proposed and sized separately rather than folded in here. This proposal does not size them, and the 449 above is not a figure for "generating the layer pages".

The first version of this count was wrong, and wrong in the document's own way

It said 1,452 rows, and it counted the wrong set twice over: it treated every row under a ## Public items heading as an export inventory row, and its filter excluded header rows by matching a list of first-column names: Item, Trait, Path, Handler, Function, Kind. Those are not all the header keys in use. Constant, Code, Event, Parser, Limit, Location, Route, Runner, Property and File are also headers, so each of their tables contributed its header row to the count as though it were data.

A pattern that appears to select a class and selects a subset is the same defect as the /?Cargo\.toml: suffix match, the basename match in the blast-radius measurement, and the key patterns behind the citation partition on the convention page. This instance was found by a reviewer, in the sizing evidence of the document arguing that hand-maintained derived values go wrong. Enumerate what a pattern matches, and what it declines to match, before trusting either.

The larger error was not arithmetic. A number counted over a set the proposal never defined reads as a measurement of the proposal, and 1,452 was doing that work in a document whose whole argument is that a hand-maintained number drifts from what it claims to describe. Define the set, then count it.

What would falsify this

Stated as concrete checks, so the next person tests rather than searches.

  1. The defects are in the derivable columns. Falsified if a sample of inventory defects turns out to be mostly wrong Meaning text. Test it by classifying the next ten found. The seven known ones are unanimous the other way, which is suggestive and not conclusive at n=7.
  2. Visibility is mechanically decidable. Falsified by an item whose effective visibility a simple parser gets wrong. This repository writes methods through macros — impl_active_flow_policy!, keeper_intent_methods!, orca_drain_accounts_from! — and a generator that reads syntax rather than expansion will not see what they produce. Test it by generating venue/adapters/ first, which is the worst case, before generating anything else.
  3. Regeneration diffs stay reviewable. Falsified if a routine commit produces a large table diff that a reviewer must read to find one real change. Test it by regenerating against the last ten merged commits and measuring the diff each would have produced.
  4. The merge does not lose prose. Falsified by any Meaning silently dropped on a rename. The generator must fail on an orphaned Meaning rather than discard it; a run that quietly drops one is the same silent-skip defect the counts check had, in a tool with write access.

Point 2 is the one most likely to fail, and it is cheap to test first.

It also decides what the citation convention should target

Generating these tables deletes the rows and every citation inside them, so converting those citations to symbol form first is work that gets thrown away. The convention page states the resulting rule and the measurement behind it: the inventory rows are the bulk of the rows but not the bulk of the citations, so prose and step lists are where the conversion pays and inventory rows should be left alone until this proposal is decided either way.

That makes this decision worth taking before the layer 02 contract opens, not because generation must be built first, but because leaving it undecided leaves the conversion without a boundary.

What stays hand-written

Responsibilities, invariants, call flows, and the audit notes. Those carry judgement, and none of them is derivable. Generation is for the part of a page that restates the compiler.

Source: docs/dev/table-generation-proposal.md in vfat-io/sickle-programs, vendored here by its sync workflow. Edit the markdown there; this page is rebuilt from it on deploy.