diff --git a/references/architecture.md b/references/architecture.md new file mode 100644 index 0000000..b070c49 --- /dev/null +++ b/references/architecture.md @@ -0,0 +1,74 @@ +# Architecture + +## State Storage: SeiDB + +Sei uses **SeiDB** for state storage. As of the IAVL removal, SeiDB state-commit (SC) is **mandatory** — the legacy IAVL backend has been fully removed and is no longer available as a fallback. + +### State Commit (SC) + +The state-commit layer is backed by **memIAVL** (an in-memory IAVL implementation with a write-ahead log and snapshotting). It is enabled via `sc-enable` in `app.toml`: + +```toml +[state-commit] +# Enable defines if the state-commit should be enabled. +sc-enable = true +``` + +**`sc-enable` must be `true`.** If it is `false`, the node panics on startup: + +``` +panic: SeiDB state-commit (SC) must be enabled; IAVL backend has been fully deprecated +``` + +There is no IAVL fallback. memIAVL is the sole state-commit backend; it does not "override" or coexist with an IAVL db backend — the IAVL backend does not exist anymore. + +### State Store (SS) + +The optional state-store layer provides historical state for queries and is configured under `[state-store]`: + +```toml +[state-store] +ss-enable = true +ss-backend = "pebbledb" +``` + +## Migration Notes + +Node operators upgrading from a version that supported the IAVL backend must ensure `sc-enable = true` before upgrading. The following have been removed and no longer take effect: + +### Removed CLI commands + +- `seid compact` — compacted the application levelDB +- `seid debug dump-iavl [height]` (with `--db-path`/`-d`, `--output-dir`, `--module`/`-m`) — dumped IAVL data +- `seid prune` (with `--home`, `--app-db-backend`, `--pruning`, `--pruning-keep-recent`, etc.) — pruned app history states +- `seid latest_version` — printed the latest app DB version + +### Removed `start` flags + +- `--separate-orphan-storage` +- `--separate-orphan-versions-to-keep` +- `--num-orphan-per-file` +- `--orphan-dir` +- `--iavl-disable-fastnode` + +### Removed `app.toml` config fields + +- `iavl-cache-size` +- `iavl-disable-fastnode` +- `no-versioning` +- `separate-orphan-storage` +- `separate-orphan-versions-to-keep` +- `num-orphan-per-file` +- `orphan-dir` +- The entire `[iavl]` section (including `iavl.pruning`, `iavl.pruning-keep-recent`, `iavl.pruning-keep-every`, `iavl.pruning-interval`) + +Only the top-level `pruning`, `pruning-keep-recent`, `pruning-keep-every`, and `pruning-interval` flags remain. + +### Snapshots / restore + +Legacy IAVL snapshot and restore via the rootmulti store are no longer supported. Attempting the legacy IAVL path returns an error: + +``` +legacy IAVL snapshots are no longer supported +legacy IAVL restore is no longer supported +``` diff --git a/references/cli/seid-cli.md b/references/cli/seid-cli.md new file mode 100644 index 0000000..bf8f143 --- /dev/null +++ b/references/cli/seid-cli.md @@ -0,0 +1,429 @@ +# seid CLI + +Reference for the `seid` node/CLI binary. This documents notable command and flag changes after the full removal of the IAVL backend (SeiDB state-commit is now mandatory). + +## Important: SeiDB state-commit is mandatory + +The IAVL backend has been fully removed. SeiDB state-commit (SC) must be enabled. + +- In `app.toml`, under `[state-commit]`, `sc-enable` **must** be `true`. +- If `sc-enable = false`, the node **panics** at startup with: + `SeiDB state-commit (SC) must be enabled; IAVL backend has been fully deprecated` +- Operators must ensure `sc-enable = true` before upgrading. IAVL configuration and commands are removed and any leftover IAVL config no longer takes effect. + +## Removed commands + +The following commands no longer exist: + +- `seid compact` — previously compacted the application levelDB. **Removed.** +- `seid debug dump-iavl [height]` (with flags `--db-path`/`-d`, `--output-dir`, `--module`/`-m`) — previously dumped IAVL data for a height. **Removed.** +- `seid prune` (with flags `--home`, `--app-db-backend`, `--pruning`, `--pruning-keep-recent`, `--pruning-keep-every`, `--pruning-interval`) — previously pruned app history states. **Removed.** +- `seid latest_version` — previously printed the latest app DB version. **Removed.** + +## Removed `seid start` flags + +The following IAVL/orphan-storage flags have been removed from `seid start`: + +- `--separate-orphan-storage` +- `--separate-orphan-versions-to-keep` +- `--num-orphan-per-file` +- `--orphan-dir` +- `--iavl-disable-fastnode` + +Top-level pruning flags (`--pruning`, `--pruning-keep-recent`, `--pruning-keep-every`, `--pruning-interval`) remain, but the separate `[iavl]` config section and the `iavl.*` pruning keys have been removed. Only the top-level pruning settings apply. + +## Removed config fields (app.toml) + +- `iavl-cache-size` — removed. +- Base config fields removed: `iavl-disable-fastnode`, `no-versioning`, `separate-orphan-storage`, `separate-orphan-versions-to-keep`, `num-orphan-per-file`, `orphan-dir`. +- The entire `[iavl]` section (`pruning`, `pruning-keep-recent`, `pruning-keep-every`, `pruning-interval`) has been removed. + +## Snapshots / restore + +Legacy IAVL snapshot and restore via the rootmulti store are no longer supported. Attempting the legacy IAVL snapshot/restore path returns an error: +`legacy IAVL snapshots/restore are no longer supported` + +Use the SeiDB-based snapshot/restore tooling instead. + + + +## seidb trace-profile-report + +Runs the `debug_traceTransactionProfile` JSON-RPC method across a range of blocks and writes a raw JSONL dump plus an aggregated summary. Useful for offline profiling of transaction execution and historical DB (store) access across a block range. + +```bash +seidb trace-profile-report \ + --endpoint \ + --start-block \ + --end-block \ + --output-dir +``` + +### Flags + +- `--endpoint` (**required**) — RPC endpoint, e.g. `http://localhost:8545`. Must expose `eth_getBlockByNumber` and `debug_traceTransactionProfile`. +- `--start-block` (**required**) — starting block number (must be positive). +- `--end-block` (**required**) — ending block number (must be positive and `>= --start-block`). +- `--output-dir` / `-o` (**required**) — directory for the output files `raw_profiles.jsonl` and `summary.json` (created if it does not exist). +- `--concurrency` / `-c` — number of concurrent `debug_traceTransactionProfile` requests (default `4`; values `<= 0` are treated as `1`). +- `--trace-config-json` — JSON object passed as the trace config to each request (default `{}`). Invalid JSON causes the command to error. +- `--max-transactions` — optional cap on the total number of transactions processed (`0` = no cap). + +### Output + +- `raw_profiles.jsonl` — one JSON record per transaction with `blockNumber`, `blockHash`, `txHash`, and either the full `result` (trace + profile) or an `error` string. +- `summary.json` — aggregated stats: tx/block/success/error counts, average and P50/P95 total & historical-DB-lookup latencies, per-phase totals (`lookupTransaction`, `loadBlock`, `replayHistoricalTxs`, `buildBlockContext`, `prepareTx`, `execution`, `traceResult`), per-module store operation totals, and top transactions/blocks by total time. + +The underlying `debug_traceTransactionProfile(hash, config)` method returns `{ "trace": ..., "profile": { "totalNanos", "historicalDbLookupNanos", "otherNanos", "phases": {...}, "store": { "modules": {...}, "stats": {...} } } }`. + + + +## seidb state-size + +Scans a memIAVL database and reports per-module state size (key/value/total bytes, key counts, prefix breakdown, and top EVM contracts). Can optionally scan a FlatKV store alongside memIAVL and fold the result into the same output. + +```bash +seidb state-size \ + --db-dir \ + [--height ] \ + [--module ] \ + [--flatkv-dir ] \ + [--export-dynamodb] \ + [--dynamodb-table ] \ + [--aws-region ] +``` + +### Flags + +- `--db-dir` / `-d` — memIAVL database directory. (Help text is now "memIAVL database directory".) +- `--height` — block height to analyze (`0` = latest available version). +- `--module` / `-m` — restrict analysis to a single module. Default: all modules. +- `--flatkv-dir` — FlatKV data directory. **Optional.** When omitted, the tool auto-detects a sibling `flatkv/` directory next to `--db-dir` (i.e. `/../flatkv`, the standard `seid` shadow-node layout: `/data/committer.db` -> `/data/flatkv`). Set explicitly to point at a FlatKV dir elsewhere. +- `--export-dynamodb` — export results to DynamoDB instead of printing to console. +- `--dynamodb-table` — DynamoDB table name (default `state_size_analysis`). +- `--aws-region` — AWS region for the DynamoDB export. + +### FlatKV integration + +FlatKV analysis is strictly additive and only runs when `--module` is empty or `evm` (FlatKV in production holds only evm keys; everything else is bucketed into a `legacy` DB). If the FlatKV directory is missing, its snapshot is unavailable, or the store fails to open, the tool logs the reason and continues — the memIAVL path always still succeeds. + +When a FlatKV directory is present: + +- **Console output** gains a `=== FlatKV state size (version N) ===` section with totals, a per-DB breakdown (`account`, `code`, `storage`, `legacy`), and a top EVM contracts table (top 100 by storage size). +- **DynamoDB export** appends a FlatKV row (module name `flatkv`) to the same batch as the memIAVL module rows. Its `PrefixBreakdown` is a JSON map keyed by bucket name (`{"account": {...}, "storage": {...}}`), matching the shape memIAVL uses. + +The FlatKV store is opened read-only via a temporary snapshot + WAL clone, so it does not contend with a live node for the FlatKV writer lock. + +### Console output notes + +- Modules are printed in alphabetical order so successive runs produce diffable output. +- Each module's prefix breakdown is marshaled as a single combined map (key/value/total bytes and key count per prefix byte). +- The top-contracts table is skipped for modules with no `0x03` (contract storage) entries. +- Progress logging cadence is every 10M keys (previously every 1M). + +### Example + +```bash +# memIAVL + auto-detected sibling flatkv/ at latest height +seidb state-size --db-dir ~/.sei/data/committer.db + +# explicit FlatKV dir and a historical height, evm module only +seidb state-size \ + --db-dir ~/.sei/data/committer.db \ + --flatkv-dir ~/.sei/data/flatkv \ + --height 12345678 \ + --module evm + +# export both memIAVL and FlatKV rows to DynamoDB +seidb state-size \ + --db-dir ~/.sei/data/committer.db \ + --export-dynamodb \ + --dynamodb-table state_size_analysis \ + --aws-region us-east-1 +``` + + +## seidb dump-flatkv + +Iterates and dumps every physical FlatKV (key, value) pair into per-bucket files (`account`, `code`, `storage`, `legacy`), formatted to match `dump-iavl` so the same diff tooling works on both. Keys and values are emitted as `Key: , Value: `, one per line, under a `Bucket at version ` header. + +```bash +seidb dump-flatkv \ + --db-dir \ + --output-dir \ + [--height ] \ + [--bucket account|code|storage|legacy] +``` + +### Flags + +- `--db-dir` / `-d` (**required**) — FlatKV database directory. +- `--output-dir` / `-o` (**required**) — output directory; one file per bucket is written. +- `--height` — FlatKV target version (`0` = latest available version). +- `--bucket` / `-b` — restrict the dump to a single bucket (`account`, `code`, `storage`, or `legacy`). Default: all buckets. When set, only that bucket's file is created. + +The store is opened via a read-only temporary snapshot + WAL clone. The FlatKV `metadataDB` and internal `_meta/*` rows are excluded. + +### Example + +```bash +seidb dump-flatkv \ + --db-dir ~/.sei/data/flatkv \ + --output-dir ./flatkv-dump \ + --height 12345678 \ + --bucket storage +``` + + + +## seidb dump-flatkv + +Iterates and dumps physical FlatKV `(key, value)` pairs into per-bucket files, formatted to match `dump-iavl` so the same diff tooling works on both. The tool operates on a temporary read-only clone of the selected snapshot + WAL, so it does not contend with a live node for the FlatKV writer lock. + +```bash +seidb dump-flatkv \ + --db-dir \ + --output-dir \ + [--height ] \ + [--bucket account|code|storage|legacy] +``` + +### Flags + +- `--db-dir` / `-d` (**required**) — FlatKV database directory. Panics if omitted. +- `--output-dir` / `-o` (**required**) — output directory; one file is written per bucket. Panics if omitted. +- `--height` — FlatKV target version. `0` (default) selects the latest available version. +- `--bucket` / `-b` — restrict the dump to a single bucket (`account`, `code`, `storage`, or `legacy`). Default: all buckets. Invalid values panic. + +### Buckets + +Physical keys are classified into four buckets in this order (`account` → `code` → `storage` → `legacy`): + +- `account` — evm nonce and codehash rows (both canonicalize to the same account row per address). +- `code` — evm contract bytecode. +- `storage` — evm contract storage slots. +- `legacy` — non-evm module keys and any evm keys with an unrecognized type prefix. + +The FlatKV metadata DB and per-DB `_meta/*` rows are intentionally excluded. Physical keys are emitted verbatim (including their `/` + type-prefix header) because they are not byte-for-byte comparable with memIAVL logical keys. + +### Output format + +Each bucket file mirrors the `dump-iavl` format: + +``` +Bucket at version +Key: , Value: +... +``` + +When `--bucket` is set, only that bucket's file is created; unselected buckets produce no file. + +### Example + +```bash +seidb dump-flatkv \ + --db-dir ~/.sei/data/flatkv \ + --output-dir ./flatkv-dump \ + --height 0 \ + --bucket storage +``` + + + +## seidb import-flatkv-from-memiavl + +Imports selected memIAVL modules into FlatKV. This is the offline migration tool for moving the EVM module's SC-layer data from memIAVL into FlatKV. It is a restore-style import: it **resets the FlatKV directory before loading** the imported rows and refuses to run over existing committed FlatKV data unless `--force` is supplied. + +```bash +seidb import-flatkv-from-memiavl \ + --modules=evm \ + --data-dir \ + --height \ + [--home ] \ + [--force] +``` + +### Flags + +- `--modules` — comma-separated module names to import (default `evm`). **Initial production scope is evm-only**; any other module name is rejected at the CLI boundary. +- `--data-dir` — Sei data directory or home directory. If the basename is `data`, its parent is used as home. +- `--home` — Sei home directory. Defaults to `$HOME/.sei`. Takes precedence over `--data-dir` when both are set. +- `--height` — memIAVL version to import. `0` (default) means the latest committed memIAVL version. +- `--force` — overwrite existing committed FlatKV data. Without it, the command refuses to run when FlatKV already has a committed version. + +### Constraints + +- **Import must run at the memiavl latest height.** Importing at `H < memiavl latest` is refused, because a subsequent `GIGA_STORAGE` startup would call `CompositeCommitStore.reconcileVersions` and silently roll memIAVL back to `H`, truncating every cosmos block in `(H, memiavlLatest]`. To import at a lower height, roll memIAVL back to `H` first (this CLI never does that rollback for you). Importing at `H > memiavl latest` is also refused. +- On failure the import is **aborted, not finalized** — FlatKV is left at its pre-import committed version so the operation can be retried without `--force`. +- After a successful import, the offline migration workflow requires keeping `evm-ss-split = false` and `sc-enable-lattice-hash = false` across the import boundary on restart, to avoid AppHash-mismatch and rootmulti startup panics. See `sei-db/state_db/sc/migration/OPERATIONS.md`. + +### Multi-validator workflow + +For a cluster, stop every validator, read each node's latest memIAVL version with `memiavl-latest-version`, pick the minimum as a uniform import height, roll any node that committed extra blocks back to that height, then run the import on every node at the same `--height`. + +### Example + +```bash +# import the evm module at the latest memiavl height +seidb import-flatkv-from-memiavl \ + --modules=evm \ + --data-dir /root/.sei/data \ + --height 12345678 + +# overwrite an existing committed FlatKV store +seidb import-flatkv-from-memiavl \ + --modules=evm \ + --data-dir /root/.sei/data \ + --height 12345678 \ + --force +``` + + +## seidb memiavl-latest-version + +Prints the latest committed memIAVL version of a stopped node to stdout. This is the read-only companion to `import-flatkv-from-memiavl`: orchestration scripts read each validator's version to pick a single uniform import height across a multi-validator cluster. + +```bash +seidb memiavl-latest-version --data-dir [--home ] +``` + +### Flags + +- `--data-dir` — Sei data directory or home directory. If the basename is `data`, its parent is used as home. +- `--home` — Sei home directory. Defaults to `$HOME/.sei`. Takes precedence over `--data-dir` when both are set. + +### Example + +```bash +seidb memiavl-latest-version --data-dir /root/.sei/data +``` + + + +## seidb migrate-evm-status + +Reports the on-disk FlatKV EVM migration state of a FlatKV directory as JSON. Intended for integration test drivers polling for migration completion from the host: the tool reads the migration bookkeeping keys directly from FlatKV instead of adding a custom RPC or grepping node logs. + +```bash +seidb migrate-evm-status --db-dir [--height ] +``` + +### Flags + +- `--db-dir` / `-d` (**required**) — FlatKV database directory. Panics if omitted. +- `--height` — FlatKV target version. `0` (default) selects the latest available version. + +The store is opened read-only via a temporary snapshot + WAL clone, so it does not contend with a live node for the FlatKV writer lock and gives a stable view even if the live writer rolls snapshots mid-run. + +### What it reads + +It reads two reserved keys from the FlatKV `migration` store: + +- `migration-version` — an 8-byte big-endian `uint64` written exactly once on the migration bump block. Absent or `0` means the FlatKV EVM migration (MigrateEVM, version 1) has not yet completed. +- `migration-boundary` — the in-flight resume cursor `(module, key)`. Present iff the boundary is strictly between not-started and complete. + +### JSON output + +```json +{ + "version_at": 12345, + "migration_version": 1, + "migrate_evm_complete": true, + "boundary_present": false, + "boundary_hex": "...", + "version_raw_hex": "0000000000000001" +} +``` + +- `version_at` — the FlatKV version the read resolved against. +- `migration_version` — decoded `migration-version` value (`0` when the key is absent). +- `migrate_evm_complete` — `true` when `migration_version >= 1` (Version1_MigrateEVM). This is the field test drivers poll on. +- `boundary_present` — whether the in-flight boundary key exists. +- `boundary_hex` — hex of the boundary bytes, present only when `boundary_present` is true. +- `version_raw_hex` — hex of the raw `migration-version` bytes, present only when the key exists. + +### Example + +```bash +# poll until migration completes +seidb migrate-evm-status --db-dir /root/.sei/data/state_commit/flatkv \ + | jq -r '.migrate_evm_complete' +``` + + +### Example + +```bash +seidb trace-profile-report \ + --endpoint http://localhost:8545 \ + --start-block 1000 \ + --end-block 1100 \ + --output-dir ./trace-report \ + --concurrency 8 \ + --trace-config-json '{"timeout":"60s"}' \ + --max-transactions 500 +``` + + + +## seid tendermint gen-autobahn-config + +Generates an Autobahn (GigaRouter) JSON config file from per-node pubkey/address files. + +```bash +seid tendermint gen-autobahn-config [node-dirs...] --output +``` + +- Takes one or more node directories as positional args (at least one required). +- For each node directory, it reads three files: + - `validator_pubkey.txt` — validator public key in `validator:` format + - `node_pubkey.txt` — p2p node public key in `node:ed25519:public:` format + - `autobahn_address.txt` — network address in `host:port` format +- `--output` / `-o` (**required**) — output file path for the generated autobahn config. If omitted, the command errors with `--output flag is required`. + +The generated JSON contains a `validators` array (one entry per node dir with `validator_key`, `node_key`, `address`) plus default block/mempool settings (e.g. `max_gas_per_block = 50_000_000`, `max_txs_per_block = 5_000`, `mempool_size = 5_000`, `block_interval = 400ms`, `view_timeout = 1500ms`, `dial_interval = 10s`, `allow_empty_blocks = false`). + +### Where the pubkey files come from + +The two pubkey text files are written automatically as a side effect of saving keys: + +- Saving the validator private key file also writes `validator_pubkey.txt` (`validator:`) in the same directory. +- Saving the node key also writes `node_pubkey.txt` (`node:ed25519:public:`) in the same directory. + +The `autobahn_address.txt` file must be provided/generated separately (e.g. `echo "$NODE_IP:26656" > /autobahn_address.txt`). + +### Wiring the config into a node + +Point the node at the generated config via a top-level `autobahn-config-file` key in `config.toml` (must appear before any `[section]` header so it parses as a top-level key): + +```toml +autobahn-config-file = "/root/.sei/config/autobahn.json" +``` + +### Example + +```bash +seid tendermint gen-autobahn-config \ + build/generated/node_0 \ + build/generated/node_1 \ + build/generated/node_2 \ + --output ~/.sei/config/autobahn.json +``` + + +## Example: starting a node + +Ensure `sc-enable = true` in `app.toml` before starting: + +```toml +[state-commit] +sc-enable = true +``` + +Then start the node: + +```bash +seid start --home ~/.sei +``` + +If state-commit is disabled, the node will panic on startup rather than falling back to IAVL. diff --git a/references/ecosystem/node-operations.md b/references/ecosystem/node-operations.md new file mode 100644 index 0000000..3345815 --- /dev/null +++ b/references/ecosystem/node-operations.md @@ -0,0 +1,221 @@ +# Node Operations + +Operational reference for running a Sei node (`seid`), focused on SeiDB state storage configuration and CLI commands. + +## SeiDB state-commit is mandatory + +As of the IAVL removal, SeiDB state-commit (SC) is **required**. The legacy IAVL backend has been fully removed. + +- `sc-enable` must be `true` in `app.toml`. +- If `sc-enable = false`, the node **panics** on startup with: + ``` + panic: SeiDB state-commit (SC) must be enabled; IAVL backend has been fully deprecated + ``` +- Before upgrading, node operators must ensure `sc-enable = true`. There is no IAVL fallback. +- Legacy IAVL snapshot/restore via the rootmulti store is no longer supported; snapshot/restore of the legacy IAVL path returns an error (`legacy IAVL snapshots/restore are no longer supported`). + +## app.toml: [state-commit] + +```toml +[state-commit] +# Enable defines if the state-commit should be enabled. +# Must be true — the node will panic if this is false. +sc-enable = true + +# Defines the SC store directory; defaults to the application home directory if unset. +# sc-directory = "" + +# ZeroCopy defines if memiavl should return slices pointing to mmap-ed buffers directly (zero-copy). +# zero-copy = false + +# AsyncCommitBuffer defines the size of the asynchronous commit queue. +# async-commit-buffer = 100 + +# SnapshotInterval defines the block interval the memiavl snapshot is taken (0 = disabled by default in tests). +# snapshot-interval = 10000 +``` + +## app.toml: [state-store] + +State-store (SS) is the historical query layer and is separate from state-commit. + +```toml +[state-store] +# ss-enable defines whether the state-store should be enabled for historical queries. +ss-enable = true + +# ss-backend defines the backend database used for state-store. +ss-backend = "pebbledb" +``` + +## Removed config fields + +The following IAVL-related config fields have been **removed**. They no longer take effect and should be deleted from `app.toml` when upgrading: + +- `iavl-cache-size` (was `iavl-cache-size = 781250`) +- `iavl-disable-fastnode` +- `no-versioning` +- `separate-orphan-storage` +- `separate-orphan-versions-to-keep` +- `num-orphan-per-file` +- `orphan-dir` +- The entire `[iavl]` section, including `pruning`, `pruning-keep-recent`, `pruning-keep-every`, and `pruning-interval` under `[iavl]`. + +Pruning is now controlled only by the top-level `pruning` flags: + +```toml +pruning = "nothing" # default | nothing | everything | custom +pruning-keep-recent = "0" +pruning-keep-every = "0" +pruning-interval = "0" +``` + +## Removed CLI commands and flags + +The following IAVL-related commands have been **removed** from `seid`: + +- `seid compact` — compacted the application levelDB. +- `seid debug dump-iavl [height]` (and flags `--db-path`/`-d`, `--output-dir`, `--module`/`-m`). +- `seid prune` (and flags `--home`, `--app-db-backend`, `--pruning`, `--pruning-keep-recent`, etc.). +- `seid latest_version` — printed the latest app DB version. + +Removed `seid start` flags related to IAVL/orphan storage: + +- `--separate-orphan-storage` +- `--separate-orphan-versions-to-keep` +- `--num-orphan-per-file` +- `--orphan-dir` +- `--iavl-disable-fastnode` + +## Migration checklist + +1. Set `sc-enable = true` under `[state-commit]` in `app.toml` (required — node panics otherwise). +2. Remove the `[iavl]` section and all removed fields listed above. +3. Configure top-level `pruning` flags if you relied on the old `[iavl]` pruning keys. +4. Stop using removed CLI commands (`compact`, `debug dump-iavl`, `prune`, `latest_version`) and removed orphan-storage start flags. +5. Start the node normally: + ```bash + seid start --home ~/.sei + ``` + + + +## Autobahn (GigaRouter) config + +Autobahn wiring is enabled by pointing the node at a generated JSON config via a **top-level** `autobahn-config-file` field in `config.toml`: + +```toml +# Must appear before any [section] header so the TOML parser reads it as a top-level key. +autobahn-config-file = "/root/.sei/config/autobahn.json" +``` + +When `autobahn-config-file` is set, the node loads the file at startup and wires up the GigaRouter. Currently only validator nodes are supported (a signing validator key must be present); observer/non-validator support is not yet available. + +### Pubkey side-effect files + +Saving keys now also writes autobahn-compatible pubkey text files alongside the key files, in the same directory: + +- Saving the validator private key (`priv_validator_key.json`) also writes **`validator_pubkey.txt`** in `validator:` format. +- Saving the node key (`node_key.json`) also writes **`node_pubkey.txt`** in `node:ed25519:public:` format. + +These files are consumed by the config generator below. + +### Generating the config + +Use `seid tendermint gen-autobahn-config` to produce the JSON config. Each node directory passed as an argument must contain: + +- `validator_pubkey.txt` (`validator:`) +- `node_pubkey.txt` (`node:ed25519:public:`) +- `autobahn_address.txt` (a single `host:port` line, e.g. the node's P2P address) + +Syntax: + +```bash +seid tendermint gen-autobahn-config [node-dirs...] --output [--persistent-state-dir ] +``` + +- `--output` / `-o` (**required**): output file path for the generated autobahn config. +- `--persistent-state-dir` (default `data/autobahn`): directory to persist autobahn consensus and data WALs across restarts. Relative paths are resolved against the node's `--home` dir at load time; absolute paths pass through unchanged. Pass `--persistent-state-dir=` (empty) to disable persistence and run both consensus and data layers **in-memory only**. +- Requires at least one node directory argument. + +By default (`data/autobahn`), Autobahn persists both the consensus and data layer WALs to disk under a shared on-disk root, with distinct subdirectories per layer, so state survives node restarts. Persistence is on without operator action. + +Example (4-node cluster): + +```bash +seid tendermint gen-autobahn-config \ + build/generated/node_0 \ + build/generated/node_1 \ + build/generated/node_2 \ + build/generated/node_3 \ + --output ~/.sei/config/autobahn.json +``` + +Then set `autobahn-config-file` in `config.toml` to that path (placed before any `[section]` header). The generated config includes each validator's `validator_key`, `node_key`, and `address`, plus defaults such as `max_gas_per_block = 50000000`, `max_txs_per_block = 5000`, `mempool_size = 5000`, `block_interval = 400ms`, `view_timeout = 1500ms`, and `dial_interval = 10s`. + + + +## FlatKV EVM in-flight migration + +SeiDB supports an in-flight (online) migration of EVM state from memiavl into FlatKV, driven by the `sc-write-mode` config under `[state-commit]`. The in-flight migration modes are `migrate_evm`, `migrate_bank`, and `migrate_all_but_bank`; each drains keys from the old backend (memiavl) into the new backend (FlatKV) over multiple blocks. + +### app.toml: [state-commit] migration fields + +```toml +[state-commit] +# sc-write-mode selects the SC backend routing. In-flight migration modes: +# migrate_evm | migrate_bank | migrate_all_but_bank +# (also: memiavl_only, evm_migrated, flatkv_only, test_only_dual_write, etc.) +sc-write-mode = "migrate_evm" + +# sc-keys-to-migrate-per-block controls how many keys the in-flight migration +# drains from memiavl into FlatKV per block. Must be > 0; defaults to 1024 +# when unset. Default 1024 is appropriate for production drains; lower it +# (e.g. 256) to spread the migration across more blocks. +sc-keys-to-migrate-per-block = 1024 +``` + +Notes: + +- `sc-keys-to-migrate-per-block` must be `> 0`. An absent app.toml entry preserves the default of `1024`; explicitly setting it to `0` fails `StateCommitConfig.Validate` ("keys-to-migrate-per-block must be > 0") and brings the node down at startup when write-mode is a migration mode. +- Flipping `sc-write-mode` into a migration mode across a validator quorum changes how EVM data contributes to the AppHash, so it must be done with a coordinated stop / edit / restart across all validators — flipping one node while the rest stay in the old mode diverges the AppHash and halts consensus. +- Once the migration completes, operators flip `sc-write-mode` from `migrate_evm` to `evm_migrated` so subsequent restarts do not spin up the migration manager. This flip is lossless (same version, same FlatKV root, same reads). + +### Behavior during in-flight migration + +While a migration is in flight (`NotStarted` / `InProgress`): + +- **Reads** for unmigrated keys consult the old DB first and fall back to the new DB if not found — brand-new keys created after migration starts are routed to the new DB, so read transparency holds across the boundary. +- **Writes** to existing not-yet-migrated keys stay in the old DB until the iterator reaches them; writes to already-migrated keys and to brand-new keys go to the new DB (avoids the migration chasing an ever-growing key tail). +- **Iteration** is forwarded to the old-DB iterator. Caveat: keys already migrated out of the old DB are silently skipped, so iteration results may be incomplete during `InProgress` and must only be used for best-effort work (this self-heals once migration completes). Once migration is `Complete`, iteration is refused (the old DB has been retired). + +### Polling migration status with `seidb migrate-evm-status` + +Use the `seidb migrate-evm-status` subcommand to poll on-disk FlatKV migration state as JSON, without a custom RPC or grepping node logs: + +```bash +seidb migrate-evm-status --db-dir [--height ] +``` + +- `--db-dir` / `-d` (**required**): FlatKV database directory (e.g. `/root/.sei/data/state_commit/flatkv`). +- `--height` (default `0`): FlatKV target version; `0` selects the latest available version. + +The tool hardlink-clones the latest snapshot + WAL into a temp dir before opening, so it can read concurrently with a live node without contending for the FlatKV writer lock. Example output: + +```json +{ + "version_at": 1234, + "migration_version": 1, + "migrate_evm_complete": true, + "boundary_present": false +} +``` + +- `migrate_evm_complete` is `true` once the `migration-version` key reaches `1` (Version1_MigrateEVM). +- `boundary_present` is `true` while the migration cursor is still in flight (strictly between not-started and complete). + +This is the recommended way to poll for migration completion from the host across a multi-validator cluster: repeatedly run `migrate-evm-status` against each validator's FlatKV dir until every node reports `migrate_evm_complete: true`. + +### Relationship to the offline import workflow + +The offline `seidb import-flatkv-from-memiavl` path bulk-imports FlatKV from a memiavl snapshot before startup. The in-flight `migrate_evm` mode plus `migrate-evm-status` polling is the **online** alternative: the node keeps producing blocks while draining keys per block, and operators watch progress with `migrate-evm-status` rather than waiting for an offline import to finish. diff --git a/skill/references/cli/seid-cli.md b/skill/references/cli/seid-cli.md index 256a1d0..6d933ca 100644 --- a/skill/references/cli/seid-cli.md +++ b/skill/references/cli/seid-cli.md @@ -240,11 +240,184 @@ For writes: sign first, then submit via `eth_sendRawTransaction`. Prefer a walle ### Sei-Specific JSON-RPC Behaviour - **Finality:** `safe`, `finalized`, and `latest` are equivalent on Sei (instant single-block finality). -- **Proofs:** Sei uses IAVL trees, not Merkle Patricia Tries. Proofs returned by proof-bearing endpoints are IAVL proofs. +- **Proofs:** Sei uses IAVL trees, not Merkle Patricia Tries. Proofs returned by proof-bearing endpoints are IAVL proofs. `eth_getProof` now unwraps additional KVStore wrappers (`cachekv`, Giga cache, `tracekv`, and prefix stores) to reach the underlying proof-capable queryable store, so it succeeds across more node/store configurations (classic IAVL, store/v2 memiavl, and future proof-capable roots) rather than only classic IAVL. If no proof-capable store can be found it errors with `cannot find a proof-capable queryable KV store`. + + +- **`eth_getProof` storage key input constraints:** Storage keys must be **hex-encoded** (e.g. `0x0000...0000616263`, a 32-byte slot). They are decoded and left-padded to 32 bytes internally. Raw byte strings are no longer accepted — a malformed (non-hex) key is rejected with `invalid storage key "": ...`. A request may include at most **1024 storage keys** (`MaxStorageKeysPerProof`); exceeding this returns `too many storage keys: got , max 1024`. + + ```bash + # Hex-encode the storage slot before passing it to eth_getProof + curl -s "$EVM_RPC" \ + -H 'Content-Type: application/json' \ + --data '{"jsonrpc":"2.0","method":"eth_getProof","params":["0x
",["0x0000000000000000000000000000000000000000000000000000000000000001"],"latest"],"id":1}' | jq + ``` +- **`eth_getBlockByNumber` for future/unknown heights:** A numeric block number above the node's safe latest watermark (non-existent/future height) returns `result: null`, matching the Ethereum JSON-RPC spec — not a JSON-RPC error. (Previously this returned error `-32000`, e.g. `requested height 1000 is not yet available; safe latest is 128`.) + + +- **`eth_getTransactionByBlockHashAndIndex` / `eth_getTransactionByBlockNumberAndIndex` for out-of-range index:** (v6.5) An index beyond the number of transactions in the block returns `result: null`, matching the Ethereum JSON-RPC spec — instead of erroring or returning unexpected data as in earlier versions. - **Filter limits:** Open-ended log queries return up to 10,000 logs; closed-range queries cover up to 2,000 blocks. -- **Deprecated:** `sei_*` and `sei2_*` namespaced methods are deprecated — use standard `eth_*` and `debug_*` methods. +- **Deprecated + gated:** `sei_*` and `sei2_*` namespaced methods are deprecated and scheduled for removal — migrate to standard `eth_*` and `debug_*` methods. They are gated by the `[evm].enabled_legacy_sei_apis` list in `app.toml` (env/flag `evm.enabled_legacy_sei_apis`). Only methods named in that list are served on the EVM HTTP endpoint. `seid init` / defaults enable **only three helpers**: `sei_getSeiAddress`, `sei_getEVMAddress`, `sei_getCosmosTx`. Every other gated `sei_*` / `sei2_*` method — and any unknown `sei_*` name (fails closed) — returns HTTP 200 with a JSON-RPC error (`code: -32601`, `data: "legacy_sei_deprecated"`) unless explicitly allowlisted. To enable a legacy method, add its exact name to `enabled_legacy_sei_apis` under `[evm]`. Successful allowlisted responses set the `Sei-Legacy-RPC-Deprecation` HTTP header (JSON body unchanged) as a deprecation signal. +- **`sei2_*` namespace:** Block-related methods (`sei2_getBlockByHash`, `sei2_getBlockByNumber`, `sei2_getBlockReceipts`, `sei2_getBlockTransactionCountByHash`, `sei2_getBlockTransactionCountByNumber`, plus `sei2_getBlockByHashExcludeTraceFail` / `sei2_getBlockByNumberExcludeTraceFail`) mirror the `sei` block payloads but include bank transfers (HTTP only). There is no `sei2` transaction or filter API. These are gated by the same allowlist and off by default. +- **Removed block-trace endpoints:** `sei_traceBlockByNumberExcludeTraceFail` and `sei_traceBlockByHashExcludeTraceFail` have been **removed** entirely — they are no longer served and are no longer valid entries in `enabled_legacy_sei_apis`. Use the standard `debug_traceBlockByNumber` / `debug_traceBlockByHash` methods instead. - **`debug_traceTransaction`:** Only available if the RPC node exposes debug methods. If unavailable, fall back to standard RPC queries. + +- **`debug_traceTransactionProfile`:** Sei-specific debug method (`debug_traceTransactionProfile(hash, config)`, only when the node exposes debug methods). Returns the standard transaction trace plus a `profile` object for latency analysis. Params: transaction hash and a trace config object (same shape as `debug_traceTransaction`, e.g. `{"timeout":"60s"}`; pass `{}` for defaults). Response shape: + - `trace` — the normal trace result (identical to `debug_traceTransaction`). + - `profile.totalNanos` — total time spent handling the request. + - `profile.historicalDbLookupNanos` — time spent in historical DB lookups (sum of `get`/`has`/`iterator`/`iteratorNext` store access nanos). + - `profile.otherNanos` — `totalNanos` minus historical-lookup and execution time. + - `profile.phases` — per-phase timings: `lookupTransactionNanos`, `loadBlockNanos`, `replayHistoricalTxsNanos`, `buildBlockContextNanos`, `prepareTxNanos`, `executionNanos`, `traceResultNanos`. + - `profile.store.modules.` — per-module store access with `stats` (per-op `count`/`totalNanos` for `get`/`has`/`set`/`delete`/`iterator`/`iteratorNext`/`iteratorValue`) and `iterators` (each with `start`, `end`, `ascending`, sampled `keys`, `nextCount`, `totalNanos`, `truncated`). Per-tx sampling is capped (16 iterators, 64 keys each) to bound the response size; overflow sets `truncated: true`. + + ```bash + curl -s "$EVM_RPC" \ + -H 'Content-Type: application/json' \ + --data '{"jsonrpc":"2.0","method":"debug_traceTransactionProfile","params":["0x...",{"timeout":"60s"}],"id":1}' | jq + ``` + + To batch this across a block range and produce aggregate reports, use the `seidb trace-profile-report` command (writes `raw_profiles.jsonl` + `summary.json`): + + ```bash + seidb trace-profile-report \ + --endpoint http://localhost:8545 \ + --start-block 100 --end-block 200 \ + -o ./trace-report \ + -c 4 \ + --trace-config-json '{}' \ + --max-transactions 0 + ``` + + Flags: `--endpoint` (RPC URL, required), `--start-block`/`--end-block` (positive block numbers, required), `--output-dir`/`-o` (output directory, required), `--concurrency`/`-c` (concurrent requests, default 4), `--trace-config-json` (JSON trace config, default `{}`), `--max-transactions` (optional cap, `0` = no cap). + + +### seidb FlatKV Import (memiavl → FlatKV migration) + +Offline tooling for migrating the EVM module's SC-layer data from memiavl into FlatKV storage. Run these only while `seid` is stopped (they open the node's data directory directly). + +#### `seidb memiavl-latest-version` + +Read-only. Prints the latest committed memiavl version of a stopped node. Use it to pick a single, uniform import height across a multi-validator cluster. + +```bash +seidb memiavl-latest-version --data-dir /root/.sei/data +``` + +Flags: +- `--home` — Sei home directory. Defaults to `$HOME/.sei`. +- `--data-dir` — Sei data directory or home directory. If the basename is `data`, its parent is used as home. + +#### `seidb import-flatkv-from-memiavl` + +Imports selected memiavl modules into FlatKV. Initial production scope is **evm-only** — any other module name is rejected. This is a restore-style import: it **resets FlatKV** before loading the imported rows and refuses to run over existing committed FlatKV data unless `--force` is supplied. + +```bash +seidb import-flatkv-from-memiavl --modules=evm --data-dir /root/.sei/data --height + +# Overwrite an existing committed FlatKV store +seidb import-flatkv-from-memiavl --modules=evm --data-dir /root/.sei/data --height --force +``` + +Flags: +- `--home` — Sei home directory. Defaults to `$HOME/.sei`. +- `--data-dir` — Sei data directory or home directory. If the basename is `data`, its parent is used as home. +- `--modules` — Comma-separated module names to import. Default `evm`; only `evm` is supported in the initial scope. +- `--height` — memiavl version to import. `0` means latest. +- `--force` — overwrite existing committed FlatKV data. + +**Import must run at the memiavl latest height.** If `--height H` is below the latest committed memiavl version, the command refuses to run: a subsequent `GIGA_STORAGE` startup would call `CompositeCommitStore.reconcileVersions` and silently roll memiavl back to `H`, truncating every cosmos block in `(H, latest]`. To import at an older `H`, first roll memiavl back to `H` yourself (`seid rollback`), then re-run the import. A height above latest is also rejected. Use `seidb memiavl-latest-version` to determine the correct height. On failure the import aborts without finalizing, so FlatKV is left at its pre-import version and can be retried without `--force`. + +#### Post-import startup constraints (MigrateEVM V0 → V1) + +The import moves only SC-layer EVM data into FlatKV; SS history for EVM stays in the existing combined cosmos pebbledb. Across the import boundary, keep the following in `app.toml` to avoid AppHash / startup panics: +- `evm-ss-split = false` — otherwise rootmulti panics with `EVM SS directory ... does not exist but Cosmos SS already has history`. +- `sc-enable-lattice-hash = false` — turning it on would fold the FlatKV LtHash into the AppHash and fail the replay check (`state.AppHash does not match AppHash after replay`). `dual_write` does not require lattice hash; only `split_write` does. + +See `sei-db/state_db/sc/migration/OPERATIONS.md` for the full operational roadmap. + + +#### `seidb migrate-evm-status` + +Reports the on-disk FlatKV EVM migration state of a FlatKV directory as JSON. Use it to poll for migration completion from the host during the in-flight FlatKV EVM migration (`sc-write-mode = migrate_evm`), without needing a custom RPC handler or grepping node logs. Reads the `migration-version` and `migration-boundary` keys from the FlatKV migration store via a read-only clone (hardlink-clones the latest snapshot + copies the WAL into a temp dir), so it can run concurrently with a live node. + +```bash +# Report status against the latest available FlatKV version +seidb migrate-evm-status --db-dir /root/.sei/data/state_commit/flatkv + +# Report status at a specific FlatKV version +seidb migrate-evm-status --db-dir /root/.sei/data/state_commit/flatkv --height +``` + +Flags: +- `--db-dir` / `-d` — FlatKV database directory (required). +- `--height` — FlatKV target version; `0` (default) selects the latest available version. + +JSON output fields: +- `version_at` — the FlatKV version the status was read at. +- `migration_version` — the on-disk migration version (`0` = MemiavlOnly / not yet complete, `1` = MigrateEVM complete). +- `migrate_evm_complete` — `true` once the EVM migration has finished (`migration_version >= 1`). +- `boundary_present` — `true` while the migration is in flight (boundary cursor still present). +- `boundary_hex` / `version_raw_hex` — raw hex of the boundary and version keys when present. + +Example poll loop against a validator's FlatKV dir: + +```bash +seidb migrate-evm-status --db-dir /root/.sei/data/state_commit/flatkv \ + | jq -r '.migrate_evm_complete' +``` + + + +#### `seidb evm-logical-digest` + +Computes a **backend-independent digest of EVM logical state** (account / code / storage canonical buckets) so a memIAVL node and a FlatKV node can be compared at the same chain height. It strips each value's serialization-version + per-key `blockHeight` header and digests only the logical payload (storage word / bytecode / balance+nonce+codeHash), so a freshly migrated FlatKV node and a memIAVL-only node produce the same digest when the underlying EVM state matches. Run only while `seid` is stopped (FlatKV opens a read-only clone; memIAVL reads snapshot/`current` directly). + +The per-bucket accumulator is an order-independent XOR of `sha256(len(key)||key||len(val)||val)`, so iteration order does not matter. Output includes per-bucket `bucket_digest` lines and one `FINAL_DIGEST account+code+storage+legacy ... digest=` line for cross-backend comparison. FlatKV's internal `migration/migration-version` marker (which a memIAVL-only node never owns) is omitted from the final comparison. + +```bash +# FlatKV digest at a height (WAL-replays to it) +seidb evm-logical-digest --backend flatkv \ + --db-dir /root/.sei/data/state_commit/flatkv --height 213200000 + +# memIAVL digest at the same height (0 = current symlink), default semantic mode. +# memIAVL does NOT replay WAL here; it opens snapshot-/evm or current/evm. +seidb evm-logical-digest --backend memiavl \ + --db-dir /root/.sei/data/state_commit/memiavl --height 213200000 + +# Translator-based memIAVL digest (runs leaves through flatkv.ImportTranslator) +seidb evm-logical-digest --backend memiavl \ + --db-dir /root/.sei/data/state_commit/memiavl --height 213200000 \ + --memiavl-normalization translator + +# Inspect one bucket instead of the global digest (e.g. shard storage rows by next 2 bytes) +seidb evm-logical-digest --backend flatkv -d /root/.sei/data/state_commit/flatkv --height 213200000 \ + --inspect-bucket storage --key-prefix 03 --shard-next-bytes 2 + +# List account rows with backend version metadata +seidb evm-logical-digest --backend flatkv -d /root/.sei/data/state_commit/flatkv --height 213200000 \ + --inspect-bucket account --list --list-limit 50 --details + +# Hunt the single diverging entry: when two bucket_digest values differ by exactly one +# row, XOR those two 32-byte hex values and pass the result; matching rows print as FOUND-HASH. +seidb evm-logical-digest --backend flatkv -d /root/.sei/data/state_commit/flatkv --height 213200000 \ + --find-hash <32-byte-hex> +``` + +Flags: +- `--backend` — backend to read: `flatkv` | `memiavl` (required). +- `--db-dir` / `-d` — for `flatkv`, the FlatKV data dir; for `memiavl`, the memiavl root dir (contains `current/` and `snapshot-*`) (required). +- `--height` — target version. `flatkv` WAL-replays to it; `memiavl` resolves `snapshot-/evm` (`0` = `current` symlink). +- `--memiavl-normalization` — memiavl digest/inspect normalization: `semantic` (default; independent raw EVM key/value decoder, does not call `flatkv.ImportTranslator`) | `independent` (alias of `semantic`) | `translator` (routes each leaf through `flatkv.ImportTranslator`, i.e. the current migration mapping). +- `--inspect-bucket` — inspect one normalized bucket (`account` | `code` | `storage` | `legacy`) instead of printing the global digest. +- `--key-offset` — inspect mode: byte offset into the physical key before applying `--key-prefix` / sharding (non-negative). +- `--key-prefix` — inspect mode: hex prefix, relative to `--key-offset`, used to filter physical keys. +- `--shard-next-bytes` — inspect mode: group matching keys by this many bytes after `--key-prefix` (non-negative). +- `--list` — inspect mode: list matching key / logical-value pairs instead of per-shard `bucket_digest` values. +- `--list-limit` — inspect mode: maximum pairs to print with `--list` (default `1000`; `<=0` means unlimited). +- `--details` — inspect list mode: include backend-specific version metadata (e.g. `block_height` / `leaf_version`). +- `--find-hash` — optional 32-byte hex per-entry hash to hunt for; prints every entry whose `sha256(len(key)||key||len(val)||val)` matches (as `FOUND-HASH`). + + ## Agent Workflow 1. Classify the task: install · wallet · read query · payload generation · pointer lookup · tx lookup · transaction submission · raw JSON-RPC. diff --git a/skill/references/contracts/common-errors.md b/skill/references/contracts/common-errors.md index 348de6e..12c4a95 100644 --- a/skill/references/contracts/common-errors.md +++ b/skill/references/contracts/common-errors.md @@ -202,6 +202,47 @@ forge verify-contract \ Existing `ibc/...` balances are **not** stuck: they remain in the bank module and still transfer between Sei accounts and through their ERC-20 pointers. Only the route off the chain is closed. See [../ecosystem/ibc-bridging.md](../ecosystem/ibc-bridging.md). + +--- + +## Pagination / Query Errors + +Sei enforces hard caps on pagination parameters for module (gRPC/REST/`seid q ...`) queries. Requests exceeding these bounds are rejected with an `InvalidArgument` error. + +| Bound | Value | Meaning | +|-------|-------|---------| +| `MaxLimit` | `1000` | Max results per page | +| `MaxOffset` | `10000` | Max offset for offset-based paging | +| `MaxScanLimit` | `10000` | Max store entries scanned past the page end | + +### `limit N exceeds maximum allowed limit 1000` +**Cause**: `pagination.limit` set above `MaxLimit` (1000). `limit` is no longer effectively unbounded. +**Fix**: Set `pagination.limit` to `1000` or less and page through results using the returned `next_key`. +```bash +# Wrong: limit too large +seid q bank total --page-limit 5000 --node https://rpc.sei-apis.com + +# Correct: page in chunks of <= 1000 and follow next_key +seid q bank total --page-limit 1000 --node https://rpc.sei-apis.com +``` + +### `offset N exceeds maximum allowed offset 10000` +**Cause**: `pagination.offset` set above `MaxOffset` (10000). Deep offset paging is no longer allowed. +**Fix**: Do not skip past 10000 entries with `offset`. Use **key-based pagination** instead — follow the `next_key` (`pagination.key`) returned by each page rather than incrementing `offset`. + +### `scanned more than 10000 entries ... use key-based pagination instead` +**Cause**: Offset-based (lazy) pagination walked more than `MaxScanLimit` (10000) store entries without filling the page — typically a sparse filter, a large offset, or `count_total=true` over a large store. +**Fix**: Switch to **key-based pagination** (`pagination.key` / follow `next_key`), narrow the query with a more specific key prefix, and avoid `count_total=true` on large datasets. +```bash +# Wrong: forces a full-store scan to count everything +seid q bank total --count-total --node https://rpc.sei-apis.com + +# Correct: page with next_key, no total count +seid q bank total --page-limit 1000 --page-key --node https://rpc.sei-apis.com +``` + +> **Note**: `count_total` is no longer auto-enabled when `limit` is omitted. If you need `total` populated, you must explicitly set `pagination.count_total=true` (subject to the scan-limit cap above); otherwise `total` is `0`. + ### `IBC denom not recognized` **Cause**: A consumer doesn't know what an `ibc/HASH` denom represents. These denoms still exist and are still valid on Sei. **Fix**: Query the denom trace — this is a local query and still works: diff --git a/skill/references/contracts/tokens.md b/skill/references/contracts/tokens.md index e7208b1..4413b58 100644 --- a/skill/references/contracts/tokens.md +++ b/skill/references/contracts/tokens.md @@ -113,6 +113,23 @@ Native tokens (not smart contracts) that live in the Cosmos bank module: | IBC USDC | `ibc/...` | Arrived via IBC from Noble, etc.; IBC is now closed, so no new inbound and no route out | | Factory tokens | `factory//` | Created via TokenFactory | + +### Listing a Creator's TokenFactory Denoms + +Query all denoms created by an address via CLI, gRPC, or REST. Results are **paginated** (a default page limit applies) using the standard Cosmos SDK pagination flags: + +```bash +seid query tokenfactory denoms-from-creator \ + --page 1 --limit 100 --count-total +``` + +Supported pagination flags: `--page`, `--limit`, `--offset`, `--page-key`, `--count-total`, `--reverse`. + +- gRPC/REST `DenomsFromCreator` accepts a `pagination` (`PageRequest`) and returns a `pagination` (`PageResponse`) alongside `denoms`. To page through large result sets, pass the returned `next_key` back as `--page-key` (or `PageRequest.Key`). +- REST endpoint accepts pagination query parameters (e.g. `?pagination.limit=100&pagination.key=...`). +- The same standard pagination caps apply as documented in [common-errors.md](../common-errors.md) — requesting more than the max page limit is clamped. +- The CosmWasm query path (`GetAllDenomsFromCreator`) is unbounded and returns **all** denoms in a single call, relying on gas metering rather than page limits. + ## Token Visibility by Wallet Type | Token type | EVM wallets (MetaMask) | Cosmos wallets (Compass) | diff --git a/skill/references/ecosystem/node-operations.md b/skill/references/ecosystem/node-operations.md index 4c93172..040be93 100644 --- a/skill/references/ecosystem/node-operations.md +++ b/skill/references/ecosystem/node-operations.md @@ -158,7 +158,7 @@ ss-prune-interval = 600 SeiDB has two layers: **State Commit (SC)** — a memiavl Merkle tree that holds Cosmos module state and computes the app hash — and **State Store (SS)** — versioned raw key/values for historical queries (`ss-enable = true` is required for any RPC node). - **RocksDB SS backend** (optional): faster for iteration-heavy work (`debug_trace*`, large archive queries). Build with `make build-rocksdb && make install-rocksdb`, then set `ss-backend = "rocksdb"`. RocksDB RPC nodes must state-sync on first start. -- **Giga SS Store** (optional, RPC nodes): splits the **State Store** so EVM state lives in its own SS DB. Controlled by a single bool — `evm-ss-split = true` (Sei v6.5+; older releases used per-key `evm-ss-write-mode`/`evm-ss-read-mode`). Requires a **fresh state sync** — flipping it on a node with existing data fails startup safety checks. **SC config is left untouched.** See the [Giga SS Store Migration Guide](https://docs.sei.io/node/giga-storage-migration). +- **Giga SS Store** (optional, RPC nodes): splits the **State Store** so EVM state lives in its own SS DB. As of Sei v6.5+ this is controlled by a single consolidated `evm-ss-mode` config field (the previously separate SS write/read mode settings — `evm-ss-write-mode`/`evm-ss-read-mode` — have been merged into this one field). Requires a **fresh state sync** — flipping it on a node with existing data fails startup safety checks. **SC config is left untouched.** See the [Giga SS Store Migration Guide](https://docs.sei.io/node/giga-storage-migration). - **Giga Storage (SC FlatKV routing)** is a *separate*, broader option that routes EVM **State Commit** data through FlatKV, controlled by the single `sc-write-mode` key: ```toml @@ -172,11 +172,290 @@ SeiDB has two layers: **State Commit (SC)** — a memiavl Merkle tree that holds sc-keys-to-migrate-per-block = 1024 ``` - The migration is staged: `migrate_evm` drains EVM data in the background and settles at `evm_migrated`; later modes migrate the remaining modules. + The migration is staged: `migrate_evm` drains EVM data in the background and settles at `evm_migrated`; later modes migrate the remaining modules. `sc-keys-to-migrate-per-block` must be `> 0` (defaults to `1024` when unset); lower it to spread the in-flight migration across more blocks. + + Poll migration completion per validator with the `seidb migrate-evm-status` subcommand, which reads the on-disk FlatKV migration state and prints it as JSON: + + ```bash + seid migrate-evm-status --db-dir /root/.sei/data/state_commit/flatkv [--height ] + ``` + + | Flag | Default | Purpose | + |---|---|---| + | `--db-dir` / `-d` | *(required)* | FlatKV database directory | + | `--height` | `0` (latest) | FlatKV target version; `0` selects the latest available version | + + Output fields: `version_at`, `migration_version`, `migrate_evm_complete` (true once `migration_version >= 1`), and `boundary_present` (true while the migration is still in flight). It reads through a read-only temp clone of the snapshot + WAL, so it is safe to run against a live node — operators/test drivers poll it until `migrate_evm_complete: true` on every validator before flipping `sc-write-mode` to `evm_migrated`. - **Giga Executor** (`[giga_executor] enabled`) is a *separate* feature — an evmone-based EVM interpreter for throughput. Don't conflate it with Giga Storage. > Minimum gas price, block gas limit, and SSTORE/storage gas are governance-adjustable — confirm live values at https://docs.sei.io/evm/differences-with-ethereum, and set `minimum-gas-prices` at or above the mainnet floor (`0usei` is local-dev only). + + +### seidb `trace-profile-report` (batch transaction profiling) + +Batch-runs the `debug_traceTransactionProfile` JSON-RPC method across a block range and writes two files to the output directory: `raw_profiles.jsonl` (one JSON record per traced transaction) and `summary.json` (aggregate stats — average/p50/p95 total & historical-DB-lookup latencies, per-phase totals, per-module store-access totals, top transactions, and top blocks). + +`debug_traceTransactionProfile(hash, config)` returns `{ trace, profile }`, where `profile` includes `totalNanos`, `historicalDbLookupNanos`, `otherNanos`, per-phase timings (`lookupTransactionNanos`, `loadBlockNanos`, `replayHistoricalTxsNanos`, `buildBlockContextNanos`, `prepareTxNanos`, `executionNanos`, `traceResultNanos`), and per-module store access stats/iterators. Enable it as a `debug_*` method on the EVM JSON-RPC endpoint (`8545`). + +```bash +seid trace-profile-report \ + --endpoint http://localhost:8545 \ + --start-block 1000000 \ + --end-block 1000100 \ + --output-dir ./trace-report \ + --concurrency 4 \ + --trace-config-json '{}' \ + --max-transactions 0 +``` + +Flags: + +| Flag | Default | Purpose | +|---|---|---| +| `--endpoint` | *(required)* | RPC endpoint, e.g. `http://localhost:8545` | +| `--start-block` | *(required, >0)* | Starting block number | +| `--end-block` | *(required, >=start)* | Ending block number | +| `--output-dir` / `-o` | *(required)* | Directory for `raw_profiles.jsonl` and `summary.json` | +| `--concurrency` / `-c` | `4` | Concurrent `traceTransactionProfile` requests | +| `--trace-config-json` | `{}` | JSON object passed as the trace config | +| `--max-transactions` | `0` (no cap) | Optional cap on the number of transactions processed | + +> Offline analysis tool — point it at an archive/RPC node with `debug_*` enabled. Each request has a 120s client timeout. The command collects tx hashes via `eth_getBlockByNumber` across the range, then traces them concurrently. + + + + +### seidb FlatKV analysis tooling + +When a node runs Giga Storage (SC FlatKV routing — see `sc-write-mode` above), EVM State Commit data lives in a FlatKV store (a sibling `flatkv/` directory next to the memIAVL `committer.db`). Two seidb subcommands inspect it. Both operate on a **read-only temp clone** of the selected snapshot + changelog (snapshot files are hardlinked, changelog files are byte-copied) so they never contend for the live FlatKV writer lock — safe to run against a running node. + +#### `dump-flatkv` — dump physical (key, value) pairs per bucket + +Iterates every physical FlatKV (key, value) pair and writes one file per bucket (`account`, `code`, `storage`, `legacy`) into the output directory, formatted identically to `dump-iavl` so the same diff tooling works on both. Each file starts with a `Bucket at version ` header followed by `Key: , Value: ` lines. Physical keys are emitted verbatim (with their `/` + type-prefix header). Internal metadata rows are excluded. + +```bash +seid dump-flatkv \ + --db-dir /path/to/data/flatkv \ + --output-dir ./flatkv-dump \ + --height 0 \ + --bucket storage +``` + +| Flag | Default | Purpose | +|---|---|---| +| `--db-dir` / `-d` | *(required)* | FlatKV database directory | +| `--output-dir` / `-o` | *(required)* | Output directory (one file per bucket) | +| `--height` | `0` (latest) | FlatKV target version; `0` selects the latest available version | +| `--bucket` / `-b` | *(all)* | Restrict dump to a single bucket: `account`, `code`, `storage`, or `legacy` | + +Bucket classification of physical EVM keys: nonce + codehash → `account`, code → `code`, storage → `storage`; any non-EVM module (or unrecognized EVM type prefix) → `legacy`. + +#### `state-size --flatkv-dir` — fold FlatKV into state-size analysis + +The existing `state-size` command now scans FlatKV alongside memIAVL. Supply `--flatkv-dir` explicitly, or leave it unset to auto-detect a sibling `flatkv/` directory next to `--db-dir` (e.g. `/data/committer.db` → `/data/flatkv`). FlatKV is only scanned when `--module` is empty or `evm` (FlatKV holds only EVM keys). FlatKV analysis is strictly additive — any FlatKV open/scan failure is logged and skipped, leaving the memIAVL analysis intact. + +```bash +seid state-size \ + --db-dir /path/to/data/committer.db \ + --flatkv-dir /path/to/data/flatkv \ + --height 0 +``` + +Relevant flags: + +| Flag | Default | Purpose | +|---|---|---| +| `--db-dir` / `-d` | *(required)* | memIAVL database directory | +| `--flatkv-dir` | auto-detect `/../flatkv` | FlatKV data directory (empty + no sibling ⇒ FlatKV skipped) | +| `--module` / `-m` | *(all)* | Module to analyze; FlatKV scanned only when empty or `evm` | +| `--height` | `0` (latest) | Block height / version | +| `--export-dynamodb` | `false` | Export to DynamoDB instead of printing; FlatKV is added as a `flatkv` module row in the same batch | + +Console output notes (as of this change): modules are printed alphabetically for diffable runs, the prefix breakdown is marshaled as a full map (key/value/total bytes + key count per prefix byte), and scan progress logs every **10M** keys (previously 1M). The FlatKV section reports total keys/size, a per-DB breakdown (`account`/`code`/`storage`/`legacy`), and the top EVM contracts by storage size. + +> Both commands are offline analysis tools. `--height 0` uses the latest snapshot; an explicit height selects the newest snapshot at or below that version. + + + + + +#### `evm-logical-digest` — compare memIAVL vs FlatKV EVM state at the same height + +Computes a **backend-independent digest of EVM logical state** (account / code / storage canonical buckets) so a memIAVL node and a FlatKV node can be verified to hold identical EVM state at the same chain height during/after a Giga Storage migration. This is the tool operators use to prove a migrated FlatKV node matches a memiavl-only truth node. + +Why "logical": every FlatKV value embeds a per-key `blockHeight` stamp (the height the key was last written/migrated), so a byte-for-byte physical digest diverges even when the underlying EVM state is identical. `evm-logical-digest` strips the serialization-version + blockHeight header on both sides and digests only the logical payload (storage word / bytecode / balance+nonce+codehash). The per-bucket accumulator is an XOR of `sha256(len(key)||key||len(val)||val)`, so it is **order-independent** — pebble global order vs leaf-index scan order does not matter. + +```bash +# FlatKV digest at a height (WAL-replays to it). +seid evm-logical-digest --backend flatkv \ + --db-dir /root/.sei/data/state_commit/flatkv --height 213200000 + +# memIAVL digest at the same height (default semantic normalization). +# memiavl does NOT replay WAL here; it opens snapshot-/evm (0 = current/evm). +seid evm-logical-digest --backend memiavl \ + --db-dir /root/.sei/data/state_commit/memiavl --height 213200000 +``` + +Both runs print per-bucket `bucket_digest=` lines and one `FINAL_DIGEST account+code+storage+legacy count=... digest=...` line. The nodes match when the two `FINAL_DIGEST` lines are equal. + +> **Marker adjustment:** FlatKV owns a FlatKV-only `migration/migration-version` row that a memiavl-only node never has. The FlatKV run automatically **omits that marker from the legacy bucket** in its final result (printed as `flatkv_marker_adjustment: ...`) so the comparison is apples-to-apples against memiavl-only output. + +| Flag | Default | Purpose | +|---|---|---| +| `--backend` | *(required)* | `flatkv` or `memiavl` | +| `--db-dir` / `-d` | *(required)* | flatkv: the flatkv data dir. memiavl: the memiavl root dir (contains `current/` and `snapshot-*`) | +| `--height` | `0` | Target version. flatkv WAL-replays to it; memiavl resolves `snapshot-/evm` (`0` = `current` symlink) | +| `--memiavl-normalization` | `semantic` | memiavl normalization: `semantic`/`independent` (raw EVM key/value decoder, does **not** call `flatkv.ImportTranslator`) or `translator` (feeds each leaf through `flatkv.ImportTranslator` = current migration mapping) | +| `--inspect-bucket` | *(unset)* | Inspect one normalized bucket (`account`\|`code`\|`storage`\|`legacy`) instead of printing the global digest | +| `--key-offset` | `0` | Inspect mode: byte offset into the physical key before applying `--key-prefix` / sharding | +| `--key-prefix` | *(empty)* | Inspect mode: hex prefix (relative to `--key-offset`) used to filter physical keys | +| `--shard-next-bytes` | `0` | Inspect mode: group matching keys by this many bytes after `--key-prefix` | +| `--list` | `false` | Inspect mode: list matching key/logical-value pairs instead of per-shard `bucket_digest` values | +| `--list-limit` | `1000` | Inspect list mode: max pairs to print; `<=0` means unlimited | +| `--details` | `false` | Inspect list mode: include backend-specific version metadata (e.g. `block_height=` / `leaf_version=`) | +| `--find-hash` | *(unset)* | 32-byte hex per-entry hash to hunt for; prints every matching entry as `FOUND-HASH ...` | + +**Translator mode** proves FlatKV state matches the current migration mapping (`flatkv.ImportTranslator`) and is useful when debugging the importer: + +```bash +seid evm-logical-digest --backend memiavl \ + --db-dir /root/.sei/data/state_commit/memiavl --height 213200000 \ + --memiavl-normalization translator +``` + +**Inspect a single bucket** instead of the global digest (e.g. list storage rows under a key prefix, sharded by the next 2 bytes): + +```bash +seid evm-logical-digest --backend flatkv -d --height H \ + --inspect-bucket storage --key-prefix 03 --shard-next-bytes 2 + +seid evm-logical-digest --backend flatkv -d --height H \ + --inspect-bucket account --list --list-limit 50 --details +``` + +**Hunt the single diverging entry** between two runs: when two `bucket_digest` values differ by exactly one row, XOR those two 32-byte hex values and pass the result — every matching row prints as `FOUND-HASH`: + +```bash +seid evm-logical-digest --backend flatkv -d --height H --find-hash <32-byte-hex> +``` + +> Offline analysis tool. Like the other seidb FlatKV tools it reads a read-only clone, so it is safe against a live node. The digest is XOR-order-independent, so leaf/iteration order never affects correctness. + + + + + +### `seid tendermint gen-autobahn-config` (Autobahn/Giga consensus config) + +Generates an Autobahn consensus config file from one or more validator directories. + +```bash +seid tendermint gen-autobahn-config ... \ + --output autobahn.json \ + --persistent-state-dir data/autobahn +``` + +| Flag | Default | Purpose | +|---|---|---| +| `--output` / `-o` | *(required)* | Output file path for the autobahn config | +| `--persistent-state-dir` | `data/autobahn` | Directory to persist autobahn **consensus and data WALs** across restarts. Relative paths resolve against the node's `--home` dir at load time; absolute paths pass through unchanged. Pass `--persistent-state-dir=` (empty) to disable persistence and run **in-memory only**. | + +> **Persistence is on by default.** As of this change, Autobahn persists both the consensus and data-layer WALs to disk (a shared on-disk root under `data/autobahn` with distinct subdirectories per layer — `inner`/`blocks`/`commitqcs` for consensus, `globalblocks`/`fullcommitqcs` for data). Previously the data WAL ran in-memory only with no cross-restart persistence. To opt back into the old in-memory-only behavior, generate the config with `--persistent-state-dir=` (empty value). + + + +### Offline EVM memiavl→FlatKV migration (`import-flatkv-from-memiavl`) + +Moving the EVM module's State Commit data from memIAVL into FlatKV can be done offline with two seidb subcommands. This is the operator workflow behind the staged Giga Storage `migrate_evm` → `evm_migrated` transition when performed as a coordinated offline import. + +#### `memiavl-latest-version` — read the latest committed memiavl version + +Read-only. Prints the latest committed memiavl version of a **stopped** node. Use it to pick a single, uniform import height across a multi-validator cluster (read each validator after stopping `seid`, then take the minimum). + +```bash +seid memiavl-latest-version --data-dir /root/.sei/data +# or: seid memiavl-latest-version --home /root/.sei +``` + +| Flag | Purpose | +|---|---| +| `--home` | Sei home directory. Defaults to `$HOME/.sei` | +| `--data-dir` | Sei data directory or home directory. If the basename is `data`, its parent is used as home | + +#### `import-flatkv-from-memiavl` — import selected modules into FlatKV + +Imports selected memIAVL modules into FlatKV. **Initial scope is EVM-only** — `--modules=evm` is the only accepted value; any other module name is rejected at the CLI boundary. The import **resets FlatKV** before loading the rows and **refuses to overwrite committed FlatKV data unless `--force` is supplied**. + +```bash +# Node MUST be stopped first (pkill / systemctl stop seid). +seid import-flatkv-from-memiavl \ + --modules=evm \ + --data-dir /root/.sei/data \ + --height \ + [--force] +``` + +| Flag | Default | Purpose | +|---|---|---| +| `--home` | `$HOME/.sei` | Sei home directory | +| `--data-dir` | *(unset)* | Sei data directory or home directory (basename `data` ⇒ parent is home) | +| `--modules` | `evm` | Comma-separated module names; only `evm` is supported | +| `--height` | `0` (latest) | memiavl version to import; `0` resolves to memiavl latest | +| `--force` | `false` | Overwrite existing committed FlatKV data | + +#### Operational constraints (critical) + +1. **Import at memiavl latest height only.** The CLI refuses to import at `H < memiavl-latest`: a subsequent GIGA_STORAGE startup would call `CompositeCommitStore.reconcileVersions` and **silently roll memiavl back to `H`, truncating every cosmos block in `(H, memiavl-latest]`**. If you genuinely need a non-latest `H`, roll memiavl back to `H` yourself first (`seid rollback --home --num-blocks `) — this tool never rolls memiavl back on your behalf. Importing at `H > memiavl-latest` is also refused. + + For a multi-validator cluster, read every validator's `memiavl-latest-version` after stopping, pick the **minimum** as the uniform import height, roll any validator that committed extra blocks back to that height, then run the import on all validators at the same height. (When rolling back after a `pkill -9`, also drop the consensus WAL — `rm -rf /cs.wal /tendermint/cs.wal` — or restart panics with `last height in WAL is N, want N-1`.) + +2. **Keep `evm-ss-split = false` across the import boundary.** The import moves only **SC-layer** EVM data into FlatKV; EVM **SS** history stays in the combined cosmos pebbledb. Switching SS to split mode mid-life triggers the rootmulti startup panic `EVM SS directory ... does not exist but Cosmos SS already has history`. A split-SS transition is a separate state-sync workflow. + +3. **Keep `sc-enable-lattice-hash = false` across the import boundary.** Pre-import the chain persisted `AppHash = memiavl-only` for every block up to the import height. Turning lattice hash on now would fold the FlatKV LtHash into the AppHash, and the startup replay check would fail with `state.AppHash does not match AppHash after replay`. (`dual_write` does not require lattice hash; only `split_write` does. A real production rollout coordinates this transition via a chain upgrade at an agreed height.) + +4. **Import is one-way and abortable, not partial.** On any error the tool aborts without finalizing — FlatKV is left at its pre-import committed version, so you can re-run without `--force`. A successful import writes a FlatKV snapshot so the imported data survives restart. + +> Post-import startup: apply your Giga Storage `sc-write-mode` (e.g. `dual_write`/`evm_migrated` per your rollout) but keep `evm-ss-split = false` and `sc-enable-lattice-hash = false` as above. See `sei-db/state_db/sc/migration/OPERATIONS.md` for the full MigrateEVM (V0→V1) failure-mode catalog and recovery tooling roadmap. + + + +--- + +## Legacy `sei_*` / `sei2_*` JSON-RPC gating (`app.toml [evm]`) + +All `sei_*` and `sei2_*` JSON-RPC methods (EVM HTTP endpoint on `8545` only — not the Cosmos REST API on `1317`) are **deprecated and scheduled for removal**. New integrations should use `eth_*` / `debug_*` methods. Access to these gated methods is controlled by `[evm].enabled_legacy_sei_apis` in `app.toml`. + +```toml +[evm] +# Only methods listed here are served on the EVM HTTP endpoint. +# seid init / DefaultConfig pre-fills just these three address/Cosmos helpers: +enabled_legacy_sei_apis = ["sei_getSeiAddress", "sei_getEVMAddress", "sei_getCosmosTx"] +``` + +- **Env var / flag:** `evm.enabled_legacy_sei_apis`. +- **Default:** only the three helpers `sei_getSeiAddress`, `sei_getEVMAddress`, `sei_getCosmosTx` are enabled out of the box. Every other gated `sei_*` and `sei2_*` method must be **explicitly added to the list**, or it fails closed. +- **Behavior when a method is not allowlisted:** the node returns **HTTP 200** with a JSON-RPC error — code `-32601`, `data = "legacy_sei_deprecated"`, and a message noting the method is not enabled and is deprecated. The inner handler is never invoked. Unknown `sei_*` / `sei2_*` names (typos, future methods) also fail closed. Matching is case-insensitive against canonical method names. +- **Behavior when a method is allowlisted:** the call passes through unchanged; the response may include the HTTP header `Sei-Legacy-RPC-Deprecation` as a deprecation signal (JSON body is not mutated). +- **`sei2_*` namespace:** block-only methods that mirror `sei_*` block payloads but include bank transfers (HTTP only) — `sei2_getBlockByHash`, `sei2_getBlockByNumber`, `sei2_getBlockReceipts`, `sei2_getBlockTransactionCountByHash`, `sei2_getBlockTransactionCountByNumber`, and the `sei2_*ExcludeTraceFail` variants. There is no `sei2` transaction or filter API. They are gated by the same `enabled_legacy_sei_apis` list. +- **Removed endpoints:** `sei_traceBlockByNumberExcludeTraceFail` and `sei_traceBlockByHashExcludeTraceFail` have been **removed entirely** — they are no longer valid entries in `enabled_legacy_sei_apis` and the `sei` debug namespace is no longer registered. Use `debug_traceBlockByNumber` / `debug_traceBlockByHash` instead. + +To enable more legacy methods, add their exact names to the array, e.g.: + +```toml +[evm] +enabled_legacy_sei_apis = [ + "sei_getSeiAddress", + "sei_getEVMAddress", + "sei_getCosmosTx", + "sei_getBlockByNumber", + "sei_getLogs", + "sei2_getBlockByNumber", +] +``` + +> Because the list is an allowlist, upgrading a node keeps only the three default helpers enabled unless you explicitly re-add the legacy methods your integrations still depend on. Plan migrations to `eth_*` / `debug_*` accordingly. + --- ## Commonly Used Ports diff --git a/skill/references/ecosystem/rpc-providers.md b/skill/references/ecosystem/rpc-providers.md index 0735f30..0d8db15 100644 --- a/skill/references/ecosystem/rpc-providers.md +++ b/skill/references/ecosystem/rpc-providers.md @@ -117,9 +117,11 @@ Standard Ethereum JSON-RPC methods supported on Sei EVM: - `eth_estimateGas`, `eth_gasPrice` — note Sei's 50 gwei minimum. Sei-specific methods (varies by provider, check before relying): -- `sei_getEVMAddress` / `sei_getSeiAddress` — dual-address lookup. +- `sei_getEVMAddress` / `sei_getSeiAddress` / `sei_getCosmosTx` — the **three default-enabled** legacy helpers (dual-address + Cosmos tx lookup). - `debug_traceTransaction` — trace support depends on provider; archive providers have it. +> **Legacy `sei_*` / `sei2_*` are gated and deprecated.** As of the `[evm].enabled_legacy_sei_apis` config (`evm.enabled_legacy_sei_apis` flag/env in `app.toml`), most `sei_*` and all `sei2_*` JSON-RPC methods are **gated behind an allowlist** and **default to disabled**. Only the three helpers above (`sei_getSeiAddress`, `sei_getEVMAddress`, `sei_getCosmosTx`) are enabled out of the box on a `seid init` node. Any other `sei_*`/`sei2_*` method not explicitly listed returns a JSON-RPC error `code: -32601` with `data: "legacy_sei_deprecated"` (HTTP 200) — and unknown `sei_*` names fail closed. **Do not assume `sei_*`/`sei2_*` methods are available on a given endpoint.** All of them are deprecated and scheduled for removal; migrate to `eth_*` / `debug_*` equivalents. Successful allowlisted responses may carry an `Sei-Legacy-RPC-Deprecation` HTTP header as a deprecation signal. (The `sei2_*` namespace mirrors `sei_*` block payloads but includes bank transfers; HTTP only, no `sei2` transaction or filter API.) Note: the `sei_traceBlockByNumberExcludeTraceFail` and `sei_traceBlockByHashExcludeTraceFail` endpoints have been **removed** — use `debug_traceBlockByNumber` and `debug_traceBlockByHash` instead. They are no longer valid entries in `enabled_legacy_sei_apis`. + Methods that may **not** be supported on every endpoint: - `eth_subscribe` (WebSockets) — provider-dependent; Sei Foundation supports WS at `wss://evm-ws.sei-apis.com` (verify). - `debug_*` and `trace_*` — typically only on archive nodes / paid tiers. @@ -157,5 +159,41 @@ Self-hosting archive: see [node-operations.md](node-operations.md). Disk require - **Always set `gasPrice ≥ 50 gwei`** in calls submitted via these endpoints; under-priced txs will be rejected by the RPC or evicted from mempool. - **Cosmos and EVM RPCs are separate.** A `sei1...`-targeted operation needs the Cosmos Tendermint RPC (`rpc.sei-apis.com`); a `0x...`-targeted call needs the EVM RPC (`evm-rpc.sei-apis.com`). - **Block tags**: `latest`, `pending`, `earliest` are supported; `safe` and `finalized` are not (Sei has instant finality, so `latest` is functionally equivalent to `finalized`). + + +- **`eth_getBlockByNumber` for future / non-existent numeric heights returns `result: null`** (per the Ethereum JSON-RPC spec), not a JSON-RPC error. Earlier builds returned error `-32000` (e.g. `requested height 1000 is not yet available; safe latest is 128`); current builds map a numeric block number above the node's safe latest watermark to `null`. Handle a `null` block result rather than expecting an error for out-of-range numeric heights. + + +- **`eth_getTransactionByBlockHashAndIndex` and `eth_getTransactionByBlockNumberAndIndex` return `result: null` for an out-of-range index** (as of v6.5, sei-chain [#3367](https://github.com/sei-protocol/sei-chain/pull/3367)). Earlier builds could error or return unexpected data when the transaction index exceeded the number of transactions in the block; current builds return `null`. Handle a `null` result rather than expecting an error when probing indices at or beyond the block's transaction count. +- **`eth_getProof` now works across more node/store configurations.** Proof lookup unwraps additional KVStore wrappers (cachekv, Giga cache, tracekv, and prefix stores) to reach any proof-capable queryable store, so it succeeds beyond just classic IAVL nodes. Older builds returned error `-32000 "cannot find EVM IAVL store"` on non-IAVL backends; current builds resolve any proof-capable queryable KV store (classic IAVL, store/v2 memiavl, etc.). +- **`eth_getProof` storage keys must be hex-encoded, and are capped at 1024 per request.** Each storage key in the `storageKeys` array must be a valid hex string (e.g. `"0x0000...0001"`); it is decoded and left-padded to 32 bytes. Raw byte-string keys are no longer accepted — a non-hex key returns an `invalid storage key "": ...` error. Requests with more than `MaxStorageKeysPerProof` (1024) keys are rejected with `too many storage keys: got , max 1024`. Example: `eth_getProof(address, ["0x0000000000000000000000000000000000000000000000000000000000000001"], blockTag)`. + + +## Cosmos pagination hard caps (`PageRequest`) + +Cosmos REST/LCD (`rest.sei-apis.com`), gRPC (`grpc.sei-apis.com`), and Tendermint-RPC-backed queries that accept a `PageRequest` (`pagination.limit`, `pagination.offset`, `pagination.key`, `pagination.count_total`, `pagination.reverse`) now enforce **hard caps**. Requests that exceed them fail with a gRPC `InvalidArgument` error (HTTP 400 over REST) — they are *not* silently clamped. + +| Cap | Value | Field | Error on exceed | +|---|---|---|---| +| `MaxLimit` | **1000** (per page) | `pagination.limit` | `limit exceeds maximum allowed limit 1000` | +| `MaxOffset` | **10000** | `pagination.offset` | `offset exceeds maximum allowed offset 10000` | +| `MaxScanLimit` | **10000** | (internal store-scan cap) | `scanned more than 10000 entries ...; use key-based pagination instead` | + +Key behavior changes (breaking): + +- **`limit` is capped at 1000 per page.** Previously `MaxLimit` was effectively unbounded (`math.MaxUint64`). A `pagination.limit` above 1000 is now **rejected**, not truncated. To retrieve a full dataset, page through it with `pagination.key` set to the previous response's `next_key`, using `limit=1000` each request — do not ask for everything in one call. +- **`offset` is capped at 10000.** Deep offset-based paging is no longer possible; switch to key-based (cursor) pagination via `pagination.key` for anything beyond the first ~10k records. +- **Offset-based (lazy) pagination caps its store scan.** For sparse filters or `count_total`, the query walks at most `MaxScanLimit` (10000) entries past the page start/end. Exceeding this returns `InvalidArgument` and the `next_key` may be `nil` even when more results exist. Use **key-based pagination** for reliable traversal of sparse or large datasets. +- **`count_total` is no longer auto-enabled.** Previously, omitting `limit` (or `limit=0`) implicitly counted all records and populated `pagination.total`. Now `total` is **`0` unless you explicitly set `pagination.count_total=true`**. Code that relied on a populated `total` from a default/empty page request will now read `0` — add `count_total=true` (and expect the scan cost + `MaxScanLimit` cap it incurs). + +Example — REST query paging by key at the max page size, explicitly counting totals: + +```bash +curl "https://rest.sei-apis.com/cosmos/bank/v1beta1/supply?pagination.limit=1000&pagination.count_total=true" +# follow next_key from the response for subsequent pages: +curl "https://rest.sei-apis.com/cosmos/bank/v1beta1/supply?pagination.limit=1000&pagination.key=" +``` + +> A bare request like `.../supply` (no `count_total`) returns at most `DefaultLimit` (100) records with `total: 0` — set `pagination.count_total=true` if you actually need the count. - **Endpoint freshness**: Sei is a fast-moving project — verify endpoints monthly against [docs.sei.io/learn/rpc-providers](https://docs.sei.io/learn/rpc-providers). - For agent-driven RPC usage, see [rpc-agent-skills.md](rpc-agent-skills.md) for the canonical 17 RPC skills, retry/backoff patterns, and response-shape expectations. diff --git a/skill/references/evm/overview.md b/skill/references/evm/overview.md index 7e95fc1..c7e1740 100644 --- a/skill/references/evm/overview.md +++ b/skill/references/evm/overview.md @@ -104,8 +104,15 @@ const balance = await provider.getBalance(address); // always accurate ### State Storage (AVL vs MPT) - Sei uses a single global AVL-tree state root — there are no per-account state roots - `eth_getProof` (EIP-1186) returns proofs against the global state root, not per-account roots +- `eth_getProof` support is not limited to the classic IAVL store: the RPC unwraps known KVStore wrappers (`cachekv`, Giga cache, `tracekv`, prefix stores) to reach any proof-capable queryable store (classic IAVL, store/v2 memiavl, future proof-capable roots). This means `eth_getProof` now works across more node configurations rather than failing with `cannot find EVM IAVL store` +- `eth_getProof` storage keys must be valid **hex-encoded** values (e.g. `0x0000…0001`); they are decoded and padded to 32 bytes. Raw byte strings are no longer accepted — a malformed/non-hex key is rejected with `invalid storage key` +- `eth_getProof` enforces a maximum of **1024 storage keys per request** (`MaxStorageKeysPerProof`); exceeding this returns a `too many storage keys` error - Block hash encoding differs from Ethereum — BLOCKHASH returns Tendermint header hash, not Ethereum keccak header hash +### JSON-RPC Method Notes +- `eth_getBlockByNumber` returns `result: null` (per the Ethereum JSON-RPC spec) for numeric block heights above the node's safe latest watermark or otherwise non-existent — it no longer returns JSON-RPC error `-32000` (e.g. `requested height 1000 is not yet available; safe latest is 128`) for future/unknown numeric block numbers +- `eth_getTransactionByBlockHashAndIndex` and `eth_getTransactionByBlockNumberAndIndex` return `result: null` (per the Ethereum JSON-RPC spec) when the transaction index is out of range for the block (v6.5+) — they no longer error or return unexpected data. Handle `null` in dApp/indexer code rather than expecting an error when probing indices beyond the block's transaction count + ## What Works Unchanged - All Solidity syntax and version up to 0.8.x diff --git a/skill/references/pointers/token-factory.md b/skill/references/pointers/token-factory.md index 9395827..b073b35 100644 --- a/skill/references/pointers/token-factory.md +++ b/skill/references/pointers/token-factory.md @@ -119,6 +119,49 @@ const [pointerAddress, version, exists] = await pointerView.getNativePointer( console.log("ERC20 pointer at:", pointerAddress); ``` + +## Query Denoms by Creator + +List all TokenFactory denoms created by a given address. Results are paginated via standard Cosmos pagination flags. + +```bash +# List denoms created by an address +seid q tokenfactory denoms-from-creator sei1abc...xyz \ + --node https://rpc-testnet.sei-apis.com + +# Paginate: first 10 results +seid q tokenfactory denoms-from-creator sei1abc...xyz \ + --limit 10 \ + --node https://rpc-testnet.sei-apis.com + +# Fetch the next page using the returned next_key +seid q tokenfactory denoms-from-creator sei1abc...xyz \ + --limit 10 \ + --page-key \ + --node https://rpc-testnet.sei-apis.com + +# Get a page-numbered result and the total count +seid q tokenfactory denoms-from-creator sei1abc...xyz \ + --page 2 \ + --limit 10 \ + --count-total \ + --node https://rpc-testnet.sei-apis.com +``` + +### Pagination flags + +- `--limit` — max number of denoms per page (a default page limit applies if omitted) +- `--page` — page number to fetch (1-indexed); mutually exclusive with `--page-key` +- `--offset` — number of denoms to skip; mutually exclusive with `--page-key` +- `--page-key` — key-based cursor; pass the `next_key` from the previous response for efficient paging over large result sets +- `--count-total` — include the total denom count in the response `pagination.total` +- `--reverse` — iterate results in reverse order + +The gRPC/REST `DenomsFromCreator` query is now paginated as well: the request accepts a `pagination` PageRequest and the response returns a `pagination` PageResponse (with `next_key` and, when `--count-total` is set, `total`). + + + + ## Complete Token Launch Workflow ```bash diff --git a/skill/references/precompiles/staking-distribution.md b/skill/references/precompiles/staking-distribution.md index 8683be3..d9ceb53 100644 --- a/skill/references/precompiles/staking-distribution.md +++ b/skill/references/precompiles/staking-distribution.md @@ -56,6 +56,25 @@ function delegatorUnbondingDelegations(address delegatorAddress, uint32 pageLimi external view returns (UnbondingDelegation[] memory); ``` + +### Pagination limits + +Underlying Cosmos pagination now enforces hard caps on all paginated staking queries (`validators`, `delegatorDelegations`, `delegatorUnbondingDelegations`, etc.): + +| Bound | Cap | On exceed | +|---|---|---| +| Per-page limit | `1000` (`MaxLimit`) | `InvalidArgument`: `limit N exceeds maximum allowed limit 1000` | +| Offset | `10000` (`MaxOffset`) | `InvalidArgument`: `offset N exceeds maximum allowed offset 10000` | +| Scan iterations | `10000` (`MaxScanLimit`) | `InvalidArgument`: `scanned more than 10000 entries...` | + +Key/cursor-based behavior for these precompiles: + +- Keep per-page `pageLimit` at **≤ 1000**. Passing a larger limit returns an `InvalidArgument` error, not a truncated page. +- **Prefer cursor-based iteration** (pass `nextKey`/`pageKey` from the previous response) rather than large offsets. Offset-based paging is capped at `10000` and lazy scans are capped at `MaxScanLimit` (10000) entries past the page end. +- To iterate the full validator/delegation set, loop on `nextKey` until it returns empty (`""`/`0x`), requesting up to 1000 per page each time — do not attempt to fetch everything in one large-limit call. +- **`count_total` is no longer implicit.** A total record count is only returned when the caller explicitly requests it; otherwise the total is `0`. On sparse datasets, a full-page response with a non-nil `nextKey` may still be returned even if the scan window was reached, so always drive iteration off `nextKey` rather than a total count. + + ### Events ```solidity