Conversation
…alized Match wasm-node/javascript/bench/time-to-initialized.mjs: drive polkadot-api on top of smoldot and stop when client.getFinalizedBlock() resolves, i.e. an app actually has a finalized block in hand. Renames RESULT field initialized_ms -> finalized_ms; replaces the follow_subscribe / wait_initialized sub-phases with wait_finalized.
The two scripts only differed in DB loading. Unify them into startup.js — warm mode kicks in when LOAD_DB_DIR is set.
Same shape as the JS cleanup: one binary 'startup' with --mode cold|warm. Warm gates db_dir setup, the save-DB step, and the LOAD_DB_DIR child env on the mode flag. README rewritten around the unified CLI.
Previously only the relay was waited on, so warm-para benches could start before the collator had reached the target finalized block.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
| "type": "module", | ||
| "private": true, | ||
| "dependencies": { | ||
| "polkadot-api": "^2.1.0", |
There was a problem hiding this comment.
Not sure if we want PAPI as dependency eventually.
Using it like this right now to have setup as close as possible as dotli has.
There was a problem hiding this comment.
In general looking good. With real networks numbers are all over the place of course because they depend on which peer, network connection, when next block arrives etc.
In the future we could also look into how long the individual stages, would be interesting to know time to first peer, warp sync time spent, time from para finalized head to tip etc.
Good idea. TODO added |
gab8i
left a comment
gab8i
left a comment
There was a problem hiding this comment.
Implementation looking good! Also really useful to compare different transport layer such as tcp, wss or webrtc!
Another future feature could be to spawn smoldot within a headless browser instead of node and have numbers as close as possible to reality!
…bench default to 10 save_db.js: replace the racy "wait for chainHead_v1_follow initialized" gate with "wait for first post-init finalized event" per chain. Without this, smoldot's initial finalized header is the chain-spec checkpoint (genesis on zombienet local), so the dumped para .db captures genesis state and warm mode silently degrades to cold. startup.rs: align the default zombienet readiness gate with the typical need (one block is too thin; the network can be alive but smoldot will still race the warp/bootstrap promotion).
Previously interpreted as an absolute height threshold. On a fresh zombienet that is equivalent (current is 0), but it surprises on a reused/long-running network where the gate would return immediately. Now read each gated node's current finalized at the moment we begin gating and wait for current+N. Default unchanged (10), so fresh zombienet runs are byte-identical; only the long-running case changes.
lrubasze
force-pushed
the
lrubasze/smoldot_benchmarks
branch
from
712f3e6 to
59a670b
Compare
Adds a startup benchmark for smoldot. - Measures time-to-first-finalized-block, as seen by an app on top of smoldot. - Two modes: **cold** (no DB, warp-sync) and **warm** (pre-saved DB skips warp-sync). - Two targets: **relay** or **para**. - Runs against a local zombienet network, or any real network via a chain spec. - Reports per-phase timings and finalized-block drift; JSON output optional. Example: ```bash cargo run --release --bin startup -- \ --mode cold --target para --relay-chain-spec paseo --para-chain-spec paseo_asset_hub --iterations 5 ``` Output: ``` === startup benchmark === mode : Cold source : user-supplied spec relay chain : Paseo Testnet (paseo) para chain : Paseo Asset Hub (asset-hub-paseo) target : Para iterations : 5 warmup : 0 finalized_ms (total) mean=5229.0 median=5064.7 p95=8008.4 stddev=1752.5 min=2776.0 max=9790.7 addChain relay mean=43.9 median=43.3 p95=47.2 stddev=2.0 min=41.7 max=48.9 addChain para mean=13.0 median=13.4 p95=13.8 stddev=0.9 min=11.3 max=13.9 wait finalized block mean=5142.8 median=4981.0 p95=7921.9 stddev=1752.6 min=2690.9 max=9703.4 ``` See `benchmarks/README.md` for more details and examples.
3 participants
Adds a startup benchmark for smoldot.
Example:
Output:
See
benchmarks/README.mdfor more details and examples.