Run ClickBench against LessDB (and ClickHouse)

Run ClickBench against LessDB (and ClickHouse)

The full 100M-row / 105-column ClickBench pipeline, exactly as run on the 28-core benchmark instance in this repo. One script does everything: dataset download, MergeTree load + queries, SharedMergeTree load + queries, read stress, mixed load+query stress, crash-recovery, and the dockerized ClickHouse reference.

Run it

git clone https://github.com/maruthiprithivi/lessdb && cd lessdb
scripts/clickbench-run.sh                # downloads hits.parquet (~15 GB)
scripts/clickbench-run.sh --skip-download   # reuse the dataset
scripts/clickbench-run.sh --ch-only         # ClickHouse reference alone

Requirements on the machine: Rust (stable), docker, ~45 GB free disk, more RAM is better (62 GB used for the reference numbers below).

What it measures

• Load — 99,997,497 rows streamed in 256k-row batches; peak RSS stays

~0.6 GB for the entire load.

• OPTIMIZE — one call converges in bounded size-tiered passes

(max_merge_rows = 4M); 382 parts → 25, peak ~4.8 GB RSS.

• 43 queries — stock ClickBench SQL, verbatim, 1 warmup + 3 hot runs;

median reported with an order-independent result checksum.

• Stress — 8 workers hammering queries for 60 s; then the same while a

background stream loads the dataset into a fresh table (135k rows/s ingested without degrading query latency).

• Crash-recovery — kill -9 25 s into a WAL-enabled load, verify

recovery (WAL replay + torn-part sweep), resume to completion.

• ClickHouse reference — docker, default settings, 3-run averages.

Reference numbers (optimus0, 28c/62 GB)

Details, per-query timings, and the comparison markdown are committed in bench/results. The gap analysis and the next optimizations (page-level skipping, lazy range reads for shared parts) are in the same doc.