ClickBench on LessDB

Benchmark methodology, harness, and results for running the ClickHouse ClickBench workload against LessDB and a dockerized ClickHouse reference.

Harness

bench/src/clickbench.rs (binary less-bench) implements the full workload:

Schema: ClickHouse create.sql (105 columns, `PRIMARY KEY (CounterID,

EventDate, UserID, EventTime, WatchID)` = LessDB sort key).

43 queries: the stock queries.sql, run verbatim (ClickHouse-style

exact-case identifiers; no query rewriting).

Measurement: one warmup + three hot runs per query; median/min

reported, plus rows and an order-independent result checksum verified across the three hot runs (float aggregates are folded at 32-bit precision so rounding noise doesn't trip the check; checksum_ok=false flags genuinely order-sensitive queries — LIMIT/OFFSET without a total order — rather than failing them).

Loader: hits.parquet is streamed with a bounded batch size

(256k rows), column types leniently cast to the table schema — peak RSS stays ~0.6 GB for the whole 100M-row load.

Merge: a single post-load OPTIMIZE converges in bounded passes

(max_merge_rows = 4M input rows per pass; the ClickBench-scale merge peaks at ~4.8 GB RSS instead of holding the whole table in memory).

Reports: Markdown + JSON per engine (bench/results/clickbench-<tag>*.md,

*.json), comparisons via scripts/clickbench-compare.py (LessDB vs ClickHouse, or engine vs engine).

Orchestration

scripts/clickbench-run.sh (run on the benchmark instance) drives, in order:

dataset download (hits.parquet, ~15 GB) — --skip-download to reuse;

MergeTree: load + optimize + 43-query run;

SharedMergeTree: same pipeline, object-store parts

(--clickbench-shared);

read stress: 8 concurrent workers hammering the query set for 60 s

(--clickbench-stress);

mixed load + query stress: the query workers plus a background

stream of the dataset into a throwaway hits_load table with WAL + inline merges (--clickbench-stress-mixed);

crash-recovery under load: load with WAL on, kill -9 after 25 s,

verify recovery (WAL replay + torn-part sweep), resume to completion;

ClickHouse reference: docker run clickhouse/clickhouse-server,

INSERT INTO hits SELECT * FROM file('hits.parquet'), 43 queries × 3 via clickhouse-client --time (--ch-only to rerun this phase alone; a fully-loaded table is reused across reruns).

Environment

optimus0 (192.168.0.2): 28 cores / 62 GB RAM, shared with unrelated workloads and the self-hosted CI runners. Dataset: 99,997,497 rows, 105 columns, 14 GB parquet.

Results (2026-08-28)

Full reports: bench/results/clickbench-20260828-optimus0.md/json (MergeTree), clickbench-20260828-optimus0-shared.md/json (SharedMergeTree), clickbench-clickhouse-20260828-optimus0.md, clickbench-compare-20260828-optimus0.md, clickbench-compare-merge-vs-shared-20260828-optimus0.md.

phase	result
MergeTree load	391.1 s (~265k rows/s, flat ~0.6 GB RSS, 382 parts)
MergeTree OPTIMIZE	bounded passes → 25 parts, peak ~4.8 GB RSS
43 queries (MergeTree)	43/43 pass, total 47,211 ms (medians); 6 order-sensitive queries flagged by checksum
SharedMergeTree load	396.3 s (object-store parts, same RSS profile)
SharedMergeTree OPTIMIZE	431.2 s → 25 parts
43 queries (SharedMergeTree)	43/43 pass, total 380,229 ms
ClickHouse (docker, defaults)	total 12,388 ms (3-run averages)
read stress (8 workers, 60 s)	131 queries, 2.0 q/s, p50 3.55 s, p95 10.12 s, p99 13.88 s
mixed load + query stress (8 workers, 60 s)	129 queries, 2.1 q/s, p50 3.80 s, p95 10.25 s, p99 12.56 s — while ingesting 8.4M rows @ ~135k rows/s
crash-recovery	kill -9 mid-load → 5,242,880 rows recovered; resume to completion

MergeTree vs ClickHouse: LessDB is 3.81x slower overall (47.2 s vs 12.4 s). SharedMergeTree is 8.05x slower than local MergeTree on queries (380.2 s vs 47.2 s) while load/merge stay at parity.

Findings and headroom

Page-level parquet skipping is disabled (DataFusion Mask selection

bug, apache/datafusion#8092 — the offset index is not written; see less-storage writer properties). Row-group statistics skipping remains on. This is the largest single query-side lever.

SharedMergeTree scans materialize whole parts per query: with the

default 256 MB block cache, every query re-fetches ~8.6 GB of parts (25 × ~350 MB), hence the 8x query gap. The fix is lazy range reads through object_store (fetch row-group ranges on demand) plus a block cache sized to the hot part set — the roadmap item now has a measured number attached to it.

Late materialization for wide tables (ClickBench projects a few of

105 columns per query) and general DataFusion execution overhead make up the rest of the gap to ClickHouse.

Six ClickBench queries (LIMIT/OFFSET without a total order, float

aggregates) are order-sensitive; the harness flags them via checksum_ok instead of hiding the variance.

Along the way the harness drove four real fixes: bounded iterative

merges (max_merge_rows), ClickHouse-style identifier case handling, torn-part sweep at open, and offset-index-free part writing.

Rerunning

ssh 192.168.0.2
cd ~/lessdb-clickbench
scripts/clickbench-run.sh --skip-download        # everything except the download
scripts/clickbench-run.sh --ch-only              # ClickHouse reference alone