Real-time product analytics (ClickBench)

Real-time product analytics (ClickBench)

LessDB is built for exactly the workload ClickBench models: a web-analytics hits table — 100M+ rows, 105 columns, event timestamps, heavy group-by/filter aggregation, and range scans over a primary key.

Shape of the workload

CREATE TABLE hits (
    WatchID BIGINT, ClientIP INTEGER, CounterID INTEGER, EventTime TIMESTAMP,
    URL TEXT, Title TEXT, Referer TEXT, ResolutionWidth SMALLINT, …
) ENGINE=MergeTree
PRIMARY KEY (CounterID, EventDate, UserID, EventTime, WatchID)
TTL EventTime INTERVAL 30 DAY;

• Sort key = primary key: rows arrive sorted by `(CounterID, EventDate,

UserID, EventTime, WatchID)` in every part, so range scans over a counter + date window are sequential reads, and TTL drops whole expired parts for free.

• Pruning is three-level: part metadata (typed min/max + blooms)

before any I/O → parquet row-group statistics → pushed-down row filters. A CounterID = 62 AND EventDate BETWEEN … query touches only the parts that can possibly match.

Measured on this repo's benchmark box (28c/62GB)

99,997,497 rows loaded in 391 s at a flat ~0.6 GB RSS; 43/43 stock ClickBench queries pass (47.2 s total vs 12.4 s for a dockerized ClickHouse reference — first baseline, gap analysis in the benchmarks doc).

Try it

git clone https://github.com/maruthiprithivi/lessdb && cd lessdb
scripts/clickbench-run.sh --skip-download

or see the ClickBench playbook for the full runbook and the ClickHouse comparison harness.