Vector search inside SQL

LessDB has native embedding search (LanceDB-style): vector spaces with their own metric and index, queried through a plain SQL table function — so search hits join and filter like any other table.

Create a space and index

-- via the Rust API or MCP/CLI: a space with dimension, metric, index flavor

less vector create --dir . --name docs --dim 768 --metric cosine --index ivf_pq
less vector insert --dir . docs --file embeddings.csv    # (id, vector…, payload…)

(For SQL-first workflows the vector surface is the vector_search table function below; space management is less vector / SDK calls.)

Search + join + filter in one query

SELECT d.title, d.url, s.score
FROM vector_search('docs', [0.012, -0.4, …], 20) s
JOIN documents d ON d.id = s.id
WHERE d.published > '2025-01-01'
ORDER BY s.score;

The same pattern works against memory_* tables and ordinary MergeTree tables — vector hits are just rows.

Indexes

flat — exact scan, the reference for recall.

IVF-PQ — k-means inverted lists over residuals + product

quantization; ADC lookup tables per probed list; exact re-ranking of finalists. Cosine/dot spaces use IVF-flat (PQ for those metrics is on the roadmap; HNSW too).

Indexes train lazily (k-means++ init) and snapshot to index.bin.

Embeddings

A deterministic trigram lexical embedder ships built-in so the whole pipeline works offline with zero model downloads; production models register themselves (register_embedder). Spaces persist under <data_dir>/vectors/<space>/{meta.json,data.bin,index.bin}.

Why SQL-first matters

The LanceDB integration pattern: no separate search service, no serialization hop — SELECT … FROM vector_search(…) is a DataFusion table function, so filters, joins, aggregations, and EXPLAIN all work on search results. Agents get it through the same less_query MCP tool.