bm25q 0.0.1 is live

More queries.
Same BM25.

A faster, API-compatible BM25 library for Python. Exact by default; quality-bounded quantization when speed matters.

Start searching View source
search.py adaptive 
import bm25q

tokens = bm25q.tokenize(corpus)

engine = bm25q.BM25(
    corpus=corpus,
    backend="numba",
    quantize="adaptive",
)
engine.index(tokens)

docs, scores = engine.retrieve(
    bm25q.tokenize(query), k=10
)
quality guard passed 8-bit path
4.77× faster on MSMARCO bm25q adaptive vs official bm25s 0.3.9
2.60× faster on Natural Questions bm25q adaptive vs official bm25s 0.3.9
0.474% worst BEIR metric delta adaptive vs exact retrieval · all 15 datasets
53,359 queries in paired validation full BEIR sweep · unchanged settings

Designed for the hot path

Spend less time moving scores.

bm25q keeps the sparse eager-scoring design and familiar API, then reduces memory traffic where retrieval spends its time.

01

Adaptive precision

Each query gets a proven overflow bound. Safe queries use the one-byte accumulator; the rest automatically take the higher-precision path.

02

Exact when you want it

Floating-point retrieval remains the default. Quantization is an explicit option, never a hidden behavior change.

03

One import change

Keep the same tokenization, indexing, retrieval, save/load, and Hugging Face workflows under the bm25q namespace.

Measured, not guessed

Faster on the collections that hurt.

Same machine, same index, one thread, every scored query, and k=1000. Values are relative to the official bm25s 0.3.9 release; the fastest bar fills each group and raw QPS stays visible at the right.

0.474% maximum relative change against exact retrieval in NDCG@10 or Recall@1000 across all 15 BEIR datasets
Relative retrieval throughput
bm25s 0.3.9 = 1.00×
11-dataset aggregate37,353 queries
2.91× faster
bm25s 0.3.91 1.00×214.7
bm25q exact3 1.14×244.6
BM25S-FQ preview2 1.92×412.8
bm25q adaptive3 2.91×624.4
Natural Questions3,452 queries · 2.68M docs
2.60× faster
bm25s 0.3.91 1.00×214.7
bm25q exact3 1.35×288.8
BM25S-FQ preview2 1.96×421.5
bm25q adaptive3 2.60×558.2
MSMARCO6,980 queries · 8.84M docs
4.77× faster
bm25s 0.3.91 1.00×56.4
bm25q exact3 1.18×66.7
BM25S-FQ preview2 2.32×130.8
bm25q adaptive3 4.77×268.9

Local AMD EPYC 7302 full-query run, Python 3.10, one retrieval thread. Relative values are derived from the listed QPS. Hardware and query mix affect absolute throughput.

  1. 1

    Official bm25s 0.3.9: repository, 0.3.9 source, Numba implementation, PyPI, and paper.

  2. 2

    BM25S-F / BM25S-FQ preview: source snapshot, retrieval implementation, and benchmark definition.

  3. 3

    bm25q 0.0.1: release source, adaptive implementation, and full methodology, results, and quality validation.

Ready in minutes

Change the import.
Keep the workflow.

Install the recommended extras, select the adaptive backend, and use the rest of the API as you already do.

Open on PyPI Read the README
1import bm25q
2
3retriever = bm25q.BM25(
4  backend="numba",
5  quantize="adaptive",
6)

Search at memory speed

Put fewer bytes between
the query and the answer.

Install bm25q