Overview

Semantic search for the wiki using sqlite-vec + BGE-M3 + bge-reranker-v2-m3, entirely local, following a Hindsight/TencentDB hybrid + RRF pattern.

Memo

Applied (2026-08-13) — ① registered the wiki-embed-search skill; ② integrated automatic indexing into wiki-sync.sh (09:00/21:00: start the servers → re-index; new wiki documents are reflected automatically); ③ PoC measurement: 58 documents → 512 chunks in about two minutes, with accurate semantic search (muse-glimmer ranked first for the relevant query, and “mining pool configuration” ranked the bitaxe document first); ④ wrote a bitaxe entity document to fix data coverage. ARM64 was measured with the sqlite-vec aarch64 wheel. Usage: python3 [local path] "query".

Detailed Research

Wiki Embedding Search — Research and Design for Local Hybrid Search

1. Current State (Measured 2026-08-13)

Item	Current state
Search tool	search_files — ripgrep keyword/regex search, with no embeddings
Index	log_index.md generated monthly plus the existing structure
Vector DB / embedding / reranker	❌ None
Wiki size	75 Markdown files (46 entities / 12 concepts / the rest logs and designs)

Limitation: keyword matching only. A document written with a synonym, or a document where the exact words are forgotten—such as “that model we investigated back then”—cannot be found. This leaves room for semantic search.

2. Patterns from Existing Memory-System Research

The existing research documents use a common search structure:

System	Storage	Embedding	Reranker	Fusion
Hindsight	PostgreSQL + pgvector	bge-small-en-v1.5	ms-marco-MiniLM-L-6-v2 cross-encoder	Semantic + BM25 + Graph + Temporal → RRF + reranking
TencentDB Agent Memory	SQLite + sqlite-vec	BGE-M3 and other local/remote models	—	BM25 + vector + RRF

→ Shared conclusion: hybrid keyword + vector retrieval, RRF fusion, and a cross-encoder reranker are the standard pattern for local search quality. Both systems assume a fully local deployment.

3. Systems That Can Be Served Locally

Vector storage

System	Form	ARM64	Fit for a 75-document wiki
sqlite-vec (Mozilla Builders)	SQLite extension, file-embedded	✅ aarch64 wheel measured (0.1.9)	✅ Good fit — zero infrastructure, one database file
ChromaDB	Embedded Python database	✅	✅ Simple, but metadata-filter recall issues at larger scale; irrelevant for this wiki
LanceDB	Embedded Lance format	✅	✅ Designed for scale; excessive for this wiki
Qdrant	Server-based, Rust	✅ arm64 image measured	⚠️ Strong scalability, but adds one process to operate
pgvector	PostgreSQL extension	✅	❌ Requires DB infrastructure; excessive for this wiki even though Hindsight already uses it

Embedding models

Model	Size	Features
BGE-M3 (BAAI)	568M (~2.3 GB)	100+ languages and strong Korean performance; dense + sparse + multi-vector in one model
bge-reranker-v2-m3	~568M	Cross-encoder reranker, scoring query/document pairs
nomic-embed-text	137M	Lightweight and English-oriented
Qwen3-Embedding	Various	Newer candidate, for later review

Serving paths on GB10: ① llama.cpp embedding server, already available on GB10 with a native ARM64 build; ② Ollama embedding API; ③ an embedding_api open-source server optimized for BGE-M3 + reranker on CPU and compatible with the OpenAI API shape.

4. Recommended Design — Wiki Hybrid Search

[Indexing — integrated into wiki-sync.sh, automatic at 09:00/21:00]
Scan wiki Markdown files
 → chunk by section (## headings, about 500 characters)
 → local BGE-M3 embeddings (llama.cpp server, OpenAI-compatible)
 → store in sqlite-vec: path / section / vector / metadata (tags, date)
 → one index database (tens of MB) + incremental updates only

[Search — wiki-search script]
① Query → BGE-M3 embedding → vector top-K (semantic search)
② ripgrep/BM25 keyword top-K (preserve exact matching)
③ RRF (Reciprocal Rank Fusion)
④ optional bge-reranker-v2-m3 reranking → output top 5
 → Hermes reads the result documents and answers

[Operations]
- Entirely local on GB10 — no external API, zero external cost
- Hermes integration: expose `wiki-search "question"` as a skill
- Keep existing ripgrep search as a fallback

Why These Components

• sqlite-vec: a server such as Qdrant is excessive for a 75-document wiki. TencentDB validated the same combination. One file makes backup and movement simple.
• BGE-M3: strong Korean and multilingual performance for a wiki mixing Korean and English, plus sparse retrieval that can supplement keyword search.
• Reranker: reranking runs only at search time, so indexing adds no cost. Re-score the top 20–30 candidates and return the top 5; BGE reranking is feasible on the GB10 CPU.
• RRF: a standard fusion approach used by both Hindsight and TencentDB, without requiring weight tuning.

5. Expected Effects and Risks

Effects

• Semantic search: “the agent memory system we investigated back then” can automatically connect to Hindsight/TencentDB documents.
• Synonyms and mixed Korean/English documents become searchable, closing ripgrep's blind spots.
• Better wiki exploration at the start of research should reduce duplicate investigations and missed documents.

Risks / cost

• BGE-M3 stays resident at about 2.5 GB, which is negligible on a 128 GB GB10.
• When integrated with wiki-sync, indexing can update only changed documents in seconds.
• The reranker costs CPU inference only at search time, roughly a few hundred milliseconds for 20 candidates.
• Initial setup requires one or two scripts (indexer and searcher) plus embedding-server startup.

6. Adoption Decision

• For a 75-document wiki, sqlite-vec + BGE-M3 + optional reranker is the best fit: zero infrastructure, fully local, and measured on ARM64.
• Hindsight/TencentDB validated the same pattern in benchmarks (WideSearch +51%, PersonaMem 48→76%).
• PoC sequence: ① start the llama.cpp embedding server; ② prototype the indexer and searcher; ③ index the entire wiki; ④ register the Hermes skill; ⑤ integrate into wiki-sync after review.

7. PoC Measurement (2026-08-13, GB10)

Configuration

• Embedding: llama.cpp llama-server + BGE-M3 Q8 (634 MB), port 8082; --ubatch-size 2048 is required because the default 512 returns a 500 error on long chunks.
• Reranker: llama.cpp + bge-reranker-v2-m3 FP16 (1.16 GB), port 8083 with --reranking.
• Storage: SQLite + sqlite-vec v0.1.9 (aarch64 wheel).
• Scripts: wiki-index.py, wiki-search.py, and start-embed-server.sh.

Indexing

• Target: 58 entity documents, excluding navigation documents → 503 chunks, indexed in about two minutes with batch size 8.

Search Quality Test (Vector + Keyword RRF + Reranker)

Query	Top result	Judgment
“A local agent model made by Meta”	muse-glimmer-30b	✅ Correct
“The agent memory system we investigated back then”	tencentdb/hindsight	✅ Correct
“A Korean speech synthesis model”	qwen3-tts	✅ Correct
“Mining pool configuration”	No related document	⚠️ Data missing

Findings During the PoC

1. Vector-only search was very accurate; keyword OR fallback added noise under RRF, which was fixed with AND-first retrieval and vector weight 2.0.
2. Document-level deduplication was necessary; chunks from one document could dominate the results, so only the best chunk per document is kept.
3. Navigation documents must be excluded; list documents such as research-backlog scored too highly in the reranker, so a skip list is required.
4. Data coverage is a real limit: bitaxe/miner knowledge was not in the wiki, only in skills and logs. It was correct for search to return nothing; the wiki needed an entity document such as bitaxe.
5. The llama.cpp embedding server uses a default --ubatch-size of 512; long inputs need 2048 to avoid a 500 error.

Usage

# Start/stop the embedding and reranker servers
bash [로컬]
bash [로컬] stop

# Index after wiki changes
python3 [로컬]

# Search
python3 [로컬] "query"

Sources

• sqlite-vec: https://github.com/asg017/sqlite-vec (aarch64 wheel measured, v0.1.9)
• Qdrant: https://qdrant.tech (arm64 image measured)
• BGE-M3: https://huggingface.co/BAAI/bge-m3
• bge-reranker-v2-m3: https://huggingface.co/BAAI/bge-reranker-v2-m3
• embedding_api: https://github.com/smallOpenSource/embedding_api
• Local vector database comparison (2026): https://aliteq.com/best-vector-database-for-local-rag-2026
• Reference: the local hybrid + RRF + reranker pattern