Transparent hash(content)+model embedding cache in Wolbarg 0.4 — cut provider cost and latency on repeated text with optional LRU and TTL.

What is it?

Wolbarg 0.4 wraps your embedding provider with a transparent cache keyed by:

hash(content) + model

Identical text for the same model skips the provider call. A model change always forces a miss — critical so you never mix vectors from different embedding spaces during recall.

Why it exists

Agent memory workloads repeat text constantly:

Re-ingesting the same docs / chunks
Agents restating the same preference or fact
Batch jobs that overlap corpora across runs
Dedupe near-match paths that re-embed candidates

Without a cache, every remember / ingest / near-dedupe path pays full provider latency and token cost. The cache is on by default in 0.4 (disable if you need every call to hit the network).

Configuration

import { wolbarg, openaiEmbedding } from "wolbarg";

const ctx = wolbarg({
  organization: "my-org",
  database: { provider: "sqlite", url: "./memory.db" },
  embedding: openaiEmbedding({
    apiKey: process.env.OPENAI_API_KEY!,
    model: "text-embedding-3-small",
  }),
  embeddingCache: {
    enabled: true,      // default
    ttlMs: undefined,   // no expiry by default
    maxEntries: 10_000, // optional LRU bound
  },
});

Disable entirely:

embeddingCache: { enabled: false }

Options

Field	Default	Meaning
`enabled`	`true`	Master switch
`ttlMs`	unset	Lazy expiry on read when set
`maxEntries`	unset	LRU eviction via `last_used_at` when set

How it works

Every single and batch embed path checks the cache first.
Hits return stored vectors immediately (no provider round-trip).
Misses call the provider — batch paths only request uncached items.
Results persist in the embedding_cache table (schema v3+) on SQLite, or an in-memory fallback when a durable store is unavailable.
LRU eviction (if maxEntries is set) drops least-recently-used rows by last_used_at.
TTL (if set) is checked lazily on read — expired entries miss and re-embed.

Cache key discipline

Content is hashed after the same normalization the write path uses for identity where applicable.
Model id is part of the key — changing text-embedding-3-small → text-embedding-3-large never reuses old vectors.
Provider API keys / base URLs are not part of the key; if you point two incompatible endpoints at the same model string, use distinct model labels or disable the cache.

Cost & latency impact

Published embedding-cache microbench (embedding-cache.json):

Metric	Value
Workload	100 chunks, 20 unique, 2 passes (mock provider)
Uncached provider calls	100
Cached provider calls	20
Call reduction	90%
Cache hits / misses	180 / 20

v4 stress suite headlines (repeated-text spot):

Backend	Cold avg	Hot avg	Speedup
SQLite	0.34 ms	0.23 ms	1.47×
Postgres	0.77 ms	0.66 ms	1.18×

Reproduce:

npx tsx benchmark/embedding-cache-bench.ts

Interaction with other 0.4 features

Feature	Interaction
Memory upsert	Exact matches skip re-embed; near matches re-embed (cache helps paraphrases that hash differently but share substrings only when text matches exactly)
Concurrency	Cache reads are cheap; provider misses still participate in normal write locking when results are persisted
Schema	Requires schema with `embedding_cache` table (auto-migrated)

Operational notes

Cache is correctness-preserving for identical content+model; it does not change recall ranking for distinct strings.
Clearing the memory DB does not always imply you want to drop the cache — but a full wipe / new file typically recreates schema cleanly.
For compliance environments that forbid persisting embeddings of PII, disable the cache or scope database lifecycle carefully.

Embedding cache