Why it matters to you
When you ask LIA “what did I tell you about Paul's restaurant?”, it doesn't look for the exact words of your sentence. It searches by meaning. Each memory, each journal entry and each passage of your documents is turned into a sequence of numbers, an embedding, which represents what it means. Two texts that are close in meaning have close embeddings, and their similarity is measured by a score.
That leaves one decision: from what score is a memory close enough to be useful? Too low, and LIA slips unrelated memories into its answer: the answer gets muddled, and you pay tokens for noise. Too high, and it misses what it actually knows. This threshold looks like a technical detail; it's what decides whether LIA remembers well.
What I wanted
I wanted one threshold per use, chosen on realistic tests. Finding a useful memory, spotting a duplicate in your interests or choosing a passage from a document aren't the same question. And I wanted each threshold to be chosen by the cost of its errors.
How it works
Each embedding model encodes meaning in its own way. When LIA moved to Google's embedding model, which no longer brings two unrelated texts together just because they're written in the same language, all the thresholds were recalibrated: the old model's values no longer meant anything. For each one, I measured on test sets what it wrongly lets through and what it misses.
- Searching or deduplicating. To recall a memory, a moderate threshold is enough. To decide that two interests are really one, they have to be almost identical. A memory even carries two embeddings: one for its content, one for the keywords that should bring it back.
- The cost of an error. To deduplicate your interests, two thresholds made exactly as many errors. The lower one wrongly merged distinct topics, like Android and iOS: your interest in one disappeared into the other, with no way back. The higher one let a few duplicates through, visible and easy to fix. I kept the higher one, because counting errors wasn't enough: I had to look at which ones they were. The reason is written next to the value, in the code.
- Six languages, one threshold. For knowledge spaces, the threshold was measured in each of LIA's six languages, and the best value was the same everywhere. That works out well: a document in German can answer a question asked in English, and a per-language threshold wouldn't have known which one to pick. A small bonus also rewards exact words, so that a product name or a reference doesn't get lost.
- A threshold that learns. For your journal, a fixed value didn't suit everyone: scores depend on what each person writes and on how many entries there are. So this threshold adjusts to you, by one small step per day at most, between two bounds it can't cross. A switch lets you freeze it at its default value.
An embedding still has to arrive
A well-chosen threshold is useless if the embedding isn't computed. This computation goes through an online service, which limits the number of calls per minute. So LIA's background tasks start at staggered times, so they don't all fire in the same second. If the service saturates anyway, LIA retries a limited number of times, and its own throttling, when in doubt, lets the request through so that your answer keeps its memory.
The limits
- No threshold perfectly separates what's relevant from the rest. Two distinct topics can look very alike, and two phrasings of the same idea much less so. The threshold remains a compromise, and that's why I set it on the side of the less serious error.
- Embeddings from two different models can't be compared with each other: changing models means recalibrating everything.
- Search by meaning currently depends on a Google service. It doesn't work offline.
Read more: Your documents, one question away: knowledge spaces; Showing the model what matters: the services that lighten LIA's work.