# Choosing an embedding model for short, messy, multilingual text By Somil Jain · 2026-10-15 · https://mutedmiles.space/post/choosing-an-embedding-model 5 embedding models, each at its own safe threshold 11,027 labelled pairs from 25,808 generated sentences 3.3× faster with the same model on a different engine 14 ms to embed one short question, on CPU only Every number in this post comes from my own runs in September and October 2026. Where something was not measured, I say so. ## 1. What I was trying to find out I wanted to know which embedding model fits my own kind of text. That text is short, written in English, Hindi and the mix of the two that people actually type, full of filler words ("um", "so", "please"), and spelled inconsistently. The task is simple to state: given a sentence I already have, is a new sentence asking **the same thing**? Three constraints shaped everything: - **It has to run on ordinary CPUs.** No GPU, and it runs on every input, so every millisecond and every megabyte is paid again and again. - **A false match is worse than a missed one.** Two sentences about the same topic are not the same question. - **It has to work across scripts.** The same Hindi sentence turns up in Devanagari, in Roman letters or in a mix. So "which model scores highest on a benchmark" was never the question. The question was: which model lets me pick a distance below which a match is safe, still finds the repeats I care about, and runs on the hardware I have. ## 2. Where I started, and why I stopped trusting it I began with a small multilingual MiniLM and a match distance of 0.25. That number had been chosen from a single pair of opposite requests ("cancel" against "upgrade"). It held up until I added one more opposite pair. "Put my order on hold" against "release my order from hold" came out at **0.114**, well inside the threshold. One calibration point is not evidence that a threshold is safe. A small story Around then I loosened the match distance to 0.25 while testing on a made-up question set for a property developer. A question about when the project launches was matched to a stored sentence about something else. Nothing crashed and nothing looked wrong in the logs; the system simply returned the wrong neighbour with confidence. I pulled the distance back to 0.2. It was the first time the number on the dial felt like a safety setting and not a tuning knob. ## 3. Building a test that could say no I generated 25,808 sentences across 8 domains and two languages (English and Hindi), with the filler words and politeness that people really type. From them I sampled 11,027 labelled pairs, in classes that a good model must treat differently: Pair class | Should they match? | Same template, different filler | Yes, and it should be easy | Same domain and intent, different wording | Yes, but the labels are generous, so read this as a floor | Same topic, but a different question ("What is your refund policy?" against "Where is my refund?") | Never | Opposite actions (cancel against upgrade) | Never | Different domains | Never | The same question in English and in Hindi | Informational only | Here is what my starting threshold of 0.25 would have done on that test: MiniLM at distance 0.25 Share of each pair class that would be matched. Orange bars are wrong matches. Genuine rewordingsshould match 14.8% Same topic, different questionmust never match 5.9% Opposite actionsmust never match 29.6% No threshold fixed both. To push the wrong matches to zero, the distance had to drop to 0.125, where genuine-paraphrase recall fell to **2.2%**. What I took from that: embeddings measure topical similarity, not "is this the same question". Nearness is a good detector for near-duplicates (the same question again, spelled or phrased slightly differently) and a poor one for intent. A small story The text was creative with a made-up company's name. It turned up as प्रतीक, an ordinary Hindi word that means "symbol", then as "progresses प्रतीक", then as "Progressive setting". A product called "A Stage Called Tomorrow" came back as "स्टेज फॉर टुमारो". The right match was in the stored set all along; the new sentence just never looked like my spelling. In the end I wrote every stored sentence in the forms a messy source would produce, and added a small list of "this spelling means that name" rules that run before anything is embedded. ## 4. Five models, each at its own safe threshold Cosine distances are not comparable between models, because each spreads its space differently. So I read every model at the highest threshold that gave **zero wrong matches** on both of the "never" classes, and counted the repeat questions it still found (out of 2,500 easy pairs): Repeat questions found at each model's safe threshold Out of 2,500 easy pairs. Chips show the safe threshold, time per text and vector size. jina-v3 2,295 · 91.8% threshold 0.150 30.6 ms 1024 dims EmbeddingGemma 300Mwhat I chose 1,654 · 66.2% threshold 0.100 4.8 ms 768 dims bekko 1,586 · 63.4% threshold 0.150 4.5 ms 384 dims MiniLMwhere I started 1,512 · 60.5% threshold 0.075 1.5 ms 384 dims multilingual-e5-base 894 · 35.8% threshold 0.025 7.0 ms 768 dims Four things stood out: - The best finder was not the right model for me. jina-v3 found every pair the Gemma model found, and 783 more than MiniLM. But it takes about 20 times longer per text (about 29 ms extra on every input), occupies 2.2 GB against 241 MB, and whether that pays off depends on how much each correct match saves you. - A strong cross-language model can be the wrong tool. jina-v3 matched 117 of 1,000 English and Hindi pairs of the same question; the others matched 0 to 3. If your system must keep languages apart, that is a risk, not a feature. - Models disagree about what a repeat is. Only 26% of the wanted pairs were found by all five models, and 7.8% by none. The middle 66% is where the choice of model decides the result. - A tighter threshold costs less than it seems. At a 0.5% error budget, jina-v3 and MiniLM both sat at 0.150 and the gap was 91.8% against 86.9%, not 31 points. The test has limits I would state plainly: the sentences are template-generated, not real text; there are only 27 opposite-action pairs, so each "zero wrong" threshold is decided by one or two pairs; and the "same intent" labels are over-generous. My conclusion at the time was that the model was not the main lever. Tighten the threshold, and add a cheap word check that blocks a match when the sentence says "my", "right now" or "status". ## 5. Same model, different engine I settled on EmbeddingGemma 300M, cut to 256 numbers (it is trained so that its first 256 numbers still work). One property made that practical: my document search already used the same model at the same size, so one vector can serve two jobs and an input is embedded once. Then I asked a different question: is the PyTorch engine I was running it on the right one for CPUs? I compared it with the same model exported to ONNX and run through fastembed, both limited to 4 CPUs. Requests per second, one client, 4 CPUs Same model, same vectors, different engine. ONNX + fastembed 50.9 req/s PyTorch 15.3 With four clients the gap grows to 12 times, because PyTorch's four-thread pools fight over four CPUs. - **Same vectors, if you add the prompts.** EmbeddingGemma expects a short instruction before queries and before documents. Without it, fastembed's vectors had a cosine of 0.835 to PyTorch's and the top-3 rankings agreed only 62% of the time. With the instructions added by hand the cosine was **0.99999** and the rankings agreed 100%. Vectors I had already stored stayed valid. - **Faster.** 50.9 requests per second against 15.3 with one client (3.3 times), and 12 times with four clients. - **Threads decide more than you expect.** Both engines size their thread pool from the host's core count, not the container's CPU limit. On a 16-core host limited to 4 CPUs that over-subscribes the CPU. - **Bake the model in.** A model downloaded at start took 10 to 86 seconds to the first embedding; baked into the image it took about 3 seconds, at the price of an image 1.8 GB larger. - **Footprint.** The ONNX build of my search service is 3.65 GB against 5.25 GB, and holds about 0.9 GB of memory. A small story My first chart said ONNX was 12 times faster, and it looked wonderful. Then I noticed the PyTorch run was using 16 threads on a 4-CPU container, while ONNX had been pinned to 4. Fair comparison: 3.3 times with one client. Still a good result, but a quarter of the headline. I redid the chart with equal threads, and that second chart is the one in this post. 12× first chart 16 threads vs 4 threads → 3.3× fair chart equal threads, one client Measured end to end through my search service, a short question took a median of **14 ms** to embed, against 64 to 90 ms with PyTorch. One surprise from changing the engine: the right match distances moved, so a threshold tuned on one engine was wrong on the other. On the new path a wrong match sat at 0.156, so I tightened to 0.14 for a while and later settled on 0.2 with a slider I can move. ## 6. The new release: EmbeddingGemma 2 On 6 Oct 2026 Google released EmbeddingGemma 2: an Apache 2.0, 740M multimodal model whose **text part is 270M** and can be loaded without the vision (170M) and audio (300M) parts. It has 8,192 tokens of context and the same prompts and 256-number cut. I tried it the way I had tried everything else: on my own data, on my own CPU, text only. The ONNX export has a text file with empty slots for image, video and audio input, so I passed empty arrays and left the other encoders out. I tested full precision and a 4-bit file (174 MB), on 26 questions that should match stored ones, 20 that must not, and 15 document-search questions against 22 paragraphs. | EmbeddingGemma (what I use) | EmbeddingGemma 2, text only | Right paragraph ranked first (15) | 14 | 15 | Matching decisions right at the best threshold (46) | 40 | 42 | Right-vs-wrong separation (AUC) | 0.946 | 0.954 | Best distance threshold | about 0.285 | about 0.165 | Different-question sentences wrongly matched at 0.2 (of 20) | 1 | 5 | One short question | 16 ms | 22 ms (4-bit: 26 ms) | Memory | 0.86 GB | 1.18 GB | A small story I had the model running nine days after it came out. The ONNX file for the text part has empty slots for images, video and audio, so to run it text-only I fed it zero-row arrays for pictures and sound it was never going to see. It worked on the first run, which is the best kind of anticlimax. Quality was within noise at this sample size. Everything else pointed the same way: it needs its own thresholds, it is slower, it uses more memory, the 4-bit file saves disk and not time on this CPU, and switching means re-embedding everything I have stored, because the vectors are not interchangeable. **I am not switching on this evidence.** What would change that is a bigger set of real questions, a check of the ONNX output against Google's reference code, and long inputs, where the larger context window might matter. ## 7. What I would tell someone choosing a model - Test on pairs that should not match, not just ones that should. The dangerous pairs are the ones about the same topic. - Compare models at their own safe threshold. Distances are not comparable across models, and a threshold from one model is wrong for the next. - Do the cost arithmetic per input, not per match. The model runs on every sentence, so milliseconds and memory are paid every time. - Language is a safety question. A model that is good at matching across languages may be the wrong one if you need to keep languages apart. - The engine is a separate decision from the model. The same vectors came out of ONNX at several times the speed, and the thread count mattered as much as either. - Add the prompts the model was trained with, and check the engine reproduces the reference vectors before trusting a speed result. - Measure on the hardware you will run on, with the same container limits and the same thread counts for every candidate. - A bigger model that scores slightly better is not automatically better for you. My own test said a tighter threshold and a cheap word check did more than any model swap. One more story, about quiet failures When I first pointed my document search at the new setup, it returned nothing and raised no error. The index held 256-number vectors and the search was asking for the 1536-number kind. A missing setting sent the question down the wrong path, and the only symptom was a search that suddenly found nothing. The lesson: check that the vector size and the model match on both sides, and log which path each search takes. ## 8. What I did not measure - Real text at scale for the model comparison. - Memory and CPU under load, and under a CPU limit below the core count. - EmbeddingGemma 2 against Google's reference code, and on long inputs.