You ask your assistant about the per diem limit, and it responds by citing an archived email from a different department. The Qdrant collection is green, indexing finished without errors, and every query returns in milliseconds. The problem isn’t in Qdrant’s response, but in the decisions made before sending the query: what was indexed, with which model, and under which filters. Reviewing these five points helps locate the failure before changing database parameters.
The embedding model decides what you can find
Similarity only means something within the same vector space. Indexing documents with one model and formulating queries with another prevents valid vector comparison: if the dimensions match, the cosine similarity returns neighbors that are formally close but semantically irrelevant; if they don’t, Qdrant rejects the query. A latency alert might overlook this failure because the database can respond quickly with a result that looks reasonable.
The chosen model also conditions the language and the quality of retrieval. all-MiniLM-L6-v2 produces 384-dimensional vectors and was trained primarily in English; in contrast, a multilingual model like multilingual-e5-large produces 1,024-dimensional vectors and requires query: and passage: prefixes for queries and passages in retrieval tasks. The Hugging Face MTEB leaderboard allows you to compare candidates by retrieval task and language before choosing one.
If an intermediate service regenerates embeddings with a different model, the collection becomes silently mixed. FastEmbed, from the Qdrant team, allows you to centralize the model and work with retrieval-oriented options. Correcting this divergence before optimizing the search avoids measuring changes unrelated to the query.
Do a quick check: run the same test text through both scripts, apply the same prefix, and compare the dimension and the first values of the vector. If they don’t match, you’ve located a common cause of incorrect retrieval; only then should you review the distance metric without carrying over that error.
The distance metric is chosen when creating the collection
Qdrant offers cosine, dot product, and Euclidean distance, and the metric is fixed upon collection creation. Changing it later requires recreating the collection and re-indexing the data, so it is best to decide this alongside the model rather than during an incident. The equivalence between metrics depends on how the model normalizes its vectors.
OpenAI embedding models normalize their vectors, meaning cosine and dot product produce the same order. This equivalence does not extend to all models; sentence-transformers models are typically used with cosine.
from qdrant_client import QdrantClient, models
client = QdrantClient("http://localhost:6333")
client.create_collection(
collection_name="docs",
vectors_config=models.VectorParams(
size=1536, # Adjust the value to your model's dimension
distance=models.Distance.COSINE,
),
)
Port 6333 exposes the Qdrant REST API, so you can query the configuration directly. Unless you change it during collection creation, Qdrant initializes HNSW with m=16 and ef_construct=100; HNSW is the graph-based search structure introduced by Malkov and Yashunin.
For a from-scratch setup, the article RAG and semantic search with Qdrant: from theory to your first collection covers the entire process.
Run GET /collections/docs, contrast the distance field with the model’s datasheet, and fix any incompatibility before re-indexing. With the model and metric aligned, the next step is deciding how to split documents into fragments.
Chunking decides if an embedding means anything
The size and boundaries of each fragment determine what the embedding can retrieve. A block that concatenates the introduction with the appendix represents both documents without identifying either; one that is too short loses the context necessary to answer. The resulting vector looks like a mixture, not a retrievable unit.
A common starting point is several hundred tokens per fragment with a controlled overlap, although no universal rule exists. When a document has a structure of headers, tables, or lists, cutting by those boundaries preserves complete units and leaves the section available for the payload. To see the complete design of the journey, from ingestion to response, the article RAG and semantic search with Qdrant: the pipeline that finds ideas, not words breaks it down in detail.
Take ten real questions from your search engine and retrieve the chunks that should answer them. Check if any sentence was split by an arbitrary limit or if a rule appears separated from its context. If the cut hides the answer, change the size or the delimitation; afterwards, you can separate versions using payload filters.
Payload filters select the correct version
Vector search finds similar text, not current text. Without filters, a RAG system may return an older version of an internal policy because its wording is more similar to the query than that of the current document. Qdrant stores metadata in the payload and allows filtering by it within the same query.
Qdrant’s HNSW filter evaluates conditions during the graph traversal, rather than fetching candidates and discarding them later in the application. A condition like department = "finance" AND active = true reduces the eligible set before returning results. Create payload indexes for fields frequently used in match or range filters.
List three fields you currently have to discard by hand and turn them into filters; add their payload indexes where appropriate. Also store the title and section in the payload to be able to cite the source of each response. Once validity is controlled, the next problem is literals that dense search does not reliably retrieve.
Hybrid search improves retrieval of literals
Dense embeddings capture the meaning of a sentence but are often unreliable for literals such as error codes, product references, or case numbers. ERR_40921 must match as an exact string; similarity alone does not guarantee that match. Hybrid search adds an explicit path for these cases.
Qdrant supports dense and sparse vectors in the same collection using named vectors, and its Query API merges both lists using RRF or DBSF. The Qdrant team maintains BM42, a sparse model designed for this hybrid scenario.
If your interface receives identifiers sent via the search bar, route those queries through the hybrid path and compare the recall before and after with the same set of questions. The adjustment should retrieve literal cases without degrading those already solved by dense search. To verify both without mixing effects, you need an evaluation with known cases.
Measure with twenty questions before touching anything
Without an evaluation set, every RAG change is a coin flip. A minimum version fits in a CSV: twenty or thirty real questions alongside the chunk that each one should retrieve. After each change, count how many hits appear among the top five results; that ratio is your recall@5 for that list.
The calculation doesn’t require an external service: queries against the collection and an identifier comparison in a script are enough. When latency permits, increasing hnsw_ef during search can increase recall at the cost of more waiting. Do this only on a stable base; if retrieval fails due to data, segmentation, or filters, the parameter will not solve the root cause.
Change only one variable per experiment—model, chunk size, or search parameters—and re-run the same list of questions. The comparison only means something if the other variables remain frozen.
The cost of recreating a collection depends on the indexed volume and the configuration, so it’s unwise to assume it will always be cheap. Set aside an hour this week, dump twenty real questions into a CSV, and note the recall@5 of the collection as it stands. From that figure, apply a single adjustment and compare the result with the same list.