Embeddings and vector search: similarity is a retrieval aid

1. Decide what each vector represents

An embedding is a numerical representation learned to make some comparisons useful. Its coordinates are not a readable inventory of objects or a certified account of an event. CLIP learns related image and text representations from paired supervision, allowing a description to retrieve visually related material. For the translated query “Find a woman in a red shirt holding a box around 13:00”, proximity may suggest candidates but cannot prove that the shirt, person and holding relationship belong together.

Choose the represented evidence unit deliberately. A person crop emphasizes clothing but may exclude hands or the box; a whole frame retains surrounding objects but can dilute a small subject. A video clip can represent change over time only to the extent supported by its encoder, training and sampling. Averaging frame vectors does not automatically recognize carrying. Preserve crop coordinates, frame identifiers or clip intervals so reviewers can recover the actual evidence rather than treating all vectors as equivalent scene descriptions.

2. Keep similarity geometry and score direction explicit

For nonzero vectors x and y, cosine similarity is dot(x,y) divided by the product of their lengths. The dot product, also called inner product, can be influenced by stored vector magnitude as well as direction. Euclidean distance, L2, measures separation and is smaller for nearer vectors; cosine and dot-product search prefer larger scores. FAISS reports squared L2 for its L2 metric, avoiding the square root without changing the ordering. Never interpret one metric’s numerical cutoff as another metric’s cutoff.

Normalize both query and indexed vectors to unit length when using inner product to implement cosine search. On unit vectors, cosine equals dot product and squaredL2 equals 2 minus 2 times dot product. Thus dot=0.8 gives squaredL2=0.4 and L2 approximately 0.632, not 0.4. These identities require the same unit-vector assumption; zero vectors cannot be normalized meaningfully. Check finite coordinates, expected dimensions and consistent normalization. Quantization and numerical rounding can affect measured scores, and none of these geometric values is a probability that a requested action happened.

3. Version the representation, not just the index file

Equal dimensions do not imply compatible embedding spaces. A different checkpoint, image encoder, paired text encoder or training objective can change the meaning of coordinates. Compare image and query vectors only within a documented compatible model family and processing pipeline. Record resizing, crop policy, tokenizer, text language support, clip sampling, normalization and source revision. Whether translated or original-language queries work well must be measured on the chosen model; support for one language is not evidence of support for another.

Model migration should create a separately identified vector collection or rebuild the affected representations consistently, with explicit active-version rules. Do not quietly compare new query vectors against old image vectors merely because storage accepts their shape. The illustrative record below keeps source provenance and geometric assumptions visible; its candidate depths are arbitrary design examples, not tuned recommendations. Null date, timezone and window mean the search still requires user confirmation. A version label organizes evidence but does not itself establish that a model is suitable or accurate.

{
  "representation": "person_crop",
  "source": {"event_id": "evt-0042", "frame_id": "frame-0081", "region_id": "region-person-1", "revision": "r1"},
  "embedding_space": "paired-image-text-example-v1",
  "normalization": "l2_unit_nonzero",
  "metric": "inner_product",
  "unit_vector_example": {"dot": 0.8, "cosine": 0.8, "squared_l2": 0.4},
  "time": {"clock": "13:00", "date": null, "timezone": null, "window_minutes": null},
  "authorized_only": true,
  "candidate_depth": 100,
  "final_event_depth": 10,
  "execution_state": "needs_user_confirmation"
}

4. Separate nearest-neighbor recovery from event relevance

Exact search compares every eligible stored vector under the chosen metric. FAISS flat L2 and inner-product indexes provide such a baseline. Approximate nearest-neighbor search, ANN, trades search effort, memory or representation precision for incomplete recovery of that ordering. An inverted-file index searches selected partitions; a graph index explores selected connections. Quantized representations can introduce another approximation. The appropriate comparison holds the authorized corpus, embedding version, metric and query fixed, rather than attributing differences from changed inputs to index quality.

ANN neighbor recall@k measures how many of the exact top k vector identifiers appear in the approximate top k, divided by k, for a corpus containing at least k eligible vectors. Document tie handling and smaller-corpus cases. Recovering eight of ten exact neighbors gives 0.8, regardless of whether those neighbors answer the question. Event relevance recall instead uses independently judged relevant events as its denominator. Perfect neighbor recovery can return irrelevant scenes when the representation misses holding; exact search is exact geometry, not perfect event understanding.

5. Preserve constraints while expanding candidates

Candidate depth is the number retained for later inspection, not necessarily the number of distinct events shown. Many neighboring frames or crops can belong to one event and consume the budget. Reranking can reject relation errors among retrieved candidates, but cannot recover a relevant event absent from its input. Evaluate candidate depth together with event deduplication and ANN effort, such as partition probes or graph exploration. Changing one parameter may improve coverage while increasing memory use or end-to-end waiting; measure the tradeoff rather than promising a universal setting.

Apply hard permissions before retrieval and ranking through authorized partitions or a constrained strategy, and preserve camera, confirmed date, timezone, time window and valid-value restrictions throughout. Filtering a global top list afterward can discard its contents while eligible neighbors never entered it. Increasing depth is not an authorization boundary or a completeness guarantee. Resolve the original query’s missing time context before execution; do not silently select today. Recheck permissions before evidence playback, and keep missing or invalid field values distinct from valid zero and from a supported negative.

6. Test failures that similarity can conceal

Use a fixed authorized test corpus with exact-vector baselines and separate event judgments. Include small subjects, occluded hands, a red box beside another person, repeated frames and a matching shirt without a holding action. Split development and test by camera and day so adjacent observations do not leak across partitions. Compare representation choices and report where candidates disappear before reranking. An unavailable region leaves the relation unknown, not false; nearest-neighbor search always having an output does not mean a matching event exists.

Treat the person category in the query as user-described apparent presentation, not proof of gender identity or sex. Do not infer protected traits or personal identity from vectors. Expiry or deletion must invalidate source media, event revisions, embeddings and cached candidates consistently; stale neighbors cannot substantiate an answer. The checklist supports reproducible review rather than a product claim. AIFacon Detection remains planned, and these concepts establish neither an implemented vector stack nor performance, service-level, retention-policy or certification commitments.

• Test zero, nonfinite and wrong-dimension vectors, missing normalization and reversed score direction before judging retrieval quality.
• Reject incompatible query and image model versions even when their vector dimensions match.
• Measure ANN neighbor recall against exact search separately from relevant-event recall after deduplication and evidence review.
• Test restrictive authorized subsets, shallow candidate budgets and duplicated frames without relaxing hard permissions.
• Test deleted source revisions, occluded relationships and genuine no-match queries; preserve unknown states and verify fresh playback access.