Why your omnichannel RAG is hallucinating.

Retrieval-augmented generation is the default architecture for AI search and product discovery in mid-market omnichannel and digital-first commerce right now. It's also the source of the most expensive failures we see on diagnostic calls — hallucinated SKUs, wrong prices, recommendations that contradict each other a turn apart in the same session.

The problem is rarely the model. It's the retrieval layer.

The three failure patterns

Most "hallucinating" RAG systems in omnichannel commerce fall into one of three patterns. They have different recommended fixes, and confusing them is why internal teams burn months iterating on prompts when the actual issue lives somewhere else.

1. Stale embeddings, fresh catalog

Your catalog churns. Prices change, inventory flips, descriptions get edited, SKUs deprecate. If your vector embeddings are regenerated nightly (or weekly), every retrieved chunk between now and the next refresh is wrong — sometimes subtly, sometimes catastrophically.

The model dutifully answers based on what it retrieved. The retrieval was lying. Hallucinations look like the model is making things up; in practice it's faithfully reproducing yesterday's reality.

Recommended fix: event-driven embedding regeneration. The same write events that update your catalog DB (price change, SKU update, stock event) should trigger an incremental upsert into your vector store. We document the event topology and the contract your engineering team will need with the vector vendor.

2. Wrong chunk granularity

Naive RAG chunks every product into 512-token windows. For a t-shirt with a 30-word description, you've turned a single document into noise — the chunk that gets retrieved is the chunk with the closest semantic distance, not the chunk that contains the actual product.

Digital-first commerce documents have structure. Title, description, attributes, reviews, FAQs. Each layer answers different questions. Mash them into uniform chunks and the model gets the wrong layer for the question — your "is this dishwasher-safe?" query retrieves a review chunk about a different SKU because the review chunk had higher similarity to "dishwasher."

Recommended fix: structured chunking by document section. Title + attributes as one chunk, FAQs as a separate index, reviews as a third. A lightweight classifier up front routes each query to the right index before retrieval. We provide the routing rules and the index-design blueprint your team will implement against your current vector store.

3. Single-pass retrieval on multi-hop questions

"Show me dishwashers under $800 that have a quiet cycle and a 14-place setting." A vector search returns the 5 nearest neighbors to the embedding of that whole sentence. The result is roughly the right vibe — but the model has no way to verify the price filter, the feature filter, or the capacity filter against the actual SKU data.

So it confabulates. It tells you the SKU has a quiet cycle because the description mentioned "quiet" somewhere. The hallucination is the gap between semantic similarity and structured truth.

Recommended fix: hybrid retrieval. Extract structured filters (price ≤ 800, capacity ≥ 14, feature = quiet cycle) with a small LLM call up front. Apply those as hard filters against your product DB. THEN do vector retrieval on the filtered set. We hand your engineering team the prompt design for the filter-extraction call, the schema for the structured-filter contract, and the eval set to measure the lift.

The architecture blueprint we hand off

The shape of a RAG stack that doesn't hallucinate is well understood. The hard part is getting from your current architecture to that one without burning a quarter on the wrong refactor. Here's the blueprint we document on every AI Operating Partner engagement when an omnichannel client comes to us with a "broken AI search" problem:

1. Event-driven indexer. Catalog writes fire embedding regeneration on the changed SKU only.
2. Structured chunking. Title + attributes, FAQs, reviews each get their own index, with metadata for source-section attribution.
3. Query router. A 50-token classifier ("transactional / informational / comparative") picks which index to hit.
4. Hybrid filter pre-pass. Pull structured constraints first, filter the candidate set, then vector-rank the remainder.
5. Citation enforcement. The generation prompt requires the model to cite which retrieved chunk supports each claim. If it can't cite, it can't claim.

We deliver each of those five components as an architecture spec, a vendor-agnostic decision tree, and an implementation playbook with the trade-offs your team will need to weigh in week one.

The operations takeaway

At SideB, we step in as your AI Solutions Manager. We run deep structural diagnostics on your AI stack, design vendor-agnostic architecture blueprints, and provide the exact playbooks your internal engineering team needs to execute the fixes. You write the code, and we coach your team to the finish line.

If you're seeing this pattern in your stack, book a 15-minute review — we'll walk the architecture against your actual setup.