AI 101: The Manager's Glossary to AI

You don't need to write a transformer to run an AI-first org. You do need to know when a vendor is selling you architecture that will fail against your data, and when your engineering team is asking for something reasonable versus something extravagant. Four terms come up in every vendor pitch and every engineering roadmap meeting. This is the manager's version of each — accurate enough to hold the conversation, short enough to read before the meeting.

RAG (Retrieval-Augmented Generation)

What it is: the architecture pattern where the AI model doesn't answer from its training data alone. Before it generates a response, it retrieves relevant documents from your own database — product catalog, internal wiki, support tickets, contract library — and answers using those documents as ground truth.

Why it matters: without RAG, the model is answering from a snapshot of the internet that's months out of date. With RAG, it's answering from your current data. Every serious AI feature that touches your live business — search, recommendation, support agent, internal knowledge assistant — is RAG-based, whether the vendor advertises the term or not.

What can go wrong: the retrieval step is where 80% of AI failures actually live. If the retrieval returns wrong or stale documents, the model faithfully generates a wrong answer with total confidence. Most 'the AI is hallucinating' complaints are actually retrieval failures. (We wrote a longer piece on this at Why your omnichannel RAG is hallucinating.)

Tokens

What it is: the unit of text that language models actually operate on. A token is roughly ¾ of a word on average — so 100 words is ~130 tokens, and a 10-page PDF is ~5,000 tokens.

Why it matters: every AI vendor prices in tokens. Cost per 1,000 input tokens, cost per 1,000 output tokens, cost per cached token. When your engineering team says 'we're going to send the whole contract to the model,' they're saying 'we're going to pay to process every token in that contract.' Token accounting is the AI equivalent of MDR: if you don't understand the unit, you don't understand the invoice.

What can go wrong: unbounded prompts. A well-meaning engineer wires up the model to receive the full conversation history on every turn, and your token cost doubles every week. Token budgets are the AI cost control you need before the invoice arrives.

Chunks

What it is: the way you split up a long document (a product page, a policy document, a support article) into smaller pieces the retrieval system can index and match against a query. A 'chunk' is one indexed piece — typically 200–500 tokens, though the right size is a design decision, not a default.

Why it matters: chunking is where the RAG architecture makes its biggest silent trade-offs. Too big and the retrieval brings back irrelevant context; too small and it fragments the meaning of the source document. The wrong chunking strategy is the difference between a support agent that quotes the right paragraph of policy and one that quotes half a sentence from a footnote.

What can go wrong: default chunking. Most vendors ship a 512-token fixed-window chunker as a default. That default works fine for Wikipedia. It fails badly on structured commerce data (product attributes, reviews, FAQs each want their own chunk shape) or long contracts (clauses want to be preserved, not arbitrarily sliced). If your AI feature 'sort of works but gives weird answers,' chunking is the second place to look after retrieval.

Hallucinations

What it is: when the model generates a plausible-sounding answer that's factually wrong. Not a bug in the code — an inherent property of how language models work. They're optimized to produce fluent text, not necessarily true text.

Why it matters: hallucinations are the risk category that gets AI features pulled from production. A confidently-wrong price on a product page is a customer-service ticket. A confidently-wrong policy citation in a legal review is a regulatory issue.

What can go wrong (and how to guard against it): the three highest-leverage controls are (a) citation enforcement — the model must cite the retrieved document it used, and if it can't cite, it can't claim; (b) structured output — where possible, force the model to return JSON that matches a schema, not free text; (c) eval sets — a curated set of known-answer questions the AI feature is regression-tested against on every prompt change. Hallucinations don't go to zero, but they go to managed.

The Operations Takeaway

This is exactly the kind of structural work SideB is built for. We come in alongside the leaders who own this seam — CTO, VP Product, Head of Ops, CFO — and provide the steer between the roadmap and the invoice. That means reviewing the vendor contracts before the auto-renewal locks you in, auditing the configuration against what the vendor sold you, and holding the operating cadence that keeps the number honest against your live data.

Your team stays in charge of execution. Our value is the outside pattern-match — what other operators at your scale have already learned, priced, and negotiated — brought back to your specific stack every week, in your standups, on your calls with vendors. When the engagement ends, your team owns the muscle memory.

If this is a live conversation on your team right now, book a 15-minute review — we'll walk it against your actual environment.