What "AI Reads Your Docs" Actually Means: Chunking and Retrieval

Picture someone who uploads a 60-page policy PDF, asks the AI a question that is clearly answered on page 41, and gets a hedge back. The natural conclusion is that the tool is broken. It is not. The AI never read page 41. When an AI support tool answers from your docs, it does not read the whole file the way you would. It retrieves a few small pieces and answers from those. Once you understand that one mechanic, most of the advice about writing support docs stops sounding like arbitrary rules and starts making sense.

The mental model most people get wrong

I get a version of that complaint more often than almost any other question about Trigli, and it nearly always traces back to one wrong picture. The intuitive picture is that the AI has read everything you uploaded, holds it all in its head, and consults the relevant part when a question comes in. That is not what happens. The model does not see your whole knowledge base on any given question. It sees the small slice a search step decided was most relevant, and nothing else.

So the real question is never 'is the answer somewhere in my docs.' It is 'will the search step find the exact passage that holds the answer, for this specific way of asking.' The answer can be sitting in your knowledge base, perfectly correct, and still never reach the customer because the passage that holds it did not surface for their phrasing. That gap is where most 'the AI does not know our own policy' complaints actually live. This is the follow-up to /blog/build-ai-support-knowledge-base, which told you what to put in your knowledge base. This one explains the machinery underneath, because the machinery is why the structure advice works.

What actually happens when you upload a document

There are three steps, and only the middle one is invisible to most people. Understanding all three is enough to make you better at this than most teams running AI support.

Step one is chunking. Your document gets split into smaller passages, so a long PDF becomes many pieces rather than one 60-page block. This is not a quirk of one tool. Every retrieval-based AI support product does some version of it, because you cannot hand a model a 60-page document for every incoming question and expect a fast, focused answer.

Step two is embedding. Each chunk gets converted into a vector, which is just a long list of numbers that captures the meaning of that passage. Passages about refund timing land near other passages about refund timing. The point of turning text into numbers is that a computer can then measure how close two pieces of meaning are, even when they do not share the same words. This is why a customer can ask 'when do I get my money back' and the system can still find a chunk that says 'refunds are processed within five business days,' despite not one word overlapping.

Step three is retrieval. When a question comes in, the tool turns it into a vector the same way, then finds the chunks whose vectors are closest. Those few closest chunks - not the whole document, just the top matches - are what the model actually reads before it writes a reply. Everything else in your knowledge base is invisible for that question.

Why volume is not knowledge

Here is the part that surprises people, and it is the one place I most often tell someone to ignore their instinct. A bigger knowledge base does not reliably produce better answers. Past a point, it produces worse ones. The instinct is to upload more - every help article, the entire website, five years of policy PDFs - on the theory that more coverage means more correct answers. Retrieval does not reward that instinct, and I would rather you upload less.

Two things go wrong when you dump volume in. The first is dilution. When a single chunk covers six unrelated topics, its vector is a blurry average of all six, so it competes weakly against a focused chunk that is about exactly one thing. A tight 200-word refund doc has a chunk that screams 'refunds' in vector space. A 3,000-word 'everything you need to know' doc has a chunk that mumbles about refunds somewhere between shipping and account setup. The tight doc wins retrieval. This is the mechanical reason the earlier post kept saying one topic per file.

The second is contradiction. If your website says 30-day returns and an old PDF says 14-day returns, both chunks exist and both can be retrieved. Sometimes the system pulls the wrong one, and the AI is now confidently quoting a policy you retired. Retrieval has no way to know which of two plausible passages is the current truth. It just returns what is closest in meaning.

So the goal is not maximum coverage. It is maximum retrievability of the things customers actually ask about. Those are not the same target, and chasing the first one actively hurts the second.

What a retrievable chunk looks like

A good chunk is self-contained. It answers a question on its own, without needing the paragraph above it or the heading three pages back. That is the whole game, because retrieval pulls the chunk out of its surroundings and shows it to the model alone.

Consider two ways of writing the same policy. The weak version reads: 'As mentioned above, it is 30 days, and the same conditions apply.' Pulled out on its own, that chunk is useless. Thirty days for what? Which conditions? The context it leans on did not come along for the ride. The strong version reads: 'Refunds are available within 30 days of delivery. Items must be unused and in original packaging. Refunds are processed to the original payment method within five business days of approval.' That chunk answers the question by itself, no matter what surrounded it in the original document.

The pattern to internalize: write in plain declarative sentences, put the answer first, and repeat the subject instead of leaning on 'it' or 'the above.' A customer support doc is not an essay with a satisfying build. Every passage should be able to stand up alone and be quoted, because that is exactly what is going to happen to it.

Where chunking quietly breaks

A few specific formats fight the chunking step, and they fail silently, which is the dangerous part. Nothing errors. The AI just misses answers that are technically present.

Tables are the biggest one. A shipping matrix that reads perfectly to a human eye - regions down the side, speeds across the top, prices in the cells - can get flattened into a row of numbers with no labels once it is chunked. The link between 'express' and '2 days' and '$15' lives in the visual grid, and the grid does not always survive. If a table holds critical facts, paste the key ones as plain sentences alongside it: 'Express shipping is $15 and arrives in 2 business days.' Redundant for a human. Essential for retrieval.

Long legal documents are another. Terms-of-service language is written to be complete and defensible, not to answer a customer question in one passage. Retrieval over a 20-page terms doc tends to pull dense, hedged clauses that do not directly answer 'can I get a refund.' The fix is a plain-language FAQ covering the questions people actually ask, with the full legal text left out of the knowledge base entirely.

Answers buried deep in a paragraph are the quiet third one. If the real answer is in sentence four, and the first three sentences are throat-clearing about how much you value the customer, the chunk's meaning is dominated by the throat-clearing. Marketing preamble does not just waste space. It shifts what the chunk is about, and therefore whether it gets retrieved at all.

How to structure docs for retrieval

None of this requires understanding vectors. It requires writing docs the way retrieval likes to eat them. Five rules cover most of it.

1. One topic per document. A refund doc, a shipping doc, a pricing doc, an account-access doc. Not one master file. Focused docs produce focused chunks that win retrieval against blurry ones.
2. Answer first, context second. Lead each section with the fact. Put the caveats and the reassurance after it, not before it. The chunk should be about the answer, not about your brand voice.
3. Write self-contained sentences. Repeat the subject. 'Refunds are processed within five business days' beats 'they are processed within five business days' every time, because the second one loses its meaning the moment it is pulled out alone.
4. Turn critical tables into sentences. Keep the table if you like it, but paste the key facts as prose next to it so retrieval has something clean to grab.
5. Resolve contradictions before uploading. If two docs disagree on a number, retrieval cannot referee. Fix the source of truth first, then upload only the current version.

If reshaping existing docs sounds like the hard part, /blog/build-ai-support-knowledge-base includes a copy-paste prompt that turns the rewrite into an interview with an AI assistant, encoding exactly these rules for you.

Reading the symptoms

When retrieval is the problem, the symptom usually points straight at the cause once you know how to read it.

How to test retrieval in ten minutes

You do not have to guess whether your docs are retrievable. Take the ten questions your team answers most often and ask each one phrased the way a customer would phrase it, not the way your policy is written. Then check one thing per answer: did the AI pull the right fact. Not 'does it sound plausible' - does the specific number or policy match your current source doc. Each miss is a retrieval gap, and the fix is almost always to rewrite the passage that should have answered it as a cleaner, self-contained chunk.

The most useful test of all is asking something your docs do NOT cover. A retrieval-grounded tool should come up empty and flag the question for a human rather than inventing an answer, because there is no relevant chunk to retrieve. If it makes something up instead, that is a separate and more serious problem, and the setup post at /blog/set-up-ai-customer-support-stop-hallucinations covers how confidence routing is supposed to catch exactly that case.

The honest limits of retrieval

Meaning-based search closes a lot of the vocabulary gap, but not all of it. If your industry uses one word for something and your customers use a completely different one, the vectors may not land close enough and the right chunk stays hidden. The hedge is to write docs using the words customers actually use, not only your internal terminology. If people ask about 'money back' and your doc only ever says 'reimbursement,' add the customer word somewhere in the passage.

Retrieval also cannot save a doc that is wrong. If the chunk it finds is accurate but outdated, the AI quotes it with total confidence, because retrieval measures relevance, not truth. That is why keeping the knowledge base current is its own ongoing job, covered in /blog/keep-ai-support-bot-knowledge-base-current. Clean structure makes the right chunk findable. It does not make a stale chunk correct.

I would rather say this plainly than pretend clean docs make retrieval perfect: it will occasionally just miss, even on a well-written doc, for a phrasing you did not anticipate. This is the real argument for draft-and-approve as your starting posture rather than full auto-send. A human in the loop catches the miss before the customer sees it, and every miss you catch is a passage you can rewrite so it does not miss again.

How this works in Trigli

I built Trigli around exactly the mechanics in this post. It takes knowledge base sources three ways: PDF uploads, paste-in text, and URL imports. Whatever you give it gets chunked and embedded, and at answer time the AI retrieves the closest chunks and writes from those. I made it flag uncertain questions for human review rather than guess. It does not browse the open web for policy answers, and I deliberately did not let it fall back on training data for facts about your business. That constraint is the point.

One plan detail connects to the volume argument. Document counts are capped by plan, from a small number on the free tier up to unlimited on the top plan. Some people read a cap as a reason to cram everything into one file to stay under it. Do the opposite. A handful of focused, one-topic documents beats one bloated file on answer quality, so splitting docs well works with the grain of retrieval rather than against the limit. The full plan breakdown is at /pricing.

What Trigli does not do

Retrieval quality is a real strength, but it is not magic and it does not cover everything. Trigli does not process actions inside your store or backend - the AI can quote your refund policy and explain how to request one, but it does not touch Shopify to issue the refund. There is no phone or SMS channel. There is no per-agent productivity reporting, and no skill-based routing that auto-assigns tickets to specific people by tagged expertise. And one account maps to one brand, so multi-brand routing inside a single tenant is not a thing yet.

Everything in this post about chunking and retrieval applies to any retrieval-grounded AI support tool, not only this one. The mechanics are the same whether you run Trigli, a larger platform, or something you wire together yourself. Good doc structure is portable knowledge.

The short version to take with you

The AI does not read your knowledge base. It retrieves a few chunks and answers from those. That single fact explains why one focused doc beats a giant one, why self-contained sentences beat flowing prose, why tables need a plain-text twin, and why more volume can make answers worse instead of better. Structure is not busywork. It is the difference between a fact that is present and a fact that is findable, and only the findable ones ever reach a customer. The free tier is a low-stakes way to test which of yours are which: upload a few docs, run the ten-question test, and the misses are your rewrite list.