How much data does an AI assistant in the technical office need: real sizing

Reading time: about 6 minutes · Who it is for: heads of technical offices, people scoping an AI assistant pilot, owners of manufacturers wondering how much documentation they need to gather.

Verdict up front: the quality of an AI assistant in the technical office is not decided by gigabytes, but by the number of consistent, current and unambiguous sources. A corpus of 300 clean, structured pages can deliver better retrieval than 30 thousand pages full of duplicates and stale versions. The naive setup („let us just point it at the whole network drive”) fails not from a lack of data, but from too much of it with no order.

The short answer

There is no single „minimum number of documents” at which an AI assistant starts to work. That is the wrong end of the question. Retrieval augmented generation (RAG) does not learn your documentation the way a person does. For every question it retrieves a handful of the best matching fragments and builds the answer on those alone. Quality therefore depends on whether the right fragment can be found and whether it is not competing with five outdated versions of the same document.

In practice a few hundred well prepared pages are already enough for the assistant to answer sensibly about a specific product or process. The problem does not appear with too small a corpus, but when the corpus grows without control: duplicates, scans with no text layer, twenty revisions of the same instruction. Below I show how to read document volume through the lens of retrieval quality rather than sheer size.

Sizing through retrieval, not volume

The useful unit is not „number of files” or „size in GB”, but the number of fragments, that is chunks. A document is split into pieces of a few hundred tokens each, and those are what land in the index and what the search compares against a question. A 40 page manual is a few dozen chunks. Ten thousand pages of documentation is well over ten thousand chunks, and with them grows the risk that a single question turns up several similar, contradictory fragments.

The table below is a set of rough engineering thresholds, not a result from a specific client. Treat it as a risk map, not a promise.

Corpus volume	Order of chunks	Main risk	What really decides quality
up to about 300 pages	hundreds	almost none	correct split into sections
300 to 3000 pages	thousands	first duplicates and stale versions	metadata: product, version, date
3000 to 30000 pages	tens to a hundred thousand	version drift, scans with no OCR	filtering by metadata, reranking
above 30000 pages	hundreds of thousands plus	noise drowns the signal	hard curation of the corpus before indexing

The takeaway is counterintuitive: going from 3000 to 30000 pages does not improve the assistant linearly. It often makes it worse, if those extra pages are further versions and copies of the same thing. More data without order means more chances for retrieval to pull in the wrong fragment.

Why the naive setup falls apart

Teams most often start with „let us connect the whole department drive and see”. Here are the concrete reasons this usually fails:

1. Duplicates and revisions. The same instruction in version 3, 4 and 5 sits in three folders. The search does not know which one applies and cites a random one. The answer sounds credible and comes from a withdrawn version.
2. Scans with no text layer. A large share of older technical documentation is PDFs made of images. Without OCR that is a blank page to the model. The volume is there, the content in the index is not.
3. No metadata. If a chunk does not know which product, version and date it belongs to, it cannot be filtered out. A question about product A pulls in a fragment about product B, because it reads similar.
4. Chunks too large or too small. Too large mixes several topics in one fragment and blurs the match. Too small loses context, for instance a table number without its contents.
5. No reranking. A vector search on its own returns „similar”, not necessarily „right”. Without a second pass that reorders the candidates, the model gets mediocre material.

The common denominator: none of these problems is solved by adding data. They are solved by order before indexing and by retrieval configuration after it.

A practical threshold of sense

Instead of asking „how many documents do I need”, ask four things:

• Do I have a single source of truth per product or process, or several competing versions?
• Do the documents have a text layer (not just a scan)?
• Can metadata be attached to every fragment: product, version, date, department?
• Will someone maintain the corpus after go live, or will it freeze on day one?

If the answers are yes, a sensible pilot can be built on a narrow slice, for instance the documentation of a single product line. If they are no, no volume will make up for it, and a large corpus only adds noise. A small, curated scope beats a large, cluttered one.

Footprint: what it means for infrastructure

The vector store itself for tens of thousands of chunks is on the order of single gigabytes, so storage is not the bottleneck. The real costs are the compute to recompute embeddings on every documentation change and the memory of the answering model. For an assistant serving one department, running locally, a single server class GPU is usually enough. The jump in cost comes not from the number of pages, but from the number of concurrent users and how often you reindex the corpus.

What this article does not cover

I do not go here into the choice of a specific model or an embeddings benchmark, because that depends on the language of the documentation and the hardware. Nor do I discuss deployment cost in money or ROI, which is a subject for a separate business analysis. I leave aside security and the on-prem versus cloud question, which we cover in separate posts. I focus only on the relationship between how much and what kind of data and the quality of the answers.

Related

• RAG for technical documentation: how AI uses machine manuals and datasheets
• RAG without hallucinations: chunking, retrieval and what decides answer quality
• AI for managing service knowledge: how to feed the system and where the limits are
• AI in the engineering design office: what it really automates and what it just promises
• AI in technical documentation quality control: what it catches, what it misses