Privilege AIMKC2 results ↗

Enterprise RAG

Enterprise retrieval fails on the unglamorous parts: scanned documents, six source systems, permissions that change hourly, and a corpus that grows daily.

READ WHAT CHANGES ↓SOURCES · FRESHNESS · SCALE

01 / DEFINITION

The short answer first.

What is enterprise RAG?

Enterprise RAG is retrieval-augmented generation deployed across an organisation's real document estate: multiple source systems, inconsistent formats, inherited permissions, continuous change and volumes large enough that ingestion and freshness become engineering problems in their own right.

The gap between a prototype and an enterprise deployment is rarely the model or the vector store. It is the ingestion pipeline, the permission model and the operational commitment to keeping an index current — none of which are visible in a demonstration built on a clean folder of PDFs.

WORKED EXAMPLE

Resolve a policy across two enterprise repositories

A service manager asks which warranty applies to a replacement shipped last quarter. The CRM identifies the customer and shipment date, while the document repository contains the warranty revision in force at that time. A newer marketing summary describes the current offer. An enterprise RAG answer needs the right relationship between those records as well as a relevant passage.

Define which system owns each fact

Identify the source of the shipment date, the applicable policy revision and the current user's permission to view the account. Do not let a recent marketing document override a historical agreement merely because it is easier to retrieve. Use effective dates and source roles in the answer. The buyer's contract date and the publication date of a summary answer different questions.

Create an authorised test account and a similar account the service manager cannot open. Run the warranty question for both. Inspect the join between account data and retrieved documents, the passages sent to the model and the sources shown to the user. A correct policy quotation attached to the wrong customer is a material failure even when each repository was searched successfully.

Test an outage in the CRM while the policy index remains available. The system can still explain how warranties are determined, but it may lack the account-specific fact needed to select a revision. Make that uncertainty visible instead of confidently returning the newest policy. Record the dependency, responsible team and recovery behaviour so support can diagnose a partial failure without blaming the language model for every missing fact.

02 / WHAT ACTUALLY BREAKS

Problems that appear between a prototype and a deployment, and what each one costs.
ProblemWhy it appearsConsequence if ignored
Scanned documentsA real estate contains images of text, not textSilently missing content; answers with no sources
Format sprawlEmail, spreadsheets, presentations, archives, legacy formatsPartial extraction that nobody notices
Tables and structureNaive extraction destroys tabular meaningConfidently wrong numerical answers
Duplicates and versionsThe same document exists in five places at three revisionsThe model cites a superseded version
Permission inheritanceRights live in the source systems, not the indexThe index becomes the widest access path in the organisation
FreshnessThe corpus changes continuouslyAnswers drawn from material that has been superseded or deleted

Scanned material is the most consequential because its failure is silent. A pipeline that cannot read an image-based PDF produces no error; the document is simply absent from every answer, and the gap is invisible until someone notices a citation that should have existed. Measuring extraction coverage — what share of the corpus yielded usable text — is a basic ingestion metric and is rarely collected.

Version confusion is the second. In document estates where superseded drafts sit alongside executed versions, retrieving the wrong one produces an answer that is fluent, sourced and wrong. Version and status metadata belong in the index, and the retriever should prefer current material explicitly rather than by accident of ranking.

03 / INGESTION AS A PRODUCT

01

Extract, and measure coverage.

Text from every format, with optical character recognition where needed, and a recorded share of documents that yielded usable text.

02

Normalise and deduplicate.

One canonical record per document, with versions linked rather than duplicated as independent entries.

03

Chunk with structure in mind.

Boundaries that respect headings, clauses and tables, with overlap. This sets the recall ceiling for the whole system.

04

Carry permissions and provenance.

Access metadata, source, version, status and date on every chunk, from the first version of the pipeline.

Step four is the one that cannot be retrofitted cheaply. An index built without access metadata has to be rebuilt to become identity-aware, and by the time that is discovered the pipeline is usually processing millions of chunks. Carrying permissions from day one costs a schema field. See RAG access control.

Ingestion is also where most of the ongoing operational load sits. New documents, changed documents, deleted documents, re-processing after a chunking improvement, and re-embedding after a model upgrade are all continuous activities, and a full re-index of a large corpus is a scheduled project rather than a command.

04 / WHAT TO BUILD FIRST

↳ 01

One source system, one user group.

A narrow, permissioned slice that reaches real users, rather than a complete index that reaches nobody.

↳ 02

The evaluation set before the second source.

Without it there is no way to tell whether adding a source improved anything or diluted retrieval.

↳ 03

Coverage and freshness dashboards.

Share of the corpus successfully extracted, indexing lag, empty-retrieval rate. These predict user-visible quality better than any model metric.

Adding sources is not monotonically beneficial. Each new source adds candidate passages that compete for a fixed context budget, and a low-quality or redundant source can reduce answer quality measurably. That is precisely the kind of change an evaluation set makes visible and intuition does not — which is why the measurement belongs before the expansion. See RAG evaluation and private AI deployment.

05 / QUESTIONS

Why does an enterprise RAG prototype degrade in production?

Usually ingestion and volume. A prototype runs on clean, curated documents; production includes scans, legacy formats, duplicates, superseded versions and material the pipeline silently fails to read. Extraction coverage is the first number to measure.

How do you handle scanned documents?

Optical character recognition in the ingestion pipeline, with a recorded quality signal, because poor recognition produces plausible nonsense that is worse than an omission. Track the share of the corpus that yielded usable text and treat a low figure as a defect.

Should everything go into one index?

Not automatically. More sources mean more competition for a fixed context budget, and a redundant source can reduce quality. Partitioning also makes permission isolation structural rather than filter-dependent. Measure the effect of each addition.

How current does an index need to be?

As current as the decisions taken from it require. For a matter a person is actively working on, minutes; for a historical archive, days. Choose per source, and make the indexing lag a monitored metric rather than an assumption.

What is the realistic operational load of enterprise RAG?

Continuous: ingestion for new and changed documents, deletion propagation, periodic re-chunking and re-embedding as the pipeline improves, plus permission synchronisation. It is an operated service, not a project that completes.

WORK WITH PRIVILEGE AI

How we can help

Talk with Privilege AI

Tell us which enterprise repositories should contribute answers, which system owns the underlying facts and how users reach the result.

Talk to us →

CONTACT PRIVILEGE AI

Contact Us

Tell us which enterprise repositories should contribute answers, which system owns the underlying facts and how users reach the result.

When should we call?

By submitting, you agree that Privilege AI, a division of Tech1e LLC, may call or email you about this inquiry. Please leave confidential information out of your message.

Make the corpus
work.

Privilege AI builds retrieval and knowledge systems, including the ingestion and permission work that decides whether they hold up.