Making thirty years of asset documentation answerable at a UK water utility

Context

A regional water utility holds the documentary history of its network in a document management system accumulated over three decades and two mergers: as-built drawings, condition surveys, discharge permits, contractor handover packs, and the correspondence explaining why a given asset was built the way it was.

Field and planning engineers needed a specific class of answer from it, what is under this road, when was this asset last surveyed, which permit conditions apply to this discharge point, and were spending a meaningful share of each week not finding it.

The problem

The document management system had a search box. It matched filenames and whatever metadata had been typed at upload, which for pre-2011 material was frequently a site code and nothing else.

The practical consequence was escalation. An engineer who could not find a condition report within a few minutes raised a request with the asset information team, who searched more skilfully but no more automatically. That team had become a queue, and the queue had become the bottleneck on planning decisions.

The corpus itself was the harder problem. Roughly a third was scanned at inconsistent quality, some of it from microfiche. Drawings carried their meaning in title blocks and annotations rather than in prose. Superseded revisions sat alongside current ones with no consistent marker distinguishing them, and a wrong answer drawn from a superseded drawing is worse than no answer at all.

Why earlier approaches failed

An earlier enterprise search deployment indexed the text layer and returned document lists. It improved recall over filename search and changed nothing about the workload, because the work was never finding candidate documents: it was reading them to find the one paragraph that mattered.

A subsequent proof of concept put a language model over the same index. It answered fluently and was withdrawn within weeks: it cited superseded drawings without flagging them, and it could surface content from commercially restricted contractor packs to users who had no rights to the underlying documents. Both failures were architectural rather than incidental.

The pipeline we built

We rebuilt the retrieval path with revision state and access control as first-class stages rather than post-processing filters.

1. 01Corpus survey

   Three weeks sampling 2,400 documents stratified by era, source system and type. It established that four document families accounted for 78% of the questions being asked, which narrowed the parsing work substantially and pushed drawings: the hardest family, into a second phase with its own budget.

2. 02Parsing and layout recovery

   Native text extraction where the layer existed; OCR with a document-quality classifier routing poor scans to a higher-cost model. Condition reports are heavily tabular, so tables were extracted as structured records rather than flattened into prose, preserving the row-to-asset relationship that the answers depend on.

3. 03Revision resolution

   A dedicated stage inferring revision lineage from title-block metadata, document numbering conventions and issue dates, producing an explicit supersession graph. Retrieval defaults to current revisions; superseded content is retrievable only when a user asks a historical question, and is labelled as superseded in the response.

4. 04Chunking

   Structure-aware, on section and clause boundaries, with the parent document's title block and asset references attached to every chunk. Fixed 512-token windows were tested first and lost on recall by a wide margin, because they routinely severed a finding from the asset identifier that gave it meaning.

5. 05Enrichment

   Asset-identifier resolution against the asset register, so a chunk mentioning a site code, an alias and a legacy reference resolves to one asset. Date normalisation across seven observed formats. Document-type classification to allow filtering before ranking.

6. 06Retrieval

   Hybrid dense and BM25 retrieval over pgvector, with metadata pre-filtering on asset, document type and revision state, then cross-encoder reranking of the top 50. Pre-filtering carried more of the improvement than any embedding-model change tested.

7. 07Permission inheritance

   Source ACLs are indexed alongside each chunk and re-resolved per query against the directory, so the candidate set is filtered before ranking. A permissions regression suite runs synthetic users against known-restricted contractor packs on every release, and a failure blocks the release.

8. 08Grounding and abstention

   Answers cite the specific passage, document and revision. Where no retrieved passage supports a claim, the system returns what it did find and says the question is not answerable from the corpus, rather than generating a plausible sentence.

9. 09Evaluation

   310 questions collected from the asset information team's request queue, labelled with acceptable answers and supporting passages. Scored per document family and run as a CI gate.

What shipped

• A retrieval service inside the utility's Azure tenancy, integrated with the existing document management system and Entra ID.
• A query interface embedded in the planning tool engineers already use, rather than a separate destination.
• A permissions regression suite and a retrieval evaluation harness, both owned and run by the client's platform team.
• A review queue where an asset information specialist adjudicates flagged answers, with every adjudication written back to the evaluation set.

Outcomes

Each figure below carries the method behind it and the baseline it is measured against, which is the form a result has to take before it means anything.

Time to a sourced answer

Median time for an engineer to reach a cited passage, against a pre-engagement baseline drawn from timings in the request queue.

4 min 10 s → 38 s

Requests escalated to the asset information team

Weekly escalation volume, comparing the quarter before go-live with the quarter after general release.

312 → 96 per week

Answer-supported rate

Share of answers where the cited passage genuinely supports the claim, on the 310-question held-out set. Baseline is the earlier search-plus-model proof of concept.

71.6% → 94.2%

Permission regressions reaching production

Three were caught by the synthetic-user suite before release: two from a group-nesting change, one from a re-permissioned contractor folder that had not been re-indexed.

3 caught, 0 shipped

What next

Phase two extends parsing to as-built drawings, where the answer lives in the annotation layer rather than in prose, and pushes the same retrieval path into the field application used on site.