Skip to main content
VectisFlow
VectisFlow
All services

Service 02

Intelligence extraction

Most of what an organisation knows is written down somewhere no query can reach. We build the pipelines that make it retrievable, attributable and safe to act on.

The problem this solves

Your warehouse holds what someone thought to model. It does not hold the surveyor's report, the supplier contract with the non-standard indemnity clause, the eighteen months of support tickets describing the same fault in nine different vocabularies, or the recorded call where a customer explained precisely why they left.

The usual answer is a search box. Search returns documents; leaders need positions. The gap between the two is extraction, turning a corpus into a set of typed, attributed, contestable facts that a person can check and a system can aggregate.

That gap is where most GenAI pilots stall. A demonstration over fifty clean PDFs proves nothing about a corpus of four hundred thousand documents with inconsistent scanning quality, three overlapping permission models, and twenty years of superseded revisions sitting alongside current ones.

Method

How we build it

The pipeline is assembled from stages we can each evaluate in isolation. That matters when something degrades: you need to know whether retrieval got worse, the parser broke on a new document template, or the model changed underneath you.

  1. 01 · Corpus survey

    Before any code, we sample the actual corpus: formats, scan quality, revision patterns, duplication rate, and the permission model each source enforces. The survey usually reveals that two or three document families account for most of the value, which changes what gets built first.

  2. 02 · Parsing and layout recovery

    Native text where it exists, OCR where it does not, layout-aware parsing for anything tabular. Tables and appendices carry a disproportionate share of the answers and are what naive text extraction destroys first.

  3. 03 · Chunking

    Structure-aware rather than fixed-window: sections, clauses and turns, with parent-document context retained and overlap tuned against a retrieval evaluation set rather than chosen by default. Chunk boundaries are the single most under-examined decision in most RAG systems.

  4. 04 · Enrichment

    Entity resolution across sources, date and revision normalisation, document-type classification, and typed metadata extraction so retrieval can filter before it ranks. A hybrid filter on document type and date usually beats a better embedding model.

  5. 05 · Retrieval

    Hybrid dense and lexical retrieval with reranking, tuned on a labelled evaluation set built from the questions people actually ask. We report recall@k and answer-supported rate, not vibes.

  6. 06 · Grounding and attribution

    Every generated claim carries a citation back to a specific chunk of a specific revision. Answers without a supporting passage are refused rather than improvised: an abstention is a feature for this audience, not a failure.

  7. 07 · Permission inheritance

    Access is resolved at query time against the source system's own ACLs, so a retrieved passage is only ever visible to someone who could already open the document it came from. Permissions are not a filter applied after retrieval; they are part of the index.

A long archive aisle lined floor to ceiling with wooden index drawers.

Deliverables

What you are left holding.

  • A running pipeline in your environment

    Deployed on your cloud, in your network boundary, using your identity provider. No data leaves your tenancy unless you decide it should.

  • An evaluation harness

    A labelled question set, a scoring pipeline, and a regression suite that runs on every change, so the system's quality is a number your team can watch rather than an impression.

  • The decisions, written down

    Chunking strategy, retrieval configuration, model choices and their trade-offs, documented well enough that your engineers can change them after we leave.

  • Handover and enablement

    Working sessions with the team who will own it, not a slide deck. We consider the engagement finished when your people have changed something in the pipeline themselves.

The hard parts

The questions worth asking us.

If a supplier cannot answer these specifically, they have not shipped one of these systems.

How do you stop it inventing things?

Three mechanisms, layered. Retrieval is tuned so the supporting passage is actually present in context; generation is constrained to cite specific retrieved spans; and any claim without a supporting span is refused rather than produced. We then measure the residual rate on a held-out set and report it. A system that abstains 8% of the time and is right the rest is far more useful to a regulated buyer than one that always answers.

What happens when documents are contradictory or superseded?

Revision handling is a first-class stage, not a cleanup step. Documents carry effective-from and superseded-by metadata where the source system exposes it, and are inferred from content where it does not. Retrieval prefers current revisions and surfaces the conflict explicitly where two current documents disagree, because that conflict is usually the finding.

How is permission inheritance actually preserved?

The index stores the source ACL alongside each chunk and re-resolves group membership at query time against your directory. A user's retrieval set is filtered before ranking, so ranked results never leak the existence of documents they cannot open. This is checked by a permissions regression suite that runs synthetic users against known-restricted content.

Where does a human stay in the loop?

Wherever an error is expensive and reversible only by a person. Typically: a review queue for extractions above a value threshold, a feedback control on every answer that writes to the evaluation set, and a periodic sampled audit. The review effort should fall over time, if it does not, the pipeline is not improving and we would rather find that in month two than month ten.

Related engagement

How this looks in practice.

Water treatment infrastructure at dusk, concrete channels running into the distance

Water & utilities

Making thirty years of asset documentation answerable at a UK water utility

A retrieval pipeline over 380,000 asset documents: drawings, condition reports, permits and handover packs, with permission inheritance preserved from the source document management system.

UK regional water utility, ~4,500 staff · 16 weeks

Read the engagement

Usually engaged alongside

Start here

Talk to us about intelligence extraction.

Bring the estate, the constraint and the question you want answered. Forty-five minutes is usually enough to tell whether this is viable, and we would rather say so early.

Response time
One working day, from a person who has read it.