The most common request we receive after a first demonstration is to reduce how often the system says it does not know. It is an understandable reaction and usually the wrong instinct.
An answer you have to verify saves you nothing. If a system answers every question and is right most of the time, every answer must be checked, and checking costs roughly what finding the answer cost in the first place. The efficiency gain evaporates.
What abstention buys
A system that abstains where support is absent partitions its output into a set you can act on and a set you must handle yourself. The first set is the value. Its size matters less than its reliability.
This is also the behaviour that changes the conversation with risk functions. 'It is right 94% of the time' invites a question about the other 6%. 'It answers 82% of questions and, on those, cites a supporting passage 97% of the time, and declines the rest' describes a control, and controls are approvable.
How to build it
Abstention is not a prompt instruction. It is an architectural property with three parts: retrieval that reliably surfaces the supporting passage when one exists, generation constrained to cite retrieved spans, and a check that refuses output where no cited span supports the claim.
Then measure both halves. Track the abstention rate and the answer-supported rate together: one without the other is easy to game. A system can reach a perfect supported rate by declining almost everything, and that is a different failure.
- Abstention rate: share of questions where the system declines
- Answer-supported rate: of answered questions, the share whose citation genuinely supports the claim
- False abstention: questions the corpus could have answered but the system declined: the cost of tuning too conservatively
The third metric is the one teams forget, and it is the one that keeps abstention honest.
