Daniel KliewerThe book

Concept

Evaluation and information quality

How a system decides which information should be surfaced, trusted, or acted upon.

Ranking is a judgement about relevance, and relevance is only meaningful relative to a need. Page Quality and Needs Met are attempts to make that judgement explicit enough to be assessed, rather than leaving it implicit in a model's output.

The engineering version of the same question appears in every retrieval system here: what does it mean for a result to be good, and how would you know? A system that cannot answer that cannot be evaluated, only demonstrated.

This is where professional experience, research interest and implemented work need to be kept apart. The evaluation work informs the architecture; the architecture is what is actually running.

Implementation

Where this runs

A concept that never left a document is a hypothesis. These are the systems that implement it.