Reviewer effectiveness testing

What it does

It measures whether human review is working. Indicators of review quality are tracked: how often reviewers disagree with the system, how long they take, and whether their decisions hold up on later examination. Known errors are inserted into the review queue to see whether they are caught. Results are reported, and poor results lead to change in workload, interface, training or the design of the review. The business owner of the process is accountable; the second line or quality assurance runs the tests. It operates on a set cycle throughout the life of the review.

How to implement it

• Track override rate, time per case and outcome of overrides, by reviewer and over time.
• Seed the queue with cases where the system's output is known to be wrong.
• Re-review a sample independently, without sight of the system's output.
• Set thresholds, such as a minimum detection rate, with defined action.
• Ask reviewers what prevents them from disagreeing.
• Act on findings, and retest after each change.

Evidence it operates

Design: a testing plan with measures and thresholds. Operation: trend reports on review indicators; seeded-case results; independent re-review findings; changes made to the review as a result.

Where it fits

In operation, as the check on human-in-the-loop review. It is the primary control for automation bias and over-reliance and ineffective human review, and for human-agent trust exploitation, where an agent's persuasiveness is what defeats the reviewer.

Limits

Reviewers who know they are being tested may behave differently for a while. Seeded errors have to be realistic and rare enough not to be spotted as tests. A low override rate may mean the system is good and not that the reviewers are passive, which is why seeded cases are needed to tell the two apart. Measuring individuals can become surveillance and should be handled as a test of the process.