RS Glossary
Evidence Selection
Evidence selection is the process of deciding which observations, records, sources, examples, or test results will be used to evaluate a claim or support a decision.
- Review state
- Reviewed in POC
- Freshness
- Current
- Version
- 1
- evidence-selection
Remember it this wayThe evidence you choose sets the boundary of what you can conclude.
In plain language
A conclusion depends partly on what evidence is allowed into the analysis. Good selection seeks relevant, reliable, and sufficiently representative evidence, including material that could challenge the preferred answer.
Why it matters
Careful reasoning cannot rescue a distorted evidence set. Selection choices can hide failures, overrepresent convenient cases, or make a system appear safer and more capable than it is in real use.
Example
A team evaluates an AI support tool using only short, well-structured questions written by its developers. It excludes long, ambiguous requests from actual customers. The resulting accuracy score describes the selected test set, not the full service environment.
Common misconception
More evidence is not automatically better evidence. Repeated copies of one claim do not create independent support, and a large convenient sample can still omit the people, conditions, or counterexamples that matter.
Related terms
Sources
- NIST, “AI RMF Core: Measure”representative evaluation conditions, documented test sets, meaningful measurement, and consultation with affected actors.
- NIST, “Artificial Intelligence Risk Management Framework 1.0”context-dependent measurement, population differences, baselines, and risks in real-world settings.
Learn more
Public history
23 recorded events · 23 public summaries
Candidate vocabulary was imported from the Learning Lab discovery queue.
The captured candidate received a complete initial draft and entered section-level human review.
Radar reviewed the initial definition draft for human consideration.
Radar reviewed the initial plain-language-explanation draft for human consideration.
Radar reviewed the initial why-it-matters draft for human consideration.
Radar reviewed the initial example draft for human consideration.
Radar reviewed the initial common-misconception draft for human consideration.
Radar reviewed the initial remember-it-this-way draft for human consideration.
Radar reviewed the initial related-terms draft for human consideration.
Radar reviewed the initial sources draft for human consideration.
Radar reviewed the initial learn-more draft for human consideration.
A proposal was opened for section-level human review while preserving the current version.
A human reviewer opened a proposed revision while preserving the current approved or captured version.
definition approved by a human reviewer.
plain-language-explanation approved by a human reviewer.
why-it-matters approved by a human reviewer.
example approved by a human reviewer.
common-misconception approved by a human reviewer.
remember-it-this-way approved by a human reviewer.
related-terms approved by a human reviewer.
sources approved by a human reviewer.
learn-more approved by a human reviewer.
All section gates are approved; content version 1 is now a fully approved term.
Source: vault/Glossary/terms/evidence-selection/term.md · sha256:d11309e82e6fd3a4d159f9514b86df21cb800eb7a19d92e73b9ac4ad7402346f · Release: rs-glossary-glossary-team-poc-0.5.5-2026-09-02