Methodology guide
Reviewing with domain expertise
Match review work to the knowledge it requires, and make professional judgments understandable to the next reviewer.
When to use it
Start with a clear purpose.
Use domain review where a mechanical check cannot establish correctness, relevance, or practical usefulness. The review plan should identify the required experience for each task instead of assuming one credential covers every decision.
Step 01
Specify the review roles
Separate task authoring, independent scoring, and disagreement resolution. Match each role to its subject knowledge and working context; record conflicts that could affect judgment.
Step 02
Calibrate on shared examples
Ask reviewers to score a small common set independently. Compare reasons as well as scores, and clarify criteria before reviewing a larger batch.
Step 03
Record and resolve differences
Attach each judgment to a criterion and evidence. Route unresolved cases to an agreed adjudicator, then record the decision and any resulting rubric change.
Acceptance
Make the result reviewable.
Require complete review records, explicit uncertainty, and documented decisions on disputed cases. Report reviewer agreement for the actual batch; avoid treating credentials alone as evidence of output quality.