Phase 9: Reconciliation Model and Authority¶
Release 2 -- Reconciliation Data Model Phase 9 creates the data model that makes reconciliation possible: the system learns to determine when a study's annotations are authoritative.
Summary¶
This phase introduces the concept of authority into SyRF's annotation system. For any study, the platform will determine whether a single "gold-standard" answer exists -- and if not, flag the study for reconciliation. Two paths to authority:
- Auto-promotion: When a single annotator completes a study and the project only requires one annotator, their answers automatically become the gold standard
- Reconciliation pool: When two or more annotators have completed a study, it enters a pool for a reconciler to resolve
The Problem¶
Today, SyRF has no concept of a gold-standard answer. When multiple annotators answer the same questions for the same study, their answers simply coexist. There is no mechanism to:
- Determine whether a study's annotations are authoritative or still need review
- Automatically identify studies where annotators disagree
- Track which studies have been reconciled and which have not
- Produce a single, auditable answer set that can be trusted for analysis
What We Are Building¶
A ReconciliationSession entity -- a versioned gold-standard record per study. Each study gets at most one reconciliation session that tracks the authoritative answers across all stages.
Authority determination rules that automatically classify studies:
| Scenario | What Happens |
|---|---|
| One annotator completed, and the project only needs one | Auto-promoted -- their answers become the gold standard automatically |
| Two or more annotators completed | Study enters the reconciliation pool for a reconciler to review |
MinAnnotators as a readiness threshold: This setting controls when a study is ready for authority determination -- not whether reconciliation is needed. Think of it as "how many people need to annotate before we can determine a gold standard." A study with two completed sessions always requires reconciliation, regardless of the MinAnnotators setting.
Why This Matters¶
Without authority determination, there is no way to know which studies need reconciliation and which already have trustworthy answers. This phase provides the foundation that the reconciliation workflow (Phase 10) builds upon.
Auto-promotion is particularly important: for projects where each study is annotated by a single person (a common scenario), the system automatically creates gold-standard records without requiring any manual reconciliation. This means existing workflows become more structured with zero additional effort.
How It Connects¶
| Connection | Detail |
|---|---|
| Phase 8 (Project Groups) | Uses group-based permissions to identify who has the Reconcile role |
| Phase 10 (Reconciliation Workflow) | Consumes the reconciliation pool to present studies to reconcilers |
| Phase 11 (Release 2 Migration) | Backfills ReconciliationSessions for existing single-annotator studies |
| Phase 5 (Annotation Form v2) | ReconciliationSessionVersions reference specific AnnotationVersionIds from the versioned form |
For the platform architecture overview, see platform-architecture.md. For the full reconciliation technical specification, see README.md.
Phase 8 creates the permission structure. Phase 9 builds the reconciliation data model. Phase 10 delivers the reconciliation workflow. Phase 11 migrates existing data.