Stage Settings

Per-stage configuration that controls the reviewer experience -- mode selection, reviewer policies, session targets, and reconciliation behaviour.

Overview

Every stage in a SyRF project has a set of configurable settings that determine how reviewers interact with that stage. Stage Settings define what kind of work happens in a stage (screening, annotation, or both), how studies are presented to reviewers, and what policies govern reviewer throughput and reconciliation eligibility.

Project Administrators configure these settings when creating a stage and can adjust them as the project evolves. The settings form a unified panel in the Stage Admin area, accessible from the project's stage management UI.

Stage Settings are the coordination layer between several features:

• Stage Filtering defines which studies appear in a stage's study pool
• Screening Profiles define what criteria govern screening decisions
• Question Management defines what questions annotators answer
• Reconciliation defines how disagreements are resolved

Stage Settings tie these together by specifying the mode of operation, assigning profiles and question sets, and configuring the policies that govern reviewer workflow.

---

Problem Statement

Different stages in a systematic review serve fundamentally different purposes. A Title/Abstract screening stage presents studies for rapid include/exclude decisions. A Full-Text annotation stage asks reviewers to extract structured data. A reconciliation stage asks a third-party reconciler to resolve disagreements from earlier work.

Each of these workflows requires different configuration:

• Screening stages need a Screening Profile assignment, control over whether excluded studies are hidden, and agreement mode configuration.
• Annotation stages need a Question Set assignment, a target number of annotators per study, and a maximum in-progress limit to manage throughput.
• Reconciliation stages need policies governing self-reconciliation and candidate blinding.
• Combined stages (screening + annotation) need a StudySelectionMode to define the relationship between the two activities.

Without a unified settings model, these concerns scatter across disconnected configurations. Stage Settings provide a single, coherent configuration surface that adapts to the stage's purpose.

---

Solution

A unified settings panel per stage, embedded on the Stage entity within the Project aggregate, covering:

1. Review Mode -- What activities are enabled (screening, annotation, or both)
2. Study Selection Mode -- How screening and annotation combine when both are enabled
3. Screening Profile assignment -- Which profile governs screening decisions
4. Question Set assignment -- Which question set version annotators answer
5. Policy settings -- Throughput controls, visibility rules, and reconciliation configuration
6. Stage lifecycle -- Active/inactive state controlling whether reviewers can work

All settings are persisted as properties of the Stage entity, which is embedded in the Project document in MongoDB. Changes take effect immediately for subsequent study selections.

---

Scope

In Scope

• Review Mode configuration (Screening, Annotation, ScreeningAndAnnotation)
• Study Selection Mode for combined stages
• Policy settings: MaxInProgress, SessionCountTarget, HideExcludedStudiesFromReviewers, AllowSelfReconciliation
• ExcludedSessionStatsGrouping (progress bar display of team-excluded studies)
• Data Extraction toggle
• Stage active/inactive lifecycle
• Screening Profile assignment to stages
• Question Set Version assignment to stages
• Filter Set configuration (delegated to Stage Filtering)
• Stage security settings (reviewer/reconciler permissions)

Out of Scope / Future

• Reconciliation selection mode: The reconciliation value for StudySelectionMode is not yet implemented in the codebase. It is specified in the Stage Filtering design and will be added when screening annotation reconciliation ships. See the Stage Modes section.
• MinAnnotators as a stage setting: Currently, the system uses the project-level AgreementThreshold.NumberScreened as a fallback when SessionCountTarget is not explicitly set. A dedicated MinAnnotators setting for reconciliation readiness thresholds is planned as part of the Annotation Reconciliation feature. See Interaction with Reconciliation.
• Cross-stage visibility settings: Admin-configurable settings controlling whether reconcilers see answers from other stages (Blind / ShowOwnPrior / ShowReconciled). Designed in design-decisions.md Section 4 but not yet implemented.
• Per-stage rationale requirement: Whether reconcilers must provide rationale for their decisions, configurable per stage.
• Stage ordering: Stages are unordered and can be concurrent. No sequential ordering is enforced.

---

Stage Modes

A stage's Review Mode determines what activities are available. The Study Selection Mode (relevant only when both screening and annotation are enabled) determines how those activities combine in the reviewer workflow.

Review Mode

Review Mode is set at stage creation and can be changed by a Project Administrator. It is represented as a flags enum:

Review Mode	Value	Description
Screening	1	Reviewers make include/exclude decisions only
Annotation	2	Reviewers answer annotation questions only
ScreeningAndAnnotation	3	Both screening and annotation are enabled

At least one of Screening or Annotation must be selected. The UI presents these as two checkboxes; the backend stores the combined flags value.

Study Selection Mode

Study Selection Mode is only configurable when Review Mode is ScreeningAndAnnotation. When only one activity is enabled, the Study Selection Mode is derived automatically from the Review Mode.

Study Selection Mode	Value	Description
Screening	1	All studies require screening; annotation is optional
Annotation	2	All studies require annotation; screening is optional
ScreeningAndAnnotation	3	All studies require both screening and annotation

Selection Behaviour by Mode

Mode	"Get next" returns	Pool basis
Screening	Next unscreened study from Stage Study Pool, filtered by policies	Stage Study Pool minus per-reviewer suppression
Annotation	Next study needing data extraction annotations	Included studies (Final Screening Outcome = Included)
ScreeningAndAnnotation	Next study for combined screening + annotation	Stage Study Pool; reviewer screens then annotates in one session

Reconciliation Mode

A reconciliation Study Selection Mode is planned but not yet implemented in the codebase. When implemented, it will:

• Present studies from the Screening Annotation Reconciliation Pool via random assignment (no browsing)
• Respect the AllowSelfReconciliation policy (whether a reviewer who screened the study can also reconcile it)
• Ignore HideExcludedStudiesFromReviewers (reconcilers need to see excluded studies to reconcile annotations)
• Be governed by SessionCountTarget for data extraction annotation coverage, and by the Screening Profile's agreement mode for screening decision coverage

See Stage Filtering -- Reconciliation for the full design of reconciliation selection modes.

---

Settings Reference

Required Settings

These must be provided at stage creation.

Setting	Type	Description	Applies To
Name	string	Display name for the stage	All stages
ReviewMode	ReviewMode (flags enum)	Which activities are enabled: Screening, Annotation, or both	All stages
StudySelectionMode	StudySelectionMode (enum)	How screening and annotation combine when both are enabled	ScreeningAndAnnotation stages only; auto-derived otherwise

Policy Settings

All policy settings have defaults and can be adjusted after stage creation. These are selection-time rules -- they filter the reviewer's view of the study pool but never change pool membership itself.

Setting	Type	Default	Description	Applies To
HideExcludedStudiesFromReviewers	bool	false	Suppress already-excluded studies from reviewer selection and progress display. Does NOT apply in reconciliation mode (reconcilers need to see excluded studies).	Screening, Annotation, ScreeningAndAnnotation
MaxInProgress	int?	null (unlimited)	Maximum number of studies a reviewer can have in progress simultaneously. Must be a positive integer or null. When null, no limit is enforced.	All modes
SessionCountTarget	int	Falls back to project AgreementThreshold.NumberScreened	Target number of completed annotation sessions per study before it is considered fully reviewed for data extraction. Does not apply to screening decisions (those are governed by the Screening Profile's agreement mode).	Annotation, ScreeningAndAnnotation, Reconciliation
AllowSelfReconciliation	bool	false	Whether a reviewer who screened or annotated a study can also be its reconciler. When false, the system excludes the reviewer from the reconciliation assignment pool for studies they have previously worked on.	Reconciliation
ExcludedSessionStatsGrouping	enum	Unavailable	Controls how team-excluded studies appear in the reviewer's progress bar. Values: WithUnexcluded (grouped with normal studies), Separate (own section), Unavailable (hidden under "Unavailable"). Subject to constraints with HideExcludedStudiesFromReviewers.	Annotation, ScreeningAndAnnotation
Extraction	bool	false	Whether data extraction (numerical data from tables/graphs) is enabled for this stage. When enabled, system annotation questions are automatically included in the stage's question set.	Annotation, ScreeningAndAnnotation
Active	bool	true	Whether the stage is open for reviewer work. When false, the stage is in draft/disabled state.	All stages

Validation Constraints

The following cross-setting validation rules are enforced:

1. ReviewMode: At least one of Screening or Annotation must be selected.
2. MaxInProgress: Must be a positive integer (> 0) or null.
3. SessionCountTarget: Must be a positive integer (>= 1).
4. ExcludedSessionStatsGrouping + HideExcludedStudiesFromReviewers:
5. WithUnexcluded is NOT allowed when HideExcludedStudiesFromReviewers = true (cannot group with unexcluded if excluded are hidden)
6. Unavailable is NOT allowed when HideExcludedStudiesFromReviewers = false (makes no semantic sense)

Feature-Dependent Settings

These settings are assigned through their respective feature UIs but are logically part of stage configuration.

Setting	Managed By	Description
Screening Profile	Screening Profile assignment	Which profile governs screening decisions in this stage. Configured in Project Settings, linked to stages.
Question Set Version (QSV)	Question Management (Assign tab)	Which versioned question set annotators answer in this stage. Managed via the QM Assign tab with drag-and-drop ordering.
Filter Set	Stage Filtering	JSON-based rules defining the Stage Study Pool. Configured as a property of the stage, managed through the filter configuration UI.
Security Settings	Stage Security	Permission grants controlling which project members can annotate, reconcile, or administer this stage. Managed through stage security settings.
Systematic Search Filter	Legacy	Optional filter restricting the stage to studies from specific systematic searches.
Partition Filter	Legacy	Optional filter restricting the stage to a specific partition of studies.

---

User Workflows

Configuring a New Stage

1. Create stage: Project Administrator opens the Project Overview and clicks "Add Stage". They provide a name and select the Review Mode (Screening, Annotation, or both).
2. Set Study Selection Mode: If both Screening and Annotation are enabled, the admin chooses how they combine (all require screening, all require annotation, or all require both).
3. Configure policies: The admin sets SessionCountTarget (recommended: 2 for journal publication), optionally enables MaxInProgress limits, and configures team-excluded study visibility.
4. Enable Data Extraction: If needed, the admin toggles Extraction on, which automatically includes system questions.
5. Assign Screening Profile: If screening is enabled, the admin links a Screening Profile (created in Project Settings) to this stage.
6. Assign Questions: In the Question Management Assign tab, the admin selects which annotation questions (at specific versions) are active for this stage.
7. Configure Filter Set: If this stage sits in a multi-stage pipeline, the admin configures a Filter Set defining which studies are in scope (e.g., "studies included under TA Criteria profile").
8. Configure Security: The admin assigns reviewer and reconciler permissions to project members or groups for this stage.
9. Activate: The admin activates the stage, which enables reviewer access and (for the Question Management v2 lifecycle) converts draft questions to versioned Annotation Questions.

Changing Settings on an Active Stage

Most settings can be changed after a stage has active sessions. Changes take effect immediately for subsequent study selections.

Setting	Can Change?	Impact
ReviewMode	Yes	Affects which activities are available. Existing sessions remain valid.
StudySelectionMode	Yes	Changes how next studies are selected. In-progress sessions are unaffected.
MaxInProgress	Yes	New limit applies immediately. Reviewers over the new limit cannot start new studies until they complete existing ones.
SessionCountTarget	Yes	Applies to future eligibility calculations. Studies already meeting the old target are not re-evaluated.
HideExcludedStudiesFromReviewers	Yes	Immediately changes visibility of excluded studies for all reviewers.
ExcludedSessionStatsGrouping	Yes	Immediately changes progress bar display.
AllowSelfReconciliation	Yes	Applies to future reconciliation assignments.
Extraction	Yes	Toggling adds/removes system questions from the stage. Existing extraction data is preserved.
Active	Yes	Deactivating prevents new work; in-progress sessions remain but cannot advance.
Filter Set	Yes	Changes the study pool immediately. In-progress studies outside the new pool remain in-progress.
Screening Profile	Conditional	Only if no decisions have been recorded against the current profile in this stage. Profiles are immutable once used.
Question Set Version	Yes	System reconstructs ASVs per-session using PAC, carrying forward compatible AVs and flagging breaking changes. Admin answers a series of scoping questions. See Question Management — Admin Decision Framework.

Enabling and Disabling a Stage

The Active flag controls the stage lifecycle:

• Disabled (Active = false): The stage is in draft mode. Reviewers cannot access it. Admins can freely configure settings, assign questions, and set up filter rules. In the Question Management v2 lifecycle, draft questions assigned to a disabled stage remain fully mutable.
• Enabled (Active = true): The stage is open for work. Reviewers with appropriate permissions can begin screening or annotating. In the Question Management v2 lifecycle, enabling a stage triggers activation of draft questions (converting them to versioned AQs with an initial v1).

Disabling an active stage: - Prevents new study selections - Does not delete in-progress sessions - Does not revoke completed work - Allows the admin to reconfigure before re-enabling

---

Interaction with Other Features

Interaction with Stage Filtering

Stage Filtering defines the Study Pool -- the set of studies eligible for work in a stage. Stage Settings define the policies applied on top of that pool at selection time:

Stage Study Pool (from Filter Set)
  --> Apply HideExcludedStudiesFromReviewers
  --> Apply MaxInProgress cap
  --> Apply per-reviewer suppression (already-screened, already-annotated)
  --> Apply StudySelectionMode logic
  --> Return next study to reviewer

The Filter Set is a stage setting (stored on the Stage entity) but its configuration and compilation are managed by the Stage Filtering feature. See Stage Filtering.

Interaction with Screening Profiles

Each screening stage is assigned a Screening Profile that defines: - Inclusion/exclusion criteria text - Agreement mode (Single, Dual Manual, Dual Automated)

The profile's agreement mode governs how many screening decisions are needed for a study's Screening Outcome to resolve -- this is distinct from SessionCountTarget, which governs annotation session coverage. The stage references the profile; the profile is managed in Project Settings. See Screening Profiles.

Interaction with Question Management

The Question Set Version (QSV) assigned to a stage determines what questions annotators see. This assignment is managed through the QM Assign tab, not the Stage Settings panel. However, it is logically a stage configuration:

• QSV determines form content: The QSV's ordered list of AQVersionIds defines the annotation form.
• Stage enable triggers activation: Enabling a disabled stage converts any DraftAQs assigned to it into versioned AQs with an initial version.
• QSV changes on active stages: When the admin assigns a new QSV to a stage with existing sessions, they choose how to handle the transition (see Question Management).

Interaction with Reconciliation

The reconciliation feature introduces additional stage settings (some planned, some existing):

• AllowSelfReconciliation (existing): Controls whether candidates can reconcile their own studies.
• MinAnnotators (planned): A readiness threshold controlling when a study is eligible for reconciliation. When CompletedSessions >= MinAnnotators, the system evaluates the reconciliation pathway. If exactly 1 session and MinAnnotators == 1, the study is auto-promoted. If 2+ sessions, the study enters the reconciliation pool regardless of MinAnnotators. See Design Decisions D8, D19-D22.
• SessionCountTarget already serves a related role: it defines how many candidate annotation sessions are targeted per study. The distinction is that SessionCountTarget is a target (how many sessions the system tries to collect) while MinAnnotators is a threshold (when to trigger authority determination).
• Cross-stage visibility (planned): Per-stage setting controlling whether reconcilers see answers from other stages. Values: Blind, ShowOwnPrior, ShowReconciled.

Interaction with Stage Security

Stage security settings control who can perform what activities in a stage. These are managed through a separate security settings UI but are stored on the Stage entity:

• StageActivity.Annotate: Permission to annotate studies in this stage
• StageActivity.Reconcile: Permission to reconcile studies in this stage
• StageActivity.Admin: Permission to manage stage settings

Permissions are granted via project groups (members assigned to groups, groups granted stage permissions) or direct member assignment.

---

Data Model

Stage Settings are embedded on the Stage entity within the Project aggregate in the pmProject collection. This embedding ensures atomic consistency -- all stage settings are updated in a single document operation.

Current Schema (Stage entity)

public class Stage : Entity<Guid>
{
    // Identity
    public string Name { get; private set; }
    public Guid ProjectId => Project!.Id;

    // Review Mode (flags enum: Screening=1, Annotation=2, ScreeningAndAnnotation=3)
    public ReviewMode ReviewMode { get; private set; }
    public StudySelectionMode StudySelectionMode { get; private set; }

    // Policy Settings
    public int? MaxInProgress { get; private set; }          // null = unlimited
    public int SessionCountTarget { get; set; }              // fallback: project AgreementThreshold
    public bool HideExcludedStudiesFromReviewers { get; set; }
    public bool AllowSelfReconciliation { get; private set; }
    public ExcludedSessionStatsGrouping ExcludedSessionStatsGrouping { get; }

    // Feature Toggles
    public bool Extraction { get; private set; }
    public bool Active { get; private set; }

    // Question Assignment
    public HashSet<Guid> AnnotationQuestions { get; private set; }

    // Legacy Filters
    public ICollection<Guid>? SystematicSearchFilter { get; set; }
    public ICollection<Guid>? PartitionFilter { get; set; }

    // Security
    public StageSecuritySettings SecuritySettings { get; set; }
}

Enums

public enum ReviewMode
{
    Screening = 1 << 0,            // 1
    Annotation = 1 << 1,           // 2
    ScreeningAndAnnotation = Annotation | Screening  // 3
}

public enum StudySelectionMode
{
    Screening = 1 << 0,            // 1
    Annotation = 1 << 1,           // 2
    ScreeningAndAnnotation = Annotation | Screening  // 3
}

public enum ExcludedSessionStatsGrouping
{
    WithUnexcluded,   // Group excluded studies with normal incomplete/completed
    Separate,         // Show in own labelled sections
    Unavailable       // Hide under "Unavailable"
}

Schema Versioning

The Stage entity supports two schema versions for ReviewMode:

• Schema v0 (legacy): Used separate _v0Screening and _v0Annotation boolean fields. The ReviewMode property computes the combined value from these fields.
• Schema v1+ (current): Uses _v1ReviewMode directly as the ReviewMode flags enum value.

StudySelectionMode is stored as a nullable backing field (_studySelectionMode). When null, it is derived from ReviewMode (matching the same enum value). It is only explicitly set when the admin configures a non-default combination for ScreeningAndAnnotation stages.

SessionCountTarget has a nullable backing field (_sessionCountTarget). When null, it falls back to the project-level AgreementThreshold.NumberScreened value.

Planned Additions

The following fields will be added to the Stage entity as their respective features are implemented:

Field	Type	Feature
FilterSet	FilterSetDocument (JSON)	Stage Filtering
ScreeningProfileId	Guid?	Screening Profile assignment
QuestionConfig / QuestionConfigHistory	StageQuestionConfig / List<StageQuestionConfig>	Question Management v2
MinAnnotators	int	Annotation Reconciliation
CrossStageVisibility	enum (Blind/ShowOwnPrior/ShowReconciled)	Reconciliation
RequireRationale	bool	Reconciliation

---

API

Stage CRUD

Stages are managed as part of the Project resource. Stage settings are updated via JSON Patch operations.

Method	Path	Description
POST	/api/projects/{projectId}/stages	Create a new stage with initial settings
PATCH	/api/projects/{projectId}/stages/{stageId}	Update stage settings via JSON Patch
GET	/api/projects/{projectId}	Returns the project including all stages and their settings

Create Stage DTO

public class CreateStageDto
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public ReviewMode ReviewMode { get; set; }
    public StudySelectionMode StudySelectionMode { get; set; }
    public string Description { get; set; }
    public int? MaxInProgress { get; set; }
    public bool Active { get; set; }
    public Guid ProjectId { get; set; }
    public bool Extraction { get; set; }
}

Stage Update DTO

public class StageUpdateDto
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public ReviewMode ReviewMode { get; set; }
    public StudySelectionMode StudySelectionMode { get; set; }
    public string Description { get; set; }
    public int? MaxInProgress { get; set; }
    public int SessionCountTarget { get; set; }
    public bool Active { get; set; }
    public Guid ProjectId { get; set; }
    public bool Extraction { get; set; }
    public ExcludedSessionStatsGrouping ExcludedSessionStatsGrouping { get; set; }
    public bool HideExcludedStudiesFromReviewers { get; set; }
}

Stage Response DTO

The StageDto returned in project responses includes all settings fields plus the current question assignment and filter configuration.

Filter Set Management

Filter Sets are updated as part of stage configuration:

• PUT /api/projects/{projectId}/stages/{stageId} -- Updates stage settings including the Filter Set
• Validation returns 422 for invalid Filter Set rules
• Changes take effect immediately for subsequent selections

Stage Question Configuration

Managed through the Question Management feature:

• PUT /api/projects/{projectId}/stages/{stageId}/question-config -- Update the stage's active QSV
• GET /api/projects/{projectId}/stages/{stageId}/question-config -- Get current configuration
• GET /api/projects/{projectId}/stages/{stageId}/question-config/history -- Get configuration history

---

Migration Strategy

Existing Stage Configurations

All existing stages have valid settings under the current schema. No destructive migration is needed.

• Schema v0 stages (legacy boolean fields): The ReviewMode and StudySelectionMode properties compute correct values from the v0 fields. These stages continue to work without modification.
• Schema v1+ stages: Use the current field layout directly.

New Settings Adoption

New settings introduced by upcoming features follow the additive pattern:

1. Screening Profile assignment: Stages without a profile assignment continue to use the project-level screening configuration (current behaviour).
2. Filter Set: Stages without a Filter Set have an implicit empty pool = all project studies (current behaviour). Filter Sets are purely additive; admins opt in.
3. Question Set Version: The existing AnnotationQuestions HashSet continues to work. QSV assignment is introduced alongside it, with backward-compatible computed properties.
4. MinAnnotators: Falls back to the project-level AgreementThreshold.NumberScreened (via SessionCountTarget fallback).

No existing stage data is modified or deleted during any migration phase.

---

Success Criteria

1. All stage settings are configurable through a single unified settings panel in the Stage Admin UI.
2. Changes to stage settings take effect immediately for subsequent study selections (no cache delay).
3. Validation rules prevent invalid setting combinations (cross-field constraints enforced server-side).
4. Legacy stages (schema v0) continue to work identically to current behaviour without manual migration.
5. A Project Administrator can configure a new stage (mode + policies + profile + questions) in under 5 minutes.
6. SessionCountTarget defaults to a sensible value (project-level threshold) when not explicitly set.
7. MaxInProgress correctly caps concurrent in-progress studies per reviewer, respecting the HideExcludedStudiesFromReviewers setting for excluded study counting.
8. AllowSelfReconciliation correctly excludes/includes reviewers from reconciliation assignment pools.
9. Settings changes on active stages do not invalidate or delete existing in-progress sessions.
10. Stage activation/deactivation correctly controls reviewer access without data loss.

---

Related Documents

• Stage Filtering -- Filter Sets defining per-stage study pools; detailed selection behaviour by mode
• Screening Profiles -- Named screening criteria configurations assigned to stages
• Screening Annotations -- Structured exclusion reasons and reconciliation pool configuration
• Annotation Management & Reconciliation -- Question versioning, reconciliation sessions, and authority determination
• Design Decisions -- Authoritative reference for MinAnnotators (D8), reconciliation workflow (D9), self-reconciliation, and cross-stage visibility (D32)
• Question Management -- Question versioning, stage question configuration, and QSV assignment
• Product Overview -- PO-facing description of annotation reconciliation configuration