Skip to content

Question Lifecycle

Summary

Phase 4 is the heart of Release 1. It gives project administrators the ability to create, organise, and evolve their annotation questions through a clear two-phase lifecycle: free-form drafting followed by controlled, versioned editing. Once a stage is activated, every change to a question creates a new version rather than silently overwriting the previous one -- preserving the complete history of what was asked and when.

The Problem

Currently, SyRF locks annotation questions entirely once any annotator has answered them. This protects data integrity, but it means project teams are stuck with their initial question design for the life of the project -- even when a dropdown option is missing, wording is ambiguous, or a new question is needed.

When changes are made to unlocked questions (before any answers exist), the edit happens silently in place. There is no record of what the question looked like before. This means past annotations cannot be reliably linked to the exact question that was asked at the time.

What We're Building

For Project Administrators

Drafting phase (before activation): - Create and edit questions freely in the Question Management Design tab - Organise questions with drag-and-drop reordering and parent-child hierarchies - Choose from six question types: yes/no, dropdown, checklist, free text, number, and autocomplete - Assign questions to stages and preview exactly what annotators will see

Activation (going live): - When the administrator is satisfied and enables a stage, all assigned questions are "activated" - Each question receives its first version (v1) -- capturing the exact text, options, and settings - The question's fundamental structure (type, parent relationships) is locked, while content can still evolve

After activation (controlled versioning): - Editing a question's wording, options, or help text creates a new version (v2, v3, etc.) - Previous versions are preserved permanently -- the full history is always available - The administrator can see at a glance which version of each question is currently active

flowchart LR
    A[Create Draft<br/>Questions] --> B[Assign to<br/>Stage]
    B --> C[Preview<br/>Form]
    C --> D[Activate<br/>Stage]
    D --> E[Questions<br/>Versioned]
    E --> F[Edit Creates<br/>New Version]

    style A fill:#e3f2fd
    style B fill:#e3f2fd
    style C fill:#e3f2fd
    style D fill:#fff3e0
    style E fill:#e8f5e9
    style F fill:#e8f5e9

For Reviewers/Annotators

Annotators are not directly affected by the versioning system. They continue to see and answer questions as before. Behind the scenes, however, each answer now records which version of the question it was collected against -- providing the traceability that reconciliation (Phase 10) depends on.

Key Decisions

  1. Two-phase lifecycle (draft then activate) -- Rather than versioning everything from the start, we allow free-form editing during the design phase. This keeps the early experience simple and fast. Versioning only begins when it matters -- when a stage goes live and real data starts being collected.

  2. Structural properties are fixed after activation -- A question's type (yes/no, dropdown, etc.), parent relationship, and grouping behaviour cannot change after activation. Allowing these to change would invalidate all existing answers. Content (wording, options, help text) can evolve because answers remain meaningful even if the phrasing improves.

  3. Question Set snapshots -- Each stage maintains a versioned snapshot of which questions (at which versions, in which order) annotators see. When the configuration changes, a new snapshot is created. Historical snapshots are preserved, so it is always possible to reconstruct exactly what an annotator's form looked like at any point in time.

  4. Existing v2 QM UI becomes the Draft Question editor -- The question management interface that is approximately 90% complete behind a feature flag naturally serves as the drafting workspace. This maximises reuse of existing work.

How It Connects

Phase 4 depends on Phase 3 (Collection Infrastructure) for the database storage. It feeds into: - Phase 5 (Annotation Form v2) -- The form reads from versioned question sets to render questions - Phase 6 (Question Management UI) -- Adds version history viewing, version badges, and admin decision tools on top of Phase 4's versioning model - Phase 10 (Reconciliation) -- Compares answers to the same question version to produce gold-standard records

See also: Platform Architecture | Technical Spec Related briefs: Annotation Form v2 | Question Management UI