Skip to content

Release 1 Data Migration

Summary

Phase 7 safely converts all existing project data to work with the new question versioning system. Every existing question becomes "version 1" of itself, every stage gets its first question set snapshot, and the new features roll out in a controlled order using feature flags. No data is deleted -- all changes are additive and reversible.

The Problem

Phases 3-6 build a new versioning system, but existing projects were created before versioning existed. These projects have questions that have never been versioned, stages with no question set history, and no forward-compatible fields for PRISMA reporting. Without migration, existing projects would be stuck on the old system while new projects use the new one -- creating a permanent split that doubles our maintenance burden.

What We're Building

Data Backfill

  • Every existing annotation question becomes a versioned question with version number 1. This is accurate -- they have never been versioned before, so their current state is their first and only version.
  • For each stage, the system creates an initial question set snapshot reflecting the current question assignments. This starts the audit trail from "today forward."
  • New fields are added to support future PRISMA reporting (source type on searches, lifecycle status on studies). These fields start empty and will be populated in later releases.

Controlled Rollout

Features are enabled in a specific order using feature flags (on/off switches that control which features are active):

  1. Question versioning first -- Admin-facing, no impact on annotators
  2. New annotation form second -- Can be tested per-project before rolling out widely
  3. Question management UI third -- Depends on both versioning and the new form

Each feature can be turned off independently if issues are discovered, and the new database fields can be cleanly removed to restore the previous state.

Quality Checks

The migration includes automated checks against PRISMA requirements, verifying that the converted data structure supports the reporting needs defined in Phase 2.

How It Connects

Phase 7 is the culmination of Release 1. It takes the infrastructure (Phase 3), question lifecycle (Phase 4), new annotation form (Phase 5), and management UI (Phase 6) and makes them available for existing production data. After Phase 7, Release 1 is live and Release 2 (Reconciliation) can begin building on top of it.

See also: Platform Architecture