Skip to content

QM v2 Implementation History

Frozen execution context — preserved from the feat/question-management-v2 GSD workspace before that workspace was deleted. Captures the milestone-by-milestone narrative, key decisions, and status. Most-recent statuses recorded here.

Tracked under: PR #2461 / Epic #2488 / Phase issues #2489–#2497

Design specs (canonical, separate) live in docs/features/question-management/, docs/features/annotation-versioning/, docs/features/annotation-form-v2/, docs/features/reconciliation/.

Architectural decisions and implementation knowledge are captured separately in qm-v2-architecture-and-knowledge.md.

Project Overview

What this is: A full redesign of how project administrators define, version, assign, and publish annotation questions across systematic review stages. Replaces the current mutable-question editor with a versioned question lifecycle (draft → published → versioned), project-level and stage-level question sets, and an interactive publish ceremony.

Core value: Administrators can design questions freely, publish them to stages with full version tracking, and update published questions without losing annotation history — while annotators always see a consistent, version-pinned annotation form.

Architecture / Key Patterns

  • Two-phase question lifecycle: DraftQuestion (fully mutable, on Project) → AnnotationQuestion (versioned, own collection, structural props frozen)
  • PQS / SQS versioning: ProjectQuestionSet tracks ordered project-wide question refs; StageQuestionSet subsets PQS per stage. Both have mutable drafts + append-only immutable versions.
  • Autosave with publish ceremony: All draft changes autosaved (Figma / Google Docs model). Only deliberate action is Publish, which creates immutable AQVersions + PQS / SQS versions.
  • Admin Decision Framework: When publishing affects annotated questions, admin configures per-question impact handling (keep / map / re-answer).
  • Feature flags: newQuestionManagement (admin QM UI), annotationFormV2 (annotator form) — independent rollout.
  • Signal forms + zoneless: All new Angular code uses signal forms, OnPush, signal inputs / outputs — no reactive forms, no Zone.js dependency.

Tracking References

  • Epic: #2488 Question Management v2 — Full Redesign
  • Phase issues: #2489–#2497 (Phases 1–9)
  • PR: #2461 (current WIP branch)

Milestone Sequence and Current Status

Status as of 2026-04-23 (frozen at the point this content was migrated out of GSD).

ID Title Status Tests Shipped Wired?
M001 Backend Foundation — Domain entities, migration, v2 API ✅ Complete (2026-03-30) 71 .NET API works end-to-end
M002 Admin Question Management UI — Design / Assign / Preview ✅ Complete (2026-04-01) 74 Angular ⚠️ Not wired to API
M003 Annotation Form v2 — Unit tiles + signal controls ✅ Complete (2026-04-01) 46 Angular ⚠️ Not wired to API
M004 Admin Decision Framework — Impact assessment + alerts ✅ Complete (2026-04-01) 15 .NET + 40 Angular ⚠️ Not wired to API
M006 Frontend ↔ Backend Integration — ngrx + signal stores ⬜ Planned — —
M005 Production Migration & Rollout ⬜ Blocked on M006 — —

Critical execution note: M005 was originally listed before M006, but D012 (see architecture decisions) re-ordered them — M005 (production migration) cannot execute until M006 (frontend-backend wiring) ships, since you cannot migrate production projects to a model the UI doesn't talk to yet.

M001 — Backend Foundation: Data Model, Migration, and v2 API

Status: ✅ Complete (2026-03-30) Headline: Built QM v2 backend: versioned domain model, migration service, 8 API endpoints — 71 tests pass.

Vision

Create the new versioned question data model (DraftQuestion, AnnotationQuestion, AQVersion, PQS, SQS), build per-project migration tooling with dry-run and rollback, and implement the v2 API endpoints for draft CRUD, publishing, validation, and version history. All backend infrastructure needed before any frontend work begins.

Slice Delivery

Slice Subject Tests Status
S01 Domain Entities and Collection Infrastructure 33 entity / lifecycle ✅
S02 Migration Infrastructure 11 migration + 11 factory ✅
S03 v2 API Endpoints and Publishing 5 publish + 11 validation + 8 controller ✅

S01 — Domain Entities and Collection Infrastructure

Provides:

  • AnnotationQuestionV2 aggregate with FromDraft / Publish / ApplyVersionAsDraft / DiscardDraft domain methods
  • ProjectQuestionSet with DraftPQS (Add / Reorder / Remove / PromoteToPublished) and PQSVersion publish
  • StageQuestionSet with DraftSQS (Add / Remove / Contains) and SQSVersion publish
  • DraftSnapshotService with GFS retention (create, promote, prune, cap, maintain)
  • IAnnotationQuestionV2Repository interface + MongoDB implementation
  • All value objects: VersionAudit, QuestionReference, QuestionRef, QuestionOptionV2, DraftPublishDecision, PublishDecision

Key files:

  • src/libs/project-management/SyRF.ProjectManagement.Core/Model/QuestionVersioning/AnnotationQuestionV2.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Model/QuestionVersioning/ProjectQuestionSet.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Model/QuestionVersioning/StageQuestionSet.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/DraftSnapshotService.cs
  • src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/Repositories/AnnotationQuestionV2Repository.cs

Key decisions:

  • QuestionOptionV2 is standalone (no parent back-reference) unlike existing QuestionOption
  • AnnotationQuestionV2 extends AggregateRoot<Guid> for MongoContext compatibility
  • DraftSnapshotService operates on in-memory lists — caller responsible for persistence
  • Combined value objects into a single VersioningValueObjects.cs file

Patterns established:

  • QuestionVersioning namespace for all v2 question-model entities
  • Value objects as C# records for immutable types, classes for mutable types
  • AggregateRoot<Guid> for MongoDB-persisted aggregates, Entity<Guid> for embedded entities
  • DraftSnapshotService operates on in-memory List<DraftSnapshot> — persistence is caller's responsibility

S02 — Migration Infrastructure

Provides:

  • QuestionMigrationService.MigrateProject() for per-project migration
  • QuestionMigrationService.PlanRollback() for reversal planning
  • SystemQuestionFactory.CreateSystemQuestions() for 18 system question entities
  • SystemQuestionFactory.AllSystemQuestionIds for system-question GUID lookup

Key files:

  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/QuestionMigrationService.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/SystemQuestionFactory.cs

Key decisions:

  • SystemQuestionFactory uses definition records with resolver functions for version-dependent logic
  • Migration operates on in-memory entities — caller manages persistence
  • 32 production violations handled: CategoryParentMapping repairs 23 zero-annotation violations; 9 annotated deferred to production

S03 — v2 API Endpoints and Publishing

Provides:

  • 8 API endpoints for frontend consumption (draft, publish, validate, versions, question-set)
  • QuestionPublishService for atomic publish ceremony
  • CrossQuestionValidationService for pre-publish validation (7 of 16 rules implemented: AQ001–AQ007)
  • DTOs: DraftContentDto, PublishRequestDto, QuestionVersionHistoryDto, etc.

Key files:

  • src/services/api/SyRF.API.Endpoint/Controllers/QuestionManagementV2Controller.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/QuestionPublishService.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/CrossQuestionValidationService.cs

Key decisions:

  • Draft ops directly on controller (no separate QuestionDraftService)
  • PublishStage operates on in-memory entities — caller handles load / save
  • api/v2/projects route prefix for v2 endpoints
  • Validation rules return typed ValidationError with severity

Patterns established:

  • QuestionPublishService operates on in-memory entities (caller manages persistence)
  • CrossQuestionValidationService returns typed ValidationError list
  • v2 controller at api/v2/projects with DTOs co-located

Observability surfaces:

  • Structured logging on publish: StageId, ProjectId, PQS version, SQS version, AQ count
  • 400 responses include validation error details with rule codes

Definition of Done — Results

# Criterion Status Evidence
1 All domain entities defined with unit tests ✅ 33 entity tests
2 pmAnnotationQuestion collection with indexes ✅ Repository with projectId index
3 Migration service with dry-run, rollback, validation ✅ 11 migration + 11 factory tests
4 v2 API endpoints with integration tests ✅ 8 controller tests, 5 publish tests
5 Cross-question validation service (16 rules) ⚠️ Partial 7 of 16 rules implemented (AQ001–AQ007)
6 Draft snapshot GFS retention with unit tests ✅ 13 snapshot tests
7 Optimistic concurrency on Study writes ❌ Deferred Requires infrastructure work — see phase-03-collection-infrastructure-plans.md
8 All existing tests pass ✅ No regression

Lessons Learned

  • Project.AnnotationQuestions dynamically generates system questions on every access — filter with q.System for custom-only
  • UpsertCustomAnnotationQuestionDto needs IsNew=true for validation to pass on new questions
  • dotnet restore needed after worktree checkout for Lamar / Redis packages to resolve

Outstanding Follow-ups (carried into M006 / M005)

M002 — Admin Question Management UI

Status: ✅ Complete (2026-04-01) — but NOT wired to API (deferred to M006) Headline: Built three admin Question Management views (Design, Assign with publish wizard, Preview) — 74 tests pass, production build succeeds.

Vision

Build the three admin views (Design, Assign, Preview) as modern Angular components. Depends on M001 (v2 API). All behind newQuestionManagement feature flag.

Slice Delivery

Slice Subject Tests Status
S01 Design View 18 ✅
S02 Assign View and Publish Wizard 35 ✅
S03 Preview View 21 ✅

S01 — Design View

Established the core architecture: DesignV2Store with question state management, QuestionTreeComponent with recursive rendering, QuestionNodeComponent with control-type icons and status badges, PropertiesPanelComponent with editable fields, and DesignV2Component as split-panel container.

S02 — Assign View and Publish Wizard

Provides:

  • AssignV2Store for question assignment state
  • AssignV2Component + AssignTreeComponent + AssignNodeComponent for assign UI
  • PublishWizardComponent for publish ceremony (Steps 1–2 + 5; Steps 3–4 stubbed for M004)
  • assign route in v2 routing

Key files:

  • src/services/web/src/app/project/project-admin/question-management-v2/assign/assign-v2.store.ts
  • src/services/web/src/app/project/project-admin/question-management-v2/assign/assign-v2.component.ts
  • src/services/web/src/app/project/project-admin/question-management-v2/assign/publish-wizard/publish-wizard.component.ts

Key decisions:

  • Extended QuestionNode with AssignQuestionNode rather than creating a separate model
  • Checkbox cascade uses recursive helpers for ancestor / descendant traversal
  • Filter toggle based on published state (baseline for diff)
  • AssignNodeComponent passes lookup maps to children for O(1) state resolution
  • Tests use vi.useFakeTimers instead of Angular fakeAsync (Vitest environment)

Patterns established:

  • AssignV2Store at component-level provider scope (same as DesignV2Store)
  • AssignQuestionNode extends QuestionNode for assign-specific fields
  • Recursive tree node with Map-based state lookups for O(1) per-node resolution
  • TestBed.resetTestingModule() pattern for dialog tests with varied data

S03 — Preview View

Built the form simulation: PreviewV2Store with published / pending mode toggle, PreviewFormComponent rendering questions as interactive Material controls per controlType.

Cross-Cutting Architecture (M002)

  • All three views share QuestionNode model, QUESTION_CATEGORIES, CategoryKey, and category-tab patterns
  • All routes wired into questionManagementV2Routes
  • question-management-v2/ directory for all v2 components
  • SignalStore at component-level provider scope for each view
  • NgTemplateOutlet for recursive form rendering in Preview

Lessons Learned

  • Vitest + Angular requires vi.useFakeTimers instead of fakeAsync / tick
  • TestBed.overrideProvider fails after component creation — use factory-based test setup per data variation
  • SignalStore computed signals work well for derived state (assignment deltas, checkbox states)
  • Set-based ID tracking (publishedQuestionIds, pendingQuestionIds) enables clean published / pending mode switching

M003 — Annotation Form v2

Status: ✅ Complete (2026-04-01) — but NOT wired to API (deferred to M006) Headline: Rewrote annotation form with unit tiles, 6 signal controls, and conditional logic — 46 tests pass.

Vision

Complete rewrite of the 943-line annotation form with collapsible unit tiles, signal forms, virtual scrolling, and rewritten form controls. Depends on M001 (API) and M002 (Preview integration). Separate annotationFormV2 feature flag.

Slice Delivery

Slice Subject Tests Status
S01 Form Structure and Unit Tiles 29 ✅
S02 Signal Form Controls and Conditional Logic 17 ✅

What Was Built

S01 created AnnotationFormV2Store with unit management, workspace toggling, pagination, and completion state. Built UnitCardSelectorComponent (mini cards with three-state completion icons, pagination), UnitPanelComponent (collapsible panels with header actions and inline delete confirmation), and AnnotationFormV2Component (container with category tabs, workspace toolbar).

S02 created 6 signal-based form controls (text, dropdown, checkbox, radio, checklist, autocomplete), QuestionFieldComponent (@switch dispatcher), and QuestionListComponent with conditional question visibility based on parent answers.

Key Files

  • src/services/web/src/app/shared/annotation/annotation-form-v2/annotation-form-v2.store.ts
  • src/services/web/src/app/shared/annotation/annotation-form-v2/annotation-form-v2.component.ts
  • src/services/web/src/app/shared/annotation/annotation-form-v2/unit-card-selector/unit-card-selector.component.ts
  • src/services/web/src/app/shared/annotation/annotation-form-v2/unit-panel/unit-panel.component.ts
  • src/services/web/src/app/shared/annotation/annotation-form-v2/question-field/question-field.component.ts
  • src/services/web/src/app/shared/annotation/annotation-form-v2/question-list/question-list.component.ts

Key Decisions

  • AnnotationFormV2Store lives in shared/annotation/annotation-form-v2/
  • Narrowed angular.json test exclusion to allow v2 tests
  • Signal-based controls with ngModel
  • QuestionFieldComponent @switch dispatcher
  • Local signal Map for per-unit answer state

Lessons Learned

  • angular.json test exclusions need to be specific — blanket exclusions block new code in the same directory tree
  • NgTemplateOutlet is the cleanest approach for recursive template rendering in Angular standalone components
  • Signal-based local state (signal Map) works well for per-unit answer tracking without global store overhead

M004 — Admin Decision Framework

Status: ✅ Complete (2026-04-01) — but NOT wired to API (deferred to M006) Headline: Impact assessment, design-time config, publish wizard handling, session transitions, annotator alerts — 55 new tests (15 .NET + 40 Angular).

Vision

Implement the full Admin Decision Framework: annotation impact assessment, design-time impact configuration, multi-step publish wizard with impact handling, session transition engine, and annotator-facing version transition experience. This lifts the Phases 1–7 mitigation that blocks publishing to annotated stages. Depends on M002 (publish wizard shell) and M003 (annotation form).

Slice Delivery

Slice Subject Tests Status
S01 Impact Assessment 7 .NET + 3 Angular ✅
S02 Impact & Mapping Config 11 Angular ✅
S03 Publish Wizard with Impact 14 Angular ✅
S04 Session Transition Engine 8 .NET ✅
S05 Annotator Transitions 12 Angular ✅

Cross-Slice Flow

S01 provides impact data consumed by S02 (properties panel) and S03 (publish wizard). S02 provides DraftPublishDecision models used by S03 (wizard pre-population) and S04 (decision promotion). S04 provides transition results consumed by S05 (annotator alerts).

Key Files

  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/AnnotationImpactService.cs
  • src/libs/project-management/SyRF.ProjectManagement.Core/Services/SessionTransitionService.cs
  • src/services/web/src/app/shared/annotation/annotation-form-v2/services/annotation-impact.service.ts
  • src/services/web/src/app/project/project-admin/question-management-v2/design/properties-panel/impact-mapping-panel/impact-mapping-panel.component.ts
  • src/services/web/src/app/project/project-admin/question-management-v2/assign/publish-wizard/publish-wizard.component.ts
  • src/services/web/src/app/shared/annotation/annotation-form-v2/version-transition-alert/version-transition-alert.component.ts

Key Decisions

  • AnnotationImpactService operates on in-memory data (consistent pattern with M001 services)
  • DraftPublishDecision TypeScript interfaces mirror backend exactly
  • Publish wizard Steps 3–4 conditional on 'may-affect' confirmations
  • SessionTransitionService counts but doesn't mutate annotations (needs domain method — follow-up)
  • VersionTransitionAlertComponent has three visual variants (mapped / re-answer / updated)

Lessons Learned

  • OptionMapping record uses OldValue / NewValue not FromValue / ToValue — always check existing record constructors before building new code
  • Conflict detection needs session-level data not yet available — placeholder is acceptable for now
  • Separate HTML template files are the codebase convention — don't use inline templates

M006 — Frontend ↔ Backend Integration (PLANNED, runs before M005)

Status: ⬜ Planned — must execute before M005 (per D012) Headline: Wire the v2 frontend to the backend API using modern ngrx SignalStore patterns.

Vision

Wire the v2 frontend to the backend API using modern ngrx SignalStore patterns. Project-scoped root store with three withEntities collections (question / version / details). Resource signals for reads, withMutations for writes. withCallState, withUndoRedo, withDevtools. Component stores own view-specific logic and inject the root store. After this milestone, Design / Assign / Preview views render real data behind the feature flag.

Planned Slice Overview

ID Slice Risk Depends Done After this
S01 ngrx Entity Layer + Signal Store Bridge high — ⬜ Design view renders questions loaded from the v2 API. Global ngrx state has normalised v2 question entities.
S02 Autosave + Assign / Preview Store Bridges medium S01 ⬜ Edit a question field → autosave fires after debounce → entity state updates. Assign and Preview stores also bridge from ngrx.
S03 Undo / Redo Service medium S02 ⬜ Admin edits a field, presses Ctrl+Z → edit reversed, new autosave triggered with reverted content.

Why This Comes Before M005

Per D012: M005 (production migration, staged rollout, legacy removal) cannot proceed without the frontend-backend integration M006 delivers. You can't migrate production projects to a new model and roll out a new UI if the UI doesn't talk to the API yet.

M005 — Production Migration & Rollout (PLANNED, blocked on M006)

Status: ⬜ Blocked on M006 Headline: Migrate all production projects to the new question model, enable feature flags with staged rollout, monitor for issues, and document legacy code removal.

Planned Slice Overview

ID Slice Risk Depends Done After this
S01 Production Migration Execution high M006 ⬜ All production projects on the new model with validated data.
S02 Staged Rollout and Monitoring medium S01 ⬜ All users on the new QM UI and annotation form. Rollback documented.
S03 Legacy Code Removal Plan low S02 ⬜ Legacy QM code identified and a removal plan documented.

Requirements Traceability (Most Recent Status)

ID Class Owner Status Notes
R001 — Draft question CRUD core-capability M001 ✅ Advanced DraftQuestion entity with full mutability
R002 — Question versioning (AQ + AQVersion) core-capability M001 ✅ Advanced AnnotationQuestionV2 with AQVersion history
R003 — Project Question Set (PQS) versioning core-capability M001 ✅ Advanced ProjectQuestionSet with DraftPQS / PQSVersion
R004 — Stage Question Set (SQS) versioning core-capability M001 ✅ Advanced StageQuestionSet with DraftSQS / SQSVersion
R005 — Stage publish ceremony core-capability M001 ✅ Advanced QuestionPublishService + controller endpoint
R006 — Migration infrastructure operability M001 ✅ Advanced QuestionMigrationService + SystemQuestionFactory
R007 — Design view primary-user-loop M002 ✅ Advanced (UI only) Wired in M006
R008 — Assign view primary-user-loop M002 ✅ Advanced (UI only) Wired in M006
R009 — Preview view primary-user-loop M002 ✅ Advanced (UI only) Wired in M006
R010 — Annotation form v2 primary-user-loop M003 ✅ Advanced (UI only) Wired in M006
R011 — Admin Decision Framework core-capability M004 ✅ Advanced (UI only) Wired in M006
R012 — Cross-question validation (AQ001–AQ016) quality-attribute M001 + M002 ⚠️ Partial 7 of 16 rules (AQ001–AQ007). AQ008–AQ016 are follow-ups.
R013 — Dual-model backwards compatibility continuity M001 + M002 ✅ Advanced New entities alongside old, additive
R014 — Draft snapshots (GFS retention) quality-attribute M001 ✅ Advanced DraftSnapshotService with GFS retention

Out of Scope

ID Description Notes
R030 Cross-project question sharing Fields included in data model now (scope, ownerProjectId), UI deferred.
R031 Training rounds Independent of QM v2 phases, designed with version awareness.

Cross-References