fchinembiri
/
geocrop-platform.
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
geocrop-platform./apps/nextgen/.harness/changelogs/2026-07-16-p0-sync-dedupe.md

3.3 KiB
Raw Blame History

2026-07-16 — P0 sync dedupe (WT-E)

Closes P0-6 from the live audit (3fb3d84): server/src/services/SyncEngine.js tablesToSync listed 5 duplicate pairs and a 3-way settings conflict, causing redundant sync cycles and potential upload drift.

Files

File Change
server/src/database/migrations/2026-07-16-dedupe-tables.js NEW. One-shot migration. Backs up the DB to data/school.db.pre-dedupe-backup, copies any non-duplicate rows from the legacy tables to the canonical ones, drops the legacy tables. Idempotent — running it twice is a no-op.
server/src/services/SyncEngine.js Removed subjects_new, attendance_new, messages_new, calendar_events from tablesToSync (lines 30, 35, 40, 42). Added a startup check: any table in tablesToSync that doesn't exist in sqlite_master throws a clear error at engine init.
server/src/database/init.js Removed the CREATE TABLE IF NOT EXISTS blocks for the 4 dropped legacy tables.
.harness/changelogs/2026-07-16-p0-sync-dedupe.md This file.
~/.mavis/scratchpads/mvs_88aff8965ad4492d83c427623f15d3f0/p0-sync-dedupe-decision.md DECISION-0 (canonical table per pair, with live ref + row counts).

Migration contract

  • Pre-check: the script inspects sqlite_master for each canonical + legacy table and reports what it found before doing anything. If the schema doesn't match the expected shape, it aborts.
  • Backup: data/school.db is copied to data/school.db.pre-dedupe-backup BEFORE any change. The backup is left in place for the operator to delete after they confirm.
  • Copy: for each pair, the script does INSERT INTO canonical (uid, …) SELECT … FROM legacy WHERE NOT EXISTS (SELECT 1 FROM canonical WHERE canonical.uid = legacy.uid). Duplicate uids in the source are skipped (a warning is logged, not a failure — they're meant to be duplicates by construction).
  • Drop: DROP TABLE legacy after the copy completes successfully.
  • Idempotency: running the script twice is a no-op. The first run does the work; the second run sees no legacy tables to drop and no rows to copy.
  • Transaction: the whole migration is wrapped in a single db.transaction(() => { … })(). If anything fails, the DB is restored from the backup automatically.

Verification

  • data/school.db.pre-dedupe-backup exists and is openable.
  • tablesToSync has 4 fewer entries.
  • node -e "const e = require('./src/services/SyncEngine').getSyncEngine(); console.log('OK')" succeeds (startup check passes).
  • A manual POST /api/sync/force returns 200 and the sync_logs table shows each remaining table appearing exactly once.
  • Select-String -Path 'server\\src' -Pattern 'subjects_new|attendance_new|messages_new|calendar_events' returns 0 controller refs (the init.js CREATE blocks are gone; the migration script itself is the only mention).

What did NOT change (deliberately)

  • settings / system_settings / school_settings: all 3 kept. Different schemas, different purposes. A future PR can rationalize them.
  • No controller SQL was touched. The bare-named tables were already canonical in controller queries; the legacy _new tables had only stale migration-script mentions.
  • No sync_status data migration. Both versions of every pair share the same sync_status CHECK constraint and default value.