geocrop-platform./apps/nextgen/server/databaseplan.md

Offline-First Database Sync Plan

This document outlines the blueprint and instructions for linking the React Native mobile app with the Express-based web app server. It replaces direct-to-Supabase synchronization on the mobile client with web-server-mediated syncing.

---

1. Primary Intent

To redirect the React Native mobile app (nextgenmobile) sync path away from direct Supabase communication to a local-hub-mediated synchronization architecture using the Express-based web application server (server).

• Online State Definition: The system detects the mobile app is "online" if and only if the web server's API endpoint is reachable (running on port 3001).
• Offline State Definition: The system is "offline" when the web server is stopped, crashed, or network-unreachable.
• Offline-First Synchronization: The mobile app functions with full local write capability into its local SQLite database (using expo-sqlite and PowerSync). Once a connection to the web server is verified, it pushes all locally accumulated changes (marked as pending) to the server via /api/sync/push and pulls remote changes via /api/sync/pull.

---

2. Context Boundaries

In-Scope Files

• Mobile Client Sync Coordinator: sync.service.ts
• Mobile API Client Helper: api.ts
• Mobile DB Schema definitions: schemaSql.ts and database.ts
• Web App Server Sync Endpoints: sync.controller.js
• Web App Server Entry Point: index.js
• Web App Server DB Config/Sync Engine: SyncEngine.js

Synced Database Tables (23 Tables)

The synchronization covers the following entities: users, departments, classes, subjects, courses, enrollments, attendance, fee_groups, student_fees, payments, messages, events, expenses, grades, assignments, submissions, staff_records, leave_requests, item_stock, items, hostels, rooms, routes

---

3. Reasoning Constraints

• Offline Writes: Writes are recorded in local SQLite immediately. No remote calls block user interaction.
• Change Tracking: Every table row uses the standard sync columns:
  • uid: Unique identifier (UUID string) shared globally.
  • sync_status: Tracked as 'pending' (unsynced), 'synced' (up to date), or 'conflict'.
  • updated_at: Timestamp representing the last edit.
  • is_deleted: Soft deletion flag (1 = deleted, 0 = active).
• Server-Mediated Role-Based LWW: The local hub server resolves conflicts using Role-Based Last Write Wins (LWW) defined in sync.controller.js.
• Mobile-Side Pull Resolution: During a pull update, the client overwrites local database rows only if the incoming record's updated_at timestamp is newer, or if it resolves a local 'conflict' state.

---

4. Failure Behavior

• Server Unreachable / Connection Drops:
  • The API call probeBackend() (with a 3-second timeout) acts as the connection check before syncing.
  • If probeBackend() returns false or any fetch call throws a network error, the client gracefully cancels the sync cycle and transitions UI indicators to offline. Unsynced data must remain intact as 'pending'.
• Authentication Failures (401/403):
  • If the server rejects the request with a token error, the sync engine suppresses fatal crashes. It marks the device as offline (or prompts re-login) and preserves the queue.
• Partial Table Failures:
  • If syncing a particular table throws an SQL error, log it in the local sync_logs and proceed to sync other tables. Increment the table's failure retry counts but do not block the rest of the queue.

---

5. Output Context

Request Payload Contracts

• Push Endpoint (POST /api/sync/push):

  {
    "changes": {
      "messages": [
        { "uid": "msg-uuid-123", "sender_id": 1, "body": "Hello", "updated_at": "2026-07-09T23:00:00Z" }
      ]
    }
  }
  

• Pull Endpoint (GET /api/sync/pull?last_sync=ISO_TIMESTAMP):

  {
    "success": true,
    "changes": {
      "users": [...],
      "messages": [...]
    },
    "server_time": "2026-07-09T23:15:00Z"
  }
  

---

6. Quality Bar

• SQLite Transactions: Push and pull updates per table must be wrapped in transactions (db.transaction or SQLite batch execution) to prevent partial writes.
• No Direct Cloud Writes: The client must not use @supabase/supabase-js for data sync. All supabase communication goes through the server's backend sync engine.
• Soft Deletion Preservation: Checking is_deleted = 1 updates must flow correctly to prevent deleted records from reappearing.

---

7. Step-by-Step Implementation Plan

Part 1: Backend Connection Probe & Network State

1. Update api.ts to export probeBackend(timeoutMs?: number) as the source of truth for the local server's online status.
2. In sync.service.ts, modify the NetInfo event listener. Instead of relying solely on the device's internet state, combine it with a periodic call to api.probeBackend() to set the online flag.

Part 2: Mobilizing Push Sync to Web Server

1. Remove references to supabase client inside the pushChanges function in sync.service.ts.
2. Rewrite pushChanges to scan all sync-enabled SQLite tables for rows where sync_status = 'pending'.
3. Construct a single nested object of changes: { [tableName]: record[] }.
4. Send the payload to /api/sync/push using api.post('/sync/push', { changes }).
5. Upon receiving { success: true, results: { applied, rejected } }, update local database rows: set sync_status = 'synced' and last_synced_at = nowIso().

Part 3: Mobilizing Pull Sync from Web Server

1. Rewrite pullChanges in sync.service.ts to replace Supabase calls with api.get('/sync/pull', { last_sync: lastSyncTimestamp }).
2. Retrieve the changes object from the response.
3. Iterate over each table and upsert records into local SQLite.
4. Update local system settings or local storage variables with the returned server_time to use as the next last_sync timestamp.

Part 4: Conflict Resolution and LWW Implementation

1. Ensure the SQLite pull execution in sync.service.ts compares timestamps if the local row is pending. If the incoming server row is newer, update it; otherwise, flag the row as conflict.
2. Ensure the web server's sync.controller.js is correctly registering updates in sync_metadata for every table change.

Part 5: Local Database Transactional Consistency

1. Wrap the pull logic updates in sync.service.ts using powerSync.writeTransaction or transactional batch executes to ensure database state is never left half-synced if the app is closed mid-operation.

Part 6: Testing and Offline Simulation Checks

1. Simulate offline state by stopping the webapp server (npm stop or killing node process).
2. Perform inserts, updates, and deletes in the mobile app. Verify that changes are written locally and marked pending.
3. Start the webapp server. Verify probeBackend returns true, triggers the auto-sync engine, and successfully synchronizes all queued changes to /api/sync/push.
4. Verify the web server's SyncEngine uploads the school hub data successfully to Supabase.