# Offline-First Database Sync Plan

This document outlines the blueprint and instructions for linking the React Native mobile app with the Express-based web app server. It replaces direct-to-Supabase synchronization on the mobile client with web-server-mediated syncing.

---

## 1. Primary Intent
To redirect the React Native mobile app ([nextgenmobile](file:///E:/nextgenmobile)) sync path away from direct Supabase communication to a local-hub-mediated synchronization architecture using the Express-based web application server ([server](file:///E:/nextgen/next-gen/server)).
- **Online State Definition**: The system detects the mobile app is "online" if and only if the web server's API endpoint is reachable (running on port 3001).
- **Offline State Definition**: The system is "offline" when the web server is stopped, crashed, or network-unreachable.
- **Offline-First Synchronization**: The mobile app functions with full local write capability into its local SQLite database (using `expo-sqlite` and `PowerSync`). Once a connection to the web server is verified, it pushes all locally accumulated changes (marked as `pending`) to the server via `/api/sync/push` and pulls remote changes via `/api/sync/pull`.

---

## 2. Context Boundaries
### In-Scope Files
- **Mobile Client Sync Coordinator**: [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts)
- **Mobile API Client Helper**: [api.ts](file:///E:/nextgenmobile/src/lib/api.ts)
- **Mobile DB Schema definitions**: [schemaSql.ts](file:///E:/nextgenmobile/src/db/schemaSql.ts) and [database.ts](file:///E:/nextgenmobile/src/db/database.ts)
- **Web App Server Sync Endpoints**: [sync.controller.js](file:///E:/nextgen/next-gen/server/src/controllers/sync.controller.js)
- **Web App Server Entry Point**: [index.js](file:///E:/nextgen/next-gen/server/src/index.js)
- **Web App Server DB Config/Sync Engine**: [SyncEngine.js](file:///E:/nextgen/next-gen/server/src/services/SyncEngine.js)

### Synced Database Tables (23 Tables)
The synchronization covers the following entities:
`users`, `departments`, `classes`, `subjects`, `courses`, `enrollments`, `attendance`, `fee_groups`, `student_fees`, `payments`, `messages`, `events`, `expenses`, `grades`, `assignments`, `submissions`, `staff_records`, `leave_requests`, `item_stock`, `items`, `hostels`, `rooms`, `routes`

---

## 3. Reasoning Constraints
- **Offline Writes**: Writes are recorded in local SQLite immediately. No remote calls block user interaction.
- **Change Tracking**: Every table row uses the standard sync columns:
  - `uid`: Unique identifier (UUID string) shared globally.
  - `sync_status`: Tracked as `'pending'` (unsynced), `'synced'` (up to date), or `'conflict'`.
  - `updated_at`: Timestamp representing the last edit.
  - `is_deleted`: Soft deletion flag (`1` = deleted, `0` = active).
- **Server-Mediated Role-Based LWW**: The local hub server resolves conflicts using Role-Based Last Write Wins (LWW) defined in [sync.controller.js](file:///E:/nextgen/next-gen/server/src/controllers/sync.controller.js).
- **Mobile-Side Pull Resolution**: During a pull update, the client overwrites local database rows only if the incoming record's `updated_at` timestamp is newer, or if it resolves a local `'conflict'` state.

---

## 4. Failure Behavior
- **Server Unreachable / Connection Drops**:
  - The API call `probeBackend()` (with a 3-second timeout) acts as the connection check before syncing.
  - If `probeBackend()` returns `false` or any fetch call throws a network error, the client gracefully cancels the sync cycle and transitions UI indicators to offline. Unsynced data must remain intact as `'pending'`.
- **Authentication Failures (401/403)**:
  - If the server rejects the request with a token error, the sync engine suppresses fatal crashes. It marks the device as offline (or prompts re-login) and preserves the queue.
- **Partial Table Failures**:
  - If syncing a particular table throws an SQL error, log it in the local `sync_logs` and proceed to sync other tables. Increment the table's failure retry counts but do not block the rest of the queue.

---

## 5. Output Context
### Request Payload Contracts
- **Push Endpoint (`POST /api/sync/push`)**:
  ```json
  {
    "changes": {
      "messages": [
        { "uid": "msg-uuid-123", "sender_id": 1, "body": "Hello", "updated_at": "2026-07-09T23:00:00Z" }
      ]
    }
  }
  ```
- **Pull Endpoint (`GET /api/sync/pull?last_sync=ISO_TIMESTAMP`)**:
  ```json
  {
    "success": true,
    "changes": {
      "users": [...],
      "messages": [...]
    },
    "server_time": "2026-07-09T23:15:00Z"
  }
  ```

---

## 6. Quality Bar
- **SQLite Transactions**: Push and pull updates per table must be wrapped in transactions (`db.transaction` or SQLite batch execution) to prevent partial writes.
- **No Direct Cloud Writes**: The client must not use `@supabase/supabase-js` for data sync. All supabase communication goes through the server's backend sync engine.
- **Soft Deletion Preservation**: Checking `is_deleted = 1` updates must flow correctly to prevent deleted records from reappearing.

---

## 7. Step-by-Step Implementation Plan

### Part 1: Backend Connection Probe & Network State
1. Update [api.ts](file:///E:/nextgenmobile/src/lib/api.ts) to export `probeBackend(timeoutMs?: number)` as the source of truth for the local server's online status.
2. In [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts), modify the `NetInfo` event listener. Instead of relying solely on the device's internet state, combine it with a periodic call to `api.probeBackend()` to set the `online` flag.

### Part 2: Mobilizing Push Sync to Web Server
1. Remove references to `supabase` client inside the `pushChanges` function in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts).
2. Rewrite `pushChanges` to scan all sync-enabled SQLite tables for rows where `sync_status = 'pending'`.
3. Construct a single nested object of changes: `{ [tableName]: record[] }`.
4. Send the payload to `/api/sync/push` using `api.post('/sync/push', { changes })`.
5. Upon receiving `{ success: true, results: { applied, rejected } }`, update local database rows: set `sync_status = 'synced'` and `last_synced_at = nowIso()`.

### Part 3: Mobilizing Pull Sync from Web Server
1. Rewrite `pullChanges` in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts) to replace Supabase calls with `api.get('/sync/pull', { last_sync: lastSyncTimestamp })`.
2. Retrieve the changes object from the response.
3. Iterate over each table and upsert records into local SQLite.
4. Update local system settings or local storage variables with the returned `server_time` to use as the next `last_sync` timestamp.

### Part 4: Conflict Resolution and LWW Implementation
1. Ensure the SQLite pull execution in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts) compares timestamps if the local row is `pending`. If the incoming server row is newer, update it; otherwise, flag the row as `conflict`.
2. Ensure the web server's [sync.controller.js](file:///E:/nextgen/next-gen/server/src/controllers/sync.controller.js) is correctly registering updates in `sync_metadata` for every table change.

### Part 5: Local Database Transactional Consistency
1. Wrap the pull logic updates in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts) using `powerSync.writeTransaction` or transactional batch executes to ensure database state is never left half-synced if the app is closed mid-operation.

### Part 6: Testing and Offline Simulation Checks
1. Simulate offline state by stopping the webapp server (`npm stop` or killing node process).
2. Perform inserts, updates, and deletes in the mobile app. Verify that changes are written locally and marked `pending`.
3. Start the webapp server. Verify `probeBackend` returns true, triggers the auto-sync engine, and successfully synchronizes all queued changes to `/api/sync/push`.
4. Verify the web server's SyncEngine uploads the school hub data successfully to Supabase.