117 lines
7.9 KiB
Markdown
117 lines
7.9 KiB
Markdown
# Offline-First Database Sync Plan
|
|
|
|
This document outlines the blueprint and instructions for linking the React Native mobile app with the Express-based web app server. It replaces direct-to-Supabase synchronization on the mobile client with web-server-mediated syncing.
|
|
|
|
---
|
|
|
|
## 1. Primary Intent
|
|
To redirect the React Native mobile app ([nextgenmobile](file:///E:/nextgenmobile)) sync path away from direct Supabase communication to a local-hub-mediated synchronization architecture using the Express-based web application server ([server](file:///E:/nextgen/next-gen/server)).
|
|
- **Online State Definition**: The system detects the mobile app is "online" if and only if the web server's API endpoint is reachable (running on port 3001).
|
|
- **Offline State Definition**: The system is "offline" when the web server is stopped, crashed, or network-unreachable.
|
|
- **Offline-First Synchronization**: The mobile app functions with full local write capability into its local SQLite database (using `expo-sqlite` and `PowerSync`). Once a connection to the web server is verified, it pushes all locally accumulated changes (marked as `pending`) to the server via `/api/sync/push` and pulls remote changes via `/api/sync/pull`.
|
|
|
|
---
|
|
|
|
## 2. Context Boundaries
|
|
### In-Scope Files
|
|
- **Mobile Client Sync Coordinator**: [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts)
|
|
- **Mobile API Client Helper**: [api.ts](file:///E:/nextgenmobile/src/lib/api.ts)
|
|
- **Mobile DB Schema definitions**: [schemaSql.ts](file:///E:/nextgenmobile/src/db/schemaSql.ts) and [database.ts](file:///E:/nextgenmobile/src/db/database.ts)
|
|
- **Web App Server Sync Endpoints**: [sync.controller.js](file:///E:/nextgen/next-gen/server/src/controllers/sync.controller.js)
|
|
- **Web App Server Entry Point**: [index.js](file:///E:/nextgen/next-gen/server/src/index.js)
|
|
- **Web App Server DB Config/Sync Engine**: [SyncEngine.js](file:///E:/nextgen/next-gen/server/src/services/SyncEngine.js)
|
|
|
|
### Synced Database Tables (23 Tables)
|
|
The synchronization covers the following entities:
|
|
`users`, `departments`, `classes`, `subjects`, `courses`, `enrollments`, `attendance`, `fee_groups`, `student_fees`, `payments`, `messages`, `events`, `expenses`, `grades`, `assignments`, `submissions`, `staff_records`, `leave_requests`, `item_stock`, `items`, `hostels`, `rooms`, `routes`
|
|
|
|
---
|
|
|
|
## 3. Reasoning Constraints
|
|
- **Offline Writes**: Writes are recorded in local SQLite immediately. No remote calls block user interaction.
|
|
- **Change Tracking**: Every table row uses the standard sync columns:
|
|
- `uid`: Unique identifier (UUID string) shared globally.
|
|
- `sync_status`: Tracked as `'pending'` (unsynced), `'synced'` (up to date), or `'conflict'`.
|
|
- `updated_at`: Timestamp representing the last edit.
|
|
- `is_deleted`: Soft deletion flag (`1` = deleted, `0` = active).
|
|
- **Server-Mediated Role-Based LWW**: The local hub server resolves conflicts using Role-Based Last Write Wins (LWW) defined in [sync.controller.js](file:///E:/nextgen/next-gen/server/src/controllers/sync.controller.js).
|
|
- **Mobile-Side Pull Resolution**: During a pull update, the client overwrites local database rows only if the incoming record's `updated_at` timestamp is newer, or if it resolves a local `'conflict'` state.
|
|
|
|
---
|
|
|
|
## 4. Failure Behavior
|
|
- **Server Unreachable / Connection Drops**:
|
|
- The API call `probeBackend()` (with a 3-second timeout) acts as the connection check before syncing.
|
|
- If `probeBackend()` returns `false` or any fetch call throws a network error, the client gracefully cancels the sync cycle and transitions UI indicators to offline. Unsynced data must remain intact as `'pending'`.
|
|
- **Authentication Failures (401/403)**:
|
|
- If the server rejects the request with a token error, the sync engine suppresses fatal crashes. It marks the device as offline (or prompts re-login) and preserves the queue.
|
|
- **Partial Table Failures**:
|
|
- If syncing a particular table throws an SQL error, log it in the local `sync_logs` and proceed to sync other tables. Increment the table's failure retry counts but do not block the rest of the queue.
|
|
|
|
---
|
|
|
|
## 5. Output Context
|
|
### Request Payload Contracts
|
|
- **Push Endpoint (`POST /api/sync/push`)**:
|
|
```json
|
|
{
|
|
"changes": {
|
|
"messages": [
|
|
{ "uid": "msg-uuid-123", "sender_id": 1, "body": "Hello", "updated_at": "2026-07-09T23:00:00Z" }
|
|
]
|
|
}
|
|
}
|
|
```
|
|
- **Pull Endpoint (`GET /api/sync/pull?last_sync=ISO_TIMESTAMP`)**:
|
|
```json
|
|
{
|
|
"success": true,
|
|
"changes": {
|
|
"users": [...],
|
|
"messages": [...]
|
|
},
|
|
"server_time": "2026-07-09T23:15:00Z"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 6. Quality Bar
|
|
- **SQLite Transactions**: Push and pull updates per table must be wrapped in transactions (`db.transaction` or SQLite batch execution) to prevent partial writes.
|
|
- **No Direct Cloud Writes**: The client must not use `@supabase/supabase-js` for data sync. All supabase communication goes through the server's backend sync engine.
|
|
- **Soft Deletion Preservation**: Checking `is_deleted = 1` updates must flow correctly to prevent deleted records from reappearing.
|
|
|
|
---
|
|
|
|
## 7. Step-by-Step Implementation Plan
|
|
|
|
### Part 1: Backend Connection Probe & Network State
|
|
1. Update [api.ts](file:///E:/nextgenmobile/src/lib/api.ts) to export `probeBackend(timeoutMs?: number)` as the source of truth for the local server's online status.
|
|
2. In [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts), modify the `NetInfo` event listener. Instead of relying solely on the device's internet state, combine it with a periodic call to `api.probeBackend()` to set the `online` flag.
|
|
|
|
### Part 2: Mobilizing Push Sync to Web Server
|
|
1. Remove references to `supabase` client inside the `pushChanges` function in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts).
|
|
2. Rewrite `pushChanges` to scan all sync-enabled SQLite tables for rows where `sync_status = 'pending'`.
|
|
3. Construct a single nested object of changes: `{ [tableName]: record[] }`.
|
|
4. Send the payload to `/api/sync/push` using `api.post('/sync/push', { changes })`.
|
|
5. Upon receiving `{ success: true, results: { applied, rejected } }`, update local database rows: set `sync_status = 'synced'` and `last_synced_at = nowIso()`.
|
|
|
|
### Part 3: Mobilizing Pull Sync from Web Server
|
|
1. Rewrite `pullChanges` in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts) to replace Supabase calls with `api.get('/sync/pull', { last_sync: lastSyncTimestamp })`.
|
|
2. Retrieve the changes object from the response.
|
|
3. Iterate over each table and upsert records into local SQLite.
|
|
4. Update local system settings or local storage variables with the returned `server_time` to use as the next `last_sync` timestamp.
|
|
|
|
### Part 4: Conflict Resolution and LWW Implementation
|
|
1. Ensure the SQLite pull execution in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts) compares timestamps if the local row is `pending`. If the incoming server row is newer, update it; otherwise, flag the row as `conflict`.
|
|
2. Ensure the web server's [sync.controller.js](file:///E:/nextgen/next-gen/server/src/controllers/sync.controller.js) is correctly registering updates in `sync_metadata` for every table change.
|
|
|
|
### Part 5: Local Database Transactional Consistency
|
|
1. Wrap the pull logic updates in [sync.service.ts](file:///E:/nextgenmobile/src/services/sync.service.ts) using `powerSync.writeTransaction` or transactional batch executes to ensure database state is never left half-synced if the app is closed mid-operation.
|
|
|
|
### Part 6: Testing and Offline Simulation Checks
|
|
1. Simulate offline state by stopping the webapp server (`npm stop` or killing node process).
|
|
2. Perform inserts, updates, and deletes in the mobile app. Verify that changes are written locally and marked `pending`.
|
|
3. Start the webapp server. Verify `probeBackend` returns true, triggers the auto-sync engine, and successfully synchronizes all queued changes to `/api/sync/push`.
|
|
4. Verify the web server's SyncEngine uploads the school hub data successfully to Supabase.
|