# Nextgen LMS — National Scale Multi-Tenant & SuperAdmin Licensing Platform Documentation

## Executive Summary
This document provides full technical, architectural, operational, and development-testing documentation for the **Nextgen LMS Multi-Tenant Ecosystem**, the **Standalone SuperAdmin Governance Portal**, the **Cryptographic Termly Licensing Engine**, the **Multi-School Instance Spawner**, and the **Event-Sourced Sync Engine**.

> [!IMPORTANT]
> **Application Identity**: This SuperAdmin portal is part of the **NextGen LMS** platform (formerly referenced as Africa Alert). Within `superadmin/` the rename is complete: the browser title, sidebar header, `/api/health` service name, license token issuer, and `package.json` descriptions now all read NextGen LMS. Broader `client/` and `server/` surfaces still carry the Africa Alert string pending a coordinated cutover; do not consider those authoritative.

---

## 1. System Architecture Overview

```mermaid
graph TD
    subgraph SuperAdmin Governance Layer (e:\nextgen\next-gen\superadmin)
        SA_Client["Standalone SuperAdmin Client (Port 3003)"] --> SA_Server["Standalone SuperAdmin Server (Port 3002)"]
        SA_Server -->|Spawns & Provisions| Spawner["InstanceSpawner Orchestrator"]
        SA_Server -->|Issues Signed HS256 JWT Tokens| LicGen["LicenseTokenGenerator"]
        SA_Server -->|Direct Cloud Sync & Bucket Provisioning| SupabaseCloud["Supabase Cloud (@supabase/supabase-js)"]
    end

    subgraph Spawned Subscribing School Instances (Dev / Production)
        Spawner -->|Spawned Server #1| SchoolServer1["School Server #1 (Port 3010 / DB #1)"]
        Spawner -->|Spawned Server #2| SchoolServer2["School Server #2 (Port 3011 / DB #2)"]
        Spawner -->|Spawned Server #N| SchoolServerN["School Server #N (Port 301X / DB #N)"]
    end

    subgraph School Instance Architecture (Template: server/ & client/)
        SchoolServer1 --> AccessMW["Licensing & Access Middleware"]
        AccessMW -->|Valid License / Grace Period| MainAPI["School Express API"]
        AccessMW -->|Expired License| 402Block["402 Payment Required / Billing Prompt"]
        MainAPI --> PostgresRLS["PostgreSQL DB with Row Level Security (RLS)"]
    end
```

---

## 2. SuperAdmin Direct Integration with Supabase Cloud
- **Direct Supabase Link (`SupabaseSuperAdminService.js`)**:
  - SuperAdmin Server integrates with Supabase Cloud via `@supabase/supabase-js` using `SUPABASE_URL` and `SUPABASE_KEY` (the **anon / publishable** JWT).
  - **No service-role key is held by any process** — neither the SuperAdmin nor any tenant server reads `SUPABASE_SERVICE_ROLE_KEY`. The trust boundary is the Postgres function layer, not the JWT.
  - **Cloud Multi-Tenant Sync**: Tenant onboarding and license issuance call SECURITY DEFINER RPC functions (`upsert_tenant_cloud`, `upsert_license_cloud`) installed by migration `2026072900000100_superadmin_rpc_functions.js` (raw SQL in `sql/20260729_superadmin_rpc_functions.sql`). These functions validate the payload and perform the writes; they are GRANTed to `anon` and `authenticated`.
  - **Cloud RLS & Governance**: Reads (cloud tenant list, per-tenant cloud state, cloud license list) go through the anon role under the RLS read policy from migration `2026072900000000_tenant_licenses_rls_read_policy`. The application layer scopes by `tenant_id` as defence-in-depth.

### 2.1 Tenant License Distribution (Supabase-first)

> **Updated in `feature/supabase-license-fetch`** — licenses are now fetched **directly from Supabase Cloud** by the tenant server, then verified offline with the Ed25519 public key.

```
┌──────────────────┐  signed JWT   ┌──────────────────┐  anon + RPC    ┌──────────────────┐
│  SuperAdmin      │  ─────────▶  │ Supabase Cloud   │  ───────────▶  │  tenant_licenses  │
│  (port 3002)     │              │ (anon-readable)  │  upsert_*      │  table            │
└──────────────────┘              └──────────────────┘                └──────────────────┘
                                           ▲
                                           │ anon key + ?tenant_id=eq.<uuid>
                                           │ 5s timeout, AbortSignal
                                           ▼
                                  ┌──────────────────┐
                                  │ Tenant server    │
                                  │ licensingMW      │  ──▶ Verify EdDSA against
                                  │ (port 3001)      │      SUPERADMIN_PUBLIC_KEY
                                  └──────────────────┘
                                           │
                                           ▼ on success
                                  ┌──────────────────┐
                                  │ Local SQLite     │  ← write-back cache
                                  │ tenant_licenses  │     (offline resilience)
                                  └──────────────────┘
```

- **Read path on the tenant server**: `server/src/services/SupabaseLicenseService.js` constructs a Supabase client with the anon key (`SUPABASE_KEY` / `VITE_SUPABASE_ANON_KEY`) and queries `tenant_licenses` filtered by the request's `tenant_id`. 5s timeout via `AbortSignal.timeout`.
- **Middleware ordering** (`server/src/middleware/licensingMiddleware.js`):
  1. **Cache** (`LicenseCacheService` LRU + optional Redis) — TTL clamped to `exp - now`.
  2. **Supabase fetch** — `SupabaseLicenseService.fetchActiveLicense(tenantId)`. On success: verify EdDSA against `SUPERADMIN_PUBLIC_KEY`, write-back to local SQLite, cache.
  3. **Local SQLite fallback** — preserved; only reached when Supabase is unreachable.
- **`/api/license/current` response** now includes a `source` field: `supabase | local | cache | none`. The tenant UI subscribes via `client/src/store/license.ts` and renders a banner mapping each source to a tone (Supabase = emerald, local = amber, none = rose).
- **RLS policy for `tenant_licenses`**: see `server/src/database/migrations/knex/2026072900000000_tenant_licenses_rls_read_policy.js` and the equivalent `sql/20260729_tenant_licenses_rls_read_policy.sql`. The policy allows the `anon` and `authenticated` roles to SELECT; the application layer scopes every fetch by `tenant_id`. No INSERT/UPDATE/DELETE policy is granted to anon, so tenants cannot write.
- **Graceful degradation**: every response carries `X-License-Source: supabase | local | cache | none` for observability. Fallback to local on Supabase 5xx / network errors / `SUPABASE_AUTH` 401. The X-License-Status header continues to report `ACTIVE | EXPIRED | No-License | Invalid-Signature | …`.
- **UI surface**:
  - SuperAdmin `Cloud Link` page gains a `Tenant License Distribution Flow` section explaining the three steps.
  - SuperAdmin `Licensing Engine` page has a `DELIVERED VIA SUPABASE` chip and a delivery-channels note in the Trust Anchor section.
  - Tenant client renders `LicenseSourceBanner` mounted under the nav, surfacing the source (Supabase / local cache) and term status.

---

## 3. Dynamic Multi-School Instance Spawner & Provisioning
- **Template Instance**: The current `server/` (port 3001) and `client/` (port 3000) represent the template school instance in dev mode (the Vite dev server proxies `/api` to `http://127.0.0.1:3001`).
- **Instance Spawner (`InstanceSpawner.js`)**:
  - Located in `superadmin/server/src/services/InstanceSpawner.js`.
  - Spawns isolated node server & client child processes for each subscribing school.
  - Provisions an isolated database file `server/data/schools/<school_id>/school.db` (or dedicated PostgreSQL database schema per tenant).
  - Dynamically assigns unique server ports (`3010`, `3011`, `3012`...) and client ports (`5180`, `5181`...) (per-instance vite dev server inherits its target ports from the orchestrator and reads its `/api` proxy target from the spawned node backend's port via the `VITE_API_TARGET` and `VITE_WS_TARGET` env vars passed at spawn time).
  - Generates and injects cryptographically signed termly license tokens (`LICENSE_KEY`).
  - Persists the spawned-instance registry, port counter, and last 500 stdout/stderr lines per instance to a sidecar SQLite DB at `superadmin/server/data/spawner-registry.db` so the orchestrator survives restarts.
  - Monitors child process PIDs, streams stdout/stderr telemetry, and exposes remote **start / stop / restart** controls from the SuperAdmin Portal (registered in the orphaned state on startup so previously-spawned instances can be re-launched without re-onboarding).

---

## 4. Sprint Breakdown & Implementation Details

### 🚀 Sprint 1: Multi-Tenant Data Architecture
- **PostgreSQL Migration DDL**: Migrated database schema definitions from SQLite constructs to PostgreSQL compatible Knex DDL (`SERIAL`/`BIGSERIAL` primary keys, `CURRENT_TIMESTAMP`, Postgres enum/check constraints).
- **Tenant Isolation**: Injected `tenant_id` column across all domain tables (`users`, `classes`, `subjects`, `enrollments`, `courses`, `payments`, `grades`, `attendance`, `assignments`, `submissions`, `inventory`, `payroll`, `exams`, `hostels`, `transport`, `clubs`, etc.).
- **Row Level Security (RLS)**: Enforced PostgreSQL RLS policies on all domain tables:
  ```sql
  CREATE POLICY tenant_isolation_policy ON <table>
    FOR ALL
    USING (
      tenant_id = current_setting('app.current_tenant', true)
      OR current_setting('app.is_super_admin', true) = 'true'
    );
  ```

---

### 💳 Sprint 2: Super Admin & Licensing Engine (Backend)
- **Tenants & Licensing Schema**:
  - `tenants`: Primary tenant registry storing `id`, `name`, `code`, `contact_email`, `phone`, `status` (`active`, `suspended`, `trial`, `grace_period`).
  - `tenant_licenses`: Termly license registry storing `license_key`, `term_name`, `academic_year`, `start_date`, `end_date`, `grace_period_days`, `max_students`, `max_staff`, `status` (`active`, `expired`, `revoked`, `overridden`).
- **Cryptographic License Token Generator (`LicenseTokenGenerator.js`)**:
  Generates cryptographically signed **HMAC-SHA256 (HS256)** JWT tokens (`LIC-TERM1-2026-TOKEN...`) containing tenant metadata, validity windows, and learner caps to enable offline verification on local PWA/school servers.
- **Licensing Access Middleware (`licensingMiddleware.js`)**:
  - Inspects `X-Tenant-ID` / `X-Tenant-Code` header or JWT payload.
  - Validates active term license dates + grace period.
  - If a school's license is expired, allows `GET` read-only operations for data retrieval and billing navigation, but strictly blocks all non-readonly endpoints (`POST`, `PUT`, `PATCH`, `DELETE`) with `402 Payment Required`.

---

### 🖥️ Sprint 3: Super Admin Portal (Frontend & API)
- **Standalone Location**: `e:\nextgen\next-gen\superadmin\`
  - **Server**: `superadmin/server` running on **Port 3002** with isolated master key/JWT authentication (`X-SuperAdmin-Key`).
  - **Client**: `superadmin/client` running on **Port 3003** built with Vite, React, Lucide Icons, and Tailwind CSS.
- **Key Features**:
  - **National Ecosystem Telemetry**: Real-time aggregated stats (Subscribing Schools Count, Active vs Expired Licenses, Total Active Learners, Termly Revenue (rolling 90-day SUM of completed `payments.amount`, returned in USD by convention)).
  - **School Onboarding Drawer**: Form to register new subscribing institutions and issue initial term licenses.
  - **Termly License Generator**: Tool to generate signed term licenses (`Term 1`, `Term 2`, `Term 3`, `Annual`) with custom validity dates.
  - **Emergency Access Override**: Form to grant emergency extensions with rationale audit logs.

---

### 🔄 Sprint 4: Sync Engine Event Sourcing Upgrade
- **Idempotent Event Processor (`SyncEngine.js`)**:
  - Assigns/verifies UUIDv4 `event_id` and monotonic sequence numbers (`seq_num`) for every pushed edit event.
  - Maintains `sync_events` table to reject duplicate events.
- **UTC Clock Normalization**:
  - Calculates time delta between client connection timestamp and server UTC time (`server_now - client_time`).
  - Converts client timestamps to normalized UTC server time and rejects stale sequence events (`seq_num <= max_seq`).

---

## 5. API Reference Guide

### SuperAdmin Gateway API (Port 3002)

| Method | Endpoint | Description | Auth Header |
| :--- | :--- | :--- | :--- |
| `POST` | `/api/auth/login` | SuperAdmin Master Login | None |
| `GET` | `/api/health` | Liveness check (no auth) | None |
| `GET` | `/api/metrics` | Global Ecosystem Health & Licensing Telemetry | `X-SuperAdmin-Key` |
| `GET` | `/api/tenants` | List all subscribing schools with license status | `X-SuperAdmin-Key` |
| `POST` | `/api/tenants` | Onboard a new school tenant & issue license | `X-SuperAdmin-Key` |
| `POST` | `/api/tenants/:id/licenses` | Issue / renew a signed termly license | `X-SuperAdmin-Key` |
| `POST` | `/api/tenants/:id/override` | Grant emergency access override | `X-SuperAdmin-Key` |
| `GET` | `/api/instances` | List all spawned school server/client instances | `X-SuperAdmin-Key` |
| `POST` | `/api/instances/spawn` | Spawn a new school server & client instance | `X-SuperAdmin-Key` |
| `POST` | `/api/instances/:id/stop` | Stop a running school instance | `X-SuperAdmin-Key` |
| `POST` | `/api/instances/:id/start` | Start a previously-spawned stopped school instance | `X-SuperAdmin-Key` |
| `POST` | `/api/instances/:id/restart` | Restart a running or stopped school instance | `X-SuperAdmin-Key` |
| `GET` | `/api/instances/:id/logs` | Fetch live stdout/stderr logs of a school instance | `X-SuperAdmin-Key` |

---

## 6. SuperAdmin UI Component Architecture, Design System & Theme Adaptation

The SuperAdmin Portal UI (`superadmin/client`) is built to match the exact design system, UI components, typography, layout, and light/dark theme adaptation of the main **Nextgen LMS** school application (`client/src`).

### 1. Unified Modal Component Architecture (`components/ui/Modal.tsx`)
- **Single Component Pattern**: All SuperAdmin modal windows (School Onboarding, Term License Generation, Emergency Access Override, Instance Spawning, and School Profile Inspector) use the single `Modal` component copied from `client/src/components/ui/Modal.tsx`.
- **Blueprint Compliance (`modals.md`)**:
  - **Portal Rendering**: Uses React `createPortal` to mount modal elements directly to `document.body`.
  - **Backdrop Styling**: Deep Navy blur backdrop `bg-[#002147]/60 backdrop-blur-sm` with default `zIndex = 150`.
  - **Border Radius**: Custom `rounded-[2.5rem]` corner styling matching application design guidelines.
  - **Watermark & Header**: `ModalHeader` incorporates rotating background watermark icons (`DisplayWatermark`), title, uppercase tracking subtitle, and close button (`X`).
  - **Scroll Locking**: Automatically locks body background scroll upon open and unlocks on close.

### 2. School Tenant Profile Inspector Modal (`components/TenantProfileModal.tsx`)
- **Institution Overview**: Displays full school metadata (name, code, contact email, phone, tenant ID).
- **Subscribed Capacity Progress Meters**: Interactive progress bars displaying enrolled student count vs max student cap, and staff directory count vs max staff cap.
- **Cryptographic Token Viewer**: Displays active signed HS256 license token (`LIC-TERM1-2026...`) with a one-click Copy to Clipboard button.
- **Direct Action Buttons**: Quick triggers to Renew Term License, Grant Emergency Access Override, or Spawn Isolated Server Instance.

### 3. Live Telemetry & Server Logs Terminal Stream (`components/TelemetryLogsView.tsx`)
- **Terminal Styling**: Styled as a high-tech terminal window (`#050811`) with live streaming telemetry output.
- **Level Filtering**: Filter log stream by level (`ALL`, `INFO`, `WARN`, `ERROR`).
- **Log Search & Export**: Full-text log searching and one-click log file download (`.log`).
- Source: the orchestrator's persisted log buffer (`/api/instances/:id/logs`); the view auto-refreshes every 3s when the user has selected a spawned instance from the dropdown.

### 4. Toast Notification System (`components/ui/Toast.tsx`)
- Floating feedback notifications in bottom-right corner providing user feedback upon school onboarding, key generation, emergency access overrides, or instance spawner actions.

### 5. Multi-Filter Toolbar & Registry CSV Export
- Search box filtering school name, code, or contact email.
- Status dropdown filter (`ALL`, `ACTIVE`, `EXPIRED`, `OVERRIDDEN`).
- One-click CSV export button (`schools_directory_YYYY-MM-DD.csv`) (filter values mirror the documented `tenant_licenses.status` enum: `active`, `expired`, `overridden`, `revoked`, `grace_period`, `trial`).

---

## 7. Development & Testing Considerations for Local Instances

When developing and testing the application locally using the template school instance (`server/` on Port 3001, `client/` on Port 3000, with `/api` proxied to `http://127.0.0.1:3001` by Vite) alongside SuperAdmin (`superadmin/server` on Port 3002, `superadmin/client` on Port 3003), take care of the following key items:

### 1. Dev Mode Default Tenant Seeding & Fallback
- **Default Seeding**: Running `npm run db:init` automatically executes migration `2026072200000000_multitenant_licensing_rls.js`, creating the `5b39bcb5-506a-456d-b7cb-91730b397cc1` record in `tenants` and seeding a 1-year valid active term license (`LIC-DEFAULT-2026-TERM1-VALID`).
- **Seamless Local Dev Testing**: If `TENANT_ID` is not explicitly passed in environment variables or HTTP headers, `licensingMiddleware` defaults to `tenant_id = 5b39bcb5-506a-456d-b7cb-91730b397cc1`. This ensures local developers can build and test UI features without being blocked by licensing checks.

### 2. Dual-Database Engine Switch (SQLite vs PostgreSQL RLS)
- **Local SQLite Mode (default and only currently working path)**: `server/` runs on `better-sqlite3` (`data/school.db`). The PostgreSQL RLS branch of the multi-tenant migration is gated by `knex.client.config.client === 'pg'` (see `server/src/database/migrations/knex/2026072200000000_multitenant_licensing_rls.js:77`) and is bypassed when `knexfile.js` resolves to SQLite. Tenant isolation is still enforced, just at the application layer via `licensingMiddleware` and `current_setting`-style SQL filter parameters in controllers.
- **PostgreSQL RLS path (currently requires code change)**: The migration file contains the full `ALTER TABLE ... ENABLE ROW LEVEL SECURITY` and policy DDL shown in Section 4 / Sprint 1, but `server/knexfile.js:10` hardcodes `client: 'better-sqlite3'` and ignores the `DB_CLIENT` / `DATABASE_URL` environment variables. To exercise the native RLS branch, edit `server/knexfile.js` to honor `process.env.DB_CLIENT` and provide a Postgres connection, then re-run `npm run db:init`.

### 3. Testing Expired Term Licenses & HTTP 402 Restrictive Mode
- **Simulating Expired License**: To test how the application behaves when a school's license expires:
  1. Open SQLite database (`server/data/school.db`) or execute SQL:
     ```sql
     UPDATE tenant_licenses SET end_date = '2025-01-01', status = 'expired' WHERE tenant_id = '5b39bcb5-506a-456d-b7cb-91730b397cc1';
     ```
  2. Perform a `GET` request (e.g. `GET /api/students`): Notice response succeeds with header `X-License-Restriction: Read-Only Mode Enforced`.
  3. Perform a `POST` or `PUT` write request (e.g. `POST /api/grades`): Notice response is blocked with HTTP status `402 Payment Required` and JSON payload:
     ```json
     {
       "error": "Payment Required / Term License Expired",
       "code": "LICENSE_EXPIRED",
       "message": "School term license has expired. Non-readonly operations are restricted.",
       "billing_url": "/billing",
       "tenant_id": "5b39bcb5-506a-456d-b7cb-91730b397cc1"
     }
     ```

### 4. Testing Emergency Access Overrides
- To test granting an emergency override to an expired tenant:
  1. Make a request from SuperAdmin UI or API:
     ```bash
     curl -X POST http://localhost:3002/api/tenants/5b39bcb5-506a-456d-b7cb-91730b397cc1/override \
       -H "X-SuperAdmin-Key: SA-MASTER-KEY-2026-X99" \
       -H "Content-Type: application/json" \
       -d '{"reason": "Ministry emergency extension", "extension_days": 14}'
     ```
  2. Verify that write access (`POST`, `PUT`) is immediately restored for the tenant.

### 5. Testing Multi-School Instance Spawning & Port Management
- **Instance Spawner**: Test spawning new school instances via the SuperAdmin UI or calling `POST /api/instances/spawn`.
- **Directory Isolation**: Each spawned school instance creates its own isolated database in `server/data/schools/<school_id>/school.db` and is assigned a unique server port (e.g. `3010`, `3011`).
- **Child Process Execution**: Ensure `node` is available in system PATH so `InstanceSpawner` can execute `node src/index.js` cleanly.

### 6. Cloud-First License Fetch (Supabase direct read)

The new flow is testable end-to-end on dev:

1. From the SuperAdmin UI, issue a fresh term license to a tenant.

2. Verify the response headers from any tenant API call:
   ```bash
   curl -i http://localhost:3001/api/students -H "Authorization: Bearer <jwt>" \
     -H "x-tenant-id: 5b39bcb5-506a-456d-b7cb-91730b397cc1" | grep -i x-license
   # X-License-Status: ACTIVE
   # X-License-Source: supabase
   ```

3. Disable network egress to Supabase and re-issue the same request. The middleware should fall back to local SQLite and set `X-License-Source: local` while still allowing the mutation (the previous license was write-backed on the first successful fetch).

4. The vitest suite `server/tests/supabase-license-fetch.test.js` exercises the middleware's Supabase path with a stubbed `SupabaseLicenseService` module. Run it with:
   ```bash
   cd server && npm test -- supabase-license-fetch
   ```

5. The RLS policy must be applied to the Supabase project itself for cloud reads to succeed. Apply with:
   ```bash
   psql "$SUPABASE_DB_URL" -f sql/20260729_tenant_licenses_rls_read_policy.sql
   ```
   or paste the SQL into the Supabase SQL editor. The Knex migration `2026072900000000_tenant_licenses_rls_read_policy.js` is a no-op on SQLite and runs the policy on Postgres.

### 7. Testing Sprint 4 Sync Engine Event Sourcing & Clock Normalization
- **Idempotent Push Testing**:
  - Pushing an edit with `event_id = "evt-12345"` inserts a record in `sync_events`.
  - Re-sending the exact same `event_id` returns `{ isDuplicate: true }` without running duplicate SQL writes.
- **Clock Drift Testing**:
  - `normalizeClientTimestamp()` calculates server-client time delta. Sending an offline record with a past `client_timestamp` normalizes the timestamp to server UTC time and rejects sequence numbers older than `max_seq`.

---

## 8. Port & Service Architecture Summary

| Service Component | Directory Path | Listening Port | Technology Stack |
| :--- | :--- | :--- | :--- |
| **School Tenant Server (Template)** | `server/` | `3001` | Node.js / Express / SQLite / PostgreSQL |
| **School Tenant Client (Template)** | `client/` | `3000` | React / Vite / Tailwind CSS |
| **SuperAdmin Server** | `superadmin/server/` | `3002` | Node.js / Express / Supabase SDK / HS256 Crypto JWT Engine |
| **SuperAdmin Portal** | `superadmin/client/` | `3003` | React / Vite / Tailwind CSS |
| **Spawned School Instances** | `server/data/schools/<id>` | (3010+, ports and registry persisted to `spawner-registry.db`) | Node.js / Express / Dynamic Port Allocation |
