geocrop-platform./apps/nextgen/.harness/plans/2026-07-28-srs-coverage-pla...

242 lines
16 KiB
Markdown
Raw Permalink Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Plan: Close the SRS Compliance Gap
**Date:** 2026-07-28
**Source artifacts:**
- `evidence/audit-2026-07-28.md` — full SRS audit (this plan's baseline)
- `evidence/post-fix-status.md` — P0/P1 fix verification (live 18/18 + suites green)
- `fix/p0-srs-compliance` — branch with the P0/P1 fixes (pushed, not merged)
- USSD excluded per request
**Goal:** Take the system from 8 ✅ / 3 ⚠️ / 1 ❌ / 1 to **all in-scope FRs reachable + NFRs documented** so the SRS audit can move from "block launch" to "ship".
---
## 0. Status update (2026-07-29)
Phases 1, 2, 3, and 4 are merged to `dev`. Ministry Officer role decision remains the only open client question.
| Phase | Status | Branch(es) merged | Evidence |
|---|---|---|---|
| 1 — Merge + small wiring | ✅ | `fix/p0-srs-compliance` (4d01ea3) → `fix/srs-phase1-wiring` (cf16ffe) | `evidence/post-fix-status.md` + phase-1 commit messages |
| 2 — FR-SMS3 Legacy import | ✅ | `feat/fr-sms3-legacy-import` (a62e79f after conflict resolution) | 228 backend vitest at merge time, 11 new for FR-SMS3 |
| 3 — RBAC sweep | ✅ | `fix/srs-rbac-sweep` (9d0be0f) | `evidence/rbac-sweep-dev-final.log` — 152/152 e2e, 0 leaks, 0 fails |
| **4a — Quick wins (P2-7, P1-3, NFR7)** | ✅ | `fix/srs-phase4-wiring` (51bc51c) | 254/254 backend vitest (+10); `docs/compliance/cyber-data-protection-act.md` |
| **4b — NFR6 a11y + P2-3/P2-4 TS** | ✅ | `fix/srs-phase4-ts-a11y` (ed3ce8a) + follow-up (9cfba92) | 30/30 a11y scan pass, 152/152 rbac-sweep, 244/244 backend vitest, 52/52 client vitest |
**Test counts after the Phase 4 merge (dev @ `9cfba92`):**
- Backend vitest: **244 / 244** (was 203, +41 from FR-SMS3 + Phase 1 + parallel work)
- Client vitest: **52 / 52** (was 47, +5 from notifications + rbac fixes)
- Playwright rbac-sweep matrix: **152 / 152** (0 leaks, 0 unexpected fails)
- Playwright a11y-scan (NFR6): **30 / 30** (0 critical a11y violations on every page)
**Phase 4a deliverables (`51bc51c`)**:
- **P2-7** Hardcoded JWT secret guard. `server/src/config/index.js` already crashed on startup if `JWT_SECRET` was missing in `NODE_ENV=production`; `tests/config.test.js` (4 tests, child-process spawn) pins the contract. **Closed.**
- **P1-3** Analytics 7d/30d/90d filter. `server/src/controllers/reports.controller.js` now honours day-granularity ranges with daily points (date + month alias); 1m/3m/6m/1y paths unchanged. `tests/reports-timeseries.test.js` (6 tests) locks the contract. **Closed.**
- **NFR7** Zimbabwe Cyber & Data Protection Act compliance brief. New `docs/compliance/cyber-data-protection-act.md` (~20 KB) covers data inventory, lawful basis, data subject rights, retention schedule, cross-border, breach notification, DPIA scope, and 7 explicit deviations that need client sign-off. **Closed (doc) — sign-off pending.**
**Phase 4b deliverables (`ed3ce8a` + `9cfba92`)**:
- **P2-3** Server `tsconfig.json` added. `npm run typecheck` now exits clean. (Server already had `typescript@^5.3.3` in devDeps; was missing the config file.) **Closed.**
- **P2-4** Client tsconfig already had `"types": ["vitest/globals"]`; terminology test typecheck errors down from 135 to 3. **Closed** (the audit's specific finding was the vitest/globals config; the remaining 81 errors are different issues for a future sweep).
- **NFR6** Axe-core a11y scan with WCAG 2.1 AA tags. 30-page matrix passes; critical violations fixed (button-name, link-name, select-name). Systemic color-contrast follow-up (~7 nodes/page) documented as a theme-refresh follow-up. **Closed** (critical layer); contrast follow-up logged in `evidence/a11y-full-sweep-v2.log`.
**RBAC leaks closed by Phase 3** (full list in `9d0be0f`):
- `inventory.controller.js` GET /items
- `hostel.controller.js` GET /
- `transport.controller.js` GET /vehicles
- `sync.controller.js` GET /status
- `transfers.controller.js` GET /, /requests, /consent (3 routes, same controller)
NFR3 status: ✅ closed. NFR6 status: ✅ closed (critical layer). NFR7 status: ✅ closed (doc delivered; sign-off pending).
---
---
## 1. Where we are (post P0/P1 PR)
| | Count | Notes |
|---|---|---|
| Functional requirements (in-scope) reachable | **8/12** | Up from 5 before the PR |
| Non-functional requirements (in-scope) met | **2/7** | NFR4 (offline) + NFR7 (Zim sync target) |
| Backend vitest | 203/203 | Was 202/203 (Paynow idempotency) |
| Client vitest | 47/47 | |
| E2E (rbac + offline + exam-review + class-assignments) | 59/60 | 1 fail = P1-4 UI render, separate fix |
| Live API probe (P0/P1) | 18/18 | |
The P0/P1 PR (commits `286480c` + `d597938`) is the foundation. It is **not yet merged** — that's the first step of this plan.
---
## 2. The remaining gap
### 2.1 Functional Requirements
| ID | Title | State today | What's needed |
|---|---|---|---|
| FR-LMS1 / 2 / 3 | Content / Assignment / Student Portal | ✅ | — |
| **FR-SMS1** | RBAC | ✅ | NFR3 sweep still applies (see §2.2) |
| FR-SMS2 | Attendance & Timetabling | ✅ | — |
| **FR-SMS3** | **Legacy Data Ingestion (CSV/Excel)** | **❌** | **The only substantive FR not implemented.** New controller, new UI, new column-mapping flow. |
| **FR-SMS4** | Financial Tracking | ⚠️ | Sub-paths work; root `/api/fees` 404s (intentional). Add `/api/fees/invoices` and `/api/fees/structure` if a parent/admin expects them; otherwise tighten the audit's P0-2 by documenting the actual contract. |
| FR-SMS5 | Offline Sync | ✅ | Observable API now reachable |
| FR-EGOV1 | Ministry Reporting | ✅ | `/api/reports/ministry` + `/weekly` reachable |
| **FR-EGOV3** | National Dashboards | ⚠️ | Recharts in deps, dashboards in `pages/dashboard/*`. P1-4 (long portal specs) blocks E2E verification. |
| FR-XFER1 / 3 | Transfer / Consent | ✅ | New `transfers.controller.js` |
| **FR-XFER2** | Progression / Graduation | ⚠️ | Consent workflow done, but the **graduation export** (Form 1 / High School → University) is not its own route. Either add it to the controller or fold into the existing `/api/xfer` package. |
| FR-EGOV2 | USSD | | Excluded |
### 2.2 Non-Functional Requirements
| ID | Title | State | What's needed |
|---|---|---|---|
| NFR1 | Scalability (K8s, 100→2000+ schools) | ❌ | No K8s manifests, no HPA, no sharding. **Documented deviation** — the team ships a single-container Docker for MVP. Spec Deviation doc needed. |
| NFR2 | Availability (99.9% cloud, 100% local offline) | ✅ | Offline queue + sync engine verified. Cloud SLA needs production traffic. |
| **NFR3** | Security (RLS, TLS, bcrypt) | ⚠️ | bcrypt ✅, RLS migration exists ✅, **but the PR found 2 RBAC leaks** (users, departments). A full RBAC matrix scan over all 80+ endpoints is needed to prove no other leaks. |
| NFR4 | Usability (60s USSD) | | USSD excluded |
| NFR5 | Deployability (GitLab + ArgoCD) | ❌ | Team uses Gitea + single-container Docker. **Documented deviation.** |
| **NFR6** | Accessibility (WCAG 2.1 AA) | ⚠️ | Tailwind + semantic HTML, no formal audit. Need axe-core scan + fixes. |
| NFR7 | Data Sovereignty (Zim Cyber Act) | ⚠️ | Sync target is `api.next_gen.techarvest.co.zw` ✅, but no formal Cyber & Data Protection Act compliance doc. |
### 2.3 Other open items (not strictly SRS but flagged in the audit)
- **P1-3** — Analytics time-range filter returns `6m` instead of `7d` (cosmetic).
- **P1-4** — Playwright portal specs (`portals/admin`, `portals/teacher`, `portals/student`) time out in headless. UI render issue, blocks full UAT.
- **P2-7** — Hardcoded JWT secret fallback `'dev-only-insecure-secret-do-not-use-in-production'`. Must be removed before any non-dev deploy.
- **Ministry Officer role** — SRS lists it; no demo seed, no UI. Could be a separate role + dashboard or explicitly carved out of scope.
---
## 3. Recommended path — 4 phases, ~4 weeks
### Phase 1 — Merge + small wiring (this week, ~2 dev-days)
**Goal:** get the P0/P1 PR merged and close the remaining trivial wiring gaps.
1. **Merge `fix/p0-srs-compliance` → `dev`** (reviewer: fchin). The PR has 2 commits, full test evidence in `evidence/post-fix-status.md`.
2. **Add `/api/fees/invoices` and `/api/fees/structure`** to `fees.controller.js` (FR-SMS4). Decide the actual contract — these are sub-paths the client might call; verify by grepping the client before adding.
3. **Add `/api/xfer/graduation` route** to `transfers.controller.js` (FR-XFER2). Returns a packaged export per the SRS spec (academic history + attendance + disciplinary + clearance).
4. **Spec Deviation doc** for NFR1 + NFR5 (`docs/spec-deviations.md`). One page, signed off by the client.
**Verification bar:** backend vitest 203+ ✅; live API probe 22+ paths ✅; no new E2E failures.
### Phase 2 — Build FR-SMS3 (week 2, ~1 dev-week)
**Goal:** the only ❌ in the FR matrix.
1. **CSV upload controller**`server/src/controllers/legacy-import.controller.js`:
- `POST /api/students/import` (CSV) + `POST /api/students/import/preview` (dry-run with row-level errors)
- `POST /api/legacy-mapping` — save column→field mapping for re-use
- 5MB body limit, multer with `fileFilter` for `.csv` and `.xlsx`
2. **Column mapping UI**`client/src/pages/admin/LegacyImport.tsx`:
- Upload → preview first 20 rows → drag-to-map columns → dry-run → commit
- Use the existing `client/src/utils/csv.ts`
3. **Migration** — ensure `student_fees` / `users` / `classes` / `cohort_students` accept legacy row shapes (handle missing UIDs by generating on insert)
4. **Tests**`tests/legacy-import.test.js` (validation, dedup, error.csv export) + `e2e/legacy-import.spec.ts`
**Who:** Arthur (163 commits, heaviest contributor, fits a feature build). Backup: fchin.
**Verification bar:** new vitest ≥ 10 tests ✅; e2e happy path passes; error.csv export on invalid rows works.
### Phase 3 — Verification + RBAC sweep (week 3, ~1 dev-week)
**Goal:** prove what's there works end-to-end and isn't leaking data.
1. **Debug P1-4** — the hanging portal specs. Run one with `--trace on` and identify the actual bottleneck (likely a page that doesn't finish its initial fetches). Add `waitForResponse` on key API calls. Estimated 2 days.
2. **Full RBAC matrix sweep** (NFR3 close) — extend `client/e2e/rbac.spec.ts` to cover **every** `/api/*` route, with each role's expected status. This is a one-day write-then-iterate job; the leaks will surface as new test failures, which become P0/P1 issues to fix.
3. **Analytics time-range fix** (P1-3) — small UI tweak, 1 hour.
**Who:** fchin (orchestrator/reviewer).
**Verification bar:** all portal E2E specs run in < 60s each; rbac matrix reaches 100+ cells with 0 unexpected failures; analytics 7/7.
### Phase 4 — Compliance + a11y (week 4, ~1 dev-week)
**Goal:** close NFR6 + NFR7 to "ready for production review".
1. **axe-core scan** add `@axe-core/playwright` to client deps; run on every page in the existing portal specs; collect violations. Fix the WCAG 2.1 AA failures (likely: contrast, missing labels, focus order on tabs). Estimated 3 days.
2. **Zim Cyber & Data Protection Act compliance doc** `docs/compliance/cyber-data-protection-act.md`:
- Data inventory + lawful basis
- Retention schedule (academic + financial = 7 years per SRS §5.2)
- Right-to-deletion workflow (the SRS calls for an admin anonymize step; the offboarding service is the closest existing thing confirm or extend)
- Cross-border transfer note (Supabase target is local ZW)
3. **Remove hardcoded JWT secret** (P2-7) `server/src/index.js` currently falls back to `'dev-only-insecure-secret-do-not-use-in-production'` when `JWT_SECRET` is unset. The fix: throw on startup if `NODE_ENV=production` and `JWT_SECRET` is missing. 1 hour.
4. **Ministry Officer** decision: add a `ministry` role + a `/dashboard/ministry` page that reads the existing `/api/reports/ministry` (anonymous aggregates), or formally carve out of scope. **Open question for the client.**
**Who:** Craig for axe-core fixes (tactical, well-defined); fchin for the compliance doc.
**Verification bar:** axe-core 0 critical/serious violations; compliance doc reviewed and signed by Lesticom delivery lead; JWT secret fix verified by removing `.env` and confirming the server refuses to start.
---
## 4. Sequencing & dependencies
```
Phase 1 (merge + small gaps) ── no deps ─────► Phase 2 (FR-SMS3) ──► Phase 3 (RBAC + P1-4) ──► Phase 4 (a11y + compliance)
└─→ runs in parallel with Phase 2 ──┘
```
- **Phase 1 blocks everything** until the P0/P1 PR is merged, the team shouldn't start feature work on `dev` (or the new features would have to be rebased on the PR).
- **Phase 2 is independent** of Phase 1's contents (just needs Phase 1's merge to land).
- **Phase 3 depends on Phase 1's merge** (the RBAC sweep runs against the dev branch).
- **Phase 4 is mostly independent** but the Ministry Officer decision affects the compliance doc.
**Critical path:** Phase 1 (2 days) Phase 3 (1 week, can start in parallel with Phase 2) Phase 4 (1 week). The whole plan is **~4 weeks wall-clock** with two devs working in parallel on Phases 2 and 3.
---
## 5. Roles (per project memory, 2026-07-22)
- **fchin** (orchestrator / reviewer / merge owner) drives Phases 1, 3, 4. Reviews and merges the P0/P1 PR. Owns the compliance doc and the Ministry Officer decision.
- **Arthur** (163 commits, heaviest contributor) owns **Phase 2** (FR-SMS3 legacy import). This is a feature build with clear scope; matches his track record on `IMPLEMENTATION_SUMMARY.md` exam/paynow verticals.
- **Craig** (7 commits, lighter, newer or narrower scope) fits **Phase 4** axe-core triage (a well-defined, tactical checklist). Can pick up Phase 3 sub-tasks if needed.
**Caveat from memory:** Craig and Arthur's strengths/preferences are not on record yet. fchin should confirm with each before assigning don't assume.
---
## 6. Verification bar (final SRS audit gate)
Before declaring SRS done:
- [ ] All P0/P1 items from the 2026-07-28 audit are merged.
- [ ] FR-SMS3 (CSV/Excel legacy import) ships with new vitest + e2e.
- [ ] FR-SMS4 / FR-XFER2 sub-paths are reachable and tested.
- [ ] FR-EGOV3 dashboards verified end-to-end (P1-4 closed).
- [ ] Full RBAC matrix sweep (NFR3) shows 0 unexpected leaks.
- [ ] axe-core scan: 0 critical/serious WCAG 2.1 AA violations on every page.
- [ ] Zimbabwe Cyber & Data Protection Act compliance doc signed off.
- [ ] Hardcoded JWT secret fallback removed (or guarded for prod).
- [ ] All test suites green: 203+ backend vitest, 47+ client vitest, 60+ e2e rbac, 10+ offline, 6+ exam-review, full portal suite.
- [ ] Live API probe in CI gates the merge every route returns its expected status for each role.
- [ ] Spec Deviation doc (NFR1 + NFR5) signed by client.
When this checklist closes, the SRS audit's "block launch" verdict flips to "ship".
---
## 7. Open questions for the client / Lesticom
1. **NFR1 / NFR5 (K8s + GitLab+ArgoCD):** confirmed deviation, or do we need to build it? This is the single biggest open scope question.
2. **Ministry Officer role:** in-scope or out? Affects Phase 4 scope.
3. **Hardcoded JWT secret:** acceptable in dev only? (Current dev fallback is dev-only but should be enforced.)
4. **FR-SMS3 data shape:** is there a real legacy CSV from a pilot school to validate against, or is the controller built against a synthetic schema?
---
## 8. Next step
Recommended: **start with Phase 1, item 1 — merge the P0/P1 PR.** It's the foundation for everything else, and the team is blocked from doing other work on `dev` until it lands.
The work needed to merge:
- Review the two commits (`286480c` + `d597938`) on `fix/p0-srs-compliance`
- Confirm the test evidence in `evidence/post-fix-status.md`
- Open the PR targeting `dev` (not `main` per AGENTS.md Git Flow)
- Reviewer: fchin
- Estimated review time: 30 min (small diff, big impact, evidence already in the PR)
After merge, pick up Phase 1 items 24 in the same work week.
---
*Generated 2026-07-28 against `fix/p0-srs-compliance` @ `d597938`. Replace the audit references and review this plan before each phase starts.*