256 lines
17 KiB
Markdown
256 lines
17 KiB
Markdown
# Phase 2 — Analytics Dashboard Rebuild
|
||
|
||
**Date:** 2026-07-22
|
||
**Owner:** Mavis (orchestrator) + backend-expert / frontend-expert / database-expert / tester reins
|
||
**Branch (target):** `feature/analytics-dashboard-2026-07-22` (worktree from `dev`, branched after Phase 1 lands)
|
||
**Status:** plan (not started)
|
||
**Assignee:** Arthur
|
||
**Companion plans:** [Phase 1 — auxiliary roles + cohorts + assignments](./2026-07-22-phase1-cohorts-assignments.md) · [Phase 3 — offboarding](./2026-07-22-phase3-offboarding.md) · [superseded 7-feature draft](./2026-07-22-cohorts-assignments-analytics-offboarding.md)
|
||
**Parallelizable with:** Phase 3 (zero file overlap; both can develop in parallel after Phase 1 lands)
|
||
|
||
## Problem
|
||
|
||
The `/reports` page is a single screen with one filter, not a real analytics dashboard. The principal (who shares the `/dashboard/admin` route with `school_admin`) needs to answer questions like:
|
||
|
||
- How many active enrollments do we have right now, and how does that compare to last term?
|
||
- What's the attendance rate for Form 4 IGCSE 2026 cohort over the last 30 days?
|
||
- Which class has the lowest pass rate this term?
|
||
- What's the fee collection rate per cohort?
|
||
- Where are the bottlenecks in leave / payroll approval?
|
||
|
||
The current page renders 4–5 Recharts widgets from `/api/reports/summary` but is effectively a static report — no time-range filter that's actually wired, no drilldowns, no role-scoped data, no export, and `principal` is not in the allowed-roles list in some `getRoutes()` arms of `App.tsx`. A real dashboard lets a principal open the page first thing in the morning and see the state of the school in one screen.
|
||
|
||
**What this is, in one line:** turn `/reports` into a real analytics dashboard with KPI tiles, time-series, drilldowns, role-scoped tabs, and CSV export.
|
||
|
||
## Scope (in / out)
|
||
|
||
**In this plan:**
|
||
- New `GET /api/reports/kpis`, `/timeseries`, `/cohorts`, `/export` endpoints in `server/src/controllers/reports.controller.js` (extend the existing controller, no new file)
|
||
- Full rewrite of `client/src/pages/admin/Reports.tsx` — header filters, KPI tile row, tabbed sections, drilldown modals, CSV export
|
||
- Add `principal` (and `bursar` / `hr` where appropriate) to the allowed roles for `/reports` in every `getRoutes()` arm of `App.tsx`
|
||
- Wire the `useEducationTerms()` locale helpers and the existing date helpers — no hardcoded strings
|
||
- Reuse Recharts, lucide, axios, AuditService — no new libraries
|
||
|
||
**Out of this plan (explicitly):**
|
||
- A second BI tool or external integration (Metabase, etc.)
|
||
- PDF export — CSV only this round
|
||
- Real-time streaming updates (websockets / SSE) — ship a refresh button + 5-minute in-memory cache
|
||
- Replacing the existing `/api/reports/summary` and `/api/reports/dashboard` endpoints — they stay; they're consumed elsewhere
|
||
- Tightening JWT secret, CORS, request-size limit — separate open follow-ups
|
||
- Auto-creating the missing demo `sysadmin@school.com` seed — carry-over from the 2026-07-19 portal-e2e plan, do it elsewhere if at all
|
||
|
||
## Approach
|
||
|
||
### Backend — extend `server/src/controllers/reports.controller.js`
|
||
|
||
The existing controller has `summary`, `dashboard`, `enrollments`, `financial`, `attendance`, `grades`, `grades/subject-performance`, `engagement`, `export/:type`. Keep all of them. Add four new endpoints and one new middleware-helper:
|
||
|
||
**`GET /api/reports/kpis?range=6m`**
|
||
|
||
Returns:
|
||
```js
|
||
{
|
||
enrollments: { total, active, newThisTerm, byProgramme: { zimsec: N, igcse: N, ... } },
|
||
attendance: { rate30d, rate7d, trend: 'up'|'down'|'flat' },
|
||
academics: { avgMark, passRate, topSubject, bottomSubject },
|
||
finance: { collected30d, outstanding, overdueCount }, // role-gated to finance roles
|
||
staff: { activeTeachers, onLeave, pendingApprovals } // role-gated to hr/principal
|
||
}
|
||
```
|
||
|
||
The query plan for each KPI is a single SQL statement with one index scan. If a query takes >500ms on the seed, add a covering index; don't introduce a cache this round.
|
||
|
||
**`GET /api/reports/timeseries?metric=enrollments|attendance|marks|attendance&range=6m&granularity=week|month`**
|
||
|
||
Returns `{ points: [{ date, value, secondary? }] }`. Granularity `week` returns ~26 points for 6m; `month` returns ~6. Use the existing `attendance` and `enrollments` tables — both have indexed `date` / `created_at` columns. Marks are bucketed by `graded_at`.
|
||
|
||
**`GET /api/reports/cohorts`**
|
||
|
||
Returns per-cohort performance:
|
||
```js
|
||
[
|
||
{ cohortId, name, programme, level, academicYear, studentCount,
|
||
avgMark, passRate, attendanceRate, feeCollectionRate }
|
||
]
|
||
```
|
||
|
||
This endpoint assumes `student_cohorts` + `cohort_students` exist (Phase 1). If Phase 1 is delayed, gate this endpoint behind a feature flag and ship it later — the rest of the dashboard works without it.
|
||
|
||
**`GET /api/reports/export?type=overview|academics|cohorts|finance&format=csv&range=6m`**
|
||
|
||
Server-side CSV streaming using the same queries as the JSON endpoints. The existing `GET /api/reports/export/:type` route already does CSV for one report type — generalize it to accept `type` as a query param, support the four new types, and stream the response (don't buffer).
|
||
|
||
**Auth / RBAC:**
|
||
|
||
- Read paths: `school_admin | systems_admin | principal | bursar | hr` (each role sees a different subset of tabs)
|
||
- Use the `hasRole` helper from Phase 1 if it has landed; otherwise use the existing `req.user.role` checks
|
||
- The role-scoped data filtering happens **server-side** in the controller, not the client. The client just receives what its role is allowed to see.
|
||
|
||
**Performance:**
|
||
|
||
- Each KPI = one SQL statement, one index scan. Verify with `EXPLAIN QUERY PLAN` on the seed.
|
||
- Time-series = one SQL statement with `GROUP BY date(?, 'week'|'month')`.
|
||
- Per-cohort table = one SQL with `LEFT JOIN student_cohorts + cohort_students + attendance + grades + invoices`. If the JOIN is slow at the school's data size, denormalize later — but ship the JOIN first.
|
||
- No new indexes this round. Add them only if a query is actually slow.
|
||
|
||
### Frontend — full rewrite of `client/src/pages/admin/Reports.tsx`
|
||
|
||
The existing page already uses Recharts and lucide. Keep the import set; replace the layout.
|
||
|
||
**Layout:**
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────────┐
|
||
│ Header: time-range picker | cohort filter | programme | │
|
||
│ class filter [Refresh] [Export ▼] │
|
||
├─────────────────────────────────────────────────────────────┤
|
||
│ KPI tile row (5 tiles) with sparklines │
|
||
│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ │
|
||
│ │Enrol│ │Atten│ │Acad │ │Finan│ │Staff│ │
|
||
│ └─────┘ └─────┘ └─────┘ └─────┘ └─────┘ │
|
||
├─────────────────────────────────────────────────────────────┤
|
||
│ Tabs: [Overview] [Academics] [Cohorts] [Finance] [Staff] │
|
||
├─────────────────────────────────────────────────────────────┤
|
||
│ <Active tab content> │
|
||
│ │
|
||
└─────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
**Tab 1 — Overview** (default)
|
||
- Enrollment area chart (line + area fill) from `/api/reports/timeseries?metric=enrollments`
|
||
- Attendance line from `/api/reports/timeseries?metric=attendance`
|
||
- Fee collection stacked bar (only visible to finance roles; backend returns 403 for others)
|
||
|
||
**Tab 2 — Academics**
|
||
- Grade distribution bar chart (A/B/C/D/F) — data already in `/api/reports/grades`
|
||
- Subject performance horizontal bar — data already in `/api/reports/grades/subject-performance`
|
||
- Top/bottom classes table (computed server-side from the same query)
|
||
- Click a class row → drilldown modal with that class's students ranked
|
||
|
||
**Tab 3 — Cohorts** (requires Phase 1)
|
||
- Table of cohorts from `/api/reports/cohorts` with sortable columns
|
||
- Click a row → drilldown modal with that cohort's students ranked
|
||
- If `student_cohorts` doesn't exist (Phase 1 hasn't shipped), this tab is hidden, not broken
|
||
|
||
**Tab 4 — Finance** (only visible to `bursar | school_admin | systems_admin | principal`)
|
||
- Reuse the existing `/api/reports/financial` endpoint
|
||
- Add a "fee collection rate per cohort" stacked bar (uses `/api/reports/cohorts` data)
|
||
|
||
**Tab 5 — Staff** (only visible to `hr | school_admin | systems_admin | principal`)
|
||
- Teachers on leave (count + list)
|
||
- Pending leave / payroll approvals (count + drilldown)
|
||
- Staff count by department (pie chart)
|
||
|
||
**Filters (header):**
|
||
- Time range: 7d / 30d / 90d / 6m / YTD / custom (date range picker)
|
||
- Cohort: dropdown populated from `/api/cohorts?is_active=1`
|
||
- Programme: ZIMSEC / IGCSE / AS / A2 / primary / other (only enabled when cohorts are present)
|
||
- Class: dropdown populated from `/api/classes`
|
||
|
||
**Export button:** top-right, dropdown of `Overview / Academics / Cohorts / Finance`. Each option hits `/api/reports/export?type=...&format=csv&range=...&cohort=...`. Browser downloads the file.
|
||
|
||
**Routing fix (in `client/src/App.tsx`):**
|
||
- In every `getRoutes()` arm that lists `/reports` as a route, add `principal` to the `roles: []` array. Also add `bursar` and `hr` where the role is allowed to see the relevant tab.
|
||
- Verify with `git grep -n "path: '/reports'" client/src/App.tsx` before merging.
|
||
|
||
**Existing components to keep:**
|
||
- `CustomTooltip` (already in Reports.tsx) — reuse, no change
|
||
- The 4–5 existing chart widgets — delete; they're replaced by the new tabs
|
||
|
||
**New components:**
|
||
- `KpiTile.tsx` — small tile with a value, a label, a sparkline, a trend arrow
|
||
- `DrilldownModal.tsx` — generic modal that takes a title + a child for the body
|
||
- `FilterBar.tsx` — the header filter row, with the four filter controls + refresh + export
|
||
|
||
### Sync / offline
|
||
|
||
This plan adds **no new tables**. Zero sync impact. All queries are read-only against existing tables. The dashboard works offline if the data was already synced (the existing data layer handles that), but we don't add new offline-first behavior for it this round.
|
||
|
||
## Files to add / change
|
||
|
||
### Backend (modify only)
|
||
- `server/src/controllers/reports.controller.js` — add `kpis`, `timeseries`, `cohorts`, generalize `export`; add a `roleScopedKpis(user)` helper that filters the response by role
|
||
- `server/src/index.js` — no change (the new routes are added inside the existing controller)
|
||
|
||
### Frontend (new)
|
||
- `client/src/components/KpiTile.tsx`
|
||
- `client/src/components/DrilldownModal.tsx`
|
||
- `client/src/components/FilterBar.tsx`
|
||
|
||
### Frontend (modify)
|
||
- `client/src/pages/admin/Reports.tsx` — full rewrite (existing component, same path, mostly new JSX)
|
||
- `client/src/App.tsx` — add `principal`, `bursar`, `hr` to the `/reports` allowed roles in every `getRoutes()` arm
|
||
- `client/src/components/Nav.tsx` — no change (the Reports nav entry already exists for the relevant roles)
|
||
|
||
### Docs / changelogs
|
||
- `.harness/changelogs/2026-07-22-analytics-dashboard.md`
|
||
- `.harness/docs/conventions.md` — only if a new analytics pattern is introduced (likely not)
|
||
|
||
## Constraints / decisions baked in
|
||
|
||
- **Reuse, don't replace.** Keep the existing `/api/reports/summary` and `/api/reports/dashboard` endpoints — other pages consume them. Only add the four new endpoints.
|
||
- **Role-scoped data, server-side.** The backend filters by role; the client just renders what it gets. A 403 from a role-gated section is silent (the tab doesn't render), not a toast.
|
||
- **Cohorts tab is conditional.** If Phase 1 hasn't shipped when Phase 2 merges, the Cohorts tab is hidden, the rest of the dashboard works. Use a feature flag if needed; we don't need a real one if we're disciplined about merging order.
|
||
- **No new libraries.** Recharts, lucide, axios are all already in the bundle.
|
||
- **No real-time updates.** Refresh button + 5-minute in-memory cache (useState + setInterval, not Redux or any new state lib). On `?range=` change, refetch.
|
||
- **No new tables, no sync impact.** Read-only against existing schema.
|
||
- **CSV only, no PDF.** PDF export is a follow-up.
|
||
- **The "73 TypeScript warnings" issue raised in the earlier critique isn't a blocker** for this plan — we're not adding enough new TS to make the warnings materially worse, and the project compiles with `strict: false` already.
|
||
- **No principal-specific portal.** Principal shares the school_admin portal with role-scoped data, per existing convention.
|
||
|
||
## Verification (acceptance bar)
|
||
|
||
This plan is "done" when **all** of the following pass:
|
||
|
||
1. **Migrations are not needed** — this plan adds no new tables. (Confirm: `git diff --stat` on `server/src/database/` is empty for this PR.)
|
||
2. **Endpoints work:**
|
||
- `GET /api/reports/kpis?range=6m` returns the documented shape with non-zero values from the seed.
|
||
- `GET /api/reports/timeseries?metric=...&range=...&granularity=...` returns 6+ points for 6m, 26+ points for 6m+week.
|
||
- `GET /api/reports/cohorts` returns one row per cohort (or `[]` if Phase 1 hasn't shipped — the page handles both).
|
||
- `GET /api/reports/export?type=...&format=csv` downloads a non-empty CSV.
|
||
3. **Dashboard renders for the right roles:**
|
||
- `school_admin` sees all 5 tabs.
|
||
- `principal` sees all 5 tabs (after the routing fix in `App.tsx`).
|
||
- `bursar` sees Overview, Academics, Cohorts, Finance (no Staff).
|
||
- `hr` sees Overview, Academics, Cohorts, Staff (no Finance).
|
||
- `student`, `parent`, `teacher` get a 403 from the route, not a half-rendered page.
|
||
4. **Time-range filter actually changes the data** — set range to 7d, then 6m, the values change.
|
||
5. **Cohort tab drilldown** — click a cohort row, the drilldown modal shows the cohort's students ranked by some metric. The modal has a close button + Esc-to-close.
|
||
6. **CSV export downloads a non-empty file** for each of Overview / Academics / Cohorts / Finance.
|
||
7. **Performance** — every KPI query and the time-series query is <500ms on the seed. Use `EXPLAIN QUERY PLAN` to verify. If anything is slow, add a covering index and document the choice.
|
||
8. **E2E (Playwright):** one spec loads `/reports` as `school_admin`, verifies all 5 tabs render, switches time range, drills into a cohort, exports a CSV. Existing portal-smoke tests still pass.
|
||
9. **No regression:** existing admin/principal dashboards, the exam-review module, and the finance module all still load and respond.
|
||
|
||
## Execution order (this plan only)
|
||
|
||
```
|
||
PR 1: Backend endpoints (kpis, timeseries, cohorts, export)
|
||
PR 2: Frontend rewrite (Reports.tsx + KpiTile + DrilldownModal + FilterBar)
|
||
PR 3: Routing fix (App.tsx — add principal/bursar/hr to /reports) + E2E spec
|
||
```
|
||
|
||
PR 1 and PR 2 can be developed in parallel (different files). PR 3 is small and merges last because it depends on both the new endpoints and the new frontend.
|
||
|
||
If Phase 1 hasn't landed yet, PR 1's `/cohorts` endpoint is added with a `featureFlagCohorts` query param (default `false`) and the Cohorts tab in PR 2 is hidden until the flag is on. The rest of the dashboard ships.
|
||
|
||
## Out of scope (explicitly)
|
||
|
||
- A second BI tool or external integration
|
||
- PDF export this round
|
||
- Real-time streaming updates
|
||
- Replacing the existing `/api/reports/summary` and `/api/reports/dashboard` endpoints
|
||
- Auto-creating the missing demo `sysadmin@school.com` seed
|
||
- Migrating the existing `adminOrTeacher` middleware to the `hasRole` helper from Phase 1 (the migration sweep is a separate follow-up)
|
||
- Tightening JWT secret, CORS, request-size limit — separate open follow-ups
|
||
|
||
## Parallelizability with Phase 3
|
||
|
||
This plan and Phase 3 (offboarding) share **zero files**:
|
||
|
||
- Different controllers (`reports.controller.js` vs `offboarding.controller.js`)
|
||
- Different frontend pages (`Reports.tsx` vs `Offboarding.tsx` / `OffboardingWizard.tsx` / `Alumni.tsx`)
|
||
- Different tables (no new tables here, vs `offboarding_records` + `offboarding_actions` + `users.archive_status` ALTER in Phase 3)
|
||
- Different sync impact (none here, vs 2 new tables in Phase 3)
|
||
|
||
After Phase 1 lands, a dev can pick up Phase 2 in one worktree and another dev can pick up Phase 3 in another. The only coordination is the merge order: either order is fine, since the two PRs don't touch the same code.
|