# Phase 2 — Analytics Dashboard Rebuild

**Date:** 2026-07-22
**Owner:** Mavis (orchestrator) + backend-expert / frontend-expert / database-expert / tester reins
**Branch (target):** `feature/analytics-dashboard-2026-07-22` (worktree from `dev`, branched after Phase 1 lands)
**Status:** plan (not started)
**Assignee:** Arthur
**Companion plans:** [Phase 1 — auxiliary roles + cohorts + assignments](./2026-07-22-phase1-cohorts-assignments.md) · [Phase 3 — offboarding](./2026-07-22-phase3-offboarding.md) · [superseded 7-feature draft](./2026-07-22-cohorts-assignments-analytics-offboarding.md)
**Parallelizable with:** Phase 3 (zero file overlap; both can develop in parallel after Phase 1 lands)

## Problem

The `/reports` page is a single screen with one filter, not a real analytics dashboard. The principal (who shares the `/dashboard/admin` route with `school_admin`) needs to answer questions like:

- How many active enrollments do we have right now, and how does that compare to last term?
- What's the attendance rate for Form 4 IGCSE 2026 cohort over the last 30 days?
- Which class has the lowest pass rate this term?
- What's the fee collection rate per cohort?
- Where are the bottlenecks in leave / payroll approval?

The current page renders 4–5 Recharts widgets from `/api/reports/summary` but is effectively a static report — no time-range filter that's actually wired, no drilldowns, no role-scoped data, no export, and `principal` is not in the allowed-roles list in some `getRoutes()` arms of `App.tsx`. A real dashboard lets a principal open the page first thing in the morning and see the state of the school in one screen.

**What this is, in one line:** turn `/reports` into a real analytics dashboard with KPI tiles, time-series, drilldowns, role-scoped tabs, and CSV export.

## Scope (in / out)

**In this plan:**
- New `GET /api/reports/kpis`, `/timeseries`, `/cohorts`, `/export` endpoints in `server/src/controllers/reports.controller.js` (extend the existing controller, no new file)
- Full rewrite of `client/src/pages/admin/Reports.tsx` — header filters, KPI tile row, tabbed sections, drilldown modals, CSV export
- Add `principal` (and `bursar` / `hr` where appropriate) to the allowed roles for `/reports` in every `getRoutes()` arm of `App.tsx`
- Wire the `useEducationTerms()` locale helpers and the existing date helpers — no hardcoded strings
- Reuse Recharts, lucide, axios, AuditService — no new libraries

**Out of this plan (explicitly):**
- A second BI tool or external integration (Metabase, etc.)
- PDF export — CSV only this round
- Real-time streaming updates (websockets / SSE) — ship a refresh button + 5-minute in-memory cache
- Replacing the existing `/api/reports/summary` and `/api/reports/dashboard` endpoints — they stay; they're consumed elsewhere
- Tightening JWT secret, CORS, request-size limit — separate open follow-ups
- Auto-creating the missing demo `sysadmin@school.com` seed — carry-over from the 2026-07-19 portal-e2e plan, do it elsewhere if at all

## Approach

### Backend — extend `server/src/controllers/reports.controller.js`

The existing controller has `summary`, `dashboard`, `enrollments`, `financial`, `attendance`, `grades`, `grades/subject-performance`, `engagement`, `export/:type`. Keep all of them. Add four new endpoints and one new middleware-helper:

**`GET /api/reports/kpis?range=6m`**

Returns:
```js
{
  enrollments: { total, active, newThisTerm, byProgramme: { zimsec: N, igcse: N, ... } },
  attendance: { rate30d, rate7d, trend: 'up'|'down'|'flat' },
  academics:  { avgMark, passRate, topSubject, bottomSubject },
  finance:    { collected30d, outstanding, overdueCount },   // role-gated to finance roles
  staff:      { activeTeachers, onLeave, pendingApprovals }   // role-gated to hr/principal
}
```

The query plan for each KPI is a single SQL statement with one index scan. If a query takes >500ms on the seed, add a covering index; don't introduce a cache this round.

**`GET /api/reports/timeseries?metric=enrollments|attendance|marks|attendance&range=6m&granularity=week|month`**

Returns `{ points: [{ date, value, secondary? }] }`. Granularity `week` returns ~26 points for 6m; `month` returns ~6. Use the existing `attendance` and `enrollments` tables — both have indexed `date` / `created_at` columns. Marks are bucketed by `graded_at`.

**`GET /api/reports/cohorts`**

Returns per-cohort performance:
```js
[
  { cohortId, name, programme, level, academicYear, studentCount,
    avgMark, passRate, attendanceRate, feeCollectionRate }
]
```

This endpoint assumes `student_cohorts` + `cohort_students` exist (Phase 1). If Phase 1 is delayed, gate this endpoint behind a feature flag and ship it later — the rest of the dashboard works without it.

**`GET /api/reports/export?type=overview|academics|cohorts|finance&format=csv&range=6m`**

Server-side CSV streaming using the same queries as the JSON endpoints. The existing `GET /api/reports/export/:type` route already does CSV for one report type — generalize it to accept `type` as a query param, support the four new types, and stream the response (don't buffer).

**Auth / RBAC:**

- Read paths: `school_admin | systems_admin | principal | bursar | hr` (each role sees a different subset of tabs)
- Use the `hasRole` helper from Phase 1 if it has landed; otherwise use the existing `req.user.role` checks
- The role-scoped data filtering happens **server-side** in the controller, not the client. The client just receives what its role is allowed to see.

**Performance:**

- Each KPI = one SQL statement, one index scan. Verify with `EXPLAIN QUERY PLAN` on the seed.
- Time-series = one SQL statement with `GROUP BY date(?, 'week'|'month')`.
- Per-cohort table = one SQL with `LEFT JOIN student_cohorts + cohort_students + attendance + grades + invoices`. If the JOIN is slow at the school's data size, denormalize later — but ship the JOIN first.
- No new indexes this round. Add them only if a query is actually slow.

### Frontend — full rewrite of `client/src/pages/admin/Reports.tsx`

The existing page already uses Recharts and lucide. Keep the import set; replace the layout.

**Layout:**

```
┌─────────────────────────────────────────────────────────────┐
│ Header: time-range picker | cohort filter | programme |     │
│         class filter              [Refresh] [Export ▼]      │
├─────────────────────────────────────────────────────────────┤
│ KPI tile row (5 tiles) with sparklines                      │
│ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐                    │
│ │Enrol│ │Atten│ │Acad │ │Finan│ │Staff│                    │
│ └─────┘ └─────┘ └─────┘ └─────┘ └─────┘                    │
├─────────────────────────────────────────────────────────────┤
│ Tabs: [Overview] [Academics] [Cohorts] [Finance] [Staff]   │
├─────────────────────────────────────────────────────────────┤
│ <Active tab content>                                        │
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

**Tab 1 — Overview** (default)
- Enrollment area chart (line + area fill) from `/api/reports/timeseries?metric=enrollments`
- Attendance line from `/api/reports/timeseries?metric=attendance`
- Fee collection stacked bar (only visible to finance roles; backend returns 403 for others)

**Tab 2 — Academics**
- Grade distribution bar chart (A/B/C/D/F) — data already in `/api/reports/grades`
- Subject performance horizontal bar — data already in `/api/reports/grades/subject-performance`
- Top/bottom classes table (computed server-side from the same query)
- Click a class row → drilldown modal with that class's students ranked

**Tab 3 — Cohorts** (requires Phase 1)
- Table of cohorts from `/api/reports/cohorts` with sortable columns
- Click a row → drilldown modal with that cohort's students ranked
- If `student_cohorts` doesn't exist (Phase 1 hasn't shipped), this tab is hidden, not broken

**Tab 4 — Finance** (only visible to `bursar | school_admin | systems_admin | principal`)
- Reuse the existing `/api/reports/financial` endpoint
- Add a "fee collection rate per cohort" stacked bar (uses `/api/reports/cohorts` data)

**Tab 5 — Staff** (only visible to `hr | school_admin | systems_admin | principal`)
- Teachers on leave (count + list)
- Pending leave / payroll approvals (count + drilldown)
- Staff count by department (pie chart)

**Filters (header):**
- Time range: 7d / 30d / 90d / 6m / YTD / custom (date range picker)
- Cohort: dropdown populated from `/api/cohorts?is_active=1`
- Programme: ZIMSEC / IGCSE / AS / A2 / primary / other (only enabled when cohorts are present)
- Class: dropdown populated from `/api/classes`

**Export button:** top-right, dropdown of `Overview / Academics / Cohorts / Finance`. Each option hits `/api/reports/export?type=...&format=csv&range=...&cohort=...`. Browser downloads the file.

**Routing fix (in `client/src/App.tsx`):**
- In every `getRoutes()` arm that lists `/reports` as a route, add `principal` to the `roles: []` array. Also add `bursar` and `hr` where the role is allowed to see the relevant tab.
- Verify with `git grep -n "path: '/reports'" client/src/App.tsx` before merging.

**Existing components to keep:**
- `CustomTooltip` (already in Reports.tsx) — reuse, no change
- The 4–5 existing chart widgets — delete; they're replaced by the new tabs

**New components:**
- `KpiTile.tsx` — small tile with a value, a label, a sparkline, a trend arrow
- `DrilldownModal.tsx` — generic modal that takes a title + a child for the body
- `FilterBar.tsx` — the header filter row, with the four filter controls + refresh + export

### Sync / offline

This plan adds **no new tables**. Zero sync impact. All queries are read-only against existing tables. The dashboard works offline if the data was already synced (the existing data layer handles that), but we don't add new offline-first behavior for it this round.

## Files to add / change

### Backend (modify only)
- `server/src/controllers/reports.controller.js` — add `kpis`, `timeseries`, `cohorts`, generalize `export`; add a `roleScopedKpis(user)` helper that filters the response by role
- `server/src/index.js` — no change (the new routes are added inside the existing controller)

### Frontend (new)
- `client/src/components/KpiTile.tsx`
- `client/src/components/DrilldownModal.tsx`
- `client/src/components/FilterBar.tsx`

### Frontend (modify)
- `client/src/pages/admin/Reports.tsx` — full rewrite (existing component, same path, mostly new JSX)
- `client/src/App.tsx` — add `principal`, `bursar`, `hr` to the `/reports` allowed roles in every `getRoutes()` arm
- `client/src/components/Nav.tsx` — no change (the Reports nav entry already exists for the relevant roles)

### Docs / changelogs
- `.harness/changelogs/2026-07-22-analytics-dashboard.md`
- `.harness/docs/conventions.md` — only if a new analytics pattern is introduced (likely not)

## Constraints / decisions baked in

- **Reuse, don't replace.** Keep the existing `/api/reports/summary` and `/api/reports/dashboard` endpoints — other pages consume them. Only add the four new endpoints.
- **Role-scoped data, server-side.** The backend filters by role; the client just renders what it gets. A 403 from a role-gated section is silent (the tab doesn't render), not a toast.
- **Cohorts tab is conditional.** If Phase 1 hasn't shipped when Phase 2 merges, the Cohorts tab is hidden, the rest of the dashboard works. Use a feature flag if needed; we don't need a real one if we're disciplined about merging order.
- **No new libraries.** Recharts, lucide, axios are all already in the bundle.
- **No real-time updates.** Refresh button + 5-minute in-memory cache (useState + setInterval, not Redux or any new state lib). On `?range=` change, refetch.
- **No new tables, no sync impact.** Read-only against existing schema.
- **CSV only, no PDF.** PDF export is a follow-up.
- **The "73 TypeScript warnings" issue raised in the earlier critique isn't a blocker** for this plan — we're not adding enough new TS to make the warnings materially worse, and the project compiles with `strict: false` already.
- **No principal-specific portal.** Principal shares the school_admin portal with role-scoped data, per existing convention.

## Verification (acceptance bar)

This plan is "done" when **all** of the following pass:

1. **Migrations are not needed** — this plan adds no new tables. (Confirm: `git diff --stat` on `server/src/database/` is empty for this PR.)
2. **Endpoints work:**
   - `GET /api/reports/kpis?range=6m` returns the documented shape with non-zero values from the seed.
   - `GET /api/reports/timeseries?metric=...&range=...&granularity=...` returns 6+ points for 6m, 26+ points for 6m+week.
   - `GET /api/reports/cohorts` returns one row per cohort (or `[]` if Phase 1 hasn't shipped — the page handles both).
   - `GET /api/reports/export?type=...&format=csv` downloads a non-empty CSV.
3. **Dashboard renders for the right roles:**
   - `school_admin` sees all 5 tabs.
   - `principal` sees all 5 tabs (after the routing fix in `App.tsx`).
   - `bursar` sees Overview, Academics, Cohorts, Finance (no Staff).
   - `hr` sees Overview, Academics, Cohorts, Staff (no Finance).
   - `student`, `parent`, `teacher` get a 403 from the route, not a half-rendered page.
4. **Time-range filter actually changes the data** — set range to 7d, then 6m, the values change.
5. **Cohort tab drilldown** — click a cohort row, the drilldown modal shows the cohort's students ranked by some metric. The modal has a close button + Esc-to-close.
6. **CSV export downloads a non-empty file** for each of Overview / Academics / Cohorts / Finance.
7. **Performance** — every KPI query and the time-series query is <500ms on the seed. Use `EXPLAIN QUERY PLAN` to verify. If anything is slow, add a covering index and document the choice.
8. **E2E (Playwright):** one spec loads `/reports` as `school_admin`, verifies all 5 tabs render, switches time range, drills into a cohort, exports a CSV. Existing portal-smoke tests still pass.
9. **No regression:** existing admin/principal dashboards, the exam-review module, and the finance module all still load and respond.

## Execution order (this plan only)

```
PR 1: Backend endpoints (kpis, timeseries, cohorts, export)
PR 2: Frontend rewrite (Reports.tsx + KpiTile + DrilldownModal + FilterBar)
PR 3: Routing fix (App.tsx — add principal/bursar/hr to /reports) + E2E spec
```

PR 1 and PR 2 can be developed in parallel (different files). PR 3 is small and merges last because it depends on both the new endpoints and the new frontend.

If Phase 1 hasn't landed yet, PR 1's `/cohorts` endpoint is added with a `featureFlagCohorts` query param (default `false`) and the Cohorts tab in PR 2 is hidden until the flag is on. The rest of the dashboard ships.

## Out of scope (explicitly)

- A second BI tool or external integration
- PDF export this round
- Real-time streaming updates
- Replacing the existing `/api/reports/summary` and `/api/reports/dashboard` endpoints
- Auto-creating the missing demo `sysadmin@school.com` seed
- Migrating the existing `adminOrTeacher` middleware to the `hasRole` helper from Phase 1 (the migration sweep is a separate follow-up)
- Tightening JWT secret, CORS, request-size limit — separate open follow-ups

## Parallelizability with Phase 3

This plan and Phase 3 (offboarding) share **zero files**:

- Different controllers (`reports.controller.js` vs `offboarding.controller.js`)
- Different frontend pages (`Reports.tsx` vs `Offboarding.tsx` / `OffboardingWizard.tsx` / `Alumni.tsx`)
- Different tables (no new tables here, vs `offboarding_records` + `offboarding_actions` + `users.archive_status` ALTER in Phase 3)
- Different sync impact (none here, vs 2 new tables in Phase 3)

After Phase 1 lands, a dev can pick up Phase 2 in one worktree and another dev can pick up Phase 3 in another. The only coordination is the merge order: either order is fine, since the two PRs don't touch the same code.