feat: reporting.fn_inc_dashboard — INC operations dashboard read-API (migration 09)

One parameterized function returns {window, open GeoJSON, closed GeoJSON, metrics,
freshness} for the FleetOps live INC map:
- open  = all is_actionable tickets (live), filtered by cluster/status, with
  sla_state/hours_open (from tickets.inc_open_sla)
- closed= closed_at within the selected window (EAT calendar today/week/month or
  custom [from,to)), filtered by cluster/status
- metrics= open/closed counts, SLA split (open derived, closed source), by status/
  cluster, closure rate + daily series, avg mttr (minutes)
Filters combine with AND; grants to dashboard_ro/grafana_ro. Verified live
(today/month/cluster/status/custom; last-7d closed=913 matches raw).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
david kiania 2026-06-16 01:10:18 +03:00
parent f2408f113e
commit 752ac9e418
3 changed files with 171 additions and 1 deletions

View file

@ -19,6 +19,7 @@ Field-ops **INC ticket** ingestion, geocoding, and read-schema that powers the
| `migrations/06_inc_mttr_minutes.sql` | `mttr` generated column → integer **minutes** (source is decimal hours); drops the constant `is_alarm`/`is_auto_created`/`is_auto_closed` columns (kept in `raw`). `is_actionable` retained | | `migrations/06_inc_mttr_minutes.sql` | `mttr` generated column → integer **minutes** (source is decimal hours); drops the constant `is_alarm`/`is_auto_created`/`is_auto_closed` columns (kept in `raw`). `is_actionable` retained |
| `migrations/07_inc_drop_service_type.sql` | Drops the constant `service_type` column (always `inc`; kept in `raw`) | | `migrations/07_inc_drop_service_type.sql` | Drops the constant `service_type` column (always `inc`; kept in `raw`) |
| `migrations/08_inc_open_sla_view.sql` | `tickets.inc_open_sla` view — open (`is_actionable`) tickets with **derived SLA** (`hours_open`, `sla_state` vs 48h; clock = `created_at_service``first_seen_at`), plus team/cluster/`geog` for dispatch | | `migrations/08_inc_open_sla_view.sql` | `tickets.inc_open_sla` view — open (`is_actionable`) tickets with **derived SLA** (`hours_open`, `sla_state` vs 48h; clock = `created_at_service``first_seen_at`), plus team/cluster/`geog` for dispatch |
| `migrations/09_inc_dashboard_fn.sql` | `reporting.fn_inc_dashboard(cluster, status, window, from, to)` — one JSON payload (`window` / `open` GeoJSON / `closed` GeoJSON / `metrics` / `freshness`) powering the FleetOps live INC map. Open=live, closed=windowed (EAT calendar / custom); filters AND |
| `import_tickets.py` | Ingests the **newest INC CSV** from the rustfs `tickets` bucket (`automations/inc/<EAT-timestamp>.csv`) and upserts on `ticket_id`; geocodes clusters + INC locations | | `import_tickets.py` | Ingests the **newest INC CSV** from the rustfs `tickets` bucket (`automations/inc/<EAT-timestamp>.csv`) and upserts on `ticket_id`; geocodes clusters + INC locations |
| `run_migrations.py` | Applies `migrations/*.sql` in order (ledger: `tickets.schema_migrations`) | | `run_migrations.py` | Applies `migrations/*.sql` in order (ledger: `tickets.schema_migrations`) |
| `shared.py` | Minimal DB/logging helpers (self-contained — no tracksolid dependency) | | `shared.py` | Minimal DB/logging helpers (self-contained — no tracksolid dependency) |

View file

@ -34,7 +34,7 @@ What is actually built and deployed, as of the Phase-1 completion. Companion to
| 06_inc_mttr_minutes | `mttr` → integer **minutes**; drop constant `is_alarm`/`is_auto_created`/`is_auto_closed` | | 06_inc_mttr_minutes | `mttr` → integer **minutes**; drop constant `is_alarm`/`is_auto_created`/`is_auto_closed` |
| 07_inc_drop_service_type | drop constant `service_type` | | 07_inc_drop_service_type | drop constant `service_type` |
| 08_inc_open_sla_view | `tickets.inc_open_sla` view (open tickets + derived SLA) | | 08_inc_open_sla_view | `tickets.inc_open_sla` view (open tickets + derived SLA) |
| 09_inc_dashboard_fn | *(planned)* `reporting.fn_inc_dashboard` — see `docs/phase-2-dashboard.md` | | 09_inc_dashboard_fn | **built**`reporting.fn_inc_dashboard(cluster, status, window, from, to)`: one JSON payload (open GeoJSON + windowed closed GeoJSON + metrics + freshness) for the FleetOps live INC map. See `docs/phase-2-dashboard.md` |
`tickets.inc` columns: `ticket_id` (PK), `raw` (jsonb, source of truth), `tickets.inc` columns: `ticket_id` (PK), `raw` (jsonb, source of truth),
`normalized_status`/`raw_status`, `bucket`, `is_actionable`, `cluster`/`region`/ `normalized_status`/`raw_status`, `bucket`, `is_actionable`, `cluster`/`region`/

View file

@ -0,0 +1,169 @@
-- 09_inc_dashboard_fn.sql — fleettickets · INC operations dashboard read-API
-- ─────────────────────────────────────────────────────────────────────────────
-- reporting.fn_inc_dashboard powers the FleetOps live INC map (served by
-- dashboard_api). One call returns a single JSON payload:
-- { window, open: GeoJSON, closed: GeoJSON, metrics, freshness }
--
-- open = ALL currently-open (is_actionable) tickets matching cluster/status
-- (live; NOT time-filtered) — from tickets.inc_open_sla (sla_state,
-- hours_open).
-- closed = closed tickets whose closed_at falls in the selected window,
-- matching cluster/status (the timeline overlay).
-- metrics = ticket metrics over the selection (open/closed counts, SLA split,
-- by status/cluster, closure rate + series, avg mttr minutes).
--
-- Window = calendar EAT (today / ISO-week / month) unless a custom [from,to) is
-- given. Filters (cluster, status, time) combine with AND, each optional.
-- Only rows with geom are emitted as map features; metric counts use the full
-- filtered set (geocoded or not). Idempotent (CREATE OR REPLACE).
-- ─────────────────────────────────────────────────────────────────────────────
SET search_path = tickets, public;
CREATE OR REPLACE FUNCTION reporting.fn_inc_dashboard(
p_cluster text DEFAULT NULL,
p_status text DEFAULT NULL,
p_window text DEFAULT 'today',
p_from timestamptz DEFAULT NULL,
p_to timestamptz DEFAULT NULL
)
RETURNS jsonb LANGUAGE plpgsql STABLE AS $fn$
DECLARE
v_now_eat timestamp;
v_from timestamptz;
v_to timestamptz;
v_preset text;
v_days numeric;
v_result jsonb;
BEGIN
p_cluster := NULLIF(p_cluster, '');
p_status := NULLIF(p_status, '');
v_now_eat := now() AT TIME ZONE 'Africa/Nairobi';
-- ── resolve the window ──────────────────────────────────────────────────────
IF p_from IS NOT NULL OR p_to IS NOT NULL THEN
v_preset := 'custom';
v_from := COALESCE(p_from, '-infinity'::timestamptz);
v_to := COALESCE(p_to, 'infinity'::timestamptz);
ELSE
v_preset := lower(COALESCE(NULLIF(p_window, ''), 'today'));
IF v_preset = 'week' THEN
v_from := date_trunc('week', v_now_eat) AT TIME ZONE 'Africa/Nairobi';
v_to := (date_trunc('week', v_now_eat) + interval '1 week') AT TIME ZONE 'Africa/Nairobi';
ELSIF v_preset = 'month' THEN
v_from := date_trunc('month', v_now_eat) AT TIME ZONE 'Africa/Nairobi';
v_to := (date_trunc('month', v_now_eat) + interval '1 month') AT TIME ZONE 'Africa/Nairobi';
ELSE
v_preset := 'today';
v_from := date_trunc('day', v_now_eat) AT TIME ZONE 'Africa/Nairobi';
v_to := (date_trunc('day', v_now_eat) + interval '1 day') AT TIME ZONE 'Africa/Nairobi';
END IF;
END IF;
IF v_from > '-infinity'::timestamptz AND v_to < 'infinity'::timestamptz THEN
v_days := GREATEST(EXTRACT(EPOCH FROM (v_to - v_from)) / 86400.0, 1);
ELSE
v_days := NULL; -- open-ended custom window → per-day average not meaningful
END IF;
-- ── build payload ───────────────────────────────────────────────────────────
WITH open_t AS (
SELECT * FROM tickets.inc_open_sla
WHERE (p_cluster IS NULL OR cluster = p_cluster)
AND (p_status IS NULL OR normalized_status = p_status)
),
closed_t AS (
SELECT ticket_id, normalized_status, cluster, region, location_name,
assigned_team, owner, closed_at, mttr, sla_status, geo_source, geom
FROM tickets.inc
WHERE NOT COALESCE(is_actionable, false)
AND closed_at IS NOT NULL
AND closed_at >= v_from AND closed_at < v_to
AND (p_cluster IS NULL OR cluster = p_cluster)
AND (p_status IS NULL OR normalized_status = p_status)
)
SELECT jsonb_build_object(
'window', jsonb_build_object('from', v_from, 'to', v_to, 'preset', v_preset),
'open', jsonb_build_object(
'type', 'FeatureCollection',
'features', COALESCE((
SELECT jsonb_agg(jsonb_build_object(
'type', 'Feature',
'properties', jsonb_build_object(
'ticket_id', ticket_id, 'normalized_status', normalized_status,
'cluster', cluster, 'region', region, 'location_name', location_name,
'assigned_team', assigned_team, 'owner', owner, 'geo_source', geo_source,
'sla_state', sla_state, 'hours_open', hours_open),
'geometry', ST_AsGeoJSON(geom)::jsonb))
FROM open_t WHERE geom IS NOT NULL), '[]'::jsonb)
),
'closed', jsonb_build_object(
'type', 'FeatureCollection',
'features', COALESCE((
SELECT jsonb_agg(jsonb_build_object(
'type', 'Feature',
'properties', jsonb_build_object(
'ticket_id', ticket_id, 'normalized_status', normalized_status,
'cluster', cluster, 'region', region, 'location_name', location_name,
'assigned_team', assigned_team, 'owner', owner, 'geo_source', geo_source,
'closed_at', closed_at, 'mttr', mttr, 'sla_status', sla_status),
'geometry', ST_AsGeoJSON(geom)::jsonb))
FROM closed_t WHERE geom IS NOT NULL), '[]'::jsonb)
),
'metrics', jsonb_build_object(
'open_now', (SELECT count(*) FROM open_t),
'closed_in_window', (SELECT count(*) FROM closed_t),
'sla', jsonb_build_object(
'open', (SELECT jsonb_build_object(
'breached', count(*) FILTER (WHERE sla_state = 'breached'),
'at_risk', count(*) FILTER (WHERE sla_state = 'at_risk'),
'ok', count(*) FILTER (WHERE sla_state = 'ok'),
'unknown', count(*) FILTER (WHERE sla_state = 'unknown')) FROM open_t),
'closed', (SELECT jsonb_build_object(
'compliant', count(*) FILTER (WHERE sla_status = 'Compliant'),
'breached', count(*) FILTER (WHERE sla_status = 'Breached')) FROM closed_t)
),
'by_status', COALESCE((SELECT jsonb_object_agg(s, c) FROM (
SELECT COALESCE(normalized_status, '(none)') AS s, count(*) AS c FROM (
SELECT normalized_status FROM open_t
UNION ALL SELECT normalized_status FROM closed_t) u GROUP BY 1) z), '{}'::jsonb),
'by_cluster', COALESCE((SELECT jsonb_object_agg(cl, c) FROM (
SELECT COALESCE(cluster, '(none)') AS cl, count(*) AS c FROM (
SELECT cluster FROM open_t
UNION ALL SELECT cluster FROM closed_t) u GROUP BY 1) z), '{}'::jsonb),
'closure_rate', jsonb_build_object(
'per_day_avg', CASE WHEN v_days IS NULL THEN NULL
ELSE round((SELECT count(*) FROM closed_t)::numeric / v_days, 2) END,
'series', COALESCE((SELECT jsonb_agg(jsonb_build_object('day', d, 'count', c) ORDER BY d) FROM (
SELECT (closed_at AT TIME ZONE 'Africa/Nairobi')::date AS d, count(*) AS c
FROM closed_t GROUP BY 1) z), '[]'::jsonb)
),
'avg_mttr_min', (SELECT round(avg(mttr), 1) FROM closed_t WHERE mttr IS NOT NULL)
),
'freshness', (SELECT jsonb_object_agg(dataset, jsonb_build_object(
'export_type', export_type, 'exported_at', exported_at,
'records_ingested', records_ingested, 'ingested_at', ingested_at))
FROM tickets.import_meta)
) INTO v_result;
RETURN v_result;
END $fn$;
COMMENT ON FUNCTION reporting.fn_inc_dashboard(text, text, text, timestamptz, timestamptz) IS
'FleetOps INC operations dashboard: open (live) + closed (windowed) GeoJSON + '
'ticket metrics, filtered by cluster/status/time (EAT). fleettickets 09.';
-- grants (guarded: roles may not exist on a fresh DB)
DO $grants$
BEGIN
IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'dashboard_ro') THEN
GRANT EXECUTE ON FUNCTION reporting.fn_inc_dashboard(text, text, text, timestamptz, timestamptz) TO dashboard_ro;
END IF;
IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'grafana_ro') THEN
GRANT EXECUTE ON FUNCTION reporting.fn_inc_dashboard(text, text, text, timestamptz, timestamptz) TO grafana_ro;
END IF;
END $grants$;