chore(db): purge unused ops + dwh_gold schemas
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run

Drop the dormant ops (workshop / tickets / dispatch / SLA / odometer)
and dwh_gold (nightly ETL aggregates) schemas plus their dependents —
features never implemented, no live writer or scheduled refresh.

- Prod DB (already applied): DROP SCHEMA ops/dwh_gold CASCADE, plus
  tracksolid.dispatch_log, v_sla_inflight, v_utilisation_daily.
- migrations/12_drop_ops.sql + 13_drop_dwh_gold.sql (forward, all
  IF EXISTS) registered in run_migrations.py for rebuild durability.
- grafana: removed 8 now-broken panels (In-flight SLA, Idle Cost,
  Utilisation Heatmap, Row 7 Field-Service SLAs) from daily_operations;
  panel count 21 -> 13.
- docs: scrubbed CLAUDE.md, PLATFORM_OVERVIEW.html (-19KB), DATA_FLOW.md;
  pre-drop seed snapshot in docs/reports/260605_ops_purge_backup.md.

The separate tracksolid_dwh server (31.97.44.246:5888) is unrelated
and untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
david kiania 2026-06-05 18:11:03 +03:00
parent e060933c55
commit 8c5a43f3b8
8 changed files with 1152 additions and 322 deletions

View file

@ -56,7 +56,7 @@ See `docs/CONNECTIONS.md` for the full shape. Summary:
- **SSH:** `ssh -i ~/.ssh/id_ed25519 kianiadee@stage.rahamafresh.com` - **SSH:** `ssh -i ~/.ssh/id_ed25519 kianiadee@stage.rahamafresh.com`
- **DB name:** `tracksolid_db` · **DB user:** `postgres` (internal) · `tracksolid_owner` (app) · `grafana_ro` (read-only) - **DB name:** `tracksolid_db` · **DB user:** `postgres` (internal) · `tracksolid_owner` (app) · `grafana_ro` (read-only)
- **DB schemas:** `tracksolid` (live, single source of truth) · `dwh_gold` (aggregates) · `ops` (workshop / tickets / odometer) · `infrastructure`. The legacy `tracksolid_2` schema no longer exists — migrations 0206 applied 2026-04-18. - **DB schemas:** `tracksolid` (live, single source of truth) · `reporting` (map-dashboard read layer) · `infrastructure`. The legacy `tracksolid_2` schema no longer exists (migrations 0206, 2026-04-18); the `ops` and `dwh_gold` schemas were purged 2026-06-05 (migrations 12/13) as unused.
- **DB access:** `DATABASE_URL` points to `timescale_db:5432` (internal Docker network — not reachable locally). Use `docker exec` pattern above. See `docs/CONNECTIONS.md` for full reference. - **DB access:** `DATABASE_URL` points to `timescale_db:5432` (internal Docker network — not reachable locally). Use `docker exec` pattern above. See `docs/CONNECTIONS.md` for full reference.
- **DWH target DB:** `tracksolid_dwh` at `31.97.44.246:5888` (separate PostGIS instance, public IP). Users: `dwh_owner` (bronze writes + `dwh_control`), `grafana_ro` (reads bronze/silver/gold/`dwh_control`). Always connect with `sslmode=require`. Fed by the n8n `dwh_extract` + `dwh_load_bronze` workflows — see `docs/DWH_PIPELINE.md`. - **DWH target DB:** `tracksolid_dwh` at `31.97.44.246:5888` (separate PostGIS instance, public IP). Users: `dwh_owner` (bronze writes + `dwh_control`), `grafana_ro` (reads bronze/silver/gold/`dwh_control`). Always connect with `sslmode=require`. Fed by the n8n `dwh_extract` + `dwh_load_bronze` workflows — see `docs/DWH_PIPELINE.md`.
- **Container naming:** Coolify appends a random suffix. Always resolve with: - **Container naming:** Coolify appends a random suffix. Always resolve with:
@ -93,10 +93,11 @@ dwh/ # DWH migrations for tracksolid_dwh@31.97.44.246:588
# 261002_bronze_constraints_audit.sql — ON CONFLICT key assertion # 261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
# 261003_dwh_roles.sql — role contract assertion # 261003_dwh_roles.sql — role contract assertion
# 261004_dwh_observability_views.sql — freshness/failure views # 261004_dwh_observability_views.sql — freshness/failure views
migrations/ # Numbered SQL migrations 0210, applied in order by run_migrations.py migrations/ # Numbered SQL migrations 0213, applied in order by run_migrations.py
# 02 full schema · 03 webhook · 04 distance fix · 05 enhancements # 02 full schema · 03 webhook · 04 distance fix · 05 enhancements
# 06 ops/analytics · 07 views · 08 config · 09 trips enrichment # 06 ops/analytics · 07 views · 08 config · 09 trips enrichment
# 10_driver_clock_views.sql · 10_pgbouncer_auth.sql # 10_driver_clock_views.sql · 10_pgbouncer_auth.sql · 11 reporting
# 12 drop ops schema · 13 drop dwh_gold schema (both 2026-06-05)
Dockerfile # Custom image for ingest/webhook containers Dockerfile # Custom image for ingest/webhook containers
pyproject.toml # Python project + uv dependency spec pyproject.toml # Python project + uv dependency spec
backup/ # pg_dump sidecar scripts and config backup/ # pg_dump sidecar scripts and config
@ -126,12 +127,13 @@ tracksolid.alarms -- Alarm events (alarm_type, alarm_name, alarm_time
tracksolid.obd_readings -- OBD diagnostics (push only, awaiting webhook registration) tracksolid.obd_readings -- OBD diagnostics (push only, awaiting webhook registration)
tracksolid.device_events -- Power on/off tamper events (push only) tracksolid.device_events -- Power on/off tamper events (push only)
tracksolid.ingestion_log -- API call audit trail — 875 runs / 24h, 0 failures at last check (2026-04-19) tracksolid.ingestion_log -- API call audit trail — 875 runs / 24h, 0 failures at last check (2026-04-19)
tracksolid.dispatch_log -- Dispatch decisions for SLA tracking (migration 06; empty until ops integration) tracksolid.schema_migrations -- Applied migrations 0213
tracksolid.schema_migrations -- Applied files: 02,03,04,05,06 (last 06 on 2026-04-18) -- PURGED 2026-06-05 (migrations 12 + 13): the dormant `ops` schema (tickets, service_log,
dwh_gold.fact_daily_fleet_metrics -- Nightly ETL aggregates per vehicle per day (run refresh_daily_metrics) -- odometer_readings, cost_rates, kpi_targets, vw_service_forecast), tracksolid.dispatch_log,
ops.service_log -- Workshop service history (migration 06) -- and the `dwh_gold` schema (dim_vehicles, fact_daily_fleet_metrics, refresh_daily_metrics).
ops.odometer_readings -- Physical odometer captures (migration 06) -- Those workshop/dispatch/SLA/utilisation features were never implemented. Do NOT reintroduce
ops.tickets -- Ticket skeleton for ops integration (migration 06; empty) -- references to ops.* or dwh_gold.* — they no longer exist. (The separate tracksolid_dwh DB
-- at 31.97.44.246:5888 is unrelated and untouched.)
``` ```
Full DDL: `02_tracksolid_full_schema_rev.sql` + migrations `03``06`. Full DDL: `02_tracksolid_full_schema_rev.sql` + migrations `03``06`.
@ -146,8 +148,8 @@ tracksolid.v_currently_idle -- §2.2 idle lens
tracksolid.v_driver_aggregates_daily -- §3.1 + §3.2 aggression index source tracksolid.v_driver_aggregates_daily -- §3.1 + §3.2 aggression index source
tracksolid.v_fleet_km_daily -- §7 Panel 5 distance trend tracksolid.v_fleet_km_daily -- §7 Panel 5 distance trend
tracksolid.v_alarms_daily -- §7 Panel 7 alarm frequency tracksolid.v_alarms_daily -- §7 Panel 7 alarm frequency
tracksolid.v_utilisation_daily -- §7 Panel 8 utilisation heatmap (gated on dwh_gold ETL) -- v_utilisation_daily (dwh_gold) and v_sla_inflight (ops) were DROPPED 2026-06-05 with
tracksolid.v_sla_inflight -- §4.5 SLA panels (gated on ops.tickets) -- their schemas (migrations 12/13); their Grafana panels were removed from the dashboard.
``` ```
All views carry a `COMMENT ON VIEW` referencing their spec — `\d+ tracksolid.v_*` shows the provenance. All views carry a `COMMENT ON VIEW` referencing their spec — `\d+ tracksolid.v_*` shows the provenance.
@ -222,7 +224,7 @@ dwh_control.v_watermark_lag -- Grafana: extract vs. load lag per table
7. **Secrets from env only.** Connection strings, API keys, and passwords live in `.env`. Reference variable names from `docs/CONNECTIONS.md`, never values. 7. **Secrets from env only.** Connection strings, API keys, and passwords live in `.env`. Reference variable names from `docs/CONNECTIONS.md`, never values.
8. **Two developers, one incoming.** Write code and docs that a second developer (mixed technical/operations background) can follow without prior context. 8. **Two developers, one incoming.** Write code and docs that a second developer (mixed technical/operations background) can follow without prior context.
9. **Forgejo API auth:** credentials stored in macOS keychain. Retrieve with `git credential fill` (host=repo.rahamafresh.com). Use basic auth against `https://repo.rahamafresh.com/api/v1` directly — no `tea` or `gh` needed. 9. **Forgejo API auth:** credentials stored in macOS keychain. Retrieve with `git credential fill` (host=repo.rahamafresh.com). Use basic auth against `https://repo.rahamafresh.com/api/v1` directly — no `tea` or `gh` needed.
10. **Single live schema.** All live data lives in `tracksolid`. Aggregates live in `dwh_gold`; workshop/ticket integrations live in `ops`. Do not reintroduce references to the retired `tracksolid_2` schema. 10. **Single live schema.** All live data lives in `tracksolid`; the map-dashboard read layer lives in `reporting`. Do not reintroduce references to the retired `tracksolid_2`, `ops`, or `dwh_gold` schemas (the latter two purged 2026-06-05, migrations 12/13).
--- ---
@ -256,6 +258,5 @@ Latest full snapshot: `docs/reports/260412_baseline_report.md`
| MEDIUM | Investigate 44 silent devices (only 19 of 63 reporting) — SIM installed? Activated? | | MEDIUM | Investigate 44 silent devices (only 19 of 63 reporting) — SIM installed? Activated? |
| MEDIUM | Co-develop client KPI framework (see `docs/KPI_FRAMEWORK.md`) | | MEDIUM | Co-develop client KPI framework (see `docs/KPI_FRAMEWORK.md`) |
| LOW | Populate geofences — depot boundaries, city zones | | LOW | Populate geofences — depot boundaries, city zones |
| LOW | Schedule nightly ETL: `SELECT dwh_gold.refresh_daily_metrics(CURRENT_DATE - 1)` (cron or n8n) |
| HIGH | Deploy DWH bronze pipeline: apply `dwh/26100{1,2,3,4}.sql` to `tracksolid_dwh`, import + wire the two n8n workflows, verify first run via `dwh_control.v_table_freshness`. Runbook: `docs/DWH_PIPELINE.md` | | HIGH | Deploy DWH bronze pipeline: apply `dwh/26100{1,2,3,4}.sql` to `tracksolid_dwh`, import + wire the two n8n workflows, verify first run via `dwh_control.v_table_freshness`. Runbook: `docs/DWH_PIPELINE.md` |
| MEDIUM | Rotate `dwh_owner` / `grafana_ro` passwords on `tracksolid_dwh` — plaintext in `dwh/260423_dwh_ddl_v1.sql` is a pre-existing flaw to clean up separately | | MEDIUM | Rotate `dwh_owner` / `grafana_ro` passwords on `tracksolid_dwh` — plaintext in `dwh/260423_dwh_ddl_v1.sql` is a pre-existing flaw to clean up separately |

240
docs/DATA_FLOW.md Normal file
View file

@ -0,0 +1,240 @@
# Data Flow — Ingestion → Aggregation → Views → Functions → Consumers
**Scope:** the *live* fleet pipeline only (`tracksolid` + `reporting`). The `ops` and
`dwh_gold` schemas were **purged on 2026-06-05** (migrations 12/13) — those workshop /
dispatch / SLA / utilisation features were never implemented. They are shown below only as
a struck-out "removed" footnote for historical context. (The *separate* `tracksolid_dwh`
server at 31.97.44.246:5888 is unrelated and was not touched.)
**Verified against prod 2026-06-05** (TimescaleDB hypertable + continuous-aggregate
catalog, `pg_depend` view graph, ingestion `INSERT` targets, `dashboard_api` queries,
Grafana panel SQL). Key facts that surprised the docs:
- Only `position_history` (and the empty/planned `heartbeats`, `fuel_readings`,
`temperature_readings`) are **hypertables**. `trips` and `alarms` are **plain tables**.
- `tracksolid.v_mileage_daily_cagg` is a **real TimescaleDB continuous aggregate**, not a
plain view — and it currently has **no downstream consumer**.
- `reporting.v_trips` is a **matview**, refreshed every 5 min by the in-process
`dashboard_api` background loop (FIX-D02), pg advisory-lock `920145`.
---
## Mermaid
```mermaid
flowchart TD
API["Tracksolid / Jimi API<br/>poll + push webhooks · OAuth2"]
subgraph ING["Ingestion — ts_shared_rev.py: get_conn() · api_post() · clean*()"]
IM["ingest_movement_rev.py"]
IE["ingest_events_rev.py"]
WR["webhook_receiver_rev.py"]
end
API --> IM & IE & WR
subgraph L1["L1 · Base tables + hypertables — schema: tracksolid (single source of truth)"]
LP["live_positions<br/>current fix / IMEI"]
PH[("position_history<br/>HYPERTABLE · high-res GPS trail")]
TR["trips<br/>plain table — NOT a hypertable"]
DV["devices<br/>vehicle / driver registry"]
AL["alarms<br/>plain table"]
ILOG["ingestion_log · api_token_cache"]
EMPTY["heartbeats* · fuel_readings* · temperature_readings* (HYPERTABLES, empty)<br/>device_events · fault_codes · obd_readings · parking_events · lbs_readings · geofences (empty)"]
end
IM --> LP & PH & TR & DV
IM --> ILOG
IE --> AL
WR --> EMPTY
subgraph L2["L2 · Aggregation"]
CAGG[("v_mileage_daily_cagg<br/>CONTINUOUS AGGREGATE")]
VT[("reporting.v_trips<br/>MATVIEW · ix_v_trips_trip_id")]
RLOG["reporting.refresh_log"]
end
PH -->|Timescale cont-agg policy| CAGG
TR --> VT
DV --> VT
VT -->|"REFRESH CONCURRENTLY every 300s<br/>(dashboard_api loop, adv-lock 920145)"| RLOG
subgraph L3L["L3 · Reporting views — fed by v_trips matview"]
FILT["v_filter_vehicles · v_filter_drivers<br/>v_filter_cost_centres · v_filter_cities"]
SUMM["v_daily/weekly/monthly_summary<br/>v_*_cost_centre · v_trips_today"]
VLP["v_live_positions"]
end
VT --> FILT & SUMM
LP --> VLP
subgraph L3R["L3 · Grafana views — tracksolid.* (read base tables directly)"]
GV["v_fleet_today · v_fleet_status · v_active_dispatch_map<br/>v_currently_idle · v_alarms_daily · v_fleet_km_daily<br/>v_ingestion_health · v_vehicles_not_moved_today<br/>v_driver_aggregates_daily · v_fleet_trace · v_driver_clock_*"]
end
LP --> GV
TR --> GV
DV --> GV
AL --> GV
PH --> GV
subgraph L4["L4 · Functions — reporting.* (the only API entrypoints)"]
FLP["fn_live_positions(cost_centre, acc_status)"]
FVT["fn_vehicle_track(vehicle_number, hours)"]
FTM["fn_trips_for_map(veh[], driver, cc, city, start, end)"]
NP["normalize_plate(p) · helper"]
end
LP --> FLP
DV --> FLP
PH --> FVT
VT --> FTM
NP -.-> FTM
subgraph L5["L5 · Consumers"]
DAPI["dashboard_api_rev.py<br/>FastAPI :8890 · 2 workers"]
SPA["SPAs: liveposition.* · fleetintelligence.*<br/>(rustfs / S3 single-file maps)"]
GRAF["Grafana"]
end
FLP --> DAPI
FVT --> DAPI
FTM --> DAPI
FILT --> DAPI
DAPI -->|"HTTPS · fleetapi.rahamafresh.com"| SPA
GV --> GRAF
CAGG -.->|no consumer yet| NONE(["⚠ unconsumed — no panel / API reads it"])
subgraph PARK["REMOVED 2026-06-05 — purged via migrations 12 / 13 (never implemented)"]
GONE["ops.* (tickets · dispatch_log · service_log · odometer_readings · cost_rates · kpi_targets · vw_service_forecast)<br/>dwh_gold.* (dim_vehicles · fact_daily_fleet_metrics · refresh_daily_metrics)<br/>tracksolid.v_sla_inflight · tracksolid.v_utilisation_daily + their Grafana panels"]
end
classDef hyper fill:#e1f0ff,stroke:#3b82f6,color:#0b3d91;
classDef mat fill:#fff3cd,stroke:#d4a017,color:#664d03;
classDef cagg fill:#e7f7e7,stroke:#2e9e2e,color:#1e5e1e;
classDef parked fill:#f0f0f0,stroke:#999,stroke-dasharray:5 5,color:#555;
classDef warn fill:#fdecea,stroke:#d93025,color:#a52714;
class PH hyper;
class VT mat;
class CAGG cagg;
class PARK,GONE parked;
class NONE warn;
```
---
## ASCII
```
╔══════════════════════════════════════════════╗
║ TRACKSOLID / JIMI API ║
║ (poll + push webhooks, OAuth2) ║
╚════════════════════╤═════════════════════════╝
┌──────────────────────────┬───────────┴───────────┬────────────────────────────┐
▼ ▼ ▼ ▼
┌─────────────────┐ ┌────────────────────┐ ┌──────────────────────┐
│ ingest_movement │ │ ingest_events │ │ webhook_receiver │ (all via ts_shared_rev.py:
│ _rev.py │ │ _rev.py │ │ _rev.py │ get_conn() pool, api_post(),
└────────┬────────┘ └─────────┬──────────┘ └──────────┬───────────┘ clean*() , token cache)
│ │ │
│ writes │ writes │ writes (push)
▼ ▼ ▼
╔════════════════════════════════════════════════════════════════════════════════════════════╗
║ L1 · BASE TABLES + HYPERTABLES schema: tracksolid (single source of truth) ║
║ ║
║ live_positions ── current fix / IMEI (plain) api_token_cache (auth) ║
║ position_history ◀═══ HYPERTABLE (high-res GPS trail, partitioned by gps_time) ║
║ trips ── trip summaries (plain table — NOT a hypertable) ║
║ devices ── vehicle/driver registry (plain) ║
║ alarms ── alarm events (plain table) ║
║ ingestion_log ── API audit trail (plain) ║
║ heartbeats*, fuel_readings*, temperature_readings* ◀═ HYPERTABLES (empty / planned push) ║
║ device_events, fault_codes, obd_readings, parking_events, lbs_readings, geofences (empty) ║
╚══════╤══════════════════════════╤══════════════════════════════════╤════════════════════════╝
│ │ │
│ TimescaleDB │ matview refresh │ direct reads
│ cont-agg policy │ (in-app loop) │ (Grafana SQL)
▼ ▼ ▼
╔══════════════════╗ ╔════════════════════════════════╗ ╔══════════════════════════════════╗
║ L2 · AGGREGATION ║ ║ L2 · AGGREGATION (matview) ║ ║ L3 · GRAFANA VIEWS (tracksolid.*) ║
║ ║ ║ ║ ║ read base tables directly ║
║ v_mileage_daily ║ ║ reporting.v_trips [MATVIEW] ║ ║ ║
║ _cagg ║ ║ ◀── trips + devices ║ ║ v_fleet_today v_fleet_status ║
║ CONTINUOUS AGG ║ ║ unique ix_v_trips_trip_id ║ ║ v_active_dispatch_map ║
║ ◀═ position_ ║ ║ ║ ║ v_currently_idle v_alarms_daily ║
║ history ║ ║ REFRESH … CONCURRENTLY every ║ ║ v_fleet_km_daily v_ingestion_health║
║ (auto refresh ║ ║ VTRIPS_REFRESH_INTERVAL_S=300 ║ ║ v_vehicles_not_moved_today ║
║ policy) ║ ║ by dashboard_api bg asyncio ║ ║ v_driver_aggregates_daily ║
╚════════╤═════════╝ ║ loop, pg advisory-lock 920145 ║ ║ v_fleet_trace ║
│ ║ │ ║ ║ v_driver_clock_daily/_today/_attnd ║
│ ║ └──▶ reporting.refresh_log ║ ╚══════════════╤═════════════════════╝
│ ╚═══════════════╤════════════════╝ │
│ │ │
│ ▼ │
│ ╔════════════════════════════════════════════╗ │
│ ║ L3 · REPORTING VIEWS (← v_trips matview) ║ │
│ ║ v_filter_vehicles v_filter_drivers ║ │
│ ║ v_filter_cost_centres v_filter_cities ║ │
│ ║ v_daily/weekly/monthly_summary ║ │
│ ║ v_*_cost_centre v_trips_today ║ │
│ ║ v_live_positions (view, ← live_positions) ║ │
│ ╚════════════════════════╤═══════════════════╝ │
│ │ │
│ ╔══════════════════════════════════╧═══════════════════╗ │
│ ║ L4 · FUNCTIONS (schema reporting — API entrypoints) ║ │
│ ║ ║ │
│ ║ fn_live_positions(cost_centre, acc_status) ║ │
│ ║ ◀── live_positions + devices ║ │
│ ║ fn_vehicle_track(vehicle_number, hours) ║ │
│ ║ ◀── position_history (hypertable trail) ────────╫─────┘ (also reads L1)
│ ║ fn_trips_for_map(veh[],driver,cc,city,start,end) ║
│ ║ ◀── reporting.v_trips (matview) ║
│ ║ normalize_plate(p) ── helper used by the above ║
│ ╚════════════════════════════╤══════════════════════════╝
│ │
▼ ▼
(cagg currently ╔═══════════════════════════════════════════════╗
unconsumed — ║ L5 · CONSUMERS ║
no panel/API yet) ║ ║
║ dashboard_api_rev.py (FastAPI :8890, ×2 wkrs) ║
║ POST /webhook/fleet-dashboard → fn_trips_for_map + v_filter_*
║ GET /webhook/live-positions → fn_live_positions
║ GET /webhook/live-positions/track → fn_vehicle_track
║ │ ║
║ ▼ HTTPS (fleetapi.rahamafresh.com) ║
║ SPAs: liveposition.* · fleetintelligence.* ║
║ (rustfs/S3 single-file maps) ║
║ ║
║ Grafana ──SQL──▶ tracksolid.v_* (L3 right) ║
╚═══════════════════════════════════════════════╝
┌─ REMOVED 2026-06-05 · purged via migrations 12 / 13 (never implemented) ───────────────┐
│ ops.* : tickets, dispatch_log, service_log, odometer_readings, │
│ cost_rates, kpi_targets, vw_service_forecast │
│ dwh_gold.* : dim_vehicles, fact_daily_fleet_metrics, refresh_daily_metrics() │
│ tracksolid : v_sla_inflight, v_utilisation_daily (+ their Grafana panels removed) │
│ → Schemas dropped from prod. The separate tracksolid_dwh server is unrelated/untouched.│
└────────────────────────────────────────────────────────────────────────────────────────┘
```
---
## How to read it
- **L1** is where ingestion lands. Only `position_history` (and three empty/planned
hypertables) are TimescaleDB hypertables; `trips`/`alarms` are ordinary tables.
- **Two aggregation mechanisms** run in parallel: the TimescaleDB **continuous aggregate**
`v_mileage_daily_cagg` (refreshed by a Timescale policy off `position_history`), and the
**`reporting.v_trips` matview** (refreshed every 5 min by the in-process `dashboard_api`
loop — the FIX-D02 self-refresher that replaced the dead n8n job).
- **L4 functions are the only thing the API calls.** SPAs never touch tables directly:
SPA → `dashboard_api``fn_*` → (matview / tables).
- **Grafana** takes the right-hand path, reading the `tracksolid.v_*` analytics views
straight off L1.
## Caveats / housekeeping notes
1. **`v_mileage_daily_cagg` has no consumer** — nothing in the API or Grafana reads it. It
is a live continuous aggregate maintaining itself for nobody. Candidate for removal or
wiring into a panel.
2. **`ops` and `dwh_gold` were purged** (2026-06-05, migrations 12/13) along with
`v_utilisation_daily`, `v_sla_inflight`, and their Grafana panels — the features were
never implemented.
3. See the companion housekeeping audit (2026-06-05) for the full unused-object list; the
only clean ad-hoc drop is `public.trips_viz_v1`.

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,47 @@
# ops schema purge — pre-drop backup (2026-06-05)
Snapshot taken immediately before dropping the `ops` schema + `tracksolid.dispatch_log`
+ the dependent view `tracksolid.v_sla_inflight`. The dispatch/SLA/workshop features were
never implemented and are being purged (may take a different direction later).
Only two `ops` tables held rows, and both are **migration-08 seed data**
(`migrations/08_analytics_config.sql`), i.e. regenerable — this file is belt-and-suspenders.
Every other ops table (`tickets`, `service_log`, `odometer_readings`) and the view
`vw_service_forecast` were empty. `tracksolid.dispatch_log` was empty.
## ops.cost_rates (3 rows)
| rate_key | scope_type | scope_value | metric | amount | currency | effective_from | notes |
|---|---|---|---|---|---|---|---|
| fuel.nairobi | city | nairobi | fuel_per_litre | 195.00 | KES | 2026-04-27 | Placeholder pump price — confirm with Finance. |
| fuel.mombasa | city | mombasa | fuel_per_litre | 195.00 | KES | 2026-04-27 | Placeholder pump price — confirm with Finance. |
| fuel.kampala | city | kampala | fuel_per_litre | 5200.00 | UGX | 2026-04-27 | Placeholder pump price — confirm with Finance. |
## ops.kpi_targets (12 rows)
| target_id | kpi_key | scope_type | scope_value | target | amber | red | direction | effective_from | notes |
|---|---|---|---|---|---|---|---|---|---|
| 1 | utilisation_pct | global | | 70.00 | 60.00 | 50.00 | higher_is_better | 2026-04-27 | Fleet utilisation: drive_hours / engine_on_hours. |
| 2 | idle_pct | global | | 15.00 | 20.00 | 25.00 | lower_is_better | 2026-04-27 | Idle as % of engine-on time. |
| 3 | idle_pct | cost_centre | osp patrol | 15.00 | 20.00 | 25.00 | lower_is_better | 2026-04-27 | OSP patrol idle target — same as global until calibrated. |
| 4 | fuel_kes_per_100km | global | | 12.00 | 14.00 | 16.00 | lower_is_better | 2026-04-27 | Fuel litres per 100km equivalent — uses fuel_100km on devices. |
| 5 | mttr_hours | global | | 4.00 | 6.00 | 8.00 | lower_is_better | 2026-04-27 | Mean Time To Resolve, field-service ticket. |
| 6 | alarms_per_100km | global | | 2.00 | 3.00 | 5.00 | lower_is_better | 2026-04-27 | Safety event density. |
| 7 | utilisation_pct | global | | 70.00 | 60.00 | 50.00 | higher_is_better | 2026-05-01 | Fleet utilisation: drive_hours / engine_on_hours. |
| 8 | idle_pct | global | | 15.00 | 20.00 | 25.00 | lower_is_better | 2026-05-01 | Idle as % of engine-on time. |
| 9 | idle_pct | cost_centre | osp patrol | 15.00 | 20.00 | 25.00 | lower_is_better | 2026-05-01 | OSP patrol idle target — same as global until calibrated. |
| 10 | fuel_kes_per_100km | global | | 12.00 | 14.00 | 16.00 | lower_is_better | 2026-05-01 | Fuel litres per 100km equivalent — uses fuel_100km on devices. |
| 11 | mttr_hours | global | | 4.00 | 6.00 | 8.00 | lower_is_better | 2026-05-01 | Mean Time To Resolve, field-service ticket. |
| 12 | alarms_per_100km | global | | 2.00 | 3.00 | 5.00 | lower_is_better | 2026-05-01 | Safety event density. |
## What was dropped
```sql
DROP VIEW IF EXISTS tracksolid.v_sla_inflight; -- depended on ops.tickets + dispatch_log
DROP SCHEMA IF EXISTS ops CASCADE; -- tickets, service_log, odometer_readings,
-- cost_rates, kpi_targets, vw_service_forecast
DROP TABLE IF EXISTS tracksolid.dispatch_log; -- empty; only fed v_sla_inflight
```
**Not dropped in this step:** `dwh_gold` schema and `tracksolid.v_utilisation_daily`
(separate decision, pending).

View file

@ -0,0 +1,30 @@
-- 12_drop_ops.sql
-- Purge the dormant `ops` schema (workshop / tickets / dispatch / SLA / odometer)
-- and the dispatch/SLA artefacts that lived in `tracksolid`. These features were
-- never implemented and may take a different direction; the schema and views were
-- empty (or seed-only) and unused by the live map/dashboard pipeline.
--
-- Created by migrations 06 (ops schema + dispatch_log), 07 (v_sla_inflight) and
-- 08 (ops.cost_rates / ops.kpi_targets seed). Per the repo rule we do NOT rewrite
-- those applied migrations — this forward migration drops the objects instead. On
-- a fresh rebuild 06/07/08 create them, then this file removes them again.
--
-- Pre-drop snapshot of the only seeded tables: docs/reports/260605_ops_purge_backup.md
-- (the seed is also reproducible from 08_analytics_config.sql).
--
-- Every statement is IF EXISTS so the file is safe to re-apply.
--
-- NOTE: dwh_gold and tracksolid.v_utilisation_daily are intentionally NOT touched
-- here — that is a separate decision.
-- View first: it reads ops.tickets + tracksolid.dispatch_log, so it must go before
-- the objects it depends on (avoids an implicit CASCADE surprise).
DROP VIEW IF EXISTS tracksolid.v_sla_inflight;
-- The whole ops schema: tickets, service_log, odometer_readings, cost_rates,
-- kpi_targets, and the view vw_service_forecast. CASCADE clears intra-schema deps.
DROP SCHEMA IF EXISTS ops CASCADE;
-- Dispatch feature table — lived in the tracksolid schema, empty, only fed
-- v_sla_inflight (now dropped).
DROP TABLE IF EXISTS tracksolid.dispatch_log;

View file

@ -0,0 +1,23 @@
-- 13_drop_dwh_gold.sql
-- Purge the dormant `dwh_gold` aggregate schema and its dependent view. The nightly
-- ETL (dwh_gold.refresh_daily_metrics) was never scheduled, both fact/dim tables were
-- empty, and nothing in the live map/dashboard pipeline reads them. These analytics
-- may take a different direction later.
--
-- Created by migrations 02 (schema), 05 (dwh_gold expansion + refresh_daily_metrics)
-- and 07 (tracksolid.v_utilisation_daily). Per the repo rule we do NOT rewrite those
-- applied migrations — this forward migration drops the objects instead. On a fresh
-- rebuild 02/05/07 create them, then this file removes them again.
--
-- Both tables were empty at drop time, so there is no data backup (cf. the ops purge,
-- which had seed rows — docs/reports/260605_ops_purge_backup.md). Companion to
-- 12_drop_ops.sql; together they retire the unused ops + dwh_gold analytics layers.
--
-- Every statement is IF EXISTS so the file is safe to re-apply.
-- View first: it reads dwh_gold.dim_vehicles + dwh_gold.fact_daily_fleet_metrics.
DROP VIEW IF EXISTS tracksolid.v_utilisation_daily;
-- The whole dwh_gold schema: dim_vehicles, fact_daily_fleet_metrics, the
-- dim_vehicles sequence/indexes, and the refresh_daily_metrics() function.
DROP SCHEMA IF EXISTS dwh_gold CASCADE;

View file

@ -35,6 +35,8 @@ MIGRATIONS = [
"09_trips_enrichment.sql", # trips.route_geom + addresses + plate + v_trips_enriched "09_trips_enrichment.sql", # trips.route_geom + addresses + plate + v_trips_enriched
"10_pgbouncer_auth.sql", # pgbouncer role + user_lookup() for SCRAM passthrough "10_pgbouncer_auth.sql", # pgbouncer role + user_lookup() for SCRAM passthrough
"11_reporting_schema.sql", # reporting.* map-dashboard read layer (dashboard_api) "11_reporting_schema.sql", # reporting.* map-dashboard read layer (dashboard_api)
"12_drop_ops.sql", # purge dormant ops schema + dispatch_log + v_sla_inflight
"13_drop_dwh_gold.sql", # purge dormant dwh_gold schema + v_utilisation_daily
] ]
# ── Tables that must exist before the service is allowed to start ───────────── # ── Tables that must exist before the service is allowed to start ─────────────