chore(cleanup): purge n8n, Grafana, and DWH references + dead artifacts
These subsystems are retired and replaced by better alternatives (FleetNow / FleetOps SPAs via dashboard_api; in-process pooling; reporting.v_ingest_health). Remove them so the repo reflects the live stack only. Nothing running depends on the deleted artifacts. Deleted (dead artifacts): - n8n-workflows/ (retired webhook exports), grafana/ (provisioning for the removed service), dwh/ (migrations for the decommissioned external warehouse) - runbooks: DWH_PIPELINE.md, DWH_Execution_Manual.md, grafanaDeployment.md, grafanaOperationalManual.md Code/config: - run_migrations.py: drop sync_role_passwords() (its only entries were the now -dead grafana_ro + pgbouncer syncs; the guard already made it inert) - .env: remove the two unused GRAFANA_* vars - ingest_movement_rev.py / db_audit / deploy_dashboard_api_staging.sh: reword stale Grafana/grafana_ro comments Docs: scrub n8n/Grafana/DWH from CLAUDE.md, CONNECTIONS, DATA_FLOW, OPERATIONS_MANUAL, docker_commands, KPI_FRAMEWORK, PLATFORM_OVERVIEW, STAGING_FLEETOPS, and deprecation-banner the two large SQL libraries (dwh_gold was already dropped 2026-06-05). Kept deliberately: the grafana_ro DB role (now an unused read-only login), applied migration history, dated docs/reports/*, and docs/superpowers/* specs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
76f6915e61
commit
d5093f0a1c
40 changed files with 137 additions and 11716 deletions
2
.env
2
.env
|
|
@ -11,7 +11,5 @@ POSTGRES_PASSWORD=U1pm3f5SX34DXkHoW6aKFsBHOlMA9binDPNG4aT0FAcg7AubEvYm0e6kU2dZiY
|
|||
DATABASE_URL=postgres://postgres:U1pm3f5SX34DXkHoW6aKFsBHOlMA9binDPNG4aT0FAcg7AubEvYm0e6kU2dZiYrR@timescale_db:5432/tracksolid_db
|
||||
|
||||
# Grafana
|
||||
GRAFANA_ADMIN_PASSWORD=ed3aaf20707fb5af9185708ec27f5211f71b35067277993eab624abce1
|
||||
GRAFANA_DB_RO_PASSWORD=7942a1DeLgyuiCzh8XFH21sPVJqRJo737qDW1PNDEtM
|
||||
|
||||
API_BASE_URL=https://eu-open.tracksolidpro.com/route/rest
|
||||
|
|
|
|||
68
CLAUDE.md
68
CLAUDE.md
|
|
@ -28,7 +28,7 @@ docker exec -i $DB psql -U postgres -d tracksolid_db < migrations/07_your_migrat
|
|||
|
||||
Fleet telematics ingestion and analytics stack for a **telco first-line support client** operating in Nairobi, Mombasa, and Kampala. The client dispatches field technicians to install, repair, and maintain home and business broadband, handle LOS signal faults, service migrations, and maintain outside plant infrastructure. The fleet is ~80 vehicles across three cities, all tracked via Tracksolid Pro (Jimi IoT API).
|
||||
|
||||
This repository ingests the Tracksolid Pro API into a TimescaleDB/PostGIS database and visualises fleet and operational KPIs in Grafana. The pipeline is deployed on Coolify at `stage.rahamafresh.com`.
|
||||
This repository ingests the Tracksolid Pro API into a TimescaleDB/PostGIS database and serves it to the FleetNow / FleetOps SPAs (own repos) via the `dashboard_api` read layer. The pipeline is deployed on Coolify at `stage.rahamafresh.com`. (Grafana was retired 2026-06-10 — FleetOps now owns KPI visualisation.)
|
||||
|
||||
**Repository:** `https://repo.rahamafresh.com/kianiadee/tracksolid_timescale_grafana_prod.git`
|
||||
|
||||
|
|
@ -38,12 +38,11 @@ This repository ingests the Tracksolid Pro API into a TimescaleDB/PostGIS databa
|
|||
|
||||
| Layer | Technology |
|
||||
|---|---|
|
||||
| Ingestion | Python 3.12 — `ingest_movement_rev.py`, `ingest_events_rev.py`, `webhook_receiver_rev.py` |
|
||||
| Ingestion | Python 3.12 — `ingest_worker_rev.py` (merged movement + events poller), `webhook_receiver_rev.py` |
|
||||
| Shared utils | `ts_shared_rev.py` — token cache, DB pool, API signing, clean helpers |
|
||||
| Database | PostgreSQL 16 + TimescaleDB 2.15 + PostGIS 3 (`tracksolid_db`) |
|
||||
| Orchestration | Docker Compose on Coolify |
|
||||
| Visualisation | Grafana (provisioned via custom image) |
|
||||
| Workflow automation | n8n |
|
||||
| Visualisation | FleetOps / FleetNow SPAs (own repos) via `dashboard_api`. Grafana **removed** 2026-06-10 (was redundant — service, provisioning, and runbooks all deleted) |
|
||||
| API source | Tracksolid Pro / Jimi IoT Open Platform (`eu-open.tracksolidpro.com/route/rest`) |
|
||||
| Backup | pg_dump sidecar → rustfs S3 (`fleet-db` bucket), nightly |
|
||||
| Version control | Forgejo at `repo.rahamafresh.com` |
|
||||
|
|
@ -55,10 +54,9 @@ This repository ingests the Tracksolid Pro API into a TimescaleDB/PostGIS databa
|
|||
See `docs/CONNECTIONS.md` for the full shape. Summary:
|
||||
|
||||
- **SSH:** `ssh -i ~/.ssh/id_ed25519 kianiadee@stage.rahamafresh.com`
|
||||
- **DB name:** `tracksolid_db` · **DB user:** `postgres` (internal) · `tracksolid_owner` (app) · `grafana_ro` (read-only)
|
||||
- **DB name:** `tracksolid_db` · **DB user:** `postgres` (internal) · `tracksolid_owner` (app) · `dashboard_ro` (read-only, used by the staging bridge) · `grafana_ro` (legacy read-only role, retained but no longer used by any active service)
|
||||
- **DB schemas:** `tracksolid` (live, single source of truth) · `reporting` (map-dashboard read layer) · `infrastructure`. The legacy `tracksolid_2` schema no longer exists (migrations 02–06, 2026-04-18); the `ops` and `dwh_gold` schemas were purged 2026-06-05 (migrations 12/13) as unused.
|
||||
- **DB access:** `DATABASE_URL` points to `timescale_db:5432` (internal Docker network — not reachable locally). Use `docker exec` pattern above. See `docs/CONNECTIONS.md` for full reference.
|
||||
- **DWH target DB:** `tracksolid_dwh` at `31.97.44.246:5888` (separate PostGIS instance, public IP). Users: `dwh_owner` (bronze writes + `dwh_control`), `grafana_ro` (reads bronze/silver/gold/`dwh_control`). Always connect with `sslmode=require`. Fed by the n8n `dwh_extract` + `dwh_load_bronze` workflows — see `docs/DWH_PIPELINE.md`.
|
||||
- **Container naming:** Coolify appends a random suffix. Always resolve with:
|
||||
```bash
|
||||
docker ps --filter name=<service_name> --format "{{.Names}}" | head -1
|
||||
|
|
@ -69,7 +67,7 @@ See `docs/CONNECTIONS.md` for the full shape. Summary:
|
|||
### Map dashboards & read-API
|
||||
|
||||
The UIs read the **`dashboard_api`** service (FastAPI, `dashboard_api_rev.py`) at
|
||||
`https://fleetapi.rahamafresh.com` — the stable replacement for the retired n8n webhooks. It serves
|
||||
`https://fleetapi.rahamafresh.com` — the stable read-API for the map dashboards. It serves
|
||||
GeoJSON from the `reporting.*` functions (`fn_live_positions`, `fn_vehicle_track`, `fn_trips_for_map`)
|
||||
+ filter options, **plus the `/analytics/*` read endpoints** (fleet-summary, utilisation,
|
||||
driver-behaviour, fuel, filters) that power FleetOps. **`dashboard_api` is a STANDALONE
|
||||
|
|
@ -104,36 +102,30 @@ and the fleetops repo's `docs/webhook-auto-deploy.html`.
|
|||
|
||||
```
|
||||
ts_shared_rev.py # Shared: config, signing, DB pool, token cache, clean helpers
|
||||
ingest_movement_rev.py # GPS positions, trips, parking, track-list (high-res trail), device sync
|
||||
ingest_events_rev.py # Alarm events polling (fallback for webhook push)
|
||||
ingest_worker_rev.py # Merged poller entrypoint — runs movement + events in one process (the deployed `ingest_worker` service)
|
||||
ingest_movement_rev.py # GPS positions, trips, parking, track-list (high-res trail), device sync. main() split into startup_catchup()/register_jobs() for reuse; standalone entrypoint still works
|
||||
ingest_events_rev.py # Alarm events polling (fallback for webhook push). Same startup_catchup()/register_jobs() split
|
||||
webhook_receiver_rev.py # FastAPI push receiver: /pushobd /pushevent /pushtripreport etc.
|
||||
sync_driver_audit.py # One-shot: API↔DB driver/IMEI gap report + full upsert
|
||||
import_drivers_csv.py # One-shot: populate 144 X3/JC400P devices from CSV (--apply to commit)
|
||||
run_migrations.py # Applies SQL migrations in order at container startup
|
||||
docker-compose.yaml # Services: timescale_db, ingest_movement, ingest_events,
|
||||
# webhook_receiver, grafana
|
||||
grafana/ # Grafana provisioning (baked into image)
|
||||
n8n-workflows/ # n8n workflow exports (incl. dwh_extract, dwh_load_bronze)
|
||||
docker-compose.yaml # Services (4 app + db, was 7): timescale_db, ingest_worker,
|
||||
# webhook_receiver, dashboard_api, db_backup.
|
||||
# pgbouncer + grafana REMOVED 2026-06-10.
|
||||
docs/ # Reference docs (connections, API, KPIs, project context)
|
||||
docs/PLATFORM_OVERVIEW.html # Current-state platform reference (architecture, deploy, read-API,
|
||||
# full DB schema, Grafana panels) — open in a browser. Post n8n→fleetapi.
|
||||
docs/DWH_PIPELINE.md # DWH pipeline operations runbook (setup, troubleshooting)
|
||||
# full DB schema) — open in a browser.
|
||||
docs/OSM_POI_EXPORT.md # Runbook: OSM .pbf → POI GeoJSON → FleetNow map layer (Shell stations)
|
||||
docs/superpowers/ # Pitch specs and implementation plans (not deployed code)
|
||||
scripts/export_osm_pois.py # OSM .pbf → GeoJSON+CSV POI exporter (amenity/brand filter); see OSM_POI_EXPORT.md
|
||||
dwh/ # DWH migrations for tracksolid_dwh@31.97.44.246:5888
|
||||
# 260423_dwh_ddl_v1.sql — bronze/silver/gold schemas + roles
|
||||
# 261001_dwh_control.sql — watermarks + run log
|
||||
# 261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
|
||||
# 261003_dwh_roles.sql — role contract assertion
|
||||
# 261004_dwh_observability_views.sql — freshness/failure views
|
||||
migrations/ # Numbered SQL migrations 02–18, applied in order by run_migrations.py
|
||||
migrations/ # Numbered SQL migrations 02–19, applied in order by run_migrations.py
|
||||
# 02 full schema · 03 webhook · 04 distance fix · 05 enhancements
|
||||
# 06 ops/analytics · 07 views · 08 config · 09 trips enrichment
|
||||
# 10_driver_clock_views.sql · 10_pgbouncer_auth.sql · 11 reporting
|
||||
# 12 drop ops schema · 13 drop dwh_gold schema (both 2026-06-05)
|
||||
# 14 fleet segment · 15 map exclude cc · 16 live feed vehicle_type
|
||||
# 17 reporting.v_fuel_daily (FleetOps) · 18 grant reporting.* to grafana_ro
|
||||
# 19 reporting.v_ingest_health (pipeline freshness; replaces Grafana panels)
|
||||
deploy_dashboard_api_staging.sh # Staging dashboard_api bridge (8891, fleetapi.fivetitude.com); see STAGING_FLEETOPS_ARCHITECTURE.md
|
||||
scripts/dashboard_ro_role.sql + bootstrap_dashboard_ro.sh # Dedicated read-only DB role for the staging bridge
|
||||
Dockerfile # Custom image for ingest/webhook containers
|
||||
|
|
@ -141,7 +133,7 @@ pyproject.toml # Python project + uv dependency spec
|
|||
backup/ # pg_dump sidecar scripts and config
|
||||
data/ # Source CSVs (FS Logistics 144-device list, FSG vehicles)
|
||||
legacy/ # Superseded pre-_rev scripts + old pipeline notes (NOT deployed)
|
||||
docs/manuals/ # OPERATIONS_MANUAL, grafana + DWH manuals, docker commands, DB manual
|
||||
docs/manuals/ # OPERATIONS_MANUAL, docker commands, DB manual
|
||||
docs/reference/ # 01_BusinessAnalytics.md (SQL library — read before writing queries),
|
||||
# tracksolidApiDocumentation.md, 260507_pgbouncer_deployment.md
|
||||
docs/reports/ # Baseline reports, audit output, improvement reviews
|
||||
|
|
@ -155,7 +147,7 @@ docs/reports/ # Baseline reports, audit output, improvement review
|
|||
tracksolid.devices -- Device / driver / vehicle registry (63 rows; 0 driver_name populated)
|
||||
-- IMEI mix: 353549* AT4 (23), 862798* X3/JC400P (23), 865135* X3/JC400P (10), 359857* (7)
|
||||
-- Full CSV (144 devices) not yet imported — run import_drivers_csv.py --apply
|
||||
tracksolid.live_positions -- Current fix per IMEI (19 rows; refreshed every 60s by ingest_movement)
|
||||
tracksolid.live_positions -- Current fix per IMEI (19 rows; refreshed every 60s by ingest_worker / movement pipeline)
|
||||
tracksolid.position_history -- All GPS fixes (hypertable, partitioned by gps_time). ~519 rows (308 track_list + 211 poll).
|
||||
-- pg_stat_user_tables shows 0 for hypertables — always COUNT(*) directly.
|
||||
-- source: 'poll' (60s sweep) | 'track_list' (30m high-res)
|
||||
|
|
@ -165,13 +157,12 @@ tracksolid.alarms -- Alarm events (alarm_type, alarm_name, alarm_time
|
|||
tracksolid.obd_readings -- OBD diagnostics (push only, awaiting webhook registration)
|
||||
tracksolid.device_events -- Power on/off tamper events (push only)
|
||||
tracksolid.ingestion_log -- API call audit trail — 875 runs / 24h, 0 failures at last check (2026-04-19)
|
||||
tracksolid.schema_migrations -- Applied migrations 02–18 (17 reporting.v_fuel_daily, 18 grants applied 2026-06-10)
|
||||
tracksolid.schema_migrations -- Applied migrations 02–19 (19 reporting.v_ingest_health applied 2026-06-10)
|
||||
-- PURGED 2026-06-05 (migrations 12 + 13): the dormant `ops` schema (tickets, service_log,
|
||||
-- odometer_readings, cost_rates, kpi_targets, vw_service_forecast), tracksolid.dispatch_log,
|
||||
-- and the `dwh_gold` schema (dim_vehicles, fact_daily_fleet_metrics, refresh_daily_metrics).
|
||||
-- Those workshop/dispatch/SLA/utilisation features were never implemented. Do NOT reintroduce
|
||||
-- references to ops.* or dwh_gold.* — they no longer exist. (The separate tracksolid_dwh DB
|
||||
-- at 31.97.44.246:5888 is unrelated and untouched.)
|
||||
-- references to ops.* or dwh_gold.* — they no longer exist.
|
||||
```
|
||||
|
||||
Full DDL: `02_tracksolid_full_schema_rev.sql` + migrations `03`–`06`.
|
||||
|
|
@ -192,25 +183,6 @@ tracksolid.v_alarms_daily -- §7 Panel 7 alarm frequency
|
|||
|
||||
All views carry a `COMMENT ON VIEW` referencing their spec — `\d+ tracksolid.v_*` shows the provenance.
|
||||
|
||||
**DWH bronze layer (separate DB `tracksolid_dwh`)** — populated by the n8n `dwh_extract` + `dwh_load_bronze` workflows. Operational details in `docs/DWH_PIPELINE.md`.
|
||||
|
||||
```sql
|
||||
-- bronze schema mirrors tracksolid.* (16 tables, DDL in dwh/260423_dwh_ddl_v1.sql)
|
||||
bronze.devices, bronze.live_positions -- snapshot tables (TRUNCATE + reload)
|
||||
bronze.position_history, bronze.trips,
|
||||
bronze.alarms, bronze.parking_events,
|
||||
bronze.device_events, bronze.ingestion_log -- incremental (watermark + ON CONFLICT DO NOTHING)
|
||||
-- Schema drift: bronze.trips.distance_km vs source tracksolid.trips.distance_m
|
||||
-- Extract SQL divides by 1000. Cross-ref FIX-M16.
|
||||
|
||||
-- dwh_control schema tracks pipeline state + observability
|
||||
dwh_control.extract_watermarks -- one row per incremental table
|
||||
dwh_control.extract_runs -- per-run audit log (status lifecycle)
|
||||
dwh_control.v_table_freshness -- Grafana: load lag per table
|
||||
dwh_control.v_recent_failures -- Grafana: failures in last 24h
|
||||
dwh_control.v_watermark_lag -- Grafana: extract vs. load lag per table
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. API Critical Facts
|
||||
|
|
@ -290,7 +262,7 @@ Latest full snapshot: `docs/reports/260412_baseline_report.md`
|
|||
|
||||
| Priority | Item |
|
||||
|---|---|
|
||||
| HIGH | **Redeploy the Grafana service in Coolify** to apply `daily_operations_dashboard.json` — 5 panel areas (In-flight SLA, Idle Cost, Utilisation Heatmap, Row 7 Field-Service SLAs) that queried the now-dropped `v_sla_inflight`/`v_utilisation_daily` were removed. The DB views are already gone, so **live Grafana shows errors on those panels until the redeploy** (purge commit `8c5a43f`, 2026-06-05). |
|
||||
| LOW | Pipeline freshness is surfaced by `reporting.v_ingest_health` (migration 19) via dashboard_api `GET /health/ingest` — wire it into a FleetOps panel. |
|
||||
| HIGH | Run `import_drivers_csv.py --apply` — 144 X3/JC400P devices with names + plates waiting |
|
||||
| HIGH | Register webhooks: `/pushoil` `/pushtem` `/pushlbs` (auto-register on push now done — commit 257643c) |
|
||||
| HIGH | Investigate X3-63282 in Kampala — legitimate or unauthorised? |
|
||||
|
|
@ -298,5 +270,3 @@ Latest full snapshot: `docs/reports/260412_baseline_report.md`
|
|||
| MEDIUM | Investigate 44 silent devices (only 19 of 63 reporting) — SIM installed? Activated? |
|
||||
| MEDIUM | Co-develop client KPI framework (see `docs/KPI_FRAMEWORK.md`) |
|
||||
| LOW | Populate geofences — depot boundaries, city zones |
|
||||
| HIGH | Deploy DWH bronze pipeline: apply `dwh/26100{1,2,3,4}.sql` to `tracksolid_dwh`, import + wire the two n8n workflows, verify first run via `dwh_control.v_table_freshness`. Runbook: `docs/DWH_PIPELINE.md` |
|
||||
| MEDIUM | Rotate `dwh_owner` / `grafana_ro` passwords on `tracksolid_dwh` — plaintext in `dwh/260423_dwh_ddl_v1.sql` is a pre-existing flaw to clean up separately |
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
db_audit/run_audit.py — Fireside Communications Fleet Telemetry DB Audit
|
||||
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
|
||||
Runs six health checks against the production TimescaleDB.
|
||||
Writes results to tracksolid.health_checks for Grafana monitoring.
|
||||
Writes results to tracksolid.health_checks for monitoring/auditing.
|
||||
Exits with code 1 if any critical finding is detected.
|
||||
|
||||
Usage:
|
||||
|
|
|
|||
|
|
@ -8,6 +8,6 @@ CREATE TABLE IF NOT EXISTS tracksolid.health_checks (
|
|||
row_count INT
|
||||
);
|
||||
|
||||
-- Index for Grafana time-range queries
|
||||
-- Index for time-range queries
|
||||
CREATE INDEX IF NOT EXISTS health_checks_checked_at_idx
|
||||
ON tracksolid.health_checks (checked_at DESC);
|
||||
|
|
|
|||
|
|
@ -13,9 +13,9 @@
|
|||
# 4. Refresher OFF VTRIPS_REFRESH_INTERVAL_S=0 — prod owns the v_trips refresh;
|
||||
# a read-only instance must never attempt REFRESH.
|
||||
#
|
||||
# Staging reads the SAME production DB (over the internal Docker network) as
|
||||
# grafana_ro, so it is physically incapable of writing. See
|
||||
# docs/STAGING_FLEETOPS_ARCHITECTURE.md §6.
|
||||
# Staging reads the SAME production DB (over the internal Docker network) as the
|
||||
# dedicated read-only dashboard_ro role, so it is physically incapable of writing.
|
||||
# See docs/STAGING_FLEETOPS_ARCHITECTURE.md §6.
|
||||
#
|
||||
# Like prod, this is a STANDALONE bridge container (NOT Coolify-managed): it
|
||||
# reuses the webhook_receiver image + app network, bind-mounts the WIP API file,
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ ssh -i ~/.ssh/id_ed25519 kianiadee@stage.rahamafresh.com
|
|||
| Host (internal) | `timescale_db` (Docker service name) |
|
||||
| Port | `5432` |
|
||||
| App user | `tracksolid_owner` |
|
||||
| Read-only user | `grafana_ro` |
|
||||
| Read-only user | `dashboard_ro` (used by the staging bridge) · `grafana_ro` (legacy, retained but unused) |
|
||||
| Superuser | `postgres` |
|
||||
|
||||
### `.env` variable names
|
||||
|
|
@ -36,7 +36,6 @@ POSTGRES_DB=tracksolid_db
|
|||
POSTGRES_USER=...
|
||||
POSTGRES_PASSWORD=...
|
||||
DATABASE_URL=postgresql://tracksolid_owner:<password>@timescale_db:5432/tracksolid_db
|
||||
GRAFANA_DB_RO_PASSWORD=...
|
||||
```
|
||||
|
||||
### Run a query from host
|
||||
|
|
@ -77,9 +76,9 @@ docker ps --filter name=<service_name> --format "{{.Names}}" | head -1
|
|||
|
||||
# Examples
|
||||
docker ps --filter name=timescale_db --format "{{.Names}}" | head -1
|
||||
docker ps --filter name=ingest_movement --format "{{.Names}}" | head -1
|
||||
docker ps --filter name=ingest_worker --format "{{.Names}}" | head -1
|
||||
docker ps --filter name=webhook_receiver --format "{{.Names}}" | head -1
|
||||
docker ps --filter name=grafana --format "{{.Names}}" | head -1
|
||||
docker ps --filter name=dashboard_api --format "{{.Names}}" | head -1
|
||||
```
|
||||
|
||||
Current suffix (may change on redeploy): `bo3nov2ija7g8wn9b1g2paxs-19xxxxxxxxxx`
|
||||
|
|
@ -93,19 +92,3 @@ Host: https://repo.rahamafresh.com
|
|||
Repo: kianiadee/tracksolid_timescale_grafana_prod
|
||||
Remote: https://repo.rahamafresh.com/kianiadee/tracksolid_timescale_grafana_prod.git
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Grafana
|
||||
|
||||
- Deployed as Docker service `grafana`
|
||||
- Provisioning baked into image (datasources + dashboards via `grafana/Dockerfile`)
|
||||
- Admin password: `GF_SECURITY_ADMIN_PASSWORD` from `.env`
|
||||
- Default dashboard: NOC Fleet Dashboard
|
||||
|
||||
---
|
||||
|
||||
## n8n
|
||||
|
||||
- Deployed as separate Coolify service (`n8n-usoksgg8o40044g0cw08s8wc`)
|
||||
- Workflows exported to `n8n-workflows/`
|
||||
|
|
|
|||
|
|
@ -3,12 +3,15 @@
|
|||
**Scope:** the *live* fleet pipeline only (`tracksolid` + `reporting`). The `ops` and
|
||||
`dwh_gold` schemas were **purged on 2026-06-05** (migrations 12/13) — those workshop /
|
||||
dispatch / SLA / utilisation features were never implemented. They are shown below only as
|
||||
a struck-out "removed" footnote for historical context. (The *separate* `tracksolid_dwh`
|
||||
server at 31.97.44.246:5888 is unrelated and was not touched.)
|
||||
a struck-out "removed" footnote for historical context.
|
||||
|
||||
> **Update 2026-06-10:** Grafana, pgbouncer, and the separate DWH pipeline were removed.
|
||||
> The `tracksolid.v_*` analytics views still exist but no longer have an active consumer
|
||||
> (Grafana was their only reader). The two pollers were merged into one `ingest_worker`.
|
||||
|
||||
**Verified against prod 2026-06-05** (TimescaleDB hypertable + continuous-aggregate
|
||||
catalog, `pg_depend` view graph, ingestion `INSERT` targets, `dashboard_api` queries,
|
||||
Grafana panel SQL). Key facts that surprised the docs:
|
||||
catalog, `pg_depend` view graph, ingestion `INSERT` targets, `dashboard_api` queries).
|
||||
Key facts that surprised the docs:
|
||||
|
||||
- Only `position_history` (and the empty/planned `heartbeats`, `fuel_readings`,
|
||||
`temperature_readings`) are **hypertables**. `trips` and `alarms` are **plain tables**.
|
||||
|
|
@ -64,7 +67,7 @@ flowchart TD
|
|||
VT --> FILT & SUMM
|
||||
LP --> VLP
|
||||
|
||||
subgraph L3R["L3 · Grafana views — tracksolid.* (read base tables directly)"]
|
||||
subgraph L3R["L3 · Analytics views — tracksolid.* (read base tables directly · UNCONSUMED since Grafana removed 2026-06-10)"]
|
||||
GV["v_fleet_today · v_fleet_status · v_active_dispatch_map<br/>v_currently_idle · v_alarms_daily · v_fleet_km_daily<br/>v_ingestion_health · v_vehicles_not_moved_today<br/>v_driver_aggregates_daily · v_fleet_trace · v_driver_clock_*"]
|
||||
end
|
||||
LP --> GV
|
||||
|
|
@ -88,16 +91,15 @@ flowchart TD
|
|||
subgraph L5["L5 · Consumers"]
|
||||
DAPI["dashboard_api_rev.py<br/>FastAPI :8890 · 2 workers"]
|
||||
SPA["SPAs: liveposition.* · fleetintelligence.*<br/>(rustfs / S3 single-file maps)"]
|
||||
GRAF["Grafana"]
|
||||
end
|
||||
FLP --> DAPI
|
||||
FVT --> DAPI
|
||||
FTM --> DAPI
|
||||
FILT --> DAPI
|
||||
DAPI -->|"HTTPS · fleetapi.rahamafresh.com"| SPA
|
||||
GV --> GRAF
|
||||
|
||||
CAGG -.->|no consumer yet| NONE(["⚠ unconsumed — no panel / API reads it"])
|
||||
GV -.->|Grafana removed 2026-06-10| NONE
|
||||
|
||||
subgraph PARK["REMOVED 2026-06-05 — purged via migrations 12 / 13 (never implemented)"]
|
||||
GONE["ops.* (tickets · dispatch_log · service_log · odometer_readings · cost_rates · kpi_targets · vw_service_forecast)<br/>dwh_gold.* (dim_vehicles · fact_daily_fleet_metrics · refresh_daily_metrics)<br/>tracksolid.v_sla_inflight · tracksolid.v_utilisation_daily + their Grafana panels"]
|
||||
|
|
@ -148,10 +150,10 @@ flowchart TD
|
|||
╚══════╤══════════════════════════╤══════════════════════════════════╤════════════════════════╝
|
||||
│ │ │
|
||||
│ TimescaleDB │ matview refresh │ direct reads
|
||||
│ cont-agg policy │ (in-app loop) │ (Grafana SQL)
|
||||
│ cont-agg policy │ (in-app loop) │ (ad-hoc SQL)
|
||||
▼ ▼ ▼
|
||||
╔══════════════════╗ ╔════════════════════════════════╗ ╔══════════════════════════════════╗
|
||||
║ L2 · AGGREGATION ║ ║ L2 · AGGREGATION (matview) ║ ║ L3 · GRAFANA VIEWS (tracksolid.*) ║
|
||||
║ L2 · AGGREGATION ║ ║ L2 · AGGREGATION (matview) ║ ║ L3 · ANALYTICS VIEWS — UNCONSUMED ║
|
||||
║ ║ ║ ║ ║ read base tables directly ║
|
||||
║ v_mileage_daily ║ ║ reporting.v_trips [MATVIEW] ║ ║ ║
|
||||
║ _cagg ║ ║ ◀── trips + devices ║ ║ v_fleet_today v_fleet_status ║
|
||||
|
|
@ -200,7 +202,7 @@ flowchart TD
|
|||
║ SPAs: liveposition.* · fleetintelligence.* ║
|
||||
║ (rustfs/S3 single-file maps) ║
|
||||
║ ║
|
||||
║ Grafana ──SQL──▶ tracksolid.v_* (L3 right) ║
|
||||
║Grafana removed 2026-06-10 — v_* now unconsumed║
|
||||
╚═══════════════════════════════════════════════╝
|
||||
|
||||
|
||||
|
|
@ -225,16 +227,19 @@ flowchart TD
|
|||
loop — the FIX-D02 self-refresher that replaced the dead n8n job).
|
||||
- **L4 functions are the only thing the API calls.** SPAs never touch tables directly:
|
||||
SPA → `dashboard_api` → `fn_*` → (matview / tables).
|
||||
- **Grafana** takes the right-hand path, reading the `tracksolid.v_*` analytics views
|
||||
straight off L1.
|
||||
- **The `tracksolid.v_*` analytics views (L3 right) are now unconsumed.** Grafana was their
|
||||
only reader and was removed 2026-06-10; the views still exist (migration 07) but nothing
|
||||
reads them. Candidates for removal or for wiring into a FleetOps panel.
|
||||
|
||||
## Caveats / housekeeping notes
|
||||
|
||||
1. **`v_mileage_daily_cagg` has no consumer** — nothing in the API or Grafana reads it. It
|
||||
1. **`v_mileage_daily_cagg` has no consumer** — nothing in the API reads it. It
|
||||
is a live continuous aggregate maintaining itself for nobody. Candidate for removal or
|
||||
wiring into a panel.
|
||||
2. **`ops` and `dwh_gold` were purged** (2026-06-05, migrations 12/13) along with
|
||||
`v_utilisation_daily`, `v_sla_inflight`, and their Grafana panels — the features were
|
||||
never implemented.
|
||||
3. See the companion housekeeping audit (2026-06-05) for the full unused-object list; the
|
||||
2. **The `tracksolid.v_*` analytics views are unconsumed** since Grafana's removal
|
||||
(2026-06-10) — same housekeeping candidate as the cagg above.
|
||||
3. **`ops` and `dwh_gold` were purged** (2026-06-05, migrations 12/13) along with
|
||||
`v_utilisation_daily`, `v_sla_inflight`, and their (then-live) Grafana panels — the
|
||||
features were never implemented.
|
||||
4. See the companion housekeeping audit (2026-06-05) for the full unused-object list; the
|
||||
only clean ad-hoc drop is `public.trips_viz_v1`.
|
||||
|
|
|
|||
|
|
@ -1,252 +0,0 @@
|
|||
# DWH Pipeline — Operations Runbook
|
||||
|
||||
**Pipeline:** n8n extract + load into `tracksolid_dwh` bronze schema
|
||||
**Design spec:** `docs/superpowers/specs/2026-04-24-n8n-dwh-bronze-pipeline-design.md`
|
||||
**Implementation plan:** `docs/superpowers/plans/2026-04-24-n8n-dwh-bronze-pipeline.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. What This Pipeline Does
|
||||
|
||||
Every ~3 hours during active hours (7 runs/day, 05:00–23:00 EAT), n8n extracts 8 tables from the production `tracksolid_db` (Coolify internal network), writes each as a timestamped CSV to rustfs, then loads each CSV into the `bronze` schema on `tracksolid_dwh` (31.97.44.246:5888). Rustfs CSVs are moved to `dwh/processed/` after a successful load — never deleted.
|
||||
|
||||
Two n8n workflows:
|
||||
- **`dwh_extract`** — cron-triggered, iterates tables in sequence, writes CSVs, calls `dwh_load_bronze` per table.
|
||||
- **`dwh_load_bronze`** — triggered per-table by `dwh_extract`, loads one CSV inside a single transaction (insert → update watermark → update run log → move CSV).
|
||||
|
||||
---
|
||||
|
||||
## 2. Key Locations
|
||||
|
||||
| What | Where |
|
||||
|---|---|
|
||||
| Source DB | `tracksolid_db` on Coolify (internal Docker network, `timescale_db:5432`) |
|
||||
| Target DB | `tracksolid_dwh` at `31.97.44.246:5888` |
|
||||
| Rustfs bucket | `fleet-db` (same bucket used by pg_dump backups) |
|
||||
| Active CSVs | `s3://fleet-db/dwh/exports/{table}/{YYYYMMDD_HHMM}_EAT.csv` |
|
||||
| Processed CSVs | `s3://fleet-db/dwh/processed/{table}/{YYYYMMDD_HHMM}_EAT.csv` |
|
||||
| Control schema | `dwh_control` in `tracksolid_dwh` |
|
||||
| Migrations | `dwh/26*.sql` applied in numeric order |
|
||||
|
||||
---
|
||||
|
||||
## 3. First-Time Setup
|
||||
|
||||
Apply migrations to `tracksolid_dwh` in numeric order, as a superuser (not `dwh_owner`):
|
||||
|
||||
```bash
|
||||
PSQL="psql 'postgres://postgres:***@31.97.44.246:5888/tracksolid_dwh?sslmode=require'"
|
||||
|
||||
$PSQL -f dwh/260423_dwh_ddl_v1.sql # Bronze DDL, roles, schemas
|
||||
$PSQL -f dwh/261001_dwh_control.sql # Watermarks + run log
|
||||
$PSQL -f dwh/261002_bronze_constraints_audit.sql # Assertion: ON CONFLICT keys exist
|
||||
$PSQL -f dwh/261003_dwh_roles.sql # Assertion: roles + grants present
|
||||
$PSQL -f dwh/261004_dwh_observability_views.sql # Freshness/failure views
|
||||
```
|
||||
|
||||
Each migration is idempotent. Audit files (261002, 261003) raise an exception with a bullet list of what is missing if the contract is broken — re-apply the relevant predecessor file and try again.
|
||||
|
||||
### Rustfs prefixes
|
||||
|
||||
```bash
|
||||
aws --endpoint ${RUSTFS_ENDPOINT} s3api put-object \
|
||||
--bucket fleet-db --key dwh/exports/
|
||||
aws --endpoint ${RUSTFS_ENDPOINT} s3api put-object \
|
||||
--bucket fleet-db --key dwh/processed/
|
||||
```
|
||||
|
||||
### n8n credentials
|
||||
|
||||
Three credentials, all configured in the n8n UI before importing workflows:
|
||||
|
||||
| Credential | Target | User | Notes |
|
||||
|---|---|---|---|
|
||||
| `tracksolid_source` | Coolify internal → `tracksolid_db` | `grafana_ro` | Read-only; no `sslmode` needed on internal network |
|
||||
| `tracksolid_dwh_target` | `31.97.44.246:5888` → `tracksolid_dwh` | `dwh_owner` | **Must set `sslmode=require`** — public IP |
|
||||
| `rustfs_s3` | `${RUSTFS_ENDPOINT}` | `${RUSTFS_ACCESS_KEY}` | Same creds as pg_dump backup sidecar |
|
||||
|
||||
Test each credential via the n8n "Test connection" button before enabling the cron schedule.
|
||||
|
||||
---
|
||||
|
||||
## 4. Schedule
|
||||
|
||||
n8n Schedule node, Africa/Nairobi TZ: `0 5,8,11,14,17,20,23 * * *`
|
||||
|
||||
- 7 runs/day at 05:00, 08:00, 11:00, 14:00, 17:00, 20:00, 23:00 EAT
|
||||
- Overnight gap (23:00 → 05:00 = 6h) by design — device traffic minimal
|
||||
- First-of-day run carries the overnight backlog (watermark picks up where 23:00 left off)
|
||||
|
||||
---
|
||||
|
||||
## 5. What Each Table Does on Every Run
|
||||
|
||||
### Snapshot tables (TRUNCATE + full reload)
|
||||
|
||||
`bronze.devices`, `bronze.live_positions` — small state tables, "current state" semantics. Full replace every run.
|
||||
|
||||
### Incremental tables (watermark + append-with-dedup)
|
||||
|
||||
| Bronze table | Source watermark column | ON CONFLICT target |
|
||||
|---|---|---|
|
||||
| `position_history` | `recorded_at` (DB insertion time) | `(imei, gps_time)` |
|
||||
| `trips` | `updated_at` | `id` |
|
||||
| `alarms` | `updated_at` | `id` |
|
||||
| `parking_events` | `updated_at` | `id` |
|
||||
| `device_events` | `created_at` | `id` |
|
||||
| `ingestion_log` | `run_at` | `id` |
|
||||
|
||||
Watermark bounds are closed upper: `WHERE <col> > last_extracted_at AND <col> <= :run_started_at`.
|
||||
|
||||
### Schema drift to handle in extract SQL
|
||||
|
||||
- **`trips.distance_m` → `bronze.trips.distance_km`**: source stores metres, bronze expects km. Extract SQL: `SELECT ..., distance_m/1000.0 AS distance_km, ...`. Cross-reference: FIX-M16 in `CLAUDE.md`.
|
||||
|
||||
### PostGIS geometry round-trip
|
||||
|
||||
All six geometry columns (`live_positions`, `position_history`, `trips.start_geom`, `trips.end_geom`, `parking_events`, `alarms`) use EWKT serialisation:
|
||||
|
||||
```sql
|
||||
-- Extract
|
||||
SELECT ..., CASE WHEN geom IS NULL THEN NULL ELSE ST_AsEWKT(geom) END AS geom_ewkt FROM ...;
|
||||
|
||||
-- Load
|
||||
INSERT INTO bronze... (..., geom) VALUES (..., ST_GeomFromEWKT(:geom_ewkt));
|
||||
```
|
||||
|
||||
SRID 4326 is preserved inline; no separate SRID step required.
|
||||
|
||||
---
|
||||
|
||||
## 6. Verifying a Healthy Run
|
||||
|
||||
### Immediate sanity checks (after any scheduled run)
|
||||
|
||||
```sql
|
||||
-- Any failures in the last hour?
|
||||
SELECT * FROM dwh_control.v_recent_failures WHERE run_started_at > NOW() - INTERVAL '1 hour';
|
||||
|
||||
-- All tables loaded in last 4h?
|
||||
SELECT * FROM dwh_control.v_table_freshness WHERE lag > INTERVAL '4 hours';
|
||||
|
||||
-- Watermarks advancing?
|
||||
SELECT * FROM dwh_control.v_watermark_lag ORDER BY extract_lag DESC;
|
||||
```
|
||||
|
||||
### Row-count parity (spot check weekly)
|
||||
|
||||
```sql
|
||||
-- Source
|
||||
SELECT COUNT(*) FROM tracksolid.position_history;
|
||||
-- Target
|
||||
SELECT COUNT(*) FROM bronze.position_history;
|
||||
```
|
||||
|
||||
Numbers should match ± rows inserted between the two queries. Persistent gap > 1% → investigate CSV parse errors or a dropped batch.
|
||||
|
||||
### Geometry round-trip
|
||||
|
||||
```sql
|
||||
SELECT ST_AsText(geom) FROM bronze.position_history WHERE geom IS NOT NULL LIMIT 5;
|
||||
-- Should return valid POINT(lng lat), not NULL or garbage.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Troubleshooting
|
||||
|
||||
### "A table is stale (`v_table_freshness` shows lag > 4h)"
|
||||
|
||||
1. Check `v_recent_failures` for that table. If a row exists, read `error_message`.
|
||||
2. If `status='loading'` in `extract_runs` for that table, a load is in progress — wait for the next cron tick. If it stays `loading` across two ticks, the n8n executor crashed mid-transaction; the DB rolled back, and the next run will retry naturally.
|
||||
3. If no failure row and no in-progress row, the extract workflow never fired — check n8n execution logs for the cron trigger.
|
||||
|
||||
### "A CSV is stuck in `dwh/exports/`"
|
||||
|
||||
The load failed after upload. The next scheduled run will re-process it (the watermark did not advance, so the extract SQL returns the same window). Safe to leave. If multiple days of CSVs pile up, the load workflow has a persistent bug — open n8n execution logs for the specific `run_id` in `extract_runs`.
|
||||
|
||||
### "Row counts diverge more than ~1%"
|
||||
|
||||
Usually one of:
|
||||
- A retry window overlapped the PK range and some rows lost the race with a concurrent source-side write. Re-trigger the extract for that window manually (see §8).
|
||||
- CSV parse error silently dropped a row. Check `extract_runs.rows_extracted` vs. `rows_loaded` — if they differ, the loader found malformed CSV.
|
||||
|
||||
### "Geometry loaded as NULL"
|
||||
|
||||
EWKT serialisation broke on the extract side. Verify the source query has the `CASE WHEN geom IS NULL` guard — without it, `ST_AsEWKT(NULL)` returns `NULL` correctly but an empty geometry returns `'GEOMETRYCOLLECTION EMPTY'` which `ST_GeomFromEWKT` rejects.
|
||||
|
||||
### "`bronze.trips.distance_km` values are 1000× too large"
|
||||
|
||||
The extract query is missing `/1000.0` on the source `distance_m` column. See §5 "Schema drift".
|
||||
|
||||
---
|
||||
|
||||
## 8. Manual Re-Run
|
||||
|
||||
### Re-run a single table for the current window
|
||||
|
||||
1. Open n8n, go to `dwh_extract` workflow.
|
||||
2. Locate the branch for that table.
|
||||
3. Click **Execute Workflow** → selects that table only.
|
||||
4. Confirm a new row appears in `dwh_control.extract_runs` with `status='loaded'`.
|
||||
|
||||
### Back-fill a historical window
|
||||
|
||||
The extract workflow respects the watermark; to re-extract a window, rewind the watermark:
|
||||
|
||||
```sql
|
||||
-- Rewind position_history to 24h ago
|
||||
UPDATE dwh_control.extract_watermarks
|
||||
SET last_extracted_at = NOW() - INTERVAL '24 hours'
|
||||
WHERE table_name = 'position_history';
|
||||
```
|
||||
|
||||
Next scheduled run will re-extract the gap. Loads are idempotent (`ON CONFLICT DO NOTHING` on the PK), so duplicate rows are filtered at the bronze boundary.
|
||||
|
||||
### Full reseed (nuclear option)
|
||||
|
||||
```sql
|
||||
-- Restart position_history from the beginning
|
||||
UPDATE dwh_control.extract_watermarks
|
||||
SET last_extracted_at = '2026-01-01T00:00:00Z'
|
||||
WHERE table_name = 'position_history';
|
||||
```
|
||||
|
||||
The first post-reseed run will produce one very large CSV (~all history). The rustfs `exports/` prefix will briefly hold a multi-GB object. Expected; it moves to `processed/` on success.
|
||||
|
||||
---
|
||||
|
||||
## 9. Credential Rotation
|
||||
|
||||
`260423_dwh_ddl_v1.sql` commits role passwords in plaintext — a pre-existing flaw to be cleaned up separately.
|
||||
|
||||
To rotate:
|
||||
|
||||
```sql
|
||||
-- As superuser on tracksolid_dwh:
|
||||
ALTER ROLE dwh_owner PASSWORD '<new secret>';
|
||||
ALTER ROLE grafana_ro PASSWORD '<new secret>';
|
||||
```
|
||||
|
||||
Then update the matching n8n credential and Grafana datasource. Never commit the new password — store in `.env` if needed for scripts, or keep exclusively inside n8n/Grafana credential stores.
|
||||
|
||||
---
|
||||
|
||||
## 10. Known Quirks
|
||||
|
||||
| Quirk | Source | Handling |
|
||||
|---|---|---|
|
||||
| `trips.distance_m` → `bronze.trips.distance_km` | Schema drift between source and bronze | Divide in extract SQL (§5) |
|
||||
| Hypertable row counts read 0 in `pg_stat_user_tables` | TimescaleDB quirk | Always `SELECT COUNT(*)` directly |
|
||||
| `parking_events` can be empty for days | Endpoint returns empty; not a failure | Zero rows loaded is a valid run outcome |
|
||||
| First run of each day larger | Overnight backlog | Expected; plan watermark design |
|
||||
| `last_extracted_at` default `2026-01-01` | Seed value from 261001 | First run on a new table back-fills all history |
|
||||
|
||||
---
|
||||
|
||||
## 11. Out of Scope (follow-up)
|
||||
|
||||
- Silver/gold transformations — `silver` and `gold` schemas exist but contain no views.
|
||||
- Grafana dashboard panels — these views are the data source; panels TBD.
|
||||
- OBD / fault codes / fuel / temperature / LBS / heartbeats — webhooks not yet registered; add a watermark row + extract branch when they start reporting.
|
||||
- Bronze schema evolution tooling — additive changes via numbered migrations is fine for one pipeline; revisit if scope grows.
|
||||
|
|
@ -9,13 +9,13 @@
|
|||
## How to Use This Document
|
||||
|
||||
1. **Proposed** — KPI defined, not yet validated with client
|
||||
2. **Active** — Client confirmed this matters; query written; Grafana panel exists or is in progress
|
||||
2. **Active** — Client confirmed this matters; query written; FleetOps panel exists or is in progress
|
||||
3. **Baseline set** — Enough historical data exists to set a meaningful target
|
||||
4. **Retired** — No longer tracked (document reason)
|
||||
|
||||
Each active KPI should link to:
|
||||
- The SQL query (or reference to `01_BusinessAnalytics.md`)
|
||||
- The Grafana panel name/dashboard
|
||||
- The FleetOps panel name/dashboard
|
||||
- The refresh frequency
|
||||
- The person who reviews it
|
||||
|
||||
|
|
@ -25,7 +25,7 @@ Each active KPI should link to:
|
|||
|
||||
### Fleet Utilisation
|
||||
|
||||
| KPI | Status | SQL ref | Grafana panel | Reviewed by | Cadence |
|
||||
| KPI | Status | SQL ref | FleetOps panel | Reviewed by | Cadence |
|
||||
|---|---|---|---|---|---|
|
||||
| Utilisation rate (%) | Proposed | `01_BusinessAnalytics.md §2.1` | — | — | Daily |
|
||||
| Idle time % of shift | Proposed | `01_BusinessAnalytics.md §2.2` | — | — | Daily |
|
||||
|
|
@ -35,7 +35,7 @@ Each active KPI should link to:
|
|||
|
||||
### Technician Productivity *(requires job system integration)*
|
||||
|
||||
| KPI | Status | SQL ref | Grafana panel | Reviewed by | Cadence |
|
||||
| KPI | Status | SQL ref | FleetOps panel | Reviewed by | Cadence |
|
||||
|---|---|---|---|---|---|
|
||||
| Jobs completed per tech per day | Proposed | TBD | — | — | Daily |
|
||||
| First-time fix rate | Proposed | TBD | — | — | Weekly |
|
||||
|
|
@ -45,7 +45,7 @@ Each active KPI should link to:
|
|||
|
||||
### Driver Behaviour
|
||||
|
||||
| KPI | Status | SQL ref | Grafana panel | Reviewed by | Cadence |
|
||||
| KPI | Status | SQL ref | FleetOps panel | Reviewed by | Cadence |
|
||||
|---|---|---|---|---|---|
|
||||
| Speeding events per 100 km | Proposed | `01_BusinessAnalytics.md §3.1` | — | — | Weekly |
|
||||
| Harsh driving index | Proposed | `01_BusinessAnalytics.md §3.2` | — | — | Weekly |
|
||||
|
|
@ -55,7 +55,7 @@ Each active KPI should link to:
|
|||
|
||||
### Asset Health & Cost
|
||||
|
||||
| KPI | Status | SQL ref | Grafana panel | Reviewed by | Cadence |
|
||||
| KPI | Status | SQL ref | FleetOps panel | Reviewed by | Cadence |
|
||||
|---|---|---|---|---|---|
|
||||
| Estimated idle fuel cost (KES) | Proposed | `01_BusinessAnalytics.md §2.2` | — | — | Monthly |
|
||||
| Vehicles at service threshold | Proposed | TBD | — | — | Weekly |
|
||||
|
|
@ -96,6 +96,6 @@ Adjust with client after first month of live data:
|
|||
- [ ] Confirm shift hours (start, end, lunch, working days)
|
||||
- [ ] Confirm SLA tiers (home vs business customer)
|
||||
- [ ] Confirm which KPIs the ops manager wants on a daily digest
|
||||
- [ ] Confirm reporting format (Grafana link, PDF, WhatsApp summary)
|
||||
- [ ] Confirm reporting format (FleetOps link, PDF, WhatsApp summary)
|
||||
- [ ] Identify job management system / ticketing tool for MTTA/MTTR
|
||||
- [ ] Confirm vehicle categories (motorcycle, van, 4WD) for per-type benchmarks
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -240,4 +240,4 @@ client's production domains **last**.
|
|||
---
|
||||
|
||||
*Related: [`CONNECTIONS.md`](CONNECTIONS.md) · [`PLATFORM_OVERVIEW.html`](PLATFORM_OVERVIEW.html) ·
|
||||
[`DWH_PIPELINE.md`](DWH_PIPELINE.md) · root `CLAUDE.md` (§3 map dashboards, §7 fix history).*
|
||||
root `CLAUDE.md` (§3 map dashboards, §7 fix history).*
|
||||
|
|
|
|||
|
|
@ -1,385 +0,0 @@
|
|||
# DWH Execution Manual
|
||||
|
||||
> **Purpose:** A reusable playbook for building an `extract → blob → load` data warehouse bronze layer using n8n (or any equivalent orchestrator) + object storage + PostgreSQL/PostGIS. Generalised from the Fireside Tracksolid DWH pipeline (2026-04-24). Apply this pattern to future data projects to skip re-deriving the same decisions.
|
||||
>
|
||||
> **Reference implementation:** `dwh/26100{1..4}.sql` + `docs/DWH_PIPELINE.md` + the two `n8n-workflows/dwh_*.json` files. Treat those as the copy-paste template for the next project.
|
||||
|
||||
---
|
||||
|
||||
## 1. When to Use This Pattern
|
||||
|
||||
**Use it when all of these are true:**
|
||||
- You need an analytical replica of a production OLTP DB without letting analytical load hit the OLTP.
|
||||
- Source and target are separate PostgreSQL instances (possibly separate networks).
|
||||
- Data volumes are moderate: millions of rows per day, not billions.
|
||||
- You already have an orchestrator (n8n, Airflow, Prefect) and object storage (rustfs, S3, MinIO) in the stack.
|
||||
- Latency tolerance is hours, not minutes.
|
||||
|
||||
**Don't use it when:**
|
||||
- Sub-minute latency is required → use logical replication or CDC (Debezium, pg_logical, AWS DMS).
|
||||
- Volumes exceed ~100 GB/day → step up to Spark/Flink + columnar store (Iceberg, Delta).
|
||||
- Source and target are the same DB → just use materialized views or scheduled SQL.
|
||||
- You need exactly-once streaming semantics → this pattern is at-least-once + idempotent load.
|
||||
|
||||
---
|
||||
|
||||
## 2. The Core Pattern
|
||||
|
||||
```
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Source DB │──(a)──▶ Orchestr. │──(b)──▶ Object Store │──(c)──▶ Target DB │
|
||||
│ (OLTP) │ │ (extract) │ │ (CSVs) │ │ (bronze) │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
|
||||
│ ▲
|
||||
└──────────────(d)─────────────────────────────┘
|
||||
(load workflow, per CSV)
|
||||
|
||||
(a) Watermarked SELECT, closed upper bound
|
||||
(b) Atomic CSV upload with timestamped path
|
||||
(c) CSV stays until load confirms success
|
||||
(d) Load = BEGIN → INSERT ON CONFLICT → UPDATE watermark → UPDATE run log → COMMIT → move CSV
|
||||
```
|
||||
|
||||
**Why three stages and not two:**
|
||||
- **Audit trail** — every extracted CSV is a point-in-time snapshot you can replay.
|
||||
- **Decoupling** — target DB downtime doesn't lose data; the CSV waits.
|
||||
- **Observability** — blob listings are a second source of truth independent of the DB.
|
||||
|
||||
---
|
||||
|
||||
## 3. Pre-flight Checklist
|
||||
|
||||
Before writing any SQL or workflow JSON, confirm in writing:
|
||||
|
||||
- [ ] Source DB reachable from orchestrator (internal network preferred, VPN/public IP with `sslmode=require` otherwise).
|
||||
- [ ] Target DB reachable; you hold a superuser credential for one-time DDL.
|
||||
- [ ] Object storage bucket exists; credentials are configured in the orchestrator.
|
||||
- [ ] For each source table to extract, you have identified:
|
||||
- A **DB-insertion timestamp column** (not device/user-reported time), or "it's a snapshot table".
|
||||
- A **natural unique key** that already has a `PRIMARY KEY` or `UNIQUE` constraint on source (for the `ON CONFLICT` target on bronze).
|
||||
- Any **unit/column drift** between source and target (e.g., `distance_m` vs. `distance_km`).
|
||||
- [ ] Acceptable end-to-end latency (to calibrate cron cadence).
|
||||
- [ ] Security baseline: who writes bronze, who reads it, SSL requirement, password rotation cadence.
|
||||
|
||||
If any row is unchecked, pause and resolve it. Skipping this step is the #1 cause of pipelines that "worked in test but lose data in prod."
|
||||
|
||||
---
|
||||
|
||||
## 4. Phase-by-Phase Execution
|
||||
|
||||
Execute in order. Phases are independent of each other within their phase, but phases have strict dependencies.
|
||||
|
||||
### Phase A — Target DB preparation
|
||||
|
||||
Apply three types of migrations, in numeric order:
|
||||
|
||||
1. **Bronze DDL** — one table per source table. Use `IF NOT EXISTS`; make it idempotent.
|
||||
2. **Control schema** — `dwh_control.extract_watermarks` + `dwh_control.extract_runs`.
|
||||
3. **Assertion migrations** — verify roles exist, verify every `ON CONFLICT` target is backed by a PK/UNIQUE (fail loudly if not).
|
||||
|
||||
Template files: `dwh/260423_dwh_ddl_v1.sql`, `dwh/261001_dwh_control.sql`, `dwh/261002_bronze_constraints_audit.sql`, `dwh/261003_dwh_roles.sql`.
|
||||
|
||||
**Role model:**
|
||||
- `<proj>_owner` — owns schemas, writes bronze + control tables.
|
||||
- `<proj>_ro` (or `grafana_ro`) — reads everything, writes nothing.
|
||||
- Never use `postgres` or another superuser from the orchestrator.
|
||||
|
||||
**Watermark seed:** set `last_extracted_at` to a date before any real data (`'2000-01-01T00:00:00Z'` is safe) so the first run back-fills all history in a single CSV per table.
|
||||
|
||||
### Phase B — Object storage
|
||||
|
||||
Create two prefixes per table:
|
||||
|
||||
```
|
||||
s3://<bucket>/<project>/exports/{table}/ # active CSVs, in-flight
|
||||
s3://<bucket>/<project>/processed/{table}/ # loaded CSVs, never deleted (audit)
|
||||
```
|
||||
|
||||
Naming convention: `{YYYYMMDD_HHMM}_{TZ}.csv` (e.g., `20260424_1400_EAT.csv`). Timezone in the filename because "08:00" means nothing a year from now without it.
|
||||
|
||||
Retention: match whatever backup retention is already in the stack (e.g., 30 days). `processed/` should outlive `exports/`.
|
||||
|
||||
### Phase C — Orchestrator credentials
|
||||
|
||||
Three credentials:
|
||||
|
||||
| Credential | Role | Purpose |
|
||||
|---|---|---|
|
||||
| `<proj>_source` | Read-only role on source DB | Extract queries |
|
||||
| `<proj>_dwh_target` | `<proj>_owner` on target DB | Bronze writes + control updates |
|
||||
| `<proj>_s3` | IAM user with `s3:PutObject`, `s3:GetObject`, `s3:ListBucket`, `s3:DeleteObject` on the prefix | CSV upload/download/move |
|
||||
|
||||
**Always** `sslmode=require` on any public-IP DB connection. Test each credential with the orchestrator's "Test connection" button before proceeding.
|
||||
|
||||
### Phase D — Load workflow (build this BEFORE the extract workflow)
|
||||
|
||||
Building load first lets you iterate with hand-crafted CSVs in blob storage before wiring up extract. Much faster feedback loop.
|
||||
|
||||
Load workflow input (parameters):
|
||||
```json
|
||||
{
|
||||
"table": "position_history",
|
||||
"csv_path": "s3://bucket/project/exports/position_history/20260424_1400_EAT.csv",
|
||||
"run_id": 12345,
|
||||
"run_started_at": "2026-04-24T11:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
Load workflow steps:
|
||||
1. Download CSV from blob storage.
|
||||
2. Parse CSV into rows.
|
||||
3. **Open transaction.**
|
||||
4. `INSERT INTO bronze.<table> (...) VALUES (...) ON CONFLICT (<natural_key>) DO NOTHING;`
|
||||
5. `UPDATE dwh_control.extract_watermarks SET last_extracted_at = :run_started_at, last_loaded_at = NOW(), rows_loaded_last_run = <count> WHERE table_name = :table;`
|
||||
6. `UPDATE dwh_control.extract_runs SET status = 'loaded', run_finished_at = NOW(), rows_loaded = <count> WHERE run_id = :run_id;`
|
||||
7. **Commit.**
|
||||
8. Move CSV from `exports/` to `processed/` (copy-then-delete; never delete before copy confirms).
|
||||
|
||||
**Non-negotiable invariants:**
|
||||
- Steps 3–7 are one transaction. If any fails, all rollback.
|
||||
- Step 8 only runs after commit. If step 8 fails, the next run will re-load the CSV (idempotent via ON CONFLICT) — not a data loss event.
|
||||
|
||||
### Phase E — Extract workflow
|
||||
|
||||
Extract workflow steps, per table:
|
||||
|
||||
1. Read current watermark: `SELECT last_extracted_at FROM dwh_control.extract_watermarks WHERE table_name = :table;`
|
||||
2. Capture `run_started_at = NOW()` (in the target DB's clock, not the orchestrator's — reduces clock-skew bugs).
|
||||
3. `INSERT INTO dwh_control.extract_runs (table_name, run_started_at, status) VALUES (:table, :run_started_at, 'extracting') RETURNING run_id;`
|
||||
4. Query source with closed upper bound:
|
||||
```sql
|
||||
SELECT <cols>
|
||||
FROM <source_schema>.<table>
|
||||
WHERE <watermark_col> > :last_extracted_at
|
||||
AND <watermark_col> <= :run_started_at
|
||||
ORDER BY <watermark_col>;
|
||||
```
|
||||
5. Render rows as CSV. For geometry columns: `CASE WHEN geom IS NULL THEN NULL ELSE ST_AsEWKT(geom) END`.
|
||||
6. Upload CSV to `s3://bucket/project/exports/{table}/{YYYYMMDD_HHMM}_{TZ}.csv`.
|
||||
7. `UPDATE dwh_control.extract_runs SET status = 'uploaded', rows_extracted = <count>, csv_path = :path WHERE run_id = :run_id;`
|
||||
8. Call load workflow with `{table, csv_path, run_id, run_started_at}`.
|
||||
|
||||
### Phase F — Schedule + observability
|
||||
|
||||
**Cron cadence:** start with 6–8 runs/day during active hours. Fold the overnight gap where traffic is low. Example: `0 5,8,11,14,17,20,23 * * *` TZ `Africa/Nairobi`.
|
||||
|
||||
**Three observability views** (readable by the RO role):
|
||||
|
||||
- `v_table_freshness` — per-table lag from last successful load. Drives the freshness alert.
|
||||
- `v_recent_failures` — failed runs in last 24h. Zero rows = healthy.
|
||||
- `v_watermark_lag` — extract vs. load lag per table. Distinguishes "nothing to extract" from "stuck".
|
||||
|
||||
Template file: `dwh/261004_dwh_observability_views.sql`.
|
||||
|
||||
**Grafana panels** (add at minimum):
|
||||
1. Freshness panel — red if any row in `v_table_freshness` has `lag > 4h`.
|
||||
2. Failures panel — red if `v_recent_failures` has any row.
|
||||
3. Row counts panel — daily bar chart from `extract_runs`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Design Principles (Do Not Skip)
|
||||
|
||||
### 5.1 Watermark on DB insertion time, not source-reported time
|
||||
|
||||
The watermark column must be "when the target DB got the row", not "when the device/user said it happened". Device clocks skew, webhooks arrive late, and batch imports backdate records. A source-reported watermark will silently drop rows that arrive out of order. Use `recorded_at`, `created_at`, `updated_at` (with `DEFAULT NOW()`), or `ingested_at` — never `gps_time` / `event_time` / `timestamp`.
|
||||
|
||||
### 5.2 Closed upper bound
|
||||
|
||||
Extract uses `> last_extracted_at AND <= run_started_at`. The closed upper bound prevents "row committed at `NOW()` during the extract query" from appearing in two adjacent runs. Without it, some rows are double-extracted (wasteful) or missed (data loss).
|
||||
|
||||
### 5.3 Idempotent load via natural unique keys
|
||||
|
||||
Every incremental bronze table needs a PRIMARY KEY or UNIQUE that matches the source's natural unique key. `ON CONFLICT DO NOTHING` makes re-running a CSV harmless. **Do not invent surrogate keys on bronze** — they defeat the ON CONFLICT guarantee. If the source has no natural key, fix the source or accept the table as a snapshot.
|
||||
|
||||
### 5.4 Transactional load boundary
|
||||
|
||||
Insert + watermark update + run-log update are one transaction. Splitting them creates "ghost" states where watermark advanced but rows didn't load, causing silent holes.
|
||||
|
||||
### 5.5 CSV audit trail — never delete
|
||||
|
||||
Moved-to-`processed/` CSVs are cheap ($0.023/GB/month on S3-class storage). They pay for themselves the first time you need to replay a window or debug a row-count mismatch.
|
||||
|
||||
### 5.6 PostGIS round-trip via EWKT
|
||||
|
||||
`ST_AsEWKT(geom)` on extract, `ST_GeomFromEWKT(ewkt)` on load. Preserves SRID inline. Do NOT store `ST_AsText` + separate SRID column — it doubles the chance of mismatch. Guard NULLs: `CASE WHEN geom IS NULL THEN NULL ELSE ST_AsEWKT(geom) END`.
|
||||
|
||||
### 5.7 Fail loud, fail early
|
||||
|
||||
Audit migrations (roles, constraints) should `RAISE EXCEPTION` with a bullet list of what's missing. Silent success is worse than noisy failure — a missing PK surfaces three months later as "why are there duplicate trips?".
|
||||
|
||||
---
|
||||
|
||||
## 6. Snapshot vs. Incremental Decision Matrix
|
||||
|
||||
| Signal | Snapshot (TRUNCATE + reload) | Incremental (watermark + append) |
|
||||
|---|---|---|
|
||||
| Row count | < ~10k | > ~10k |
|
||||
| Meaning of "current state" | Matters | Doesn't matter; history matters |
|
||||
| Deletes in source | Common | Rare |
|
||||
| Update frequency per row | High | Low (append-mostly) |
|
||||
| Natural unique key | May not exist | Must exist |
|
||||
| Example | `devices`, `live_positions`, `geofences` | `position_history`, `trips`, `alarms`, event logs |
|
||||
|
||||
When in doubt: **snapshot is simpler**. Only escalate to incremental when the snapshot CSV would exceed a few MB per run.
|
||||
|
||||
---
|
||||
|
||||
## 7. Observability Contract
|
||||
|
||||
Every pipeline adds these three views to its control schema — no exceptions:
|
||||
|
||||
```sql
|
||||
CREATE OR REPLACE VIEW <control>.v_table_freshness AS
|
||||
SELECT table_name,
|
||||
MAX(run_finished_at) AS last_loaded_at,
|
||||
NOW() - MAX(run_finished_at) AS lag,
|
||||
COUNT(*) FILTER (WHERE run_started_at > NOW() - INTERVAL '24 hours') AS loads_last_24h
|
||||
FROM <control>.extract_runs
|
||||
WHERE status = 'loaded'
|
||||
GROUP BY table_name;
|
||||
|
||||
CREATE OR REPLACE VIEW <control>.v_recent_failures AS
|
||||
SELECT run_id, table_name, run_started_at, run_finished_at, csv_path, error_message
|
||||
FROM <control>.extract_runs
|
||||
WHERE status = 'failed' AND run_started_at > NOW() - INTERVAL '24 hours'
|
||||
ORDER BY run_started_at DESC;
|
||||
|
||||
CREATE OR REPLACE VIEW <control>.v_watermark_lag AS
|
||||
SELECT table_name, last_extracted_at, last_loaded_at, rows_loaded_last_run,
|
||||
NOW() - last_loaded_at AS load_lag,
|
||||
NOW() - last_extracted_at AS extract_lag
|
||||
FROM <control>.extract_watermarks;
|
||||
```
|
||||
|
||||
Wire a Grafana alert on each view. Test the alert by manually failing a run before go-live.
|
||||
|
||||
---
|
||||
|
||||
## 8. Schema Drift Handling
|
||||
|
||||
Schema drift between source and bronze is inevitable. Two rules:
|
||||
|
||||
1. **Detect at design time.** Diff source DDL against bronze DDL before writing any extract SQL. Unit changes (metres vs. km), renamed columns, and added nullable columns are the usual suspects.
|
||||
2. **Fix in the extract query, not the load.** Put all transformations in the SELECT so the CSV on disk already matches the bronze column names and units. The load workflow should be dumb — CSV column N goes to bronze column N.
|
||||
|
||||
Document every drift in the runbook (§5 of the operations runbook). Future developers WILL hit them.
|
||||
|
||||
---
|
||||
|
||||
## 9. Verification Gates
|
||||
|
||||
### Pre-deploy (before first cron tick)
|
||||
|
||||
- [ ] Every migration applied successfully.
|
||||
- [ ] Control tables seeded (one watermark row per incremental table).
|
||||
- [ ] Every credential's "Test connection" passes.
|
||||
- [ ] Blob storage prefixes exist.
|
||||
- [ ] Manual workflow trigger succeeds end-to-end for one table.
|
||||
|
||||
### First run (manual trigger of extract workflow)
|
||||
|
||||
- [ ] Every processed table has a row in `extract_runs` with `status='loaded'`.
|
||||
- [ ] Row-count parity with source (± in-flight writes): `SELECT COUNT(*) FROM <source>` vs. `SELECT COUNT(*) FROM bronze.<table>`.
|
||||
- [ ] Geometry columns round-trip cleanly: `SELECT ST_AsText(geom) FROM bronze.<table> LIMIT 5` returns valid POINTs.
|
||||
- [ ] All CSVs moved from `exports/` to `processed/`.
|
||||
|
||||
### Steady-state (after 24h / first full schedule cycle)
|
||||
|
||||
- [ ] `v_table_freshness` shows lag < cadence × 2 for every table.
|
||||
- [ ] `v_recent_failures` is empty.
|
||||
- [ ] Row counts in bronze growing at expected rate.
|
||||
|
||||
Only declare "done" after all three gates pass.
|
||||
|
||||
---
|
||||
|
||||
## 10. Scheduling Calibration
|
||||
|
||||
Tradeoffs:
|
||||
|
||||
| Cadence | Pros | Cons |
|
||||
|---|---|---|
|
||||
| Every 15 min | Low lag, small CSVs | High orchestrator churn, noisy alerts |
|
||||
| Every 3 h (recommended) | Predictable, fits ops windows, tolerable lag | Overnight backlog carries to morning |
|
||||
| Nightly (once/day) | Cheap, simple | Unacceptable for real-time panels |
|
||||
|
||||
Rule of thumb: cadence = 25–50% of your latency tolerance. 4h latency budget → 1-2h cadence.
|
||||
|
||||
Fold cadence around traffic patterns. Don't run 24× at 1-hour intervals if the source generates zero rows between midnight and 05:00.
|
||||
|
||||
---
|
||||
|
||||
## 11. Common Failure Modes & Recovery
|
||||
|
||||
| Failure | Symptom | Fix |
|
||||
|---|---|---|
|
||||
| CSV stuck in `exports/` | `v_recent_failures` has a row; CSV never moved | Next scheduled run retries automatically (idempotent). If persistent, open orchestrator logs by `run_id`. |
|
||||
| Table marked `loading` for >1 cadence | n8n executor crashed mid-transaction | DB rolled back. Next run retries. If stuck >2 cadences, manually re-trigger the extract. |
|
||||
| Row counts diverge > 1% | CSV parse error silently dropped rows | `rows_extracted != rows_loaded` in `extract_runs` — inspect the CSV for malformed rows. |
|
||||
| Geometry loads as NULL | EWKT serialisation broke | Check for missing `CASE WHEN geom IS NULL` guard in extract SQL. |
|
||||
| Distance/units 1000× wrong | Schema drift not caught | Check extract SQL for the unit conversion (see §8). |
|
||||
|
||||
**Back-fill a window:**
|
||||
```sql
|
||||
UPDATE <control>.extract_watermarks
|
||||
SET last_extracted_at = NOW() - INTERVAL '24 hours'
|
||||
WHERE table_name = '<table>';
|
||||
```
|
||||
Next run re-extracts the gap. `ON CONFLICT DO NOTHING` filters duplicates.
|
||||
|
||||
**Full reseed (nuclear):**
|
||||
```sql
|
||||
UPDATE <control>.extract_watermarks
|
||||
SET last_extracted_at = '2000-01-01T00:00:00Z'
|
||||
WHERE table_name = '<table>';
|
||||
```
|
||||
Next run back-fills all history in one very large CSV. Expected; it moves to `processed/` on success.
|
||||
|
||||
---
|
||||
|
||||
## 12. Security Baseline
|
||||
|
||||
- Two roles minimum: owner (writes) and RO (reads). Never use superuser from the orchestrator.
|
||||
- `sslmode=require` on every public-IP DB connection.
|
||||
- Passwords never in committed SQL — use placeholder tokens (`CHANGE_ME_BEFORE_APPLY`) and swap in-session during apply. Document rotation in the runbook.
|
||||
- Blob storage credentials scoped to the project's prefix, not the whole bucket.
|
||||
- Rotate all credentials before go-live (don't reuse the ones that were flying around in design conversations).
|
||||
|
||||
---
|
||||
|
||||
## 13. Reusability Checklist (Applying to a New Project)
|
||||
|
||||
When starting a new data project, copy the Tracksolid DWH layout and edit these points:
|
||||
|
||||
- [ ] Rename schemas: `<proj>_control` instead of `dwh_control` if multiple DWHs share a DB.
|
||||
- [ ] Adjust `<proj>_owner` / `<proj>_ro` role names.
|
||||
- [ ] Update bucket prefix: `s3://<bucket>/<project>/exports|processed/`.
|
||||
- [ ] Re-do the snapshot/incremental decision for every source table (§6).
|
||||
- [ ] Identify watermark columns and natural unique keys for every incremental table (§5.1, §5.3).
|
||||
- [ ] Map schema drift before writing extract SQL (§8).
|
||||
- [ ] Calibrate cadence to the new project's latency budget (§10).
|
||||
- [ ] Ship the three observability views (§7) — even if nobody will look at them in week one.
|
||||
- [ ] Write the runbook from the template: follow `docs/DWH_PIPELINE.md` section-for-section.
|
||||
- [ ] Run the verification gates (§9) before declaring done.
|
||||
|
||||
---
|
||||
|
||||
## 14. Reference Implementation (Tracksolid DWH)
|
||||
|
||||
These files are the copy-paste template:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `dwh/260423_dwh_ddl_v1.sql` | Bronze DDL + roles + schemas |
|
||||
| `dwh/261001_dwh_control.sql` | Control schema (watermarks + run log) |
|
||||
| `dwh/261002_bronze_constraints_audit.sql` | ON CONFLICT key assertion |
|
||||
| `dwh/261003_dwh_roles.sql` | Role contract assertion |
|
||||
| `dwh/261004_dwh_observability_views.sql` | Freshness/failure/watermark views |
|
||||
| `docs/DWH_PIPELINE.md` | Operations runbook (troubleshooting, manual re-run, rotation) |
|
||||
| `docs/superpowers/specs/2026-04-24-n8n-dwh-bronze-pipeline-design.md` | Design spec (why each decision) |
|
||||
| `docs/superpowers/plans/2026-04-24-n8n-dwh-bronze-pipeline.md` | Task-by-task implementation plan |
|
||||
| `n8n-workflows/dwh_extract.json` | Extract workflow (reference) |
|
||||
| `n8n-workflows/dwh_load_bronze.json` | Load workflow (reference) |
|
||||
|
||||
**For the next project, fork this manual first, then adapt.** Do not re-design from scratch — the seven design principles in §5 are the parts people keep getting wrong.
|
||||
|
|
@ -30,7 +30,6 @@ After adding it: `source ~/.zshrc`
|
|||
tsdb # open interactive psql prompt
|
||||
tsdb -c "\dt tracksolid.*" # list all tracksolid tables
|
||||
tsdb -c "SELECT COUNT(*) FROM tracksolid.trips;" # run a single query
|
||||
tsdb -c "SELECT dwh_gold.refresh_daily_metrics(CURRENT_DATE - 1);" # run nightly ETL
|
||||
```
|
||||
|
||||
### One-liner (for scripts and migrations)
|
||||
|
|
@ -65,25 +64,25 @@ JIMI TRACKSOLID PRO API
|
|||
+-- POLLING (Pull) +-- PUSH (Webhook)
|
||||
| (Fallback / Catch-up) | (Real-time)
|
||||
| |
|
||||
ingest_movement ingest_events webhook_receiver
|
||||
(60s/15m/daily) (5m polling) (FastAPI :8000)
|
||||
| | |
|
||||
+------------------+----------------+
|
||||
ingest_worker (movement + events) webhook_receiver
|
||||
(60s/5m/15m/daily polling) (FastAPI :8888)
|
||||
| |
|
||||
+----------------------------------+
|
||||
|
|
||||
timescale_db
|
||||
(PG16 + TimescaleDB + PostGIS)
|
||||
|
|
||||
grafana
|
||||
(Dashboards :3000)
|
||||
dashboard_api (:8890)
|
||||
(read-API → FleetNow / FleetOps SPAs)
|
||||
```
|
||||
|
||||
| Service | Purpose | Restart Policy |
|
||||
|---------|---------|----------------|
|
||||
| `timescale_db` | PostgreSQL 16 + TimescaleDB 2.15 + PostGIS 3 | always |
|
||||
| `ingest_movement` | GPS positions, trips, parking, device sync (polling) | always |
|
||||
| `ingest_events` | Alarm event polling (catch-up/fallback) | always |
|
||||
| `ingest_worker` | GPS positions, trips, parking, device sync + alarm polling (merged movement + events) | always |
|
||||
| `webhook_receiver` | Real-time push data from Jimi (OBD, faults, GPS, alarms, heartbeats, trips) | always |
|
||||
| `grafana` | Visualization dashboards (read-only DB access) | always |
|
||||
| `dashboard_api` | Read-API for the FleetNow / FleetOps SPAs | always |
|
||||
| `db_backup` | pg_dump → rustfs S3, scheduled | always |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -101,7 +100,7 @@ SELECT * FROM timescaledb_information.hypertables;
|
|||
```sql
|
||||
SELECT table_schema, table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema IN ('tracksolid', 'dwh_gold')
|
||||
WHERE table_schema IN ('tracksolid', 'reporting')
|
||||
ORDER BY table_schema, table_name;
|
||||
```
|
||||
|
||||
|
|
@ -117,8 +116,6 @@ Expected tables:
|
|||
- `tracksolid.obd_readings`
|
||||
- `tracksolid.fault_codes`
|
||||
- `tracksolid.heartbeats` (hypertable)
|
||||
- `dwh_gold.dim_vehicles`
|
||||
- `dwh_gold.fact_daily_fleet_metrics`
|
||||
|
||||
**Verify hypertables are configured:**
|
||||
```sql
|
||||
|
|
@ -457,20 +454,17 @@ curl -X POST https://<your-webhook-domain>/pushobd \
|
|||
|
||||
---
|
||||
|
||||
### 2.5 grafana
|
||||
### 2.5 dashboard_api
|
||||
|
||||
```bash
|
||||
# Verify Grafana is accessible
|
||||
curl -f https://<your-grafana-domain>/api/health
|
||||
# Expected: {"commit":"...","database":"ok","version":"11.0.0"}
|
||||
```
|
||||
# Verify the read-API is healthy
|
||||
curl -f https://fleetapi.rahamafresh.com/health
|
||||
# Expected: {"status":"ok"}
|
||||
|
||||
**Configure data source in Grafana UI:**
|
||||
- Type: PostgreSQL
|
||||
- Host: `timescale_db:5432` (internal Docker network)
|
||||
- Database: `tracksolid_db`
|
||||
- User: `grafana_ro`
|
||||
- SSL Mode: disable (internal network)
|
||||
# Ingest pipeline freshness (replaces the old Grafana health panels)
|
||||
curl -f https://fleetapi.rahamafresh.com/health/ingest
|
||||
# Expected: {"overall":"ok","endpoints":[...]}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -578,7 +572,6 @@ The `source` column ('poll' or 'push') tracks data origin where applicable.
|
|||
| `POSTGRES_DB` | Yes | timescale_db | Database name |
|
||||
| `POSTGRES_USER` | Yes | timescale_db | Database superuser |
|
||||
| `POSTGRES_PASSWORD` | Yes | timescale_db | Database password |
|
||||
| `GRAFANA_ADMIN_PASSWORD` | Yes | grafana | Grafana admin UI password |
|
||||
| `JIMI_WEBHOOK_TOKEN` | No | webhook_receiver | Webhook auth token (empty = skip validation) |
|
||||
| `DB_POOL_MAX` | No | All Python services | Max DB connections (default: 12) |
|
||||
|
||||
|
|
|
|||
|
|
@ -1,324 +0,0 @@
|
|||
# Grafana NOC Fleet Dashboard — Deployment Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Single-region NOC dashboard monitoring 80 vehicles (Nairobi, Mombasa, Kampala) via Tracksolid/Jimi GPS.
|
||||
Live location, direction arrows, speed, and driver info. Auto-refreshes every 30 seconds.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker Compose stack running (`timescale_db`, `grafana`)
|
||||
- `.env` file with all existing variables
|
||||
- Add one new variable to `.env`:
|
||||
```
|
||||
GRAFANA_DB_RO_PASSWORD=<password for grafana_ro postgres user>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
grafana/
|
||||
└── provisioning/
|
||||
├── datasources/
|
||||
│ └── tracksolid_postgres.yaml
|
||||
├── dashboards/
|
||||
│ └── noc_fleet.yaml
|
||||
└── dashboards-json/
|
||||
└── noc_fleet_dashboard.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — docker-compose.yaml (grafana service)
|
||||
|
||||
Replace the existing `grafana` service block with:
|
||||
|
||||
```yaml
|
||||
grafana:
|
||||
image: grafana/grafana:11.0.0
|
||||
restart: always
|
||||
depends_on:
|
||||
timescale_db:
|
||||
condition: service_healthy
|
||||
env_file: .env
|
||||
environment:
|
||||
- GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD}
|
||||
- GF_USERS_DEFAULT_THEME=dark
|
||||
- GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/etc/grafana/provisioning/dashboards-json/noc_fleet_dashboard.json
|
||||
volumes:
|
||||
- grafana-data:/var/lib/grafana
|
||||
- ./grafana/provisioning:/etc/grafana/provisioning:ro
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — Datasource: `grafana/provisioning/datasources/tracksolid_postgres.yaml`
|
||||
|
||||
```yaml
|
||||
apiVersion: 1
|
||||
|
||||
datasources:
|
||||
- name: TracksolidDB
|
||||
type: postgres
|
||||
uid: tracksolid_pg
|
||||
url: timescale_db:5432
|
||||
database: tracksolid_db
|
||||
user: grafana_ro
|
||||
secureJsonData:
|
||||
password: ${GRAFANA_DB_RO_PASSWORD}
|
||||
jsonData:
|
||||
sslmode: disable
|
||||
maxOpenConns: 5
|
||||
maxIdleConns: 2
|
||||
connMaxLifetime: 14400
|
||||
postgresVersion: 1600
|
||||
timescaledb: true
|
||||
editable: false
|
||||
isDefault: true
|
||||
```
|
||||
|
||||
> **Note:** `uid: tracksolid_pg` is referenced by the dashboard JSON. Never rename it after deployment.
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Dashboard Provider: `grafana/provisioning/dashboards/noc_fleet.yaml`
|
||||
|
||||
```yaml
|
||||
apiVersion: 1
|
||||
|
||||
providers:
|
||||
- name: NOC Fleet Dashboards
|
||||
orgId: 1
|
||||
folder: NOC
|
||||
type: file
|
||||
disableDeletion: true
|
||||
updateIntervalSeconds: 30
|
||||
allowUiUpdates: false
|
||||
options:
|
||||
path: /etc/grafana/provisioning/dashboards-json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — Dashboard JSON: `grafana/provisioning/dashboards-json/noc_fleet_dashboard.json`
|
||||
|
||||
### Dashboard Settings
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| UID | `noc-fleet-live` |
|
||||
| Auto-refresh | 30s |
|
||||
| Theme | Dark |
|
||||
| Schema version | 39 (Grafana 11.0.0) |
|
||||
| Editable | false |
|
||||
| Bookmark URL | `/d/noc-fleet-live` |
|
||||
|
||||
---
|
||||
|
||||
### Panel Layout
|
||||
|
||||
| Row | Panel | Type | Notes |
|
||||
|-----|-------|------|-------|
|
||||
| 1 | Total Vehicles | Stat | Count all enabled devices |
|
||||
| 1 | Online Now | Stat | GPS fix < 5 min — green |
|
||||
| 1 | Recent (5-30m) | Stat | GPS fix 5-30 min — amber |
|
||||
| 1 | Offline | Stat | GPS fix > 30 min — red |
|
||||
| 1 | Moving Now | Stat | speed > 0 AND acc on |
|
||||
| 1 | Avg Speed (km/h) | Stat | Moving vehicles only |
|
||||
| 2 | Live Vehicle Locations | Geomap | Direction arrows, color by plate |
|
||||
| 3 | Vehicle Status Table | Table | All vehicles, sorted Online first |
|
||||
| 4 | Ingestion Health | Table | Collapsed by default |
|
||||
|
||||
---
|
||||
|
||||
### SQL Queries
|
||||
|
||||
**Total Vehicles**
|
||||
```sql
|
||||
SELECT COUNT(*) AS "Total Vehicles"
|
||||
FROM tracksolid.devices WHERE enabled_flag = 1;
|
||||
```
|
||||
|
||||
**Online Now**
|
||||
```sql
|
||||
SELECT COUNT(*) AS "Online"
|
||||
FROM tracksolid.v_fleet_status
|
||||
WHERE connectivity_status = 'online';
|
||||
```
|
||||
|
||||
**Recent (5-30 min)**
|
||||
```sql
|
||||
SELECT COUNT(*) AS "Recent"
|
||||
FROM tracksolid.v_fleet_status
|
||||
WHERE connectivity_status = 'recent';
|
||||
```
|
||||
|
||||
**Offline**
|
||||
```sql
|
||||
SELECT COUNT(*) AS "Offline"
|
||||
FROM tracksolid.v_fleet_status
|
||||
WHERE connectivity_status = 'offline';
|
||||
```
|
||||
|
||||
**Moving Now**
|
||||
```sql
|
||||
SELECT COUNT(*) AS "Moving"
|
||||
FROM tracksolid.v_fleet_status
|
||||
WHERE speed > 0 AND acc_status = '1' AND connectivity_status = 'online';
|
||||
```
|
||||
|
||||
**Avg Speed**
|
||||
```sql
|
||||
SELECT ROUND(AVG(speed)::numeric, 1) AS "Avg Speed km/h"
|
||||
FROM tracksolid.v_fleet_status
|
||||
WHERE speed > 0 AND acc_status = '1' AND connectivity_status = 'online';
|
||||
```
|
||||
|
||||
**Geomap — Live Vehicle Locations**
|
||||
```sql
|
||||
SELECT
|
||||
d.imei,
|
||||
d.vehicle_number,
|
||||
d.vehicle_name,
|
||||
d.driver_name,
|
||||
d.driver_phone,
|
||||
d.city,
|
||||
d.device_group,
|
||||
lp.lat,
|
||||
lp.lng,
|
||||
lp.speed,
|
||||
lp.direction,
|
||||
lp.acc_status,
|
||||
lp.loc_desc,
|
||||
lp.gps_time,
|
||||
CASE
|
||||
WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes' THEN 'online'
|
||||
WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 'recent'
|
||||
ELSE 'offline'
|
||||
END AS connectivity_status
|
||||
FROM tracksolid.devices d
|
||||
INNER JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1
|
||||
AND lp.lat IS NOT NULL
|
||||
AND lp.lng IS NOT NULL
|
||||
ORDER BY d.vehicle_number;
|
||||
```
|
||||
|
||||
> INNER JOIN — only vehicles with a valid GPS fix appear on the map.
|
||||
> `acc_status` is TEXT ('0'/'1') in the schema — all queries use string comparisons.
|
||||
|
||||
**Vehicle Status Table**
|
||||
```sql
|
||||
SELECT
|
||||
d.vehicle_number AS "Plate",
|
||||
d.vehicle_name AS "Vehicle",
|
||||
d.driver_name AS "Driver",
|
||||
d.driver_phone AS "Driver Phone",
|
||||
d.city AS "City",
|
||||
ROUND(lp.speed::numeric, 0) AS "Speed (km/h)",
|
||||
lp.loc_desc AS "Last Location",
|
||||
lp.gps_time AS "Last Fix",
|
||||
CASE
|
||||
WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes' THEN 'Online'
|
||||
WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 'Recent'
|
||||
ELSE 'Offline'
|
||||
END AS "Status",
|
||||
EXTRACT(EPOCH FROM (NOW() - lp.gps_time))::int / 60 AS "Min Since Fix"
|
||||
FROM tracksolid.devices d
|
||||
LEFT JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1
|
||||
ORDER BY
|
||||
CASE
|
||||
WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes' THEN 0
|
||||
WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 1
|
||||
ELSE 2
|
||||
END,
|
||||
d.vehicle_number;
|
||||
```
|
||||
|
||||
> LEFT JOIN — offline vehicles (no GPS fix) still appear in the table showing their null status.
|
||||
|
||||
**Ingestion Health**
|
||||
```sql
|
||||
SELECT
|
||||
endpoint AS "Endpoint",
|
||||
TO_CHAR(run_at, 'HH24:MI DD-Mon') AS "Last Run",
|
||||
CASE WHEN success THEN 'OK' ELSE 'FAIL' END AS "Result",
|
||||
error_message AS "Error",
|
||||
seconds_ago AS "Lag (s)"
|
||||
FROM tracksolid.v_ingestion_health
|
||||
ORDER BY endpoint;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Geomap Configuration
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Basemap | Carto Dark |
|
||||
| Location mode | `coords` — fields `lat` / `lng` |
|
||||
| Marker symbol | `img/icons/marker/arrow-up.svg` (built-in Grafana 11) |
|
||||
| Rotation | field: `direction` (0° = North, clockwise) |
|
||||
| Color | field: `vehicle_number`, scheme: `palette-classic-by-name` |
|
||||
| Tooltip hidden fields | `lat`, `lng`, `imei` |
|
||||
| Direction unit override | `degree` (shows `245°` in tooltip) |
|
||||
| Initial view | East Africa center — auto-fits to vehicle positions |
|
||||
|
||||
### Connectivity Status Color Mapping
|
||||
|
||||
| Value | Color |
|
||||
|-------|-------|
|
||||
| Online / online | Green |
|
||||
| Recent / recent | Amber |
|
||||
| Offline / offline | Red |
|
||||
|
||||
### Stat Panel Thresholds
|
||||
|
||||
| Panel | Green | Amber | Red |
|
||||
|-------|-------|-------|-----|
|
||||
| Online Now | > 0 | — | = 0 |
|
||||
| Recent | — | > 0 | — |
|
||||
| Offline | = 0 | — | > 0 |
|
||||
| Moving Now | > 0 | — | — |
|
||||
| Avg Speed | < 80 km/h | 80-120 | > 120 |
|
||||
|
||||
All stat panels: `colorMode: background`, `graphMode: none`.
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — Deploy
|
||||
|
||||
```bash
|
||||
docker compose up -d grafana
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 6 — Verify
|
||||
|
||||
1. Open `http://localhost:3000/d/noc-fleet-live`
|
||||
2. Map renders with arrow markers on vehicle positions
|
||||
3. Arrows rotate per heading (northbound vehicle = arrow pointing up)
|
||||
4. Tooltip shows: plate, driver name, speed, city, connectivity status
|
||||
5. Table sorts: Online → Recent → Offline
|
||||
6. 30s auto-refresh fires (watch "Last Fix" timestamps)
|
||||
7. Check provisioning logs:
|
||||
```bash
|
||||
docker compose logs grafana | grep -i provision
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- The `grafana/` directory must be committed to git — Coolify's clone must include it
|
||||
- `:ro` mounts prevent Grafana from writing back into the repo
|
||||
- `allowUiUpdates: false` in the provider YAML means UI edits are not persisted — the JSON file is the source of truth
|
||||
- `acc_status` in `live_positions` is TEXT (`'0'`/`'1'`), not integer — all queries use string comparisons
|
||||
- `uid: tracksolid_pg` in the datasource YAML is referenced by the dashboard JSON — never rename it after deployment
|
||||
- ServiceNow ticket integration is deferred to a future phase
|
||||
|
|
@ -1,647 +0,0 @@
|
|||
# Grafana NOC Fleet Dashboard — Operational Manual
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Pre-Deployment Checklist](#1-pre-deployment-checklist)
|
||||
2. [Deploy](#2-deploy)
|
||||
3. [Post-Deployment Verification](#3-post-deployment-verification)
|
||||
4. [Dashboard Panel Verification](#4-dashboard-panel-verification)
|
||||
5. [Database Verification Queries](#5-database-verification-queries)
|
||||
6. [Troubleshooting](#6-troubleshooting)
|
||||
7. [Day-to-Day NOC Operations](#7-day-to-day-noc-operations)
|
||||
8. [Maintenance](#8-maintenance)
|
||||
|
||||
---
|
||||
|
||||
## 1. Pre-Deployment Checklist
|
||||
|
||||
Run these checks before starting Grafana for the first time.
|
||||
|
||||
### 1.1 Environment Variables
|
||||
|
||||
Confirm `.env` contains all required Grafana variables:
|
||||
|
||||
```bash
|
||||
grep -E 'GRAFANA_ADMIN_PASSWORD|GRAFANA_DB_RO_PASSWORD' .env
|
||||
```
|
||||
|
||||
Expected output — both lines must be present and non-empty:
|
||||
```
|
||||
GRAFANA_ADMIN_PASSWORD=<your admin password>
|
||||
GRAFANA_DB_RO_PASSWORD=<grafana_ro postgres password>
|
||||
```
|
||||
|
||||
If `GRAFANA_DB_RO_PASSWORD` is missing, add it before continuing:
|
||||
```bash
|
||||
echo "GRAFANA_DB_RO_PASSWORD=<password>" >> .env
|
||||
```
|
||||
|
||||
### 1.2 Verify grafana_ro User Exists in Postgres
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "\du grafana_ro"
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Role name | Attributes
|
||||
------------+---------------------------
|
||||
grafana_ro | Cannot login, ...
|
||||
```
|
||||
|
||||
If the role is missing, create it:
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
CREATE ROLE grafana_ro WITH LOGIN PASSWORD '<password>' NOSUPERUSER NOCREATEDB NOCREATEROLE;
|
||||
GRANT USAGE ON SCHEMA tracksolid TO grafana_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA tracksolid TO grafana_ro;
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA tracksolid GRANT SELECT ON TABLES TO grafana_ro;
|
||||
"
|
||||
```
|
||||
|
||||
### 1.3 Verify Provisioning Files Are Present
|
||||
|
||||
```bash
|
||||
ls -R grafana/provisioning/
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
grafana/provisioning/:
|
||||
dashboards dashboards-json datasources
|
||||
|
||||
grafana/provisioning/datasources:
|
||||
tracksolid_postgres.yaml
|
||||
|
||||
grafana/provisioning/dashboards:
|
||||
noc_fleet.yaml
|
||||
|
||||
grafana/provisioning/dashboards-json:
|
||||
noc_fleet_dashboard.json
|
||||
```
|
||||
|
||||
### 1.4 Verify Database Has Live Data
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT COUNT(*) AS devices,
|
||||
COUNT(lp.imei) AS with_position
|
||||
FROM tracksolid.devices d
|
||||
LEFT JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1;
|
||||
"
|
||||
```
|
||||
|
||||
Expected: `devices` = total enabled vehicles, `with_position` > 0 (at least some vehicles have GPS fixes).
|
||||
|
||||
---
|
||||
|
||||
## 2. Deploy
|
||||
|
||||
### Start Grafana
|
||||
|
||||
```bash
|
||||
docker compose up -d grafana
|
||||
```
|
||||
|
||||
### Confirm Container Is Running
|
||||
|
||||
```bash
|
||||
docker compose ps grafana
|
||||
```
|
||||
|
||||
Expected `STATUS`: `Up` (not `Restarting` or `Exit`).
|
||||
|
||||
### Tail Startup Logs
|
||||
|
||||
```bash
|
||||
docker compose logs --follow grafana
|
||||
```
|
||||
|
||||
Watch for these lines — they confirm provisioning loaded successfully:
|
||||
|
||||
```
|
||||
msg="Starting Grafana"
|
||||
msg="Provisioning datasource" name=TracksolidDB
|
||||
msg="Finished provisioning data sources"
|
||||
msg="Inserting/updating dashboard" name="NOC Fleet Operations — Live"
|
||||
msg="HTTP Server Listen" address=0.0.0.0:3000
|
||||
```
|
||||
|
||||
Press `Ctrl+C` to stop following once you see the server listening.
|
||||
|
||||
---
|
||||
|
||||
## 3. Post-Deployment Verification
|
||||
|
||||
### 3.1 Check Provisioning Loaded
|
||||
|
||||
```bash
|
||||
docker compose logs grafana | grep -E "provision|Inserting|dashboard|datasource"
|
||||
```
|
||||
|
||||
| What to look for | Means |
|
||||
|---|---|
|
||||
| `Provisioning datasource` with `name=TracksolidDB` | Datasource YAML was read |
|
||||
| `Finished provisioning data sources` | Datasource created successfully |
|
||||
| `Inserting/updating dashboard` with `noc-fleet-live` | Dashboard JSON was loaded |
|
||||
| `Failed to provision` or `Error` | See Troubleshooting section |
|
||||
|
||||
### 3.2 Verify Datasource via API
|
||||
|
||||
```bash
|
||||
curl -s -u admin:${GRAFANA_ADMIN_PASSWORD} \
|
||||
http://localhost:3000/api/datasources/name/TracksolidDB \
|
||||
| python3 -m json.tool | grep -E '"name"|"uid"|"type"|"url"'
|
||||
```
|
||||
|
||||
Expected:
|
||||
```json
|
||||
"name": "TracksolidDB",
|
||||
"type": "postgres",
|
||||
"uid": "tracksolid_pg",
|
||||
"url": "timescale_db:5432",
|
||||
```
|
||||
|
||||
### 3.3 Test Datasource Connection via API
|
||||
|
||||
```bash
|
||||
curl -s -u admin:${GRAFANA_ADMIN_PASSWORD} \
|
||||
-X POST http://localhost:3000/api/datasources/uid/tracksolid_pg/health \
|
||||
| python3 -m json.tool
|
||||
```
|
||||
|
||||
Expected:
|
||||
```json
|
||||
{
|
||||
"message": "Database Connection OK",
|
||||
"status": "OK"
|
||||
}
|
||||
```
|
||||
|
||||
If status is `ERROR` — see [Troubleshooting: Datasource Connection Fails](#datasource-connection-fails).
|
||||
|
||||
### 3.4 Verify Dashboard Is Registered
|
||||
|
||||
```bash
|
||||
curl -s -u admin:${GRAFANA_ADMIN_PASSWORD} \
|
||||
http://localhost:3000/api/dashboards/uid/noc-fleet-live \
|
||||
| python3 -m json.tool | grep -E '"title"|"uid"|"version"'
|
||||
```
|
||||
|
||||
Expected:
|
||||
```json
|
||||
"title": "NOC Fleet Operations — Live",
|
||||
"uid": "noc-fleet-live",
|
||||
"version": 1,
|
||||
```
|
||||
|
||||
### 3.5 Open the Dashboard in a Browser
|
||||
|
||||
Navigate to: `http://localhost:3000/d/noc-fleet-live`
|
||||
|
||||
Login with:
|
||||
- **Username:** `admin`
|
||||
- **Password:** value of `GRAFANA_ADMIN_PASSWORD` from `.env`
|
||||
|
||||
The NOC dashboard should load as the home page automatically.
|
||||
|
||||
---
|
||||
|
||||
## 4. Dashboard Panel Verification
|
||||
|
||||
Work through each panel to confirm it renders correctly.
|
||||
|
||||
### 4.1 Stat Panels (Row 1)
|
||||
|
||||
| Panel | Expected | Red Flag |
|
||||
|---|---|---|
|
||||
| Total Vehicles | Integer matching enabled device count (should be ~80) | Shows 0 or `-` |
|
||||
| Online Now | Integer ≤ Total Vehicles | Shows `-` |
|
||||
| Recent (5-30m) | Integer ≤ Total Vehicles | Shows `-` |
|
||||
| Offline | Integer ≤ Total Vehicles | Shows `-` |
|
||||
| Moving Now | Integer ≤ Online Now | Shows `-` |
|
||||
| Avg Speed (km/h) | Numeric value, green < 80 / amber 80-120 / red > 120 | Shows `No data` |
|
||||
|
||||
**Online + Recent + Offline should sum to Total Vehicles.**
|
||||
|
||||
Check the sum:
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT
|
||||
connectivity_status,
|
||||
COUNT(*) AS vehicles
|
||||
FROM tracksolid.v_fleet_status
|
||||
GROUP BY connectivity_status
|
||||
ORDER BY connectivity_status;
|
||||
"
|
||||
```
|
||||
|
||||
### 4.2 Geomap Panel
|
||||
|
||||
Work through this checklist visually:
|
||||
|
||||
- [ ] Map loads with dark Carto basemap (not a grey blank tile)
|
||||
- [ ] Arrow markers appear on the map (not dots or circles)
|
||||
- [ ] Markers are clustered around East Africa (Nairobi / Mombasa / Kampala area)
|
||||
- [ ] Arrows point in different directions — not all the same
|
||||
- [ ] Clicking a marker opens a tooltip showing: Plate, Driver, Speed, Heading, Status, Location
|
||||
- [ ] `lat` and `lng` fields are NOT visible in the tooltip (hidden by field overrides)
|
||||
- [ ] `imei` field is NOT visible in the tooltip
|
||||
|
||||
**Verify direction data exists:**
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT
|
||||
MIN(direction) AS min_dir,
|
||||
MAX(direction) AS max_dir,
|
||||
COUNT(*) FILTER (WHERE direction IS NOT NULL) AS with_direction,
|
||||
COUNT(*) AS total
|
||||
FROM tracksolid.live_positions;
|
||||
"
|
||||
```
|
||||
|
||||
If `with_direction` = 0, all arrows will point North (0°) — this means the GPS devices haven't sent bearing data yet, which is normal for parked vehicles.
|
||||
|
||||
### 4.3 Vehicle Status Table
|
||||
|
||||
- [ ] Table shows all enabled vehicles (row count matches Total Vehicles stat)
|
||||
- [ ] Rows sort: Online (green) → Recent (amber) → Offline (red)
|
||||
- [ ] "Status" column has colour-coded background
|
||||
- [ ] "Speed (km/h)" column shows colour-coded text (green/amber/red)
|
||||
- [ ] "Last Fix" column shows a readable timestamp (not a raw epoch number)
|
||||
- [ ] "Min Ago" column shows integers (minutes since last GPS fix)
|
||||
- [ ] Vehicles with no GPS fix show `null` in speed/location columns but still appear
|
||||
|
||||
**Spot-check a specific vehicle:**
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT
|
||||
d.vehicle_number, d.driver_name, lp.speed,
|
||||
lp.gps_time, lp.lat, lp.lng
|
||||
FROM tracksolid.devices d
|
||||
LEFT JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1
|
||||
LIMIT 5;
|
||||
"
|
||||
```
|
||||
|
||||
### 4.4 Ingestion Health Panel (Collapsed)
|
||||
|
||||
Expand the panel by clicking its title row.
|
||||
|
||||
- [ ] Rows appear for each polling endpoint (`live_positions`, `trips`, `alarms`, etc.)
|
||||
- [ ] "Result" column shows `OK` (green) for all active endpoints
|
||||
- [ ] "Last Run" timestamps are recent (within the last few minutes for live_positions)
|
||||
- [ ] No `FAIL` (red) entries
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT endpoint, success, seconds_ago, error_message
|
||||
FROM tracksolid.v_ingestion_health
|
||||
ORDER BY endpoint;
|
||||
"
|
||||
```
|
||||
|
||||
### 4.5 Auto-Refresh Verification
|
||||
|
||||
1. Note the "Last Fix" timestamp for any vehicle in the table
|
||||
2. Wait 30 seconds
|
||||
3. Confirm the page auto-refreshes (watch the Grafana spinner in the top-right)
|
||||
4. Confirm "Last Fix" values have updated for Online vehicles
|
||||
|
||||
---
|
||||
|
||||
## 5. Database Verification Queries
|
||||
|
||||
Run these directly against the database to validate the data powering each panel.
|
||||
|
||||
### Fleet Status Summary
|
||||
```sql
|
||||
SELECT
|
||||
connectivity_status,
|
||||
COUNT(*) AS vehicles,
|
||||
ROUND(AVG(speed)::numeric, 1) AS avg_speed_kmh
|
||||
FROM tracksolid.v_fleet_status
|
||||
GROUP BY connectivity_status
|
||||
ORDER BY connectivity_status;
|
||||
```
|
||||
|
||||
### Live Position Freshness
|
||||
```sql
|
||||
SELECT
|
||||
COUNT(*) AS total_positions,
|
||||
COUNT(*) FILTER (WHERE gps_time >= NOW() - INTERVAL '5 minutes') AS fresh_5m,
|
||||
COUNT(*) FILTER (WHERE gps_time >= NOW() - INTERVAL '30 minutes') AS fresh_30m,
|
||||
MAX(gps_time) AS newest_fix,
|
||||
MIN(gps_time) AS oldest_fix
|
||||
FROM tracksolid.live_positions;
|
||||
```
|
||||
|
||||
### Moving Vehicles
|
||||
```sql
|
||||
SELECT
|
||||
d.vehicle_number, d.driver_name,
|
||||
lp.speed, lp.direction, lp.loc_desc, lp.gps_time
|
||||
FROM tracksolid.devices d
|
||||
INNER JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1
|
||||
AND lp.speed > 0
|
||||
AND lp.acc_status = '1'
|
||||
ORDER BY lp.speed DESC;
|
||||
```
|
||||
|
||||
### Vehicles With No GPS Fix
|
||||
```sql
|
||||
SELECT
|
||||
d.vehicle_number, d.vehicle_name, d.driver_name, d.city
|
||||
FROM tracksolid.devices d
|
||||
LEFT JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1
|
||||
AND lp.imei IS NULL
|
||||
ORDER BY d.vehicle_number;
|
||||
```
|
||||
|
||||
### Ingestion Pipeline Health
|
||||
```sql
|
||||
SELECT
|
||||
endpoint,
|
||||
run_at,
|
||||
success,
|
||||
imei_count,
|
||||
rows_upserted,
|
||||
duration_ms,
|
||||
seconds_ago,
|
||||
error_message
|
||||
FROM tracksolid.v_ingestion_health
|
||||
ORDER BY endpoint;
|
||||
```
|
||||
|
||||
### Stale Positions (no update in > 1 hour)
|
||||
```sql
|
||||
SELECT
|
||||
d.vehicle_number, d.driver_name, d.city,
|
||||
lp.gps_time,
|
||||
EXTRACT(EPOCH FROM (NOW() - lp.gps_time))::int / 60 AS minutes_since_fix
|
||||
FROM tracksolid.devices d
|
||||
INNER JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.enabled_flag = 1
|
||||
AND lp.gps_time < NOW() - INTERVAL '1 hour'
|
||||
ORDER BY lp.gps_time ASC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Troubleshooting
|
||||
|
||||
### Grafana Container Keeps Restarting
|
||||
|
||||
```bash
|
||||
docker compose logs grafana --tail=50
|
||||
```
|
||||
|
||||
Common causes:
|
||||
|
||||
| Error in logs | Fix |
|
||||
|---|---|
|
||||
| `failed to connect to server` | Database not healthy yet — wait 30s and retry |
|
||||
| `permission denied` on provisioning path | Check the `./grafana/provisioning` directory exists and is readable |
|
||||
| `GF_SECURITY_ADMIN_PASSWORD not set` | Add `GRAFANA_ADMIN_PASSWORD` to `.env` |
|
||||
|
||||
### Datasource Connection Fails
|
||||
|
||||
```bash
|
||||
docker compose logs grafana | grep -i "datasource\|postgres\|connect"
|
||||
```
|
||||
|
||||
Check the password is correct:
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U grafana_ro -d tracksolid_db -c "SELECT 1;"
|
||||
```
|
||||
|
||||
If this fails with `authentication failed`, reset the password:
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c \
|
||||
"ALTER ROLE grafana_ro WITH PASSWORD '<new_password>';"
|
||||
```
|
||||
|
||||
Then update `GRAFANA_DB_RO_PASSWORD` in `.env` and restart:
|
||||
```bash
|
||||
docker compose restart grafana
|
||||
```
|
||||
|
||||
### Dashboard Shows "No Data"
|
||||
|
||||
**Step 1 — confirm the datasource UID matches:**
|
||||
```bash
|
||||
curl -s -u admin:${GRAFANA_ADMIN_PASSWORD} \
|
||||
http://localhost:3000/api/datasources \
|
||||
| python3 -m json.tool | grep uid
|
||||
```
|
||||
|
||||
Must show `"uid": "tracksolid_pg"`. If the UID is different, the dashboard JSON references the wrong datasource.
|
||||
|
||||
**Step 2 — test the query directly in Grafana:**
|
||||
1. Open the dashboard
|
||||
2. Click the panel title → Edit
|
||||
3. Switch to the Query tab
|
||||
4. Click "Run query"
|
||||
5. If it errors, the SQL or connection is the issue
|
||||
|
||||
**Step 3 — verify data exists:**
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c \
|
||||
"SELECT COUNT(*) FROM tracksolid.live_positions;"
|
||||
```
|
||||
|
||||
If this returns 0, the ingestion pipeline has not populated data yet. Check ingestion service logs:
|
||||
```bash
|
||||
docker compose logs ingest_movement --tail=30
|
||||
docker compose logs ingest_events --tail=30
|
||||
```
|
||||
|
||||
### Geomap Shows No Markers
|
||||
|
||||
**Check coordinates are valid:**
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT COUNT(*) AS total,
|
||||
COUNT(*) FILTER (WHERE lat IS NOT NULL AND lng IS NOT NULL) AS with_coords,
|
||||
COUNT(*) FILTER (WHERE lat BETWEEN -90 AND 90 AND lng BETWEEN -180 AND 180) AS valid_coords
|
||||
FROM tracksolid.live_positions;
|
||||
"
|
||||
```
|
||||
|
||||
If `with_coords` = 0: GPS data has not arrived yet. Wait for the next ingest cycle (60 seconds).
|
||||
|
||||
If `valid_coords` < `with_coords`: some coordinates are out of range — data quality issue on the device side.
|
||||
|
||||
**Check the basemap is loading:**
|
||||
|
||||
Open browser DevTools (F12) → Network tab → filter for `tile` or `carto`. If Carto tile requests are failing, there may be no internet access from the browser. Try switching the basemap to OpenStreetMap in the Grafana panel editor temporarily.
|
||||
|
||||
### Arrows All Point North (No Direction Data)
|
||||
|
||||
This is expected for parked vehicles — direction is only meaningful when the vehicle is moving. Confirm:
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT COUNT(*) FILTER (WHERE direction > 0) AS with_bearing,
|
||||
COUNT(*) AS total
|
||||
FROM tracksolid.live_positions;
|
||||
"
|
||||
```
|
||||
|
||||
If `with_bearing` = 0 even for moving vehicles, the GPS device firmware may not be sending bearing in the position payload. Check with the Tracksolid account settings.
|
||||
|
||||
### Dashboard Not Loading as Home Page
|
||||
|
||||
Confirm the environment variable is set:
|
||||
```bash
|
||||
docker compose exec grafana env | grep DASHBOARDS
|
||||
```
|
||||
|
||||
Expected:
|
||||
```
|
||||
GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/etc/grafana/provisioning/dashboards-json/noc_fleet_dashboard.json
|
||||
```
|
||||
|
||||
If missing, confirm `env_file: .env` is present in the grafana service block of `docker-compose.yaml` and restart:
|
||||
```bash
|
||||
docker compose restart grafana
|
||||
```
|
||||
|
||||
### Provisioning Changes Not Reflecting
|
||||
|
||||
Grafana polls the provisioning directory every 30 seconds. If changes to the dashboard JSON are not appearing:
|
||||
|
||||
```bash
|
||||
docker compose restart grafana
|
||||
docker compose logs grafana | grep -i "provision\|dashboard"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Day-to-Day NOC Operations
|
||||
|
||||
### Morning Health Check (run at shift start)
|
||||
|
||||
```bash
|
||||
# 1. Confirm all services are up
|
||||
docker compose ps
|
||||
|
||||
# 2. Check ingestion pipeline ran recently
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c \
|
||||
"SELECT endpoint, run_at, success, seconds_ago FROM tracksolid.v_ingestion_health ORDER BY endpoint;"
|
||||
|
||||
# 3. Check live position freshness
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c \
|
||||
"SELECT connectivity_status, COUNT(*) FROM tracksolid.v_fleet_status GROUP BY 1 ORDER BY 1;"
|
||||
```
|
||||
|
||||
### Investigating a Specific Vehicle
|
||||
|
||||
Replace `KAA 123A` with the actual plate:
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT
|
||||
d.vehicle_number, d.vehicle_name, d.driver_name, d.driver_phone,
|
||||
lp.lat, lp.lng, lp.speed, lp.direction,
|
||||
lp.acc_status, lp.gps_time, lp.loc_desc,
|
||||
EXTRACT(EPOCH FROM (NOW() - lp.gps_time))::int / 60 AS minutes_ago
|
||||
FROM tracksolid.devices d
|
||||
LEFT JOIN tracksolid.live_positions lp USING (imei)
|
||||
WHERE d.vehicle_number = 'KAA 123A';
|
||||
"
|
||||
```
|
||||
|
||||
### Checking Recent Trips for a Vehicle
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT
|
||||
t.start_time, t.end_time,
|
||||
ROUND(t.distance_m / 1000.0, 2) AS distance_km,
|
||||
t.avg_speed_kmh, t.max_speed_kmh,
|
||||
ROUND(t.driving_time_s / 3600.0, 2) AS driving_hours
|
||||
FROM tracksolid.trips t
|
||||
INNER JOIN tracksolid.devices d USING (imei)
|
||||
WHERE d.vehicle_number = 'KAA 123A'
|
||||
AND t.start_time >= NOW() - INTERVAL '24 hours'
|
||||
ORDER BY t.start_time DESC;
|
||||
"
|
||||
```
|
||||
|
||||
### Checking Recent Alarms
|
||||
|
||||
```bash
|
||||
docker compose exec timescale_db psql -U $POSTGRES_USER -d tracksolid_db -c "
|
||||
SELECT
|
||||
d.vehicle_number, d.driver_name,
|
||||
a.alarm_type, a.alarm_name, a.alarm_time, a.speed
|
||||
FROM tracksolid.alarms a
|
||||
INNER JOIN tracksolid.devices d USING (imei)
|
||||
WHERE a.alarm_time >= NOW() - INTERVAL '24 hours'
|
||||
ORDER BY a.alarm_time DESC
|
||||
LIMIT 20;
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Maintenance
|
||||
|
||||
### Restart Grafana Only
|
||||
|
||||
```bash
|
||||
docker compose restart grafana
|
||||
```
|
||||
|
||||
### Restart Full Stack
|
||||
|
||||
```bash
|
||||
docker compose down && docker compose up -d
|
||||
```
|
||||
|
||||
### Update the Dashboard JSON
|
||||
|
||||
1. Edit `grafana/provisioning/dashboards-json/noc_fleet_dashboard.json`
|
||||
2. Grafana auto-reloads within 30 seconds (no restart needed)
|
||||
3. Commit the change: `git add grafana/ && git commit -m "Update NOC dashboard"`
|
||||
|
||||
### Check Container Resource Usage
|
||||
|
||||
```bash
|
||||
docker stats grafana timescale_db --no-stream
|
||||
```
|
||||
|
||||
### Grafana Data Volume Size
|
||||
|
||||
```bash
|
||||
docker system df -v | grep grafana-data
|
||||
```
|
||||
|
||||
### View Grafana Version
|
||||
|
||||
```bash
|
||||
docker compose exec grafana grafana-server -v
|
||||
```
|
||||
|
||||
Expected: `Version 11.0.0`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Command |
|
||||
|---|---|
|
||||
| Start Grafana | `docker compose up -d grafana` |
|
||||
| Stop Grafana | `docker compose stop grafana` |
|
||||
| Restart Grafana | `docker compose restart grafana` |
|
||||
| View logs | `docker compose logs grafana --tail=50` |
|
||||
| Check provisioning | `docker compose logs grafana \| grep -i provision` |
|
||||
| Test datasource | `curl -s -u admin:$GRAFANA_ADMIN_PASSWORD http://localhost:3000/api/datasources/uid/tracksolid_pg/health` |
|
||||
| Open dashboard | `http://localhost:3000/d/noc-fleet-live` |
|
||||
| Fleet summary | `SELECT connectivity_status, COUNT(*) FROM tracksolid.v_fleet_status GROUP BY 1;` |
|
||||
| Ingestion health | `SELECT * FROM tracksolid.v_ingestion_health ORDER BY endpoint;` |
|
||||
|
|
@ -1,5 +1,13 @@
|
|||
# Tracksolid Database Manual
|
||||
|
||||
> **⚠️ Deprecation note (2026-06-10).** This manual predates two cleanups. (1) The `dwh_gold`
|
||||
> schema and its tables (`dim_vehicles`, `fact_daily_fleet_metrics`, `refresh_daily_metrics()`)
|
||||
> were **dropped on 2026-06-05** (migration 13) — any section or query below that references
|
||||
> `dwh_gold.*` is obsolete and will error; treat `tracksolid.*` (+ the `reporting.*` read layer)
|
||||
> as the single source of truth. (2) **Grafana and the n8n/DWH pipeline were removed 2026-06-10**;
|
||||
> dashboards are now the FleetNow / FleetOps SPAs served by `dashboard_api`. Mentions of Grafana
|
||||
> panels / n8n jobs below are historical.
|
||||
|
||||
**Database:** `tracksolid_db`
|
||||
**Host:** `kianiadee@stage.rahamafresh.com`
|
||||
**Container:** `timescale_db-*` (Coolify-generated suffix — changes on redeploy, use dynamic lookup below)
|
||||
|
|
@ -31,8 +39,8 @@ docker exec -it "$TS_DB" psql -U postgres -d tracksolid_db
|
|||
9. [tracksolid.obd_readings](#9-tracksolidobd_readings)
|
||||
10. [tracksolid.parking_events](#10-tracksolidparking_events)
|
||||
11. [tracksolid.fault_codes](#11-trackshopfault_codes)
|
||||
12. [dwh_gold.dim_vehicles](#12-dwh_golddim_vehicles)
|
||||
13. [dwh_gold.fact_daily_fleet_metrics](#13-dwh_goldfact_daily_fleet_metrics)
|
||||
12. ~~dwh_gold.dim_vehicles~~ — *removed 2026-06-05 (migration 13)*
|
||||
13. ~~dwh_gold.fact_daily_fleet_metrics~~ — *removed 2026-06-05 (migration 13)*
|
||||
14. [Business Intelligence Queries](#14-business-intelligence-queries)
|
||||
15. [Today's Metrics — From 00:00 Nairobi Time to Now](#15-todays-metrics--from-0000-nairobi-time-to-now)
|
||||
16. [Daily Analytics](#16-daily-analytics)
|
||||
|
|
@ -51,7 +59,7 @@ The database is organised into three schemas:
|
|||
| Schema | Purpose |
|
||||
|---|---|
|
||||
| `tracksolid` | Raw operational data ingested from the Tracksolid/Jimi Open Platform API |
|
||||
| `dwh_gold` | Pre-aggregated data warehouse layer for reporting and dashboards |
|
||||
| `reporting` | Read layer (map + analytics views/functions) consumed by `dashboard_api` |
|
||||
| `public` | PostGIS spatial reference tables (system-managed) |
|
||||
|
||||
Data is pulled from the Jimi/Tracksolid API on a continuous polling schedule. The `ingestion_log` table records every API call so you can audit pipeline health.
|
||||
|
|
@ -66,7 +74,7 @@ SELECT
|
|||
tablename,
|
||||
pg_size_pretty(pg_total_relation_size(schemaname || '.' || tablename)) AS size
|
||||
FROM pg_tables
|
||||
WHERE schemaname IN ('tracksolid', 'dwh_gold')
|
||||
WHERE schemaname IN ('tracksolid', 'reporting')
|
||||
ORDER BY pg_total_relation_size(schemaname || '.' || tablename) DESC;
|
||||
```
|
||||
|
||||
|
|
@ -89,9 +97,7 @@ UNION ALL SELECT 'tracksolid.fuel_readings', COUNT(*) FROM tracksolid.f
|
|||
UNION ALL SELECT 'tracksolid.temperature_readings', COUNT(*) FROM tracksolid.temperature_readings
|
||||
UNION ALL SELECT 'tracksolid.lbs_readings', COUNT(*) FROM tracksolid.lbs_readings
|
||||
UNION ALL SELECT 'tracksolid.geofences', COUNT(*) FROM tracksolid.geofences
|
||||
UNION ALL SELECT 'tracksolid.ingestion_log', COUNT(*) FROM tracksolid.ingestion_log
|
||||
UNION ALL SELECT 'dwh_gold.dim_vehicles', COUNT(*) FROM dwh_gold.dim_vehicles
|
||||
UNION ALL SELECT 'dwh_gold.fact_daily_fleet_metrics', COUNT(*) FROM dwh_gold.fact_daily_fleet_metrics;
|
||||
UNION ALL SELECT 'tracksolid.ingestion_log', COUNT(*) FROM tracksolid.ingestion_log;
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -283,9 +289,9 @@ ORDER BY delta_kmh DESC;
|
|||
|
||||
---
|
||||
|
||||
### Continuous route trace for Grafana map panel
|
||||
### Continuous route trace for map rendering
|
||||
|
||||
Returns ordered waypoints for all vehicles in the last hour suitable for rendering as a continuous path in Grafana's Geomap plugin. The `source` filter includes both ingestion paths to maximise trace density.
|
||||
Returns ordered waypoints for all vehicles in the last hour suitable for rendering as a continuous path on a map (e.g. the FleetNow trail layer via `dashboard_api`). The `source` filter includes both ingestion paths to maximise trace density.
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
|
|
@ -593,31 +599,16 @@ At the time of audit this table contained **0 rows**, which is consistent with t
|
|||
|
||||
---
|
||||
|
||||
## 12. dwh_gold.dim_vehicles
|
||||
## 12. dwh_gold.dim_vehicles — REMOVED
|
||||
|
||||
This is the vehicles dimension table in the data warehouse gold layer. It is intended to be a clean, enriched, business-friendly view of the fleet — combining device metadata from `tracksolid.devices` with any additional attributes needed for reporting (cost centre, vehicle category, assigned route, etc.). Dimension tables in a star schema are typically populated by an ETL job that joins and transforms raw operational tables.
|
||||
> **Removed 2026-06-05 (migration 13).** The `dwh_gold` star-schema layer (`dim_vehicles`,
|
||||
> `fact_daily_fleet_metrics`, `refresh_daily_metrics()`) was never populated and was dropped as
|
||||
> unused. Compute fleet metrics directly from `tracksolid.*` / `reporting.*` (see §14–§19).
|
||||
|
||||
At the time of audit this table contained **0 rows**. The ETL pipeline that populates the gold layer has not yet been run.
|
||||
## 13. dwh_gold.fact_daily_fleet_metrics — REMOVED
|
||||
|
||||
### Describe table structure
|
||||
|
||||
```sql
|
||||
\d dwh_gold.dim_vehicles
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. dwh_gold.fact_daily_fleet_metrics
|
||||
|
||||
This is the central fact table of the data warehouse. It is designed to hold one pre-aggregated row per vehicle per day, summarising distance driven, fuel consumed, driving time, idle time, trip count, first departure time, last return time, and alarm counts. Pre-aggregating at this level makes dashboards and management reports extremely fast — a full month's fleet summary requires scanning at most 63 vehicles × 31 days = ~2,000 rows rather than hundreds of thousands of raw trip and position records.
|
||||
|
||||
At the time of audit this table contained **0 rows**. Once the ETL job is running, this should be the primary data source for all Grafana dashboards and business reports.
|
||||
|
||||
### Describe table structure
|
||||
|
||||
```sql
|
||||
\d dwh_gold.fact_daily_fleet_metrics
|
||||
```
|
||||
> **Removed 2026-06-05 (migration 13).** See §12. The pre-aggregated daily fact table no longer
|
||||
> exists; the BI queries below run against the live `tracksolid.trips` / `position_history` tables.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -972,7 +963,7 @@ ORDER BY created_at DESC;
|
|||
|
||||
## 16. Daily Analytics
|
||||
|
||||
Daily analytics answer the question: **"What happened across the fleet today?"** All queries use `Africa/Nairobi` timezone so that a working day (typically 08:00–20:00 EAT) maps cleanly to a single calendar date. The `dwh_gold.refresh_daily_metrics()` function pre-aggregates these into `fact_daily_fleet_metrics` for fast dashboard queries — run it nightly for the previous day.
|
||||
Daily analytics answer the question: **"What happened across the fleet today?"** All queries use `Africa/Nairobi` timezone so that a working day (typically 08:00–20:00 EAT) maps cleanly to a single calendar date. (The former `dwh_gold.refresh_daily_metrics()` pre-aggregation was removed 2026-06-05 — these queries run directly against `tracksolid.trips` / `position_history`.)
|
||||
|
||||
### Day summary — all vehicles, single date
|
||||
|
||||
|
|
@ -1238,7 +1229,7 @@ ORDER BY work_date;
|
|||
|
||||
## 18. Monthly Analytics
|
||||
|
||||
Monthly analytics answer: **"How did the fleet perform this month?"** Queries use `DATE_TRUNC('month', ...)` to anchor to the first of the current month. For historical months replace with a specific date literal. When `dwh_gold.fact_daily_fleet_metrics` is fully populated these queries can be rewritten against that table for much faster execution.
|
||||
Monthly analytics answer: **"How did the fleet perform this month?"** Queries use `DATE_TRUNC('month', ...)` to anchor to the first of the current month. For historical months replace with a specific date literal.
|
||||
|
||||
### Monthly summary per vehicle
|
||||
|
||||
|
|
@ -1539,6 +1530,6 @@ The following issues were identified during the April 2026 audit. Each represent
|
|||
| 6 | `heartbeats` | 0 rows | Cannot distinguish parked-alive from powered-off | **Open** — verify tracker firmware supports heartbeat push |
|
||||
| 7 | `obd_readings` | 0 rows | No engine health data | **Open** — requires OBD cable installation + `/pushobd` webhook registration in Tracksolid account |
|
||||
| 8 | `parking_events` | 0 rows despite 358 successful API calls | No parking dwell-time reporting | **Fixed** in `ingest_movement_rev.py` [FIX-M13] — added missing `account` and `acc_type=0` params; fixed `durSecond` field mapping |
|
||||
| 9 | `dwh_gold.*` | Both tables empty | Grafana dashboards have no data | **Fixed** — migration 05 adds `refresh_daily_metrics()` ETL function; run nightly via cron or n8n |
|
||||
| 10 | `position_history` | Only 1 fix/min per vehicle from fleet sweep — route traces incomplete | Grafana map paths had gaps; speed profiles too coarse for harsh-driving detection | **Fixed** — `poll_track_list()` added [FIX-M14]; captures every device waypoint every 30 min; density increases to 2–6 fixes/min per vehicle |
|
||||
| 9 | `dwh_gold.*` | Both tables empty (never populated) | — | **Removed** 2026-06-05 (migration 13) — the gold layer was dropped as unused; metrics come from `tracksolid.*` directly |
|
||||
| 10 | `position_history` | Only 1 fix/min per vehicle from fleet sweep — route traces incomplete | Map route paths had gaps; speed profiles too coarse for harsh-driving detection | **Fixed** — `poll_track_list()` added [FIX-M14]; captures every device waypoint every 30 min; density increases to 2–6 fixes/min per vehicle |
|
||||
| 11 | `live_positions` | No on-demand refresh mechanism for specific vehicles | Alarm enrichment and stale-device recovery required waiting up to 60s | **Fixed** — `get_device_locations()` utility added [FIX-M15]; call with specific IMEIs for instant precision refresh |
|
||||
|
|
|
|||
|
|
@ -643,11 +643,6 @@ tsdb -c "\d tracksolid.trips"
|
|||
tsdb -c "\d tracksolid.live_positions"
|
||||
```
|
||||
|
||||
### Run DWH gold ETL for yesterday
|
||||
```sql
|
||||
SELECT dwh_gold.refresh_daily_metrics(CURRENT_DATE - 1);
|
||||
```
|
||||
|
||||
### Apply a migration manually (from your Mac)
|
||||
```bash
|
||||
ssh kianiadee@rahamafresh.com \
|
||||
|
|
|
|||
|
|
@ -2,6 +2,14 @@
|
|||
## Tracksolid Pro · Field Operations & Logistics Intelligence Assessment
|
||||
### April 2026
|
||||
|
||||
> **⚠️ Deprecation note (2026-06-10).** This SQL library is still the reference for query
|
||||
> *patterns*, but two things changed after it was written: (1) the `dwh_gold` pre-aggregation
|
||||
> layer (`fact_daily_fleet_metrics`, etc.) was **dropped 2026-06-05** (migration 13) — any query
|
||||
> below that reads `dwh_gold.*` will error; run the equivalent aggregation directly against
|
||||
> `tracksolid.trips` / `position_history`. (2) **Grafana and the n8n/DWH pipeline were removed
|
||||
> 2026-06-10** — panels are now FleetNow / FleetOps SPAs served by `dashboard_api`. Read
|
||||
> "Grafana panel" / "n8n" below as "dashboard" / "the ingest or ops integration layer".
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
|
@ -13,7 +21,7 @@
|
|||
4. [Real-Time Dispatch & Field-Service SLAs](#4-real-time-dispatch--field-service-slas)
|
||||
5. [Distance per Driver per Day](#5-distance-per-driver-per-day)
|
||||
6. [Business Questions Now Answerable](#6-business-questions-now-answerable)
|
||||
7. [Grafana Dashboard Blueprint](#7-grafana-dashboard-blueprint)
|
||||
7. [Dashboard Blueprint](#7-dashboard-blueprint)
|
||||
8. [What Unlocks the Remaining 30%](#8-what-unlocks-the-remaining-30)
|
||||
9. [Fleet Readiness Scorecard](#9-fleet-readiness-scorecard)
|
||||
10. [Service-Interval Forecaster](#10-service-interval-forecaster)
|
||||
|
|
@ -22,7 +30,7 @@
|
|||
|
||||
## 0. How to Use This Document
|
||||
|
||||
Every query in this document is tagged by intended consumption cadence. Build Grafana panels, alert rules, and scheduled reports against the tag — not the SQL text — so that moving a metric between dashboard and alert is a one-line change.
|
||||
Every query in this document is tagged by intended consumption cadence. Build dashboard panels, alert rules, and scheduled reports against the tag — not the SQL text — so that moving a metric between dashboard and alert is a one-line change.
|
||||
|
||||
| Tag | Meaning | Typical cadence | Owner |
|
||||
|---|---|---|---|
|
||||
|
|
@ -77,7 +85,7 @@ Once deployed, the ingestion stack populates the following data sources:
|
|||
| `tracksolid.parking_events` | Stop/idle events with address and duration | Every 15 minutes |
|
||||
| `tracksolid.alarms` | Alarm events with type, severity, location | Every 5 minutes |
|
||||
| `tracksolid.devices` | Vehicle and driver registry | Daily at 02:00 |
|
||||
| `dwh_gold.fact_daily_fleet_metrics` | Daily KPI aggregates per vehicle | Nightly ETL |
|
||||
| ~~`dwh_gold.fact_daily_fleet_metrics`~~ | *Removed 2026-06-05 (migration 13) — aggregate from `tracksolid.trips` directly* | — |
|
||||
|
||||
**Position history density** improvement with `poll_track_list` (POLL-01):
|
||||
|
||||
|
|
@ -749,11 +757,11 @@ LIMIT 5;
|
|||
|
||||
---
|
||||
|
||||
### 4.2 Dispatch Logic for n8n or API Integration
|
||||
### 4.2 Dispatch Logic for API Integration
|
||||
|
||||
The recommended workflow when a new job/ticket arrives:
|
||||
|
||||
1. **Trigger:** New job created (webhook from job management system or n8n)
|
||||
1. **Trigger:** New job created (webhook from job management system)
|
||||
2. **Force-refresh positions:** Call `get_device_locations()` for the top 10 candidate IMEIs to get sub-second fresh positions before committing
|
||||
3. **Run dispatch query** above with job coordinates
|
||||
4. **Filter by vehicle type** if the job requires specific capacity (`AND d.vehicle_category = 'van'`)
|
||||
|
|
@ -765,7 +773,7 @@ The recommended workflow when a new job/ticket arrives:
|
|||
|
||||
### 4.3 All Active Vehicles Map — Live Fleet View
|
||||
|
||||
Returns all vehicles with a position fix in the last 10 minutes, suitable for a Grafana Geomap panel with auto-refresh at 30 seconds.
|
||||
Returns all vehicles with a position fix in the last 10 minutes, suitable for a map panel with auto-refresh at 30 seconds.
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
|
|
@ -819,7 +827,7 @@ CREATE INDEX IF NOT EXISTS idx_dispatch_log_assigned_at
|
|||
ON tracksolid.dispatch_log(assigned_at DESC);
|
||||
```
|
||||
|
||||
**Population plan:** n8n or the ops integration layer writes one row per dispatch at assignment. A nightly job back-fills `first_movement_at` / `on_site_at` by joining `trips` and `live_positions` against `job_geom`.
|
||||
**Population plan:** the ops integration layer writes one row per dispatch at assignment. A nightly job back-fills `first_movement_at` / `on_site_at` by joining `trips` and `live_positions` against `job_geom`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -1035,7 +1043,9 @@ Status key: **✅ Ready** = answerable once new stack deployed | **⚙ Needs dat
|
|||
|
||||
---
|
||||
|
||||
## 7. Grafana Dashboard Blueprint
|
||||
## 7. Dashboard Blueprint
|
||||
|
||||
*(Panel layout is tool-agnostic — now implemented in the FleetNow / FleetOps SPAs via `dashboard_api`.)*
|
||||
|
||||
### Panel 1 — Real-Time Fleet Map (auto-refresh: 30s)
|
||||
- **Type:** Geomap
|
||||
|
|
@ -1114,8 +1124,6 @@ docker exec -it ingest_movement python sync_driver_audit.py
|
|||
docker exec -it ingest_movement python import_drivers_csv.py # dry-run
|
||||
docker exec -it ingest_movement python import_drivers_csv.py --apply # commit
|
||||
|
||||
# 6. Schedule nightly ETL
|
||||
# Add to cron or n8n: SELECT dwh_gold.refresh_daily_metrics(CURRENT_DATE - 1);
|
||||
```
|
||||
|
||||
**Expected state after Step 0:**
|
||||
|
|
@ -1329,7 +1337,7 @@ ORDER BY readiness_score ASC NULLS FIRST;
|
|||
| < 60 | Red — unreliable | Do not dispatch for priority jobs; service or replace |
|
||||
| NULL | Silent | Vehicle never reported — investigate install / commission |
|
||||
|
||||
The scorecard is also the cleanest Panel 2 replacement for the Grafana Fleet Status Summary.
|
||||
The scorecard is also the cleanest Panel 2 replacement for the Fleet Status Summary.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +1,17 @@
|
|||
# pgbouncer sidecar deployment
|
||||
|
||||
> **⚠️ SUPERSEDED — pgbouncer REMOVED 2026-06-10.** The pooler was deployed but
|
||||
> stayed dormant (zero app clients ever pointed at `:6432`; every service connects
|
||||
> directly to `timescale_db:5432`). In-process pooling (`ts_shared_rev`
|
||||
> `ThreadedConnectionPool`) is more than sufficient at fleet scale, and transaction-mode
|
||||
> pooling is unsafe for the advisory-lock'd `v_trips` refresher (FIX-D02). The
|
||||
> `pgbouncer` service was dropped from `docker-compose.yaml`; migration
|
||||
> `10_pgbouncer_auth.sql` (the `pgbouncer` role + `public.user_lookup()` fn) is left
|
||||
> applied but **inert**. This document is retained for history only.
|
||||
|
||||
**Date:** 2026-05-07
|
||||
**Branch:** `quality-program-2026-04-12`
|
||||
**Status:** Phase 1 deployed. Phase 2 (pgAdmin4 sidecar) was rolled back on 2026-05-08 — see git history (`bc020cb`, then reverted).
|
||||
**Status:** Phase 1 deployed, then **removed 2026-06-10** (see banner above). Phase 2 (pgAdmin4 sidecar) was rolled back on 2026-05-08 — see git history (`bc020cb`, then reverted).
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,367 +0,0 @@
|
|||
-- =============================================================
|
||||
-- TRACKSOLID DWH SETUP & PERMISSIONS
|
||||
-- Target Database: tracksolid_dwh
|
||||
-- =============================================================
|
||||
|
||||
-- 1. EXTENSIONS
|
||||
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
|
||||
CREATE EXTENSION IF NOT EXISTS pg_trgm;
|
||||
CREATE EXTENSION IF NOT EXISTS btree_gist;
|
||||
CREATE EXTENSION IF NOT EXISTS postgis; -- REQUIRED for geometry(Point,4326) columns
|
||||
|
||||
-- 2. ROLE CREATION (Idempotent)
|
||||
-- SECURITY: Passwords below are placeholders. Before applying this file:
|
||||
-- 1. Generate two strong secrets (e.g. `openssl rand -hex 24`)
|
||||
-- 2. Replace both CHANGE_ME_BEFORE_APPLY tokens in-session (do NOT commit real values)
|
||||
-- 3. Store the generated secrets in the n8n / Grafana credential stores only
|
||||
-- Rotation: `ALTER ROLE <role> PASSWORD '<new secret>'` as a superuser.
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'dwh_owner') THEN
|
||||
CREATE ROLE dwh_owner WITH LOGIN PASSWORD 'CHANGE_ME_BEFORE_APPLY';
|
||||
END IF;
|
||||
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'grafana_ro') THEN
|
||||
CREATE ROLE grafana_ro WITH LOGIN PASSWORD 'CHANGE_ME_BEFORE_APPLY';
|
||||
END IF;
|
||||
END$$;
|
||||
|
||||
-- Grant database connection
|
||||
GRANT CONNECT ON DATABASE tracksolid_dwh TO dwh_owner;
|
||||
GRANT CONNECT ON DATABASE tracksolid_dwh TO grafana_ro;
|
||||
|
||||
-- 3. SCHEMAS
|
||||
CREATE SCHEMA IF NOT EXISTS bronze AUTHORIZATION dwh_owner;
|
||||
CREATE SCHEMA IF NOT EXISTS silver AUTHORIZATION dwh_owner;
|
||||
CREATE SCHEMA IF NOT EXISTS gold AUTHORIZATION dwh_owner;
|
||||
|
||||
ALTER DATABASE tracksolid_dwh SET search_path TO bronze, silver, gold, public;
|
||||
|
||||
-- 4. PERMISSIONS & DEFAULT PRIVILEGES (Critical for Security & Automation)
|
||||
-- Schema access
|
||||
GRANT USAGE, CREATE ON SCHEMA bronze TO dwh_owner;
|
||||
GRANT USAGE, CREATE ON SCHEMA silver TO dwh_owner;
|
||||
GRANT USAGE, CREATE ON SCHEMA gold TO dwh_owner;
|
||||
GRANT USAGE ON SCHEMA bronze TO grafana_ro;
|
||||
GRANT USAGE ON SCHEMA silver TO grafana_ro;
|
||||
GRANT USAGE ON SCHEMA gold TO grafana_ro;
|
||||
GRANT USAGE ON SCHEMA public TO dwh_owner, grafana_ro;
|
||||
|
||||
-- Existing table access for Grafana
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA bronze TO grafana_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA silver TO grafana_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA gold TO grafana_ro;
|
||||
|
||||
-- FUTURE table access: Any table created by dwh_owner will automatically be readable by grafana_ro
|
||||
ALTER DEFAULT PRIVILEGES FOR ROLE dwh_owner IN SCHEMA bronze GRANT SELECT ON TABLES TO grafana_ro;
|
||||
ALTER DEFAULT PRIVILEGES FOR ROLE dwh_owner IN SCHEMA silver GRANT SELECT ON TABLES TO grafana_ro;
|
||||
ALTER DEFAULT PRIVILEGES FOR ROLE dwh_owner IN SCHEMA gold GRANT SELECT ON TABLES TO grafana_ro;
|
||||
|
||||
-- 5. BRONZE SCHEMA TABLES
|
||||
-- Run as dwh_owner to ensure correct ownership & default privileges apply
|
||||
SET ROLE dwh_owner;
|
||||
SET search_path TO bronze, public;
|
||||
|
||||
-- 5.1 DEVICES (Slowly Changing Dimension - Type 2 handled in Silver)
|
||||
CREATE TABLE IF NOT EXISTS bronze.devices (
|
||||
imei TEXT PRIMARY KEY,
|
||||
device_name TEXT,
|
||||
mc_type TEXT,
|
||||
mc_type_use_scope TEXT,
|
||||
vehicle_name TEXT,
|
||||
vehicle_number TEXT,
|
||||
vehicle_models TEXT,
|
||||
vehicle_icon TEXT,
|
||||
vin TEXT,
|
||||
engine_number TEXT,
|
||||
vehicle_brand TEXT,
|
||||
fuel_100km NUMERIC(6,2),
|
||||
driver_name TEXT,
|
||||
driver_phone TEXT,
|
||||
sim TEXT,
|
||||
iccid TEXT,
|
||||
imsi TEXT,
|
||||
account TEXT,
|
||||
customer_name TEXT,
|
||||
device_group_id TEXT,
|
||||
device_group TEXT,
|
||||
activation_time TIMESTAMPTZ,
|
||||
expiration TIMESTAMPTZ,
|
||||
enabled_flag SMALLINT DEFAULT 1 NOT NULL,
|
||||
status TEXT DEFAULT 'active'::text,
|
||||
city TEXT,
|
||||
current_mileage_km NUMERIC(12,2),
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
updated_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
last_synced_at TIMESTAMPTZ,
|
||||
vehicle_category TEXT,
|
||||
cost_centre TEXT,
|
||||
assigned_route TEXT,
|
||||
depot_geom geometry(Point,4326),
|
||||
depot_address TEXT,
|
||||
assigned_city TEXT,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.2 POSITION HISTORY (High-volume fact table)
|
||||
CREATE TABLE IF NOT EXISTS bronze.position_history (
|
||||
imei TEXT NOT NULL,
|
||||
gps_time TIMESTAMPTZ NOT NULL,
|
||||
geom geometry(Point,4326),
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
speed NUMERIC(7,2),
|
||||
direction NUMERIC(6,2),
|
||||
acc_status TEXT,
|
||||
satellite SMALLINT,
|
||||
current_mileage NUMERIC(12,2),
|
||||
recorded_at TIMESTAMPTZ DEFAULT now(),
|
||||
altitude NUMERIC(8,2),
|
||||
post_type SMALLINT,
|
||||
source TEXT DEFAULT 'poll'::text,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
PRIMARY KEY (imei, gps_time)
|
||||
);
|
||||
|
||||
-- 5.3 TRIPS (Aggregated fact table)
|
||||
CREATE TABLE IF NOT EXISTS bronze.trips (
|
||||
id BIGINT NOT NULL,
|
||||
imei TEXT NOT NULL,
|
||||
start_time TIMESTAMPTZ NOT NULL,
|
||||
end_time TIMESTAMPTZ,
|
||||
start_geom geometry(Point,4326),
|
||||
end_geom geometry(Point,4326),
|
||||
distance_km NUMERIC(12,2),
|
||||
avg_speed_kmh NUMERIC(7,2),
|
||||
max_speed_kmh NUMERIC(7,2),
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
fuel_consumed_l NUMERIC(8,2),
|
||||
idle_time_s INTEGER,
|
||||
driving_time_s INTEGER,
|
||||
trip_seq INTEGER,
|
||||
source TEXT DEFAULT 'poll'::text,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
PRIMARY KEY (id)
|
||||
);
|
||||
|
||||
-- 5.4 ALARMS (Event log fact table)
|
||||
CREATE TABLE IF NOT EXISTS bronze.alarms (
|
||||
id BIGINT PRIMARY KEY,
|
||||
imei TEXT,
|
||||
alarm_type TEXT,
|
||||
alarm_time TIMESTAMPTZ,
|
||||
geom geometry(Point,4326),
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
speed NUMERIC(7,2),
|
||||
acc_status TEXT,
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
alarm_name TEXT,
|
||||
source TEXT DEFAULT 'poll'::text,
|
||||
severity TEXT,
|
||||
geofence_id TEXT,
|
||||
geofence_name TEXT,
|
||||
acknowledged_at TIMESTAMPTZ,
|
||||
acknowledged_by TEXT,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.5 DEVICE EVENTS (Connection lifecycle)
|
||||
CREATE TABLE IF NOT EXISTS bronze.device_events (
|
||||
id BIGINT PRIMARY KEY,
|
||||
imei TEXT NOT NULL,
|
||||
event_type TEXT NOT NULL,
|
||||
event_time TIMESTAMPTZ NOT NULL,
|
||||
timezone TEXT,
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.6 DISPATCH LOG (Operational/SLA tracking)
|
||||
CREATE TABLE IF NOT EXISTS bronze.dispatch_log (
|
||||
dispatch_id BIGINT PRIMARY KEY,
|
||||
ticket_id TEXT NOT NULL,
|
||||
imei TEXT NOT NULL,
|
||||
driver_name TEXT,
|
||||
job_lat DOUBLE PRECISION NOT NULL,
|
||||
job_lng DOUBLE PRECISION NOT NULL,
|
||||
job_geom geometry(Point,4326),
|
||||
assigned_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
first_movement_at TIMESTAMPTZ,
|
||||
on_site_at TIMESTAMPTZ,
|
||||
resolved_at TIMESTAMPTZ,
|
||||
cancelled_at TIMESTAMPTZ,
|
||||
distance_km NUMERIC(8,2),
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.7 FAULT CODES (OBD/DTC diagnostics)
|
||||
CREATE TABLE IF NOT EXISTS bronze.fault_codes (
|
||||
id BIGINT PRIMARY KEY,
|
||||
imei TEXT NOT NULL,
|
||||
reported_at TIMESTAMPTZ NOT NULL,
|
||||
fault_code TEXT NOT NULL,
|
||||
status_flags INTEGER,
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
geom geometry(Point,4326),
|
||||
event_time TIMESTAMPTZ,
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.8 FUEL READINGS
|
||||
CREATE TABLE IF NOT EXISTS bronze.fuel_readings (
|
||||
imei TEXT NOT NULL,
|
||||
reading_time TIMESTAMPTZ NOT NULL,
|
||||
sensor_path TEXT,
|
||||
value NUMERIC(10,3),
|
||||
unit TEXT,
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
geom geometry(Point,4326),
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
PRIMARY KEY (imei, reading_time)
|
||||
);
|
||||
|
||||
-- 5.9 GEOFENCES (Dimension/Reference)
|
||||
CREATE TABLE IF NOT EXISTS bronze.geofences (
|
||||
id BIGINT PRIMARY KEY,
|
||||
fence_id TEXT,
|
||||
fence_name TEXT NOT NULL,
|
||||
fence_type TEXT,
|
||||
geom geometry(Geometry,4326),
|
||||
radius_m NUMERIC(10,2),
|
||||
description TEXT,
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
updated_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.10 HEARTBEATS (Device health/ping)
|
||||
CREATE TABLE IF NOT EXISTS bronze.heartbeats (
|
||||
imei TEXT NOT NULL,
|
||||
gate_time TIMESTAMPTZ NOT NULL,
|
||||
power_level SMALLINT,
|
||||
gsm_signal SMALLINT,
|
||||
acc_status SMALLINT,
|
||||
power_status SMALLINT,
|
||||
fortify SMALLINT,
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
PRIMARY KEY (imei, gate_time)
|
||||
);
|
||||
|
||||
-- 5.11 INGESTION LOG (Metadata for tracking loads)
|
||||
CREATE TABLE IF NOT EXISTS bronze.ingestion_log (
|
||||
id BIGINT PRIMARY KEY,
|
||||
run_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
endpoint TEXT NOT NULL,
|
||||
imei_count INTEGER DEFAULT 0 NOT NULL,
|
||||
rows_upserted INTEGER DEFAULT 0 NOT NULL,
|
||||
rows_inserted INTEGER DEFAULT 0 NOT NULL,
|
||||
duration_ms INTEGER DEFAULT 0 NOT NULL,
|
||||
success BOOLEAN DEFAULT true NOT NULL,
|
||||
error_code TEXT,
|
||||
error_message TEXT,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.12 LBS READINGS (Fallback positioning)
|
||||
CREATE TABLE IF NOT EXISTS bronze.lbs_readings (
|
||||
id BIGINT PRIMARY KEY,
|
||||
imei TEXT NOT NULL,
|
||||
gate_time TIMESTAMPTZ NOT NULL,
|
||||
post_type TEXT,
|
||||
lbs_data JSONB,
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.13 LIVE POSITIONS (Current state snapshot)
|
||||
CREATE TABLE IF NOT EXISTS bronze.live_positions (
|
||||
imei TEXT PRIMARY KEY,
|
||||
geom geometry(Point,4326),
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
pos_type TEXT,
|
||||
confidence SMALLINT,
|
||||
gps_time TIMESTAMPTZ,
|
||||
hb_time TIMESTAMPTZ,
|
||||
speed NUMERIC(7,2),
|
||||
direction NUMERIC(6,2),
|
||||
acc_status TEXT,
|
||||
gps_signal SMALLINT,
|
||||
gps_num SMALLINT,
|
||||
elec_quantity NUMERIC(5,2),
|
||||
power_value NUMERIC(5,2),
|
||||
battery_power_val NUMERIC(5,2),
|
||||
tracker_oil TEXT,
|
||||
temperature NUMERIC(8,2),
|
||||
current_mileage NUMERIC(12,2),
|
||||
device_status TEXT,
|
||||
expire_flag TEXT,
|
||||
activation_flag TEXT,
|
||||
loc_desc TEXT,
|
||||
recorded_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
updated_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.14 OBD READINGS (Vehicle diagnostics)
|
||||
CREATE TABLE IF NOT EXISTS bronze.obd_readings (
|
||||
id BIGINT PRIMARY KEY,
|
||||
imei TEXT,
|
||||
reading_time TIMESTAMPTZ,
|
||||
engine_rpm INTEGER,
|
||||
fuel_level_pct NUMERIC(5,2),
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
car_type SMALLINT,
|
||||
acc_state SMALLINT,
|
||||
status_flags INTEGER,
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
geom geometry(Point,4326),
|
||||
obd_data JSONB,
|
||||
coolant_temp_c NUMERIC(6,2),
|
||||
battery_voltage NUMERIC(5,2),
|
||||
intake_pressure NUMERIC(6,2),
|
||||
throttle_pct NUMERIC(5,2),
|
||||
vehicle_speed NUMERIC(7,2),
|
||||
engine_load_pct NUMERIC(5,2),
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.15 PARKING EVENTS
|
||||
CREATE TABLE IF NOT EXISTS bronze.parking_events (
|
||||
id BIGINT PRIMARY KEY,
|
||||
imei TEXT NOT NULL,
|
||||
event_type TEXT,
|
||||
start_time TIMESTAMPTZ NOT NULL,
|
||||
end_time TIMESTAMPTZ,
|
||||
duration_seconds INTEGER,
|
||||
geom geometry(Point,4326),
|
||||
address TEXT,
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 5.16 TEMPERATURE READINGS (Cold chain sensors)
|
||||
CREATE TABLE IF NOT EXISTS bronze.temperature_readings (
|
||||
imei TEXT NOT NULL,
|
||||
reading_time TIMESTAMPTZ NOT NULL,
|
||||
temperature NUMERIC(6,2),
|
||||
humidity_pct NUMERIC(5,2),
|
||||
created_at TIMESTAMPTZ DEFAULT now() NOT NULL,
|
||||
ingested_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
PRIMARY KEY (imei, reading_time)
|
||||
);
|
||||
|
||||
-- Reset role back to superuser
|
||||
RESET ROLE;
|
||||
RESET search_path;
|
||||
|
||||
-- 6. VERIFICATION GRANTS (Ensure Grafana can query immediately)
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA bronze TO grafana_ro;
|
||||
GRANT USAGE ON SCHEMA bronze TO grafana_ro;
|
||||
|
|
@ -1,163 +0,0 @@
|
|||
"imei","device_name","mc_type","mc_type_use_scope","vehicle_name","vehicle_number","vehicle_models","vehicle_icon","vin","engine_number","vehicle_brand","fuel_100km","driver_name","driver_phone","sim","iccid","imsi","account","customer_name","device_group_id","device_group","activation_time","expiration","enabled_flag","status","city","current_mileage_km","created_at","updated_at","last_synced_at","vehicle_category","cost_centre","assigned_route","depot_geom","depot_address","assigned_city"
|
||||
"353549090553685","Daniel Omondi - KMFF 099Z","AT4","personal","KMFF 099Z","KMFF 099Z","Motorbike","mtc",NULL,NULL,NULL,NULL,"Robert","0112794067","759336150","89254021334258404099","639021335840409","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-09-23 05:50:30+00","2040-09-23 23:59:59+00",1,"1",NULL,"2354.70","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"353549090561720","Wireless_Git","AT4","personal",NULL,NULL,NULL,"bus",NULL,NULL,NULL,NULL,NULL,NULL,"0701211913","89254021374215155053","639021371515505","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2025-06-09 09:12:50+00","2035-06-09 23:59:59+00",1,"1",NULL,"5992.43","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"353549090566281","KDR 592N","AT4","personal",NULL,NULL,NULL,"bus",NULL,NULL,NULL,NULL,NULL,NULL,"0797680464","89254021334258159693","639021335815969","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2024-11-08 04:01:30+00","2034-11-08 23:59:59+00",1,"1",NULL,"7771.90","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"353549090566885","Wireless GPS","AT4","personal",NULL,NULL,NULL,"bus",NULL,NULL,NULL,NULL,NULL,NULL,"0768445963","89254021334212352574","639021331235257","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2024-10-15 13:16:57+00","2034-10-15 23:59:59+00",1,"1",NULL,"17036.41","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"353549090567685","Daniel Kipkirui - KMFF 162Z","AT4","personal","KMFF 162Z","KMFF 162Z","Motorbike","mtc",NULL,NULL,NULL,NULL,"edwine","0112795498","742532058","89254021264260388966","639021266038896","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-09-23 05:09:39+00","2040-09-23 23:59:59+00",1,"1",NULL,"462.33","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"353549090567701","Wireless","AT4","personal",NULL,NULL,NULL,"bus",NULL,NULL,NULL,NULL,NULL,NULL,"0790176094","89254021394215205906","639021391520590","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2024-11-08 04:04:44+00","2034-11-08 23:59:59+00",1,"1",NULL,"16896.20","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081885410","Allan Owana - KDK 780K","GT06E","aotomobile","KDK 780K","KDK 780K","Probox","automobile",NULL,NULL,NULL,NULL,"Allan Owana",NULL,"703616117","89254021234222499854","639021232249985","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2019-06-19 09:32:22+00","2039-06-19 23:59:59+00",1,"1",NULL,"128853.11","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081886467","Gideon Kiprono - KCQ 215F","GT06E","aotomobile","KCQ 215F","OHS","Probox","automobile",NULL,NULL,NULL,"0.00","Gideon Kiprono",NULL,"746763076","89254021084186499865","639021088649986","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2019-06-30 09:30:00+00","2039-06-30 23:59:59+00",1,"1",NULL,"141057.46","2026-04-23 10:56:37.983314+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081886871","Kamonde KBA 467S","GT06E","aotomobile",NULL,NULL,NULL,"bus",NULL,NULL,NULL,NULL,NULL,NULL,"0746763083","89254021084186499873","639021088649987","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2019-06-30 09:09:14+00","2039-06-30 23:59:59+00",1,"1",NULL,"74183.36","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081886905","Kennedy Chege - KCQ 618K","GT06E","aotomobile","KCQ 618K","KCQ 618K","Probox","automobile",NULL,NULL,NULL,NULL,"Kennedy Chege",NULL,"746763132","89254021084186499923","639021088649992","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2019-06-30 07:08:35+00","2039-06-30 23:59:59+00",1,"1",NULL,"215608.19","2026-04-23 10:35:37.678371+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081887069","Wright Oseko - KCG 668W","GT06E","aotomobile","KCG 668W","KCG 668W","Probox","automobile",NULL,NULL,NULL,NULL,"Wright Oseko",NULL,"746763106","89254021084186499915","639021088649991","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2019-06-30 06:17:43+00","2039-06-30 23:59:59+00",1,"1",NULL,"239001.19","2026-04-23 11:00:08.769463+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081887192","Ndegwa Dancun - KCG 669W","GT06E","aotomobile","KCG 669W","KCG 669W","Probox","automobile",NULL,NULL,NULL,NULL,"Ndegwa Dancun",NULL,"746760191","89254021084186499501","639021088649950","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2019-06-15 10:26:15+00","2039-06-15 23:59:59+00",1,"1",NULL,"199191.85","2026-04-23 10:34:29.074112+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081891566","Simon Kamau - KCE 090R","GT06E","aotomobile","KCE 090R","KCE 090R","Probox","automobile",NULL,NULL,NULL,NULL,"Simon Kamau",NULL,"746760404","89254021084186499527","639021088649952","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2019-06-16 07:06:15+00","2039-06-16 23:59:59+00",1,"1",NULL,"215592.36","2026-04-23 10:30:55.739184+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081891590","Garage - KCE 699F","GT06E","aotomobile","KCE 699F","KCE 699F","Probox","automobile",NULL,NULL,NULL,NULL,"Garage",NULL,"746760215","89254021084186499519","639021088649951","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2019-06-16 11:11:24+00","2039-06-16 23:59:59+00",1,"1",NULL,"207814.05","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081891632","John Ondego - KCA 542Q","GT06E","aotomobile","KCA 542Q","KCA 542Q","Probox","automobile",NULL,NULL,NULL,NULL,"John Ondego",NULL,"746760038","89254021084186499485","639021088649948","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2019-06-15 09:17:53+00","2039-06-15 23:59:59+00",1,"1",NULL,"178914.47","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081891798","Garage - KCH 167M","GT06E","aotomobile","KCH 167M","KCH 167M","Probox","automobile",NULL,NULL,NULL,NULL,"Garage",NULL,"746760102","89254021084186499493","639021088649949","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2019-06-16 10:18:57+00","2039-06-16 23:59:59+00",1,"1",NULL,"168840.95","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081892101","Cornelius Kimutai - KCU 938R","GT06E","aotomobile","KCU 938R","KCU 938R","Van","automobile",NULL,NULL,NULL,NULL,"Cornelius Kimutai",NULL,"746759919","89254021084186499451","639021088649945","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2019-06-12 08:13:48+00","2039-06-12 23:59:59+00",1,"1",NULL,"149558.50","2026-04-23 10:29:21.507861+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081892309","Nicholas Erastus - KCQ 581M","GT06E","aotomobile","KCQ 581M","KCQ 581M","Probox","automobile",NULL,NULL,NULL,NULL,"Nicholas Erastus",NULL,"700023776","89254021084178504672","639021087850467","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2019-06-09 09:39:40+00","2039-06-09 23:59:59+00",1,"1",NULL,"209105.89","2026-04-23 10:40:40.169684+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081892440","KAZ 489Z","GT06E","aotomobile",NULL,NULL,NULL,"bus",NULL,NULL,NULL,NULL,NULL,NULL,"0700023806","89254021084178504698","639021087850469","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2019-06-09 10:04:39+00","2039-06-09 23:59:59+00",1,"1",NULL,"38197.20","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857081892762","Nicholas","GT06E","aotomobile",NULL,NULL,"Station Wagon","bus",NULL,NULL,"Toyota",NULL,NULL,NULL,"0746760503","89254021274233125361","639021273312536","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2019-06-16 08:31:46+00","2039-06-16 23:59:59+00",1,"1",NULL,"51048.97","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082037185","Amani Kazungu - KCY 084X","GT06E","aotomobile","KCY 084X","KCY 084X","Probox","automobile",NULL,NULL,NULL,NULL,"Amani Kazungu",NULL,"757338522","89254021154287000597","639021158700059","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-07-13 09:42:28+00","2040-07-13 23:59:59+00",1,"1",NULL,"172298.81","2026-04-23 10:51:08.665273+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082038977","Wilfred Kinyanjui - KCU 729C","GT06E","aotomobile","KCU 729C","KCU 729C","Crane","truck",NULL,NULL,NULL,NULL,"Wilfred Kinyanjui",NULL,"110094469","89254021164215938057","639021161593805","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-04-05 09:26:00+00","2040-04-05 23:59:59+00",1,"1",NULL,"172487.09","2026-04-23 10:24:33.914628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082040981","Amani Sulubu - KCY 090X","GT06E","aotomobile","KCY 090X","KCY 090X","Probox","automobile",NULL,NULL,NULL,NULL,"Amani Sulubu",NULL,"793375853","89254021064168004164","639021066800416","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-07-13 07:25:16+00","2040-07-13 23:59:59+00",1,"1",NULL,"166028.15","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082042052","Gabriel Musumba - KCE 690F","GT06E","aotomobile","KCE 690F","KCE 690F","Probox","automobile",NULL,NULL,NULL,NULL,"Gabriel Musumba",NULL,"110094466","89254021164215938024","639021161593802","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2020-04-03 17:30:13+00","2040-04-03 23:59:59+00",1,"1",NULL,"192693.23","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082042854","Elias Baya - KCZ 476E","GT06E","aotomobile","KCZ 476E","KCZ 476E","Probox","automobile",NULL,NULL,NULL,NULL,"Elias Baya",NULL,"110941187","89254021164224352993","639021162435299","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-08-09 05:06:42+00","2040-08-09 23:59:59+00",1,"1",NULL,"217595.68","2026-04-23 10:33:56.216621+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082042953","KCU 865Q Vanguard",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,"unknown",NULL,NULL,"2026-04-23 13:24:33.293453+00","2026-04-23 13:24:33.293453+00",NULL,NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082044280","Lawrence Kijogi - KCY 080X","GT06E","aotomobile","KCY 080X","KCY 080X","Probox","automobile",NULL,NULL,NULL,NULL,"Lawrence Kijogi",NULL,"708155933","89254029851005131222","639029850513122","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-07-13 11:05:02+00","2040-07-13 11:05:02+00",1,"1",NULL,"169740.37","2026-04-23 14:52:58.983571+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082046145","Joseph Kabandi - KCY 076X","GT06E","aotomobile","KCY 076X","KCY 076X","Probox","automobile",NULL,NULL,NULL,NULL,"Joseph Kabandi",NULL,"110850007","89254021164223447158","639021162344715","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-07-13 08:31:26+00","2040-07-13 23:59:59+00",1,"1",NULL,"122254.48","2026-04-23 10:47:40.895504+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082896911","Hamisi Pande - KDD 689Y","GT06E","aotomobile","KDD 689Y","KDD 689Y","Probox","automobile",NULL,NULL,NULL,NULL,"Hamisi Pande",NULL,"112714612","89254021214211314660","639021211131466","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-09-17 11:50:53+00","2041-09-17 23:59:59+00",1,"1",NULL,"163435.74","2026-04-23 10:26:09.922447+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082897091","Peter Mbugua - KDK 728K","GT06E","aotomobile","KDK 728K","KDK 728K","Probox","automobile",NULL,NULL,NULL,NULL,"Peter Mbugua",NULL,"790262984","89254021234222500396","639021232250039","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-12-14 11:31:57+00","2042-12-14 23:59:59+00",1,"1",NULL,"131109.26","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082897257","Cassius Wakiyo - KDB 323M","GT06E","aotomobile","KDB 323M","KDB 323M","Probox","automobile",NULL,NULL,NULL,NULL,"Cassius Wakiyo",NULL,"746428882","89254021234222500818","639021232250081","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-08-29 15:07:26+00","2041-08-29 15:07:26+00",1,"1",NULL,"121688.92","2026-04-23 10:28:26.388654+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082897737","John Makori - KDB 585E","GT06E","aotomobile","KDB 585E","KDB 585E","Probox","automobile",NULL,NULL,NULL,NULL,"John Makori",NULL,"114596734","89254021214211145262","639021211114526","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-08-29 14:29:28+00","2041-08-29 14:29:28+00",1,"1",NULL,"156765.03","2026-04-23 10:38:57.445964+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082897794","Mutuku Joseph - KDC 739F","GT06E","aotomobile","KDC 739F","KDC 739F","Probox","automobile",NULL,NULL,NULL,NULL,"Mutuku Joseph","115019037","115019037","89254021224222632356","639021222263235","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-04-10 14:55:32+00","2041-04-10 14:55:32+00",1,"1",NULL,"205169.79","2026-04-23 10:30:22.530563+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082898008","Samuel Ng'ang'a - KDE 264M","GT06E","aotomobile","KDE 264M","KDE 264M","Probox","automobile",NULL,NULL,NULL,NULL,"Samuel Ng'ang'a",NULL,"711731539","89254021264260342245","639021266034224","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-10-28 09:43:11+00","2041-10-28 23:59:59+00",1,"1",NULL,"126584.24","2026-04-23 11:35:59.816581+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082898016","Job Ngare - KDM 309S","GT06E","aotomobile","KDM 309S","KDM 309S","Probox","automobile",NULL,NULL,NULL,NULL,"Job Ngare",NULL,"706895756","89254021324273007563","639021327300756","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-08-15 13:45:26+00","2033-08-15 23:59:59+00",1,"1",NULL,"107726.56","2026-04-23 11:20:25.939244+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082898073","Mutuku Antony - KDK 732K","GT06E","aotomobile","KDK 732K","KDK 732K","Probox","automobile",NULL,NULL,NULL,NULL,"Mutuku Antony",NULL,"793026954","89254021234222387539","639021232238753","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-12-20 06:33:12+00","2042-12-20 23:59:59+00",1,"1",NULL,"82096.79","2026-04-23 14:52:07.094447+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082898487","Dan Watila - KDE 638J","GT06E","aotomobile","KDE 638J","KDE 638J","Probox","automobile",NULL,NULL,NULL,NULL,"Dan Watila",NULL,"116242996","89254021334258404214","639021335840421","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-10-21 15:50:06+00","2041-10-21 23:59:59+00",1,"1",NULL,"123872.36","2026-04-23 10:31:45.186653+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082900168","KDD 913G_Ruth Mazda",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,"unknown",NULL,NULL,"2026-04-23 15:09:48.575568+00","2026-04-23 15:09:48.575568+00",NULL,NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082900341","Simon Munda - KCZ 154S","GT06E","aotomobile","KCZ 154S","KCZ 154S","Probox","automobile",NULL,NULL,NULL,NULL,"Simon Munda",NULL,"757236135","89254021154296723312","639021159672331","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-09-23 17:12:51+00","2040-09-23 23:59:59+00",1,"1",NULL,"186504.10","2026-04-23 10:45:21.454595+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082900358","Geoffrey Too - KDM 308S","GT06E","aotomobile","KDM 308S","KDM 308S","Probox","automobile",NULL,NULL,NULL,NULL,"Geoffrey Too",NULL,"796527601","89254021264260126572","639021266012657","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-10-21 15:25:28+00","2041-10-21 23:59:59+00",1,"1",NULL,"142216.91","2026-04-23 12:35:06.661934+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082900697","George Ochieng' - KDD 684Y","GT06E","aotomobile","KDD 684Y","KDD 684Y","Probox","automobile",NULL,NULL,NULL,NULL,"George Ochieng'",NULL,"114879518","89254021214211314678","639021211131467","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-09-17 10:53:11+00","2041-09-17 23:59:59+00",1,"1",NULL,"152820.07","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082902461","Sadique Wakayula - KDC 490Q","GT06E","aotomobile","KDC 490Q","KDC 490Q","Crane","truck",NULL,NULL,NULL,NULL,"Sadique Wakayula",NULL,"757556468","89254021154296722488","639021159672248","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-05-22 11:27:30+00","2041-05-22 11:27:30+00",1,"1",NULL,"183009.52","2026-04-23 11:16:03.730519+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082902503","Felix Andole - KDC 207R","GT06E","aotomobile","KDC 207R","KDC 207R","Probox","automobile",NULL,NULL,NULL,NULL,"Felix Andole",NULL,"794820817","89254021224270993254","639021227099325","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-05-15 11:38:24+00","2041-05-15 11:38:24+00",1,"1",NULL,"208724.46","2026-04-23 15:32:46.935568+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082907973","Felix Muema - KCZ 223P","GT06E","aotomobile","KCZ 223P","KCZ 223P","Probox","automobile",NULL,NULL,NULL,NULL,"Felix Muema",NULL,"757843826","89254021154287138371","639021158713837","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-08-22 14:01:25+00","2040-08-22 23:59:59+00",1,"1",NULL,"222126.36","2026-04-23 10:26:48.220151+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082908500","Santoes Omondi - KCZ 181P","GT06E","aotomobile","KCZ 181P","KCZ 181P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Santoes Omondi",NULL,"701211974","89254021374215155087","639021371515508","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-08-23 08:58:55+00","2040-08-23 23:59:59+00",1,"1",NULL,"221339.62","2026-04-23 10:48:09.537346+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082910589","Patric Bett - KDA 609E","GT06E","aotomobile","KDA 609E","KDA 609E","Probox","automobile",NULL,NULL,NULL,NULL,"Patric Bett",NULL,"797622637","89254021154296722496","639021159672249","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2020-10-26 15:46:41+00","2040-10-26 23:59:59+00",1,"1",NULL,"194618.69","2026-04-23 10:34:25.350862+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082910886","Makanda Andrew - KCZ 155P","GT06E","aotomobile","KCZ 155P","KCZ 155P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Makanda Andrew",NULL,"745067338","89254021154287138397","639021158713839","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-08-23 11:52:35+00","2040-08-23 23:59:59+00",1,"1",NULL,"231065.89","2026-04-23 11:36:31.150282+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082911983","Brian Ngetich - KDA 717B","GT06E","aotomobile","KDA 717B","KDA 717B","Probox","automobile",NULL,NULL,NULL,NULL,"Brian Ngetich","795188807","795188807","89254021214211145288","639021211114528","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-08-29 07:21:43+00","2041-08-29 07:21:43+00",1,"1",NULL,"145404.96","2026-04-23 10:36:11.774166+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082912239","Dickson Jaoko - KDK 815R","GT06E","aotomobile","KDK 815R","KDK 815R","Probox","automobile",NULL,NULL,"Probox","0.00","Sammy",NULL,"706392117","89254021234296021287","639021239602128","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-06-21 07:14:51+00","2033-06-21 23:59:59+00",1,"1",NULL,"77008.75","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082912486","Moses Wambua - KCZ 751V","GT06E","aotomobile","KCZ 751V","KCZ 751V","Probox","automobile",NULL,NULL,NULL,NULL,"Moses Wambua","0792756503","792756503","89254021154296723437","639021159672343","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-09-23 10:14:28+00","2040-09-23 23:59:59+00",1,"1",NULL,"139762.20","2026-04-23 10:41:00.207177+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082916826","Denis Kazungu - KDM 794R","GT06E","aotomobile","KDM 794R","KDM 794R","Probox","automobile",NULL,NULL,NULL,NULL,"Denis Kazungu",NULL,"705700971","89254021324273006854","639021327300685","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2023-08-21 06:38:00+00","2033-08-21 23:59:59+00",1,"1",NULL,"79639.71","2026-04-23 20:18:46.496567+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082918012","Charles Nyambane - KCB 711C","GT06E","aotomobile","KCB 711C","KCB 711C","Probox","automobile",NULL,NULL,NULL,NULL,"Charles Nyambane",NULL,"793704231","89254021154287138363","639021158713836","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2020-09-21 10:48:35+00","2040-09-21 23:59:59+00",1,"1",NULL,"159597.27","2026-04-23 10:25:52.843474+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082918038","Mbuvi Kioko - KCC 199P","GT06E","aotomobile","KCC 199P","KCC 199P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Mbuvi Kioko",NULL,"797318126","89254021154287138389","639021158713838","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-08-22 15:26:27+00","2040-08-22 23:59:59+00",1,"1",NULL,"222106.80","2026-04-23 12:09:05.609075+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082918186","KDD 977T Fielder",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,"unknown",NULL,NULL,"2026-04-23 10:36:25.832836+00","2026-04-23 10:36:25.832836+00",NULL,NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"359857082925330","Noel Merengeni - KCY 838X","GT06E","aotomobile","KCY 838X","KCY 838X","Probox","automobile",NULL,NULL,NULL,NULL,"Noel Merengeni",NULL,"794873610","89254021154296723429","639021159672342","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2020-10-26 16:36:37+00","2040-10-26 23:59:59+00",1,"1",NULL,"194429.24","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050288212","Nicholas Erastus - KCQ 581M","JC400P","aotomobile","KCQ 581M","KCQ 581M","Probox","automobile",NULL,NULL,NULL,NULL,"Nicholas Erastus",NULL,"746979531",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-11-02 09:07:41+00","2041-11-02 23:59:59+00",1,"1",NULL,"40898.98","2026-04-23 13:05:18.326254+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050288261","Patric Bett - KDA 609E","JC400P","aotomobile","KDA 609E","KDA 609E","Probox","automobile",NULL,NULL,NULL,NULL,"Patric Bett","112693340","790176509",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2021-10-23 11:50:11+00","2041-10-23 23:59:59+00",1,"1",NULL,"18538.42","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050288303","Elias Baya - KCZ 476E","JC400P","aotomobile","KCZ 476E","KCZ 476E","Probox","automobile",NULL,NULL,NULL,NULL,"Elias Baya",NULL,"115870439",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-11-06 08:50:28+00","2041-11-06 23:59:59+00",1,"1",NULL,"116091.42","2026-04-23 17:46:09.993791+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050288345","Santoes Omondi - KCZ 181P","JC400P","aotomobile","KCZ 181P","KCZ 181P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Santoes Omondi",NULL,"768446105",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2021-11-06 10:17:51+00","2041-11-06 23:59:59+00",1,"1",NULL,"107462.79","2026-04-23 10:29:45.563231+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050288360","Brian Ngetich - KDA 717B","JC400P","aotomobile","KDA 717B","KDA 717B","Probox","automobile",NULL,NULL,NULL,NULL,"Brian Ngetich",NULL,"717867861",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2021-11-05 08:47:08+00","2041-11-05 23:59:59+00",1,"1",NULL,"17808.56","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050521521","John Kimeria - KDS 525D","JC400P","aotomobile","KDS 525D","KDS 525D","Crane","truck",NULL,NULL,NULL,NULL,"John Kimeria",NULL,"752958416",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-11-26 07:58:13+00","2033-11-26 23:59:59+00",1,"1",NULL,"19354.92","2026-04-23 10:28:34.917147+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050521612","Denis Kazungu - KDM 794R","JC400P","aotomobile","KDM 794R","KDM 794R","Probox","automobile",NULL,NULL,NULL,NULL,"Denis Kazungu",NULL,"704113731",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-22 07:52:12+00","2042-01-22 23:59:59+00",1,"1",NULL,"4350.75","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050521752","Simon Munda - KCZ 154S","JC400P","aotomobile","KCZ 154S","KCZ 154S","Probox","automobile",NULL,NULL,NULL,NULL,"Simon Munda",NULL,"113805921",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 08:14:37+00","2042-01-16 23:59:59+00",1,"1",NULL,"4698.02","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522065","Gideon Kiprono - KCQ 215F","JC400P","aotomobile","KCQ 215F","KCQ 215F","Probox","automobile",NULL,NULL,NULL,NULL,"Gideon Kiprono",NULL,"113343715",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2022-01-16 07:10:16+00","2042-01-16 23:59:59+00",1,"1",NULL,"8111.98","2026-04-23 18:23:51.445608+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522107","Cassius Wakiyo - KDB 323M","JC400P","aotomobile","KDB 323M","KDB 323M","Probox","automobile",NULL,NULL,NULL,NULL,"Cassius Wakiyo",NULL,"114149576",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-22 08:18:15+00","2042-01-22 23:59:59+00",1,"1",NULL,"23316.09","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522719","Mbuvi Kioko - KCZ 199P","JC400P","aotomobile","KCZ 199P","KCZ 199P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Mbuvi Kioko",NULL,"768218655",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 22:07:24+00","2042-01-16 23:59:59+00",1,"1",NULL,"16973.89","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522743","Charles Nyambane - KCB 711C","JC400P","aotomobile","KCB 711C","KCB 711C","Probox","automobile",NULL,NULL,NULL,NULL,"Charles Nyambane",NULL,"768657106",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-12-22 21:53:57+00","2033-12-22 23:59:59+00",1,"1",NULL,"12133.75","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522859","Garage - KCH 167M","JC400P","aotomobile","KCH 167M","KCH 167M","Probox","automobile",NULL,NULL,NULL,NULL,"Garage",NULL,"706740252",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-15 08:23:21+00","2042-01-15 23:59:59+00",1,"1",NULL,"6934.86","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522883","Dan Watila - KDE 638J","JC400P","aotomobile","KDE 638J","KDE 638J","Probox","automobile",NULL,NULL,NULL,NULL,"Dan Watila",NULL,"112615393",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-15 07:17:21+00","2042-01-15 23:59:59+00",1,"1",NULL,"14483.01","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050522891","Lawrence Kijogi - KCY 080X","JC400P","aotomobile","KCY 080X","KCY 080X","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Lawrence Kijogi",NULL,"113287191",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 10:51:30+00","2042-01-16 23:59:59+00",1,"1",NULL,"11585.33","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523014","Samuel Muriithy - KDR 594N","JC400P","aotomobile","KDR 594N","KDR 594N","Probox","automobile",NULL,NULL,NULL,NULL,"Samuel Muriithy",NULL,"790175423",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-21 15:54:32+00","2033-12-21 23:59:59+00",1,"1",NULL,"27275.43","2026-04-23 10:26:17.747928+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523063","Kelvin Wambugu - KDR 594N","JC400P","aotomobile","KDR 594N","KDR 594N","Probox","automobile",NULL,NULL,NULL,NULL,"Kelvin Wambugu",NULL,"701211876",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-22 19:24:51+00","2043-12-22 19:24:51+00",1,"1",NULL,"32698.94","2026-04-23 15:31:08.065856+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523139","Mike Wanaswa - KDT 724R","JC400P","aotomobile","KDT 724R","KDT 724R","Probox","automobile",NULL,NULL,NULL,NULL,"Mike Wanaswa",NULL,"790175045",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-22 22:28:09+00","2043-12-22 22:28:09+00",1,"1",NULL,"29559.82","2026-04-23 11:16:37.277518+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523204","Amani Kazungu - KCY 084X","JC400P","aotomobile","KCY 084X","KCY 084X","Probox","automobile",NULL,NULL,NULL,NULL,"Amani Kazungu",NULL,"707892547",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 06:18:20+00","2042-01-16 23:59:59+00",1,"1",NULL,"66955.70","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523295","Emmanuel Luseno - KDS 453 Y","JC400P","aotomobile","KDS 453 Y","KDS 453 Y","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Emmanuel Luseno",NULL,"700242474",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-22 14:39:50+00","2033-12-22 23:59:59+00",1,"1",NULL,"37098.35","2026-04-23 11:29:48.369147+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523337","Victor Kimutai - KDS 919Y","JC400P","aotomobile","KDS 919Y","KDS 919Y","Probox","automobile",NULL,NULL,NULL,NULL,"Victor Kimutai",NULL,"700242527",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-22 18:00:49+00","2043-12-22 18:00:49+00",1,"1",NULL,"50756.64","2026-04-23 10:27:13.522675+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523386","George Ochieng' - KDD 684Y","JC400P","aotomobile","KDD 684Y","KDD 684Y","Probox","automobile",NULL,NULL,NULL,NULL,"George Ochieng'",NULL,"785586834",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-22 06:36:08+00","2042-01-22 23:59:59+00",1,"1",NULL,"33979.83","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523527","Allan Owana - KDK 780K","JC400P","aotomobile","KDK 780K","KDK 780K","Probox","automobile",NULL,NULL,NULL,NULL,"Allan Owana",NULL,"792375024",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2022-12-03 10:43:41+00","2042-12-03 23:59:59+00",1,"1",NULL,"109564.95","2026-04-23 10:25:24.360765+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523618","Geoffrey Too - KDM 308S","JC400P","aotomobile","KDM 308S","KDM 308S","Probox","automobile",NULL,NULL,NULL,NULL,"Geoffrey Too",NULL,"701211625",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-08-15 15:42:32+00","2033-08-15 23:59:59+00",1,"1",NULL,"26496.50","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523626","Major Simiyu - KDS 949Y","JC400P","aotomobile","KDS 949Y","KDS 949Y","Probox","automobile",NULL,NULL,NULL,NULL,"Major Simiyu",NULL,"701211892",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-22 18:05:16+00","2033-12-22 23:59:59+00",1,"1",NULL,"37042.97","2026-04-23 10:51:18.245194+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523816","Job Ngare - KDM 309S","JC400P","aotomobile","KDM 309S","KDM 309S","Probox","automobile",NULL,NULL,NULL,NULL,"Job Ngare",NULL,"707936781",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-08-15 14:05:52+00","2033-08-15 23:59:59+00",1,"1",NULL,"54320.21","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050523949","Joseph Kabandi - KCY 076X","JC400P","aotomobile","KCY 076X","KCY 076X","Probox","automobile",NULL,NULL,NULL,NULL,"Joseph Kabandi",NULL,"113288492",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 05:52:54+00","2042-01-16 23:59:59+00",1,"1",NULL,"14427.50","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524012","Moses Wambua - KCZ 751V","JC400P","aotomobile","KCZ 751V","KCZ 751V","Probox","automobile",NULL,NULL,NULL,NULL,"Moses Wambua",NULL,"113313797",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 07:40:10+00","2042-01-16 23:59:59+00",1,"1",NULL,"26551.46","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524087","Felix Muema - KCZ 223P","JC400P","aotomobile","KCZ 223P","KCZ 223P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Felix Muema",NULL,"113973875",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 13:02:24+00","2042-01-16 23:59:59+00",1,"1",NULL,"11543.26","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524368","862798050524368","JC400P","aotomobile",NULL,NULL,NULL,"automobile",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-10-29 09:24:53+00","2042-10-29 23:59:59+00",1,"1",NULL,"169208.91","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524384","Hamisi Pande - KDD 689Y","JC400P","aotomobile","KDD 689Y","KDD 689Y","Probox","automobile",NULL,NULL,NULL,"0.00","Hamisi Pande",NULL,"701211744",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-22 05:49:19+00","2042-01-22 23:59:59+00",1,"1",NULL,"13685.18","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524392","Ndegwa Dancun - KCG 669W","JC400P","aotomobile","KCG 669W","KCG 669W","Probox","automobile",NULL,NULL,NULL,NULL,"Ndegwa Dancun",NULL,"113799173",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 09:43:10+00","2042-01-16 23:59:59+00",1,"1",NULL,"13638.25","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524426","Amani Sulubu - KCY 090X","JC400P","aotomobile","KCY 090X","KCY 090X","Probox","automobile",NULL,NULL,NULL,NULL,"Amani Sulubu",NULL,"113823350",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2022-01-16 08:56:25+00","2042-01-16 23:59:59+00",1,"1",NULL,"14243.83","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524533","Leonard Nzai - KDM 306S","JC400P","aotomobile","KDM 306S","KDM 306S","Probox","automobile",NULL,NULL,NULL,NULL,"Leonard Nzai",NULL,"703487162",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-08-21 08:22:12+00","2033-08-21 23:59:59+00",1,"1",NULL,"68942.41","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524558","Mutuku Joseph - KDC 739F","JC400P","aotomobile","KDC 739F","KDC 739F","Probox","automobile",NULL,NULL,NULL,NULL,"Mutuku Joseph",NULL,"100858817",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-22 10:38:25+00","2042-01-22 23:59:59+00",1,"1",NULL,"23711.63","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524566","Makanda Andrew - KCZ 155P","JC400P","aotomobile","KCZ 155P","KCZ 155P","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Makanda Andrew",NULL,"758781444",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-22 09:47:33+00","2042-01-22 23:59:59+00",1,"1",NULL,"31663.30","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524608","Peter Mbugua - KDK 728K","JC400P","aotomobile","KDK 728K","KDK 728K","Probox","automobile",NULL,NULL,NULL,NULL,"Peter Mbugua",NULL,"706742413",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-12-03 12:11:32+00","2042-12-03 23:59:59+00",1,"1",NULL,"7219.31","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524657","Felix Andole - KDC 207R","JC400P","aotomobile","KDC 207R","KDC 207R","Probox","automobile",NULL,NULL,NULL,NULL,"Felix Andole",NULL,"758689195",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-22 07:17:47+00","2042-01-22 23:59:59+00",1,"1",NULL,"46233.99","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524681","Mutuku Antony - KDK 732K","JC400P","aotomobile","KDK 732K","KDK 732K","Probox","automobile",NULL,NULL,NULL,NULL,"Mutuku Antony",NULL,"796275746",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-12-06 13:37:49+00","2042-12-06 23:59:59+00",1,"1",NULL,"14993.36","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524707","Garage - KCE 699F","JC400P","aotomobile","KCE 699F","KCE 699F","Probox","automobile",NULL,NULL,NULL,NULL,"Garage",NULL,"110525751",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-15 07:58:49+00","2042-01-15 23:59:59+00",1,"1",NULL,"34715.97","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050524897","Cornelius Kimutai - KCU 938R","JC400P","aotomobile","KCU 938R","KCU 938R","Van","automobile",NULL,NULL,NULL,NULL,"Cornelius Kimutai",NULL,"114924404",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-22 09:03:40+00","2042-01-22 23:59:59+00",1,"1",NULL,"12668.43","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525068","Samuel Ng'ang'a - KDE 264M","JC400P","aotomobile","KDE 264M","KDE 264M","Probox","automobile",NULL,NULL,NULL,NULL,"Samuel Ng'ang'a",NULL,"768658564",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-12-22 13:33:42+00","2033-12-22 23:59:59+00",1,"1",NULL,"12299.13","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525225","Sadique Wakayula - KDC 490Q","JC400P","aotomobile","KDC 490Q","KDC 490Q","Crane","truck",NULL,NULL,NULL,NULL,"Sadique Wakayula",NULL,"768652386",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-12-22 20:52:08+00","2043-12-22 20:52:08+00",1,"1",NULL,"19138.05","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525266","Dickson Jaoko - KDK 815R","JC400P","aotomobile","KDK 815R","KDK 815R","Probox","automobile",NULL,NULL,NULL,NULL,"Dickson Jaoko",NULL,"706665867",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-06-21 07:50:00+00","2033-06-21 23:59:59+00",1,"1",NULL,"63754.71","2026-04-23 13:50:24.21992+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525423","Makori John - KDB 585E","JC400P","aotomobile","KDB 585E","KDB 585E","Probox","automobile",NULL,NULL,NULL,NULL,"Makori John",NULL,"701211724",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-15 10:59:19+00","2042-01-15 23:59:59+00",1,"1",NULL,"48804.83","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525589","Simon Kamau - KCE 090R","JC400P","aotomobile","KCE 090R","KCE 090R","Probox","automobile",NULL,NULL,NULL,NULL,"Simon Kamau",NULL,"796276387",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-19 10:10:04+00","2042-01-19 23:59:59+00",1,"1",NULL,"15874.39","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525605","John Ondego - KCA 542Q","JC400P","aotomobile","KCA 542Q","KCA 542Q","Probox","automobile",NULL,NULL,NULL,NULL,"John Ondego",NULL,"110526783",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-15 05:56:11+00","2042-01-15 23:59:59+00",1,"1",NULL,"23976.94","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525613","Kennedy Chege - KCQ 618K","JC400P","aotomobile","KCQ 618K","KCQ 618K","Probox","automobile",NULL,NULL,NULL,NULL,"Kennedy Chege",NULL,"729994247",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-16 05:21:05+00","2042-01-16 23:59:59+00",1,"1",NULL,"12804.24","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525670","Gabriel Musumba - KCE 690F","JC400P","aotomobile","KCE 690F","KCE 690F","Probox","automobile",NULL,NULL,NULL,NULL,"Gabriel Musumba",NULL,"701211996",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2022-01-15 06:40:01+00","2042-01-15 23:59:59+00",1,"1",NULL,"20110.93","2026-04-24 05:34:23.167312+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525753","Noel Merengeni - KCY 838X","JC400P","aotomobile","KCY 838X","KCY 838X","Probox","automobile",NULL,NULL,NULL,NULL,"Noel Merengeni",NULL,NULL,NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2022-01-15 05:24:00+00","2042-01-15 23:59:59+00",1,"1",NULL,"14596.59","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525837","Kennedy Ondieki - KCU 237Z","JC400P","aotomobile","KCU 237Z","KCU 237Z","Probox","automobile",NULL,NULL,NULL,NULL,"Kennedy Ondieki",NULL,"113669852",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2023-12-21 19:32:44+00","2033-12-21 23:59:59+00",1,"1",NULL,NULL,"2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050525951","Wright Oseko - KCG 668W","JC400P","aotomobile","KCG 668W","KCG 668W","Probox","automobile",NULL,NULL,NULL,NULL,"Wright Oseko",NULL,"741943212",NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2022-01-15 09:36:45+00","2042-01-15 23:59:59+00",1,"1",NULL,"13116.00","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050526165","Wilfred Kinyanjui - KCU 729C","JC400P","aotomobile","KCU 729C","KCU 729C","Crane","truck",NULL,NULL,NULL,NULL,"Wilfred Kinyanjui",NULL,"790564929",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2023-11-26 10:17:19+00","2033-11-26 23:59:59+00",1,"1",NULL,"24270.20","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050526231","Rashid Hassan - KDM 840V","JC400P","aotomobile","KDM 840V","KDM 840V","Probox","automobile",NULL,NULL,NULL,NULL,"Rashid Hassan",NULL,"790175526",NULL,NULL,"Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2023-12-22 22:36:15+00","2043-12-22 22:36:15+00",1,"1",NULL,"45418.38","2026-04-23 10:29:41.575467+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798050526256","Ian Dancun - KDT 923R","JC400P","aotomobile","KDT 923R","KDT 923R","Probox","automobile",NULL,NULL,NULL,NULL,"Ian Dancun",NULL,"794873610",NULL,NULL,"Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2023-12-22 19:37:24+00","2043-12-22 19:37:24+00",1,"1",NULL,"11093.11","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052707888","Benjamin Ananda - KDV 438W","JC400P","aotomobile","KDV 438W","KDV 438W","Probox","automobile",NULL,NULL,NULL,NULL,"Benjamin Ananda",NULL,"758047312","89254021414206816980","639021410681698","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-12-15 07:39:23+00","2035-12-15 23:59:59+00",1,"1",NULL,"8720.87","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052707896","John Mbugua - KDW 573B","JC400P","aotomobile","KDW 573B","KDW 573B","Probox","automobile",NULL,NULL,NULL,NULL,"John Mbugua",NULL,NULL,"89254021414206816725","639021410681672","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-01-30 14:48:17+00","2036-01-30 23:59:59+00",1,"1",NULL,"515.16","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052707946","Tom Wekesa/OSP-KCY 930Y_CAM","JC400P","aotomobile",NULL,NULL,NULL,"automobile",NULL,NULL,NULL,NULL,NULL,NULL,"0758047806","89254021414206816766","639021410681676","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2026-01-20 21:02:13+00","2036-01-20 23:59:59+00",1,"1",NULL,"10079.17","2026-04-23 10:25:24.363965+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052708035","862798052708035","JC400P","aotomobile",NULL,NULL,"Probox","automobile",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,"fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group",NULL,NULL,1,"1",NULL,NULL,"2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052708068","Dominic Wambua - KDV 683Z","JC400P","aotomobile","KDV 683Z","KDV 683Z","Probox","automobile",NULL,NULL,NULL,NULL,"Dominic Wambua",NULL,"758048043","89254021414206816964","639021410681696","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-01-24 09:20:09+00","2036-01-24 23:59:59+00",1,"1",NULL,"4438.55","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052708076","Albert Mutwiri - KDV 437W","JC400P","aotomobile",NULL,"KDV 437W","Probox","automobile",NULL,NULL,NULL,NULL,"Albert Mutwiri",NULL,"758047094","89254021414206816782","639021410681678","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-12-13 15:03:30+00","2035-12-13 23:59:59+00",1,"1",NULL,"5575.64","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052708167","Levine Wasike - KDV 439W","JC400P","aotomobile","KDV 439W","KDV 439W","Probox","automobile",NULL,NULL,NULL,NULL,"Levine Wasike",NULL,"758046738","89254021414206816741","639021410681674","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-12-13 19:49:29+00","2035-12-13 23:59:59+00",1,"1",NULL,"4601.08","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052708282","Godffrey Nandwa - KCN 496A","JC400P","aotomobile","KCN 496A","KCN 496A","Probox","automobile",NULL,NULL,NULL,NULL,"Godffrey Nandwa",NULL,"758047934","89254021414206816865","639021410681686","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2026-01-25 18:55:54+00","2036-01-25 23:59:59+00",1,"1",NULL,"7040.60","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713654","Garage/ISP_KCL 502T_CAM","JC400P","aotomobile",NULL,NULL,NULL,"automobile",NULL,NULL,NULL,NULL,NULL,NULL,"0780215879","89254035061001753803","639035060175380","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-09-02 10:09:57+00","2035-09-02 23:59:59+00",1,"1",NULL,"5199.72","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713696","862798052713696","JC400P","aotomobile",NULL,NULL,"Probox","automobile",NULL,NULL,NULL,NULL,NULL,NULL,NULL,"89254021394215205906","639021391520590","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-09-02 10:20:58+00","2035-09-02 23:59:59+00",1,"1",NULL,"6214.49","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713761","Management_Mazda - KDU 613A","JC400P","aotomobile","KDU 613A","KDU 613A","Mazda","automobile",NULL,NULL,NULL,NULL,"Management_Mazda",NULL,"790176786","89254021394215205955","639021391520595","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-07-09 15:49:26+00","2035-07-09 23:59:59+00",1,"1",NULL,"9262.78","2026-04-23 16:40:48.879666+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713779","Benard Kimutai - KDN 759G","JC400P","aotomobile","KDN 759G","KDN 759G","Probox","automobile",NULL,NULL,NULL,NULL,"Benard Kimutai",NULL,"752143258","89254035061001753860","639035060175386","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-08-23 11:15:59+00","2035-08-23 23:59:59+00",1,"1",NULL,"5344.24","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713811","James Onyango - KDU 613B","JC400P","aotomobile","KDU 613B","KDU 613B","Probox","automobile",NULL,NULL,NULL,NULL,"James Onyango",NULL,"790176542","89254021394215205880","639021391520588","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-09 19:24:14+00","2035-07-09 23:59:59+00",1,"1",NULL,"9657.42","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713837","Kennedy Ondieki - KCU 237Z","JC400P","aotomobile","KCU 237Z","KCU 237Z","Probox","automobile",NULL,NULL,NULL,NULL,"Kennedy Ondieki",NULL,"113669852","89254021414206327855","639021410632785","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-10-08 14:55:23+00","2035-10-08 23:59:59+00",1,"1",NULL,"9346.02","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052713985","Timothy Gitau - KDT 916R","JC400P","aotomobile","KDT 916R","KDT 916R","Probox","automobile",NULL,NULL,NULL,NULL,"Timothy Gitau",NULL,"768696668","89254021394274518892","639021397451889","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-08-02 18:21:23+00","2035-08-02 23:59:59+00",1,"1",NULL,"19998.22","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052714066","862798052714066","JC400P","aotomobile",NULL,NULL,"Probox","automobile",NULL,NULL,NULL,NULL,NULL,NULL,NULL,"89254021414206378684","639021410637868","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-11-21 17:44:44+00","2035-11-21 23:59:59+00",1,"1",NULL,"10755.28","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"862798052715220","Rofas Njagi - KDT 728R","JC400P","aotomobile","KDT 728R","KDT 728R","Probox","automobile",NULL,NULL,NULL,NULL,"Rofas Njagi",NULL,"704573658","89254021334258495873","639021335849587","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-16 07:09:25+00","2035-07-16 23:59:59+00",1,"1",NULL,"16385.58","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061035133","Major Simiyu - KDS 949Y","X3","aotomobile","KDS 949Y","KDS 949Y","Probox","automobile",NULL,NULL,NULL,NULL,"Major Simiyu",NULL,"768696642","89254021394274518918","639021397451891","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2025-08-02 13:14:33+00","2035-08-02 23:59:59+00",1,"1",NULL,"25089.98","2026-04-23 12:07:56.044395+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061035653","Richardson Komu - KDT 923R","X3","aotomobile","KDT 923R","KDT 923R","Probox","automobile",NULL,NULL,NULL,NULL,"Richardson Komu",NULL,"768697292","89254021394274518942","639021397451894","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-08-02 08:11:46+00","2035-08-02 23:59:59+00",1,"1",NULL,"23556.65","2026-04-23 10:24:50.340401+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061035778","John Kimeria - KDS 525D","X3","aotomobile","KDS 525D","KDS 525D","Crane","truck",NULL,NULL,NULL,NULL,"John Kimeria",NULL,"790176738","89254021394215205922","639021391520592","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-11 05:50:36+00","2035-07-11 23:59:59+00",1,"1",NULL,"17653.96","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061036164","Brian Njenga - KMFF 113Z","X3","aotomobile","KMFF 113Z","KMFF 113Z","Motorbike","mtc",NULL,NULL,NULL,NULL,"Brian Njenga",NULL,"768696705","89254021394274518850","639021397451885","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-31 10:06:46+00","2035-07-31 23:59:59+00",1,"1",NULL,"22990.33","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061037980","Emmanuel Luseno - KDS 453Y","X3","aotomobile","KDS 453Y","KDS 453Y","Pick-Up","automobile",NULL,NULL,NULL,NULL,"Emmanuel Luseno",NULL,"790176734","89254021394215205856","639021391520585","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-15 06:30:34+00","2035-07-15 23:59:59+00",1,"1",NULL,"42609.03","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061042261","Kelvin Wambugu - KDR 592N","X3","aotomobile","KDR 592N","KDR 592N","Probox","automobile",NULL,NULL,NULL,NULL,"Kelvin Wambugu",NULL,"797680464","89254021334258159693","639021335815969","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-10 10:23:44+00","2035-07-10 23:59:59+00",1,"1",NULL,"18755.66","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061043079","Mike Wanaswa - KDT 724R","X3","aotomobile","KDT 724R","KDT 724R","Probox","automobile",NULL,NULL,NULL,NULL,"Mike Wanaswa",NULL,"768696664","89254021394274518959","639021397451895","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2025-08-02 12:16:11+00","2035-08-02 23:59:59+00",1,"1",NULL,"27470.11","2026-04-23 11:16:35.682194+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061043426","Geoffrey Karanja - KMGS 239H","X3","aotomobile","KMGS 239H","KMGS 239H","Motorbike","mtc",NULL,NULL,NULL,NULL,"Geoffrey Karanja",NULL,"768696658","89254021394274518926","639021397451892","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-08-22 13:32:25+00","2035-08-22 23:59:59+00",1,"1",NULL,"21267.01","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061047435","Management_Mazda - KDU 613A","X3","aotomobile","KDU 613A","KDU 613A","Mazda","automobile",NULL,NULL,NULL,NULL,"Management_Mazda",NULL,"790175971","89254021394215205971","639021391520597","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-09 08:02:26+00","2035-07-09 23:59:59+00",1,"1",NULL,"9761.38","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061048276","Victor Kimutai - KDS 919Y","X3","aotomobile","KDS 919Y","KDS 919Y","Probox","automobile",NULL,NULL,NULL,NULL,"Victor Kimutai",NULL,"768696755","89254021394274518900","639021397451890","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2025-08-02 07:38:01+00","2035-08-02 23:59:59+00",1,"1",NULL,"23296.79","2026-04-23 10:54:41.63532+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061048300","KMGR 409U HENRY JAZZ",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,"unknown",NULL,NULL,"2026-04-24 04:30:20.231102+00","2026-04-24 04:30:20.231102+00",NULL,NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061048466","Samuel Muriithy - KDR 594N","X3","aotomobile","KDR 594N","KDR 594N","Probox","automobile",NULL,NULL,NULL,NULL,"Samuel Muriithy",NULL,"797680395","89254021334258159628","639021335815962","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-07-24 09:37:31+00","2035-07-24 23:59:59+00",1,"1",NULL,"27634.10","2026-04-23 11:43:39.178819+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061048615","Office-KMDG 902Z","X3","aotomobile",NULL,NULL,NULL,"automobile",NULL,NULL,NULL,NULL,NULL,NULL,"0768697276","89254021394274518876","639021397451887","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-31 09:59:43+00","2035-07-31 23:59:59+00",1,"1",NULL,"5721.21","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061048953","Timothy Gitau - KDT 916R","X3","aotomobile","KDT 916R","KDT 916R","Probox","automobile",NULL,NULL,NULL,NULL,"Timothy Gitau",NULL,"768697056","89254021394274518967","639021397451896","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2025-08-02 08:48:05+00","2035-08-02 23:59:59+00",1,"1",NULL,"28536.23","2026-04-23 10:53:31.102315+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061049001","Parked - KMGK 596V","X3","aotomobile","KMGK 596V","KMGK 596V","Motorbike","mtc",NULL,NULL,NULL,NULL,"Parked",NULL,"768697064","89254021394274518884","639021397451888","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-31 08:40:18+00","2035-07-31 23:59:59+00",1,"1",NULL,"20612.89","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061053714","Samuel Kihara - KMEL 225X","X3","aotomobile","KMEL 225X","KMEL 225X","Motorbike","mtc",NULL,NULL,NULL,NULL,"Samuel Kihara",NULL,"768696832","89254021394274518934","639021397451893","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-08-02 13:51:47+00","2035-08-02 23:59:59+00",1,"1",NULL,"26897.18","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061053748","Rashid Hassan - KDM 840V","X3","aotomobile","KDM 840V","KDM 840V","Probox","automobile",NULL,NULL,NULL,NULL,"Rashid Hassan",NULL,"768445963","89254021334212352574","639021331235257","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-07-10 13:54:11+00","2035-07-10 23:59:59+00",1,"1",NULL,"26612.42","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061054548","James Onyango - KDU 613B","X3","aotomobile","KDU 613B","KDU 613B","Probox","automobile",NULL,NULL,NULL,NULL,"James Onyango",NULL,"790175997","89254021394215205948","639021391520594","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-07-09 07:11:15+00","2035-07-09 23:59:59+00",1,"1",NULL,"13446.05","2026-04-23 10:26:24.667167+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061054555","Rofas Njagi - KDT 728R","X3","aotomobile","KDT 728R","KDT 728R","Probox","automobile",NULL,NULL,NULL,NULL,"Rofas Njagi",NULL,"790176726","89254021394215205823","639021391520582","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-07-16 06:44:33+00","2035-07-16 23:59:59+00",1,"1",NULL,"27250.80","2026-04-23 10:25:21.085437+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061559538","FRED KMGW 538W HULETI",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,"unknown",NULL,NULL,"2026-04-23 10:42:18.5831+00","2026-04-23 10:42:18.5831+00",NULL,NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061562722","John Mbugua - KDW 573B","X3","aotomobile","KDW 573B","KDW 573B","Probox","automobile",NULL,NULL,NULL,NULL,"John Mbugua",NULL,"758052508","89254021414206816832","639021410681683","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2026-01-30 06:53:57+00","2036-01-30 23:59:59+00",1,"1",NULL,"4488.19","2026-04-23 10:25:38.887433+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061562847","Levine Wasike - KDV 439W","X3","aotomobile","KDV 439W","KDV 439W","Probox","automobile",NULL,NULL,NULL,NULL,"Levine Wasike",NULL,"758047032","89254021414206816840","639021410681684","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2025-12-13 11:14:14+00","2035-12-13 23:59:59+00",1,"1",NULL,"7880.92","2026-04-23 10:35:50.779597+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061563282","X3-63282","X3","aotomobile",NULL,NULL,NULL,"automobile",NULL,NULL,NULL,NULL,NULL,NULL,NULL,"8925610001837573427F","641101970467668","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-02-14 07:20:10+00","2036-02-14 23:59:59+00",1,"1",NULL,"4758.32","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061563415","Barack Orwa - KDW 781E","X3","aotomobile","KDW 781E","KDW 781E","Vazel","automobile",NULL,NULL,NULL,NULL,"Barack Orwa",NULL,"758052541","89254021414206816931","639021410681693","Fireside_MSA","Fireside Group MSA","9d0927d235e44fe7abf254902fc68921","Default group","2026-01-13 12:37:42+00","2036-01-13 23:59:59+00",1,"1",NULL,"4165.95","2026-04-23 11:22:00.676215+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061563423","Joel Ntumba - UMA 826AB","X3","aotomobile","UMA 826AB","UMA 826AB","Motorbike","mtc",NULL,NULL,NULL,NULL,"Joel Ntumba",NULL,"119051036","89254021414206652690","639021410665269","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-01-28 13:55:39+00","2036-01-28 23:59:59+00",1,"1",NULL,"1174.05","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061563597","Dominic Wambua - KDV 683Z","X3","aotomobile","KDV 683Z","KDV 683Z","Probox","automobile",NULL,NULL,NULL,NULL,"Dominic Wambua",NULL,"758052405","89254021414206816733","639021410681673","Fireside@HQ","Fireside Telematics","6ef0b0fc2d964b358b70dc2cfcbc5b7e","Default group","2026-01-30 06:55:35+00","2036-01-30 23:59:59+00",1,"1",NULL,"6790.53","2026-04-23 10:25:40.125927+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061563639","Benjamin Ananda - KDV 438W","X3","aotomobile","KDV 438W","KDV 438W","Probox","automobile",NULL,NULL,NULL,NULL,"Benjamin Ananda",NULL,"758047065","89254021414206816683","639021410681668","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-12-13 16:02:37+00","2035-12-13 23:59:59+00",1,"1",NULL,"14446.33","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061564280","Rodin Kiberu - UMA 011EK","X3","aotomobile","UMA 011EK","UMA 011EK","Motorbike","mtc",NULL,NULL,NULL,NULL,"Rodin Kiberu",NULL,"118081642","89254021414206817244","639021410681724","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-01-28 13:13:57+00","2036-01-28 23:59:59+00",1,"1",NULL,"841.39","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061564470","Silvanus Kipkorir - KDV 064S","X3","aotomobile","KDV 064S","KDV 064S","Probox","automobile",NULL,NULL,NULL,NULL,"Silvanus Kipkorir",NULL,"113669866","89254021414206378718","639021410637871","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-11-21 16:49:17+00","2035-11-21 23:59:59+00",1,"1",NULL,"23869.16","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061568968","X3-68968","X3","aotomobile",NULL,NULL,NULL,"automobile",NULL,NULL,NULL,NULL,NULL,NULL,NULL,"89254021414206816915","639021410681691","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-03-11 06:19:14+00","2036-03-11 23:59:59+00",1,"1",NULL,"16.23","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061569123","Albert Mutwiri - KDV 437W","X3","aotomobile","KDV 437W","KDV 437W","Probox","automobile",NULL,NULL,NULL,NULL,"Albert Mutwiri",NULL,"758047101","89254021414206816881","639021410681688","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-12-13 14:26:17+00","2035-12-13 23:59:59+00",1,"1",NULL,"13032.60","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061569131","UMA 418EK","X3","aotomobile","UMA 418EK","UMA 418EK",NULL,"automobile",NULL,NULL,NULL,NULL,"UG",NULL,"256792997053","8925610001837573385F","641101970467664","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-02-26 08:15:44+00","2036-02-26 23:59:59+00",1,"1",NULL,"2333.45","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061569479","UMA 382EK","X3","aotomobile","UMA 382EK","UMA 382EK",NULL,"automobile",NULL,NULL,NULL,NULL,"UG",NULL,"256792997079","8925610001837573419F","641101970467667","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2026-02-26 08:21:10+00","2036-02-26 23:59:59+00",1,"1",NULL,"1954.86","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061578553","X3-78553",NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,"unknown",NULL,NULL,"2026-04-23 15:30:19.981271+00","2026-04-23 15:30:19.981271+00",NULL,NULL,NULL,NULL,NULL,NULL,NULL
|
||||
"865135061581904","Robert Kipruto - KDV 072L","X3","aotomobile","KDV 072L","KDV 072L","Probox","automobile",NULL,NULL,NULL,NULL,"Robert Kipruto",NULL,"114149576","89254021264261503993","639021266150399","fireside","Fireside Group HQ","2f1acaef6c884214b4598719180fe68d","Default group","2025-11-21 15:30:29+00","2035-11-21 23:59:59+00",1,"1",NULL,"15252.84","2026-04-23 10:23:56.546784+00","2026-04-24 07:43:45.210628+00","2026-04-24 07:43:45.210628+00",NULL,NULL,NULL,NULL,NULL,NULL
|
||||
|
|
|
@ -1,69 +0,0 @@
|
|||
-- =============================================================
|
||||
-- TRACKSOLID DWH CONTROL SCHEMA
|
||||
-- Target Database: tracksolid_dwh
|
||||
-- Purpose: Watermarks + per-run audit log for the n8n DWH pipeline
|
||||
-- Applies after: 260423_dwh_ddl_v1.sql (requires dwh_owner role + grafana_ro role)
|
||||
-- =============================================================
|
||||
|
||||
BEGIN;
|
||||
|
||||
-- 1. CONTROL SCHEMA
|
||||
-- Owned by dwh_owner to match bronze/silver/gold ownership convention from 260423.
|
||||
CREATE SCHEMA IF NOT EXISTS dwh_control AUTHORIZATION dwh_owner;
|
||||
|
||||
GRANT USAGE ON SCHEMA dwh_control TO grafana_ro;
|
||||
|
||||
-- 2. PERMISSIONS (dwh_owner writes, grafana_ro reads)
|
||||
-- Existing default privileges from 260423 only cover bronze/silver/gold; extend to dwh_control.
|
||||
ALTER DEFAULT PRIVILEGES FOR ROLE dwh_owner IN SCHEMA dwh_control
|
||||
GRANT SELECT ON TABLES TO grafana_ro;
|
||||
|
||||
-- 3. EXTRACT WATERMARKS
|
||||
-- One row per incremental table. Updated by Workflow 2 after a successful load commit.
|
||||
-- last_extracted_at is the UPPER bound used in the most recent successful extract,
|
||||
-- so the next run uses `WHERE <ts_col> > last_extracted_at AND <ts_col> <= :run_started_at`.
|
||||
SET ROLE dwh_owner;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS dwh_control.extract_watermarks (
|
||||
table_name TEXT PRIMARY KEY,
|
||||
last_extracted_at TIMESTAMPTZ NOT NULL DEFAULT '2026-01-01T00:00:00Z',
|
||||
last_loaded_at TIMESTAMPTZ,
|
||||
rows_loaded_last_run INT,
|
||||
updated_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 4. EXTRACT RUN AUDIT LOG
|
||||
-- One row per table per cron tick. Lifecycle: extracting → uploaded → loading → loaded (or failed).
|
||||
-- Failures retain error_message; the CSV stays in dwh/exports/ for the next run to pick up.
|
||||
CREATE TABLE IF NOT EXISTS dwh_control.extract_runs (
|
||||
run_id BIGSERIAL PRIMARY KEY,
|
||||
table_name TEXT NOT NULL,
|
||||
run_started_at TIMESTAMPTZ NOT NULL,
|
||||
run_finished_at TIMESTAMPTZ,
|
||||
rows_extracted INT,
|
||||
rows_loaded INT,
|
||||
csv_path TEXT,
|
||||
status TEXT CHECK (status IN ('extracting','uploaded','loading','loaded','failed')),
|
||||
error_message TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_extract_runs_table_time
|
||||
ON dwh_control.extract_runs (table_name, run_started_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_extract_runs_status_time
|
||||
ON dwh_control.extract_runs (status, run_finished_at DESC);
|
||||
|
||||
-- 5. SEED WATERMARKS
|
||||
-- One row per incremental table. Snapshot tables (devices, live_positions) do not need
|
||||
-- watermarks and are intentionally omitted.
|
||||
INSERT INTO dwh_control.extract_watermarks (table_name) VALUES
|
||||
('position_history'),
|
||||
('trips'),
|
||||
('alarms'),
|
||||
('parking_events'),
|
||||
('device_events'),
|
||||
('ingestion_log')
|
||||
ON CONFLICT (table_name) DO NOTHING;
|
||||
|
||||
RESET ROLE;
|
||||
|
||||
COMMIT;
|
||||
|
|
@ -1,63 +0,0 @@
|
|||
-- =============================================================
|
||||
-- BRONZE CONSTRAINTS AUDIT
|
||||
-- Target Database: tracksolid_dwh
|
||||
-- Purpose: Assert that every ON CONFLICT target used by Workflow 2
|
||||
-- (dwh_load_bronze) is backed by a PRIMARY KEY or UNIQUE
|
||||
-- constraint in the bronze schema. Fails loudly if a future
|
||||
-- DDL edit removes a key the ingestion pipeline depends on.
|
||||
-- Applies after: 260423_dwh_ddl_v1.sql
|
||||
-- Idempotent: pure assertion, no DDL changes.
|
||||
-- =============================================================
|
||||
|
||||
BEGIN;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
missing TEXT := '';
|
||||
expected RECORD;
|
||||
BEGIN
|
||||
-- Each row asserts: bronze.<table> has a PK/UNIQUE matching <cols>.
|
||||
-- If the pipeline's ON CONFLICT clause ever diverges from this list,
|
||||
-- update both here and the n8n load workflow in lockstep.
|
||||
FOR expected IN
|
||||
SELECT * FROM (VALUES
|
||||
('devices', 'imei'),
|
||||
('live_positions', 'imei'),
|
||||
('position_history', 'imei,gps_time'),
|
||||
('trips', 'id'),
|
||||
('alarms', 'id'),
|
||||
('parking_events', 'id'),
|
||||
('device_events', 'id'),
|
||||
('ingestion_log', 'id')
|
||||
) AS t(table_name, cols)
|
||||
LOOP
|
||||
IF NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pg_constraint c
|
||||
JOIN pg_class r ON r.oid = c.conrelid
|
||||
JOIN pg_namespace n ON n.oid = r.relnamespace
|
||||
CROSS JOIN LATERAL (
|
||||
SELECT string_agg(a.attname, ',' ORDER BY k.ord) AS keycols
|
||||
FROM unnest(c.conkey) WITH ORDINALITY AS k(attnum, ord)
|
||||
JOIN pg_attribute a
|
||||
ON a.attrelid = c.conrelid AND a.attnum = k.attnum
|
||||
) AS cols
|
||||
WHERE n.nspname = 'bronze'
|
||||
AND r.relname = expected.table_name
|
||||
AND c.contype IN ('p','u')
|
||||
AND cols.keycols = expected.cols
|
||||
) THEN
|
||||
missing := missing
|
||||
|| format(E'\n - bronze.%s missing PK/UNIQUE on (%s)',
|
||||
expected.table_name, expected.cols);
|
||||
END IF;
|
||||
END LOOP;
|
||||
|
||||
IF length(missing) > 0 THEN
|
||||
RAISE EXCEPTION E'Bronze constraint audit FAILED:%s', missing;
|
||||
END IF;
|
||||
|
||||
RAISE NOTICE 'Bronze constraint audit OK: all 8 ON CONFLICT targets backed by PK/UNIQUE.';
|
||||
END$$;
|
||||
|
||||
COMMIT;
|
||||
|
|
@ -1,66 +0,0 @@
|
|||
-- =============================================================
|
||||
-- DWH ROLES AUDIT
|
||||
-- Target Database: tracksolid_dwh
|
||||
-- Purpose: Assert that the n8n DWH pipeline's role contract holds:
|
||||
-- - dwh_owner exists (writes bronze + dwh_control)
|
||||
-- - grafana_ro exists (reads bronze + silver + gold + dwh_control)
|
||||
-- - grafana_ro has CONNECT on the database
|
||||
-- - grafana_ro has USAGE on every schema it needs
|
||||
-- Applies after: 260423_dwh_ddl_v1.sql, 261001_dwh_control.sql
|
||||
-- Idempotent: pure assertion, no CREATE ROLE or GRANT statements.
|
||||
--
|
||||
-- Why this file exists: 260423 creates both roles and grants bronze/silver/gold;
|
||||
-- 261001 grants dwh_control. This file is a single checkpoint that verifies
|
||||
-- those prior migrations were applied in the right order, and fails loudly
|
||||
-- if anything is missing before the pipeline goes live.
|
||||
--
|
||||
-- Password rotation and sslmode=require enforcement are out-of-band:
|
||||
-- rotate via ALTER ROLE ... PASSWORD ... in a psql superuser session,
|
||||
-- enforce SSL via the n8n credential (sslmode=require) — not SQL-level.
|
||||
-- =============================================================
|
||||
|
||||
BEGIN;
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
missing TEXT := '';
|
||||
r RECORD;
|
||||
BEGIN
|
||||
-- 1. Roles exist
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'dwh_owner') THEN
|
||||
missing := missing || E'\n - role dwh_owner missing (expected from 260423)';
|
||||
END IF;
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'grafana_ro') THEN
|
||||
missing := missing || E'\n - role grafana_ro missing (expected from 260423)';
|
||||
END IF;
|
||||
|
||||
-- 2. grafana_ro CONNECT on this database
|
||||
IF NOT has_database_privilege('grafana_ro', current_database(), 'CONNECT') THEN
|
||||
missing := missing
|
||||
|| format(E'\n - grafana_ro lacks CONNECT on database %s',
|
||||
current_database());
|
||||
END IF;
|
||||
|
||||
-- 3. grafana_ro USAGE on every schema the pipeline / dashboards touch
|
||||
FOR r IN
|
||||
SELECT unnest(ARRAY['bronze','silver','gold','dwh_control']) AS schema_name
|
||||
LOOP
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_namespace WHERE nspname = r.schema_name) THEN
|
||||
missing := missing
|
||||
|| format(E'\n - schema %s missing (expected from 260423/261001)',
|
||||
r.schema_name);
|
||||
ELSIF NOT has_schema_privilege('grafana_ro', r.schema_name, 'USAGE') THEN
|
||||
missing := missing
|
||||
|| format(E'\n - grafana_ro lacks USAGE on schema %s',
|
||||
r.schema_name);
|
||||
END IF;
|
||||
END LOOP;
|
||||
|
||||
IF length(missing) > 0 THEN
|
||||
RAISE EXCEPTION E'DWH roles audit FAILED:%s', missing;
|
||||
END IF;
|
||||
|
||||
RAISE NOTICE 'DWH roles audit OK: dwh_owner + grafana_ro present with expected grants.';
|
||||
END$$;
|
||||
|
||||
COMMIT;
|
||||
|
|
@ -1,79 +0,0 @@
|
|||
-- =============================================================
|
||||
-- DWH OBSERVABILITY VIEWS
|
||||
-- Target Database: tracksolid_dwh
|
||||
-- Purpose: Surface pipeline health for Grafana dashboards. Three views,
|
||||
-- one concern each:
|
||||
-- v_table_freshness — how long since each table was last loaded
|
||||
-- v_recent_failures — failed runs in the last 24h
|
||||
-- v_watermark_lag — per-table watermark vs. now
|
||||
-- Applies after: 261001_dwh_control.sql
|
||||
-- Readability: owned by dwh_owner → grafana_ro inherits SELECT via the
|
||||
-- ALTER DEFAULT PRIVILEGES set in 261001. Explicit GRANT below
|
||||
-- covers the case where defaults were set AFTER this file runs.
|
||||
-- =============================================================
|
||||
|
||||
BEGIN;
|
||||
|
||||
SET ROLE dwh_owner;
|
||||
|
||||
-- 1. FRESHNESS
|
||||
-- One row per table that has ever loaded successfully. `lag` drives the
|
||||
-- freshness panel; `loads_last_24h` sanity-checks the cron cadence.
|
||||
CREATE OR REPLACE VIEW dwh_control.v_table_freshness AS
|
||||
SELECT
|
||||
table_name,
|
||||
MAX(run_finished_at) AS last_loaded_at,
|
||||
NOW() - MAX(run_finished_at) AS lag,
|
||||
COUNT(*) FILTER (WHERE run_started_at > NOW() - INTERVAL '24 hours') AS loads_last_24h
|
||||
FROM dwh_control.extract_runs
|
||||
WHERE status = 'loaded'
|
||||
GROUP BY table_name;
|
||||
|
||||
COMMENT ON VIEW dwh_control.v_table_freshness IS
|
||||
'Per-table load lag. Alert when lag > 4h during active hours (05:00–23:00 EAT).';
|
||||
|
||||
-- 2. RECENT FAILURES
|
||||
-- Failures retain error_message; the CSV stays in dwh/exports/ for the next
|
||||
-- scheduled run to retry. Panel should show run_id so operators can grep logs.
|
||||
CREATE OR REPLACE VIEW dwh_control.v_recent_failures AS
|
||||
SELECT
|
||||
run_id,
|
||||
table_name,
|
||||
run_started_at,
|
||||
run_finished_at,
|
||||
csv_path,
|
||||
error_message
|
||||
FROM dwh_control.extract_runs
|
||||
WHERE status = 'failed'
|
||||
AND run_started_at > NOW() - INTERVAL '24 hours'
|
||||
ORDER BY run_started_at DESC;
|
||||
|
||||
COMMENT ON VIEW dwh_control.v_recent_failures IS
|
||||
'Failed extract/load runs in the last 24h. Alert on any row.';
|
||||
|
||||
-- 3. WATERMARK LAG
|
||||
-- Distinguishes "pipeline ran but found nothing" (load_lag small, extract_lag
|
||||
-- growing) from "pipeline is stuck" (both lags growing). Snapshot tables are
|
||||
-- not in extract_watermarks so they do not appear here — that is intentional.
|
||||
CREATE OR REPLACE VIEW dwh_control.v_watermark_lag AS
|
||||
SELECT
|
||||
table_name,
|
||||
last_extracted_at,
|
||||
last_loaded_at,
|
||||
rows_loaded_last_run,
|
||||
NOW() - last_loaded_at AS load_lag,
|
||||
NOW() - last_extracted_at AS extract_lag
|
||||
FROM dwh_control.extract_watermarks;
|
||||
|
||||
COMMENT ON VIEW dwh_control.v_watermark_lag IS
|
||||
'Per-table watermark position vs. now. Incremental tables only (6 rows).';
|
||||
|
||||
RESET ROLE;
|
||||
|
||||
-- Explicit grants: defensive in case ALTER DEFAULT PRIVILEGES from 261001
|
||||
-- was not in effect when these views were created.
|
||||
GRANT SELECT ON dwh_control.v_table_freshness TO grafana_ro;
|
||||
GRANT SELECT ON dwh_control.v_recent_failures TO grafana_ro;
|
||||
GRANT SELECT ON dwh_control.v_watermark_lag TO grafana_ro;
|
||||
|
||||
COMMIT;
|
||||
Binary file not shown.
File diff suppressed because it is too large
Load diff
|
|
@ -1,5 +0,0 @@
|
|||
FROM grafana/grafana:11.0.0
|
||||
|
||||
# Bake provisioning files into the image so Coolify bind mounts are not needed.
|
||||
# Grafana substitutes ${ENV_VAR} references in provisioning files at startup.
|
||||
COPY provisioning /etc/grafana/provisioning
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,589 +0,0 @@
|
|||
{
|
||||
"title": "NOC Fleet Operations — Live",
|
||||
"uid": "noc-fleet-live",
|
||||
"schemaVersion": 39,
|
||||
"version": 1,
|
||||
"refresh": "30s",
|
||||
"time": { "from": "now-1h", "to": "now" },
|
||||
"timepicker": {
|
||||
"refresh_intervals": ["10s", "30s", "1m", "5m"]
|
||||
},
|
||||
"editable": false,
|
||||
"tags": ["noc", "fleet", "live"],
|
||||
"fiscalYearStartMonth": 0,
|
||||
"graphTooltip": 0,
|
||||
"panels": [
|
||||
{
|
||||
"id": 1,
|
||||
"type": "stat",
|
||||
"title": "Total Vehicles",
|
||||
"gridPos": { "x": 0, "y": 0, "w": 4, "h": 3 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false },
|
||||
"textMode": "auto"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [{ "color": "text", "value": null }]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT COUNT(*) AS \"Total Vehicles\"\nFROM tracksolid.devices WHERE enabled_flag = 1;",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"type": "stat",
|
||||
"title": "Online Now",
|
||||
"description": "GPS fix within last 5 minutes",
|
||||
"gridPos": { "x": 4, "y": 0, "w": 4, "h": 3 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false },
|
||||
"textMode": "auto"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "red", "value": null },
|
||||
{ "color": "green", "value": 1 }
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT COUNT(*) AS \"Online\"\nFROM tracksolid.v_fleet_status\nWHERE connectivity_status = 'online';",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"type": "stat",
|
||||
"title": "Recent (5-30 min)",
|
||||
"description": "GPS fix between 5 and 30 minutes ago",
|
||||
"gridPos": { "x": 8, "y": 0, "w": 4, "h": 3 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false },
|
||||
"textMode": "auto"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "text", "value": null },
|
||||
{ "color": "yellow", "value": 1 }
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT COUNT(*) AS \"Recent\"\nFROM tracksolid.v_fleet_status\nWHERE connectivity_status = 'recent';",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"type": "stat",
|
||||
"title": "Offline",
|
||||
"description": "GPS fix older than 30 minutes",
|
||||
"gridPos": { "x": 12, "y": 0, "w": 4, "h": 3 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false },
|
||||
"textMode": "auto"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "red", "value": 1 }
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT COUNT(*) AS \"Offline\"\nFROM tracksolid.v_fleet_status\nWHERE connectivity_status = 'offline';",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"type": "stat",
|
||||
"title": "Moving Now",
|
||||
"description": "Vehicles with speed > 0 and engine on",
|
||||
"gridPos": { "x": 16, "y": 0, "w": 4, "h": 3 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false },
|
||||
"textMode": "auto"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "text", "value": null },
|
||||
{ "color": "green", "value": 1 }
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT COUNT(*) AS \"Moving\"\nFROM tracksolid.v_fleet_status\nWHERE speed > 0 AND acc_status = '1' AND connectivity_status = 'online';",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 6,
|
||||
"type": "stat",
|
||||
"title": "Avg Speed (km/h)",
|
||||
"description": "Average speed of currently moving vehicles",
|
||||
"gridPos": { "x": 20, "y": 0, "w": 4, "h": 3 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"colorMode": "background",
|
||||
"graphMode": "none",
|
||||
"justifyMode": "auto",
|
||||
"orientation": "auto",
|
||||
"reduceOptions": { "calcs": ["lastNotNull"], "fields": "", "values": false },
|
||||
"textMode": "auto"
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "thresholds" },
|
||||
"unit": "velocitykmh",
|
||||
"thresholds": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 80 },
|
||||
{ "color": "red", "value": 120 }
|
||||
]
|
||||
}
|
||||
},
|
||||
"overrides": []
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT ROUND(AVG(speed)::numeric, 1) AS \"Avg Speed km/h\"\nFROM tracksolid.v_fleet_status\nWHERE speed > 0 AND acc_status = '1' AND connectivity_status = 'online';",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 7,
|
||||
"type": "geomap",
|
||||
"title": "Live Vehicle Locations",
|
||||
"gridPos": { "x": 0, "y": 3, "w": 24, "h": 16 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"basemap": {
|
||||
"config": { "theme": "dark" },
|
||||
"name": "Basemap",
|
||||
"type": "carto"
|
||||
},
|
||||
"controls": {
|
||||
"mouseWheelZoom": true,
|
||||
"showAttribution": true,
|
||||
"showDebug": false,
|
||||
"showScale": true,
|
||||
"showZoom": true
|
||||
},
|
||||
"layers": [
|
||||
{
|
||||
"config": {
|
||||
"showLegend": false,
|
||||
"style": {
|
||||
"color": {
|
||||
"field": "vehicle_number",
|
||||
"fixed": "dark-green",
|
||||
"mode": "field"
|
||||
},
|
||||
"opacity": 1,
|
||||
"rotation": {
|
||||
"field": "direction",
|
||||
"fixed": 0,
|
||||
"max": 360,
|
||||
"min": -360,
|
||||
"mode": "field"
|
||||
},
|
||||
"size": {
|
||||
"fixed": 18,
|
||||
"max": 15,
|
||||
"min": 2,
|
||||
"mode": "fixed"
|
||||
},
|
||||
"symbol": {
|
||||
"fixed": "img/icons/marker/arrow-up.svg",
|
||||
"mode": "fixed"
|
||||
},
|
||||
"symbolAlign": {
|
||||
"h": "center",
|
||||
"v": "center"
|
||||
}
|
||||
}
|
||||
},
|
||||
"filterData": { "id": "byRefId", "options": "A" },
|
||||
"location": {
|
||||
"latitude": "lat",
|
||||
"longitude": "lng",
|
||||
"mode": "coords"
|
||||
},
|
||||
"name": "Vehicles",
|
||||
"tooltip": true,
|
||||
"type": "markers"
|
||||
}
|
||||
],
|
||||
"tooltip": { "mode": "details" },
|
||||
"view": {
|
||||
"allLayers": true,
|
||||
"id": "coords",
|
||||
"lat": -1.5,
|
||||
"lon": 36.5,
|
||||
"zoom": 6
|
||||
}
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"color": { "mode": "palette-classic-by-name" }
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "lat" },
|
||||
"properties": [
|
||||
{ "id": "custom.hideFrom", "value": { "legend": true, "tooltip": true, "viz": false } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "lng" },
|
||||
"properties": [
|
||||
{ "id": "custom.hideFrom", "value": { "legend": true, "tooltip": true, "viz": false } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "imei" },
|
||||
"properties": [
|
||||
{ "id": "custom.hideFrom", "value": { "legend": true, "tooltip": true, "viz": false } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "direction" },
|
||||
"properties": [
|
||||
{ "id": "unit", "value": "degree" },
|
||||
{ "id": "displayName", "value": "Heading" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "speed" },
|
||||
"properties": [
|
||||
{ "id": "unit", "value": "velocitykmh" },
|
||||
{ "id": "displayName", "value": "Speed" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "vehicle_number" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Plate" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "driver_name" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Driver" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "driver_phone" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Phone" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "vehicle_name" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Vehicle" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "connectivity_status" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Status" },
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"online": { "color": "green", "index": 0, "text": "Online" },
|
||||
"recent": { "color": "yellow", "index": 1, "text": "Recent" },
|
||||
"offline": { "color": "red", "index": 2, "text": "Offline" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "acc_status" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "ACC" },
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"1": { "text": "On", "color": "green", "index": 0 },
|
||||
"0": { "text": "Off", "color": "red", "index": 1 }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "gps_time" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Last Fix" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "loc_desc" },
|
||||
"properties": [
|
||||
{ "id": "displayName", "value": "Location" }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT\n d.imei,\n d.vehicle_number,\n d.vehicle_name,\n d.driver_name,\n d.driver_phone,\n d.city,\n d.device_group,\n lp.lat,\n lp.lng,\n lp.speed,\n lp.direction,\n lp.acc_status,\n lp.loc_desc,\n lp.gps_time,\n CASE\n WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes' THEN 'online'\n WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 'recent'\n ELSE 'offline'\n END AS connectivity_status\nFROM tracksolid.devices d\nINNER JOIN tracksolid.live_positions lp USING (imei)\nWHERE d.enabled_flag = 1\n AND lp.lat IS NOT NULL\n AND lp.lng IS NOT NULL\nORDER BY d.vehicle_number;",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"type": "table",
|
||||
"title": "Vehicle Status",
|
||||
"gridPos": { "x": 0, "y": 19, "w": 24, "h": 10 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"cellHeight": "sm",
|
||||
"footer": { "countRows": false, "fields": "", "reducer": ["sum"], "show": false },
|
||||
"showHeader": true,
|
||||
"sortBy": []
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"custom": {
|
||||
"align": "auto",
|
||||
"cellOptions": { "type": "auto" },
|
||||
"filterable": true,
|
||||
"inspect": false
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Status" },
|
||||
"properties": [
|
||||
{
|
||||
"id": "custom.cellOptions",
|
||||
"value": { "type": "color-background", "mode": "basic" }
|
||||
},
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"Online": { "color": "green", "index": 0 },
|
||||
"Recent": { "color": "yellow", "index": 1 },
|
||||
"Offline": { "color": "red", "index": 2 }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Speed (km/h)" },
|
||||
"properties": [
|
||||
{
|
||||
"id": "custom.cellOptions",
|
||||
"value": { "type": "color-text" }
|
||||
},
|
||||
{
|
||||
"id": "color",
|
||||
"value": { "mode": "thresholds" }
|
||||
},
|
||||
{
|
||||
"id": "thresholds",
|
||||
"value": {
|
||||
"mode": "absolute",
|
||||
"steps": [
|
||||
{ "color": "green", "value": null },
|
||||
{ "color": "yellow", "value": 80 },
|
||||
{ "color": "red", "value": 120 }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Last Fix" },
|
||||
"properties": [
|
||||
{ "id": "unit", "value": "dateTimeAsLocal" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Min Since Fix" },
|
||||
"properties": [
|
||||
{ "id": "custom.width", "value": 110 },
|
||||
{ "id": "displayName", "value": "Min Ago" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Driver Phone" },
|
||||
"properties": [
|
||||
{ "id": "custom.width", "value": 130 }
|
||||
]
|
||||
},
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Plate" },
|
||||
"properties": [
|
||||
{ "id": "custom.width", "value": 110 }
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT\n d.vehicle_number AS \"Plate\",\n d.vehicle_name AS \"Vehicle\",\n d.driver_name AS \"Driver\",\n d.driver_phone AS \"Driver Phone\",\n d.city AS \"City\",\n ROUND(lp.speed::numeric, 0) AS \"Speed (km/h)\",\n lp.loc_desc AS \"Last Location\",\n lp.gps_time AS \"Last Fix\",\n CASE\n WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes' THEN 'Online'\n WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 'Recent'\n ELSE 'Offline'\n END AS \"Status\",\n EXTRACT(EPOCH FROM (NOW() - lp.gps_time))::int / 60 AS \"Min Since Fix\"\nFROM tracksolid.devices d\nLEFT JOIN tracksolid.live_positions lp USING (imei)\nWHERE d.enabled_flag = 1\nORDER BY\n CASE\n WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes' THEN 0\n WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 1\n ELSE 2\n END,\n d.vehicle_number;",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"type": "table",
|
||||
"title": "Ingestion Health",
|
||||
"collapsed": true,
|
||||
"gridPos": { "x": 0, "y": 29, "w": 24, "h": 8 },
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"options": {
|
||||
"cellHeight": "sm",
|
||||
"footer": { "countRows": false, "fields": "", "reducer": ["sum"], "show": false },
|
||||
"showHeader": true
|
||||
},
|
||||
"fieldConfig": {
|
||||
"defaults": {
|
||||
"custom": {
|
||||
"align": "auto",
|
||||
"cellOptions": { "type": "auto" },
|
||||
"filterable": false,
|
||||
"inspect": false
|
||||
}
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"matcher": { "id": "byName", "options": "Result" },
|
||||
"properties": [
|
||||
{
|
||||
"id": "custom.cellOptions",
|
||||
"value": { "type": "color-background", "mode": "basic" }
|
||||
},
|
||||
{
|
||||
"id": "mappings",
|
||||
"value": [
|
||||
{
|
||||
"type": "value",
|
||||
"options": {
|
||||
"OK": { "color": "green", "index": 0 },
|
||||
"FAIL": { "color": "red", "index": 1 }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"targets": [
|
||||
{
|
||||
"datasource": { "type": "postgres", "uid": "tracksolid_pg" },
|
||||
"rawSql": "SELECT\n endpoint AS \"Endpoint\",\n TO_CHAR(run_at, 'HH24:MI DD-Mon') AS \"Last Run\",\n CASE WHEN success THEN 'OK' ELSE 'FAIL' END AS \"Result\",\n error_message AS \"Error\",\n seconds_ago AS \"Lag (s)\"\nFROM tracksolid.v_ingestion_health\nORDER BY endpoint;",
|
||||
"format": "table",
|
||||
"refId": "A"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
apiVersion: 1
|
||||
|
||||
providers:
|
||||
- name: NOC Fleet Dashboards
|
||||
orgId: 1
|
||||
folder: NOC
|
||||
type: file
|
||||
disableDeletion: true
|
||||
updateIntervalSeconds: 30
|
||||
allowUiUpdates: false
|
||||
options:
|
||||
path: /etc/grafana/provisioning/dashboards-json
|
||||
|
|
@ -1,20 +0,0 @@
|
|||
apiVersion: 1
|
||||
|
||||
datasources:
|
||||
- name: TracksolidDB
|
||||
type: postgres
|
||||
uid: tracksolid_pg
|
||||
url: timescale_db:5432
|
||||
database: tracksolid_db
|
||||
user: grafana_ro
|
||||
secureJsonData:
|
||||
password: ${GRAFANA_DB_RO_PASSWORD}
|
||||
jsonData:
|
||||
sslmode: disable
|
||||
maxOpenConns: 5
|
||||
maxIdleConns: 2
|
||||
connMaxLifetime: 14400
|
||||
postgresVersion: 1600
|
||||
timescaledb: true
|
||||
editable: false
|
||||
isDefault: true
|
||||
|
|
@ -512,7 +512,7 @@ def poll_track_list():
|
|||
|
||||
Impact on reporting:
|
||||
- position_history row density increases from ~1/min to ~1–4/min per device
|
||||
- Route traces in Grafana become accurate continuous paths
|
||||
- Route traces on the map become accurate continuous paths
|
||||
- Speed profile queries gain meaningful resolution (avg over 10s intervals
|
||||
vs 60s intervals) — enables hard-braking / harsh-acceleration detection
|
||||
- v_mileage_daily_cagg continuous aggregate gains finer odometer deltas
|
||||
|
|
|
|||
|
|
@ -1,91 +0,0 @@
|
|||
{
|
||||
"name": "Jimi Alarm Push",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "pushalarm",
|
||||
"responseMode": "responseNode",
|
||||
"options": {}
|
||||
},
|
||||
"id": "webhook-pushalarm",
|
||||
"name": "Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [260, 300],
|
||||
"webhookId": "pushalarm"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"respondWith": "json",
|
||||
"responseBody": "={{ JSON.stringify({\"code\": 0, \"msg\": \"success\"}) }}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "respond-pushalarm",
|
||||
"name": "Respond to Jimi",
|
||||
"type": "n8n-nodes-base.respondToWebhook",
|
||||
"typeVersion": 1.1,
|
||||
"position": [480, 300]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://tshook.rahamafresh.com/pushalarm",
|
||||
"sendBody": true,
|
||||
"contentType": "form-urlencoded",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "token",
|
||||
"value": "={{ $('Webhook').item.json.body.token }}"
|
||||
},
|
||||
{
|
||||
"name": "data_list",
|
||||
"value": "={{ $('Webhook').item.json.body.data_list }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 30000
|
||||
}
|
||||
},
|
||||
"id": "http-pushalarm",
|
||||
"name": "Forward to Webhook Receiver",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [700, 300]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Respond to Jimi",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Respond to Jimi": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Forward to Webhook Receiver",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "Jimi Tracksolid"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,91 +0,0 @@
|
|||
{
|
||||
"name": "Jimi Fault Code Push",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "pushfaultinfo",
|
||||
"responseMode": "responseNode",
|
||||
"options": {}
|
||||
},
|
||||
"id": "webhook-pushfaultinfo",
|
||||
"name": "Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [260, 300],
|
||||
"webhookId": "pushfaultinfo"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"respondWith": "json",
|
||||
"responseBody": "={{ JSON.stringify({\"code\": 0, \"msg\": \"success\"}) }}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "respond-pushfaultinfo",
|
||||
"name": "Respond to Jimi",
|
||||
"type": "n8n-nodes-base.respondToWebhook",
|
||||
"typeVersion": 1.1,
|
||||
"position": [480, 300]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://tshook.rahamafresh.com/pushfaultinfo",
|
||||
"sendBody": true,
|
||||
"contentType": "form-urlencoded",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "token",
|
||||
"value": "={{ $('Webhook').item.json.body.token }}"
|
||||
},
|
||||
{
|
||||
"name": "data_list",
|
||||
"value": "={{ $('Webhook').item.json.body.data_list }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 30000
|
||||
}
|
||||
},
|
||||
"id": "http-pushfaultinfo",
|
||||
"name": "Forward to Webhook Receiver",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [700, 300]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Respond to Jimi",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Respond to Jimi": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Forward to Webhook Receiver",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "Jimi Tracksolid"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,91 +0,0 @@
|
|||
{
|
||||
"name": "Jimi GPS Push",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "pushgps",
|
||||
"responseMode": "responseNode",
|
||||
"options": {}
|
||||
},
|
||||
"id": "webhook-pushgps",
|
||||
"name": "Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [260, 300],
|
||||
"webhookId": "pushgps"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"respondWith": "json",
|
||||
"responseBody": "={{ JSON.stringify({\"code\": 0, \"msg\": \"success\"}) }}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "respond-pushgps",
|
||||
"name": "Respond to Jimi",
|
||||
"type": "n8n-nodes-base.respondToWebhook",
|
||||
"typeVersion": 1.1,
|
||||
"position": [480, 300]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://tshook.rahamafresh.com/pushgps",
|
||||
"sendBody": true,
|
||||
"contentType": "form-urlencoded",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "token",
|
||||
"value": "={{ $('Webhook').item.json.body.token }}"
|
||||
},
|
||||
{
|
||||
"name": "data_list",
|
||||
"value": "={{ $('Webhook').item.json.body.data_list }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 30000
|
||||
}
|
||||
},
|
||||
"id": "http-pushgps",
|
||||
"name": "Forward to Webhook Receiver",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [700, 300]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Respond to Jimi",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Respond to Jimi": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Forward to Webhook Receiver",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "Jimi Tracksolid"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,91 +0,0 @@
|
|||
{
|
||||
"name": "Jimi Heartbeat Push",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "pushhb",
|
||||
"responseMode": "responseNode",
|
||||
"options": {}
|
||||
},
|
||||
"id": "webhook-pushhb",
|
||||
"name": "Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [260, 300],
|
||||
"webhookId": "pushhb"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"respondWith": "json",
|
||||
"responseBody": "={{ JSON.stringify({\"code\": 0, \"msg\": \"success\"}) }}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "respond-pushhb",
|
||||
"name": "Respond to Jimi",
|
||||
"type": "n8n-nodes-base.respondToWebhook",
|
||||
"typeVersion": 1.1,
|
||||
"position": [480, 300]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://tshook.rahamafresh.com/pushhb",
|
||||
"sendBody": true,
|
||||
"contentType": "form-urlencoded",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "token",
|
||||
"value": "={{ $('Webhook').item.json.body.token }}"
|
||||
},
|
||||
{
|
||||
"name": "data_list",
|
||||
"value": "={{ $('Webhook').item.json.body.data_list }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 30000
|
||||
}
|
||||
},
|
||||
"id": "http-pushhb",
|
||||
"name": "Forward to Webhook Receiver",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [700, 300]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Respond to Jimi",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Respond to Jimi": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Forward to Webhook Receiver",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "Jimi Tracksolid"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,91 +0,0 @@
|
|||
{
|
||||
"name": "Jimi OBD Push",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "pushobd",
|
||||
"responseMode": "responseNode",
|
||||
"options": {}
|
||||
},
|
||||
"id": "webhook-pushobd",
|
||||
"name": "Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [260, 300],
|
||||
"webhookId": "pushobd"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"respondWith": "json",
|
||||
"responseBody": "={{ JSON.stringify({\"code\": 0, \"msg\": \"success\"}) }}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "respond-pushobd",
|
||||
"name": "Respond to Jimi",
|
||||
"type": "n8n-nodes-base.respondToWebhook",
|
||||
"typeVersion": 1.1,
|
||||
"position": [480, 300]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://tshook.rahamafresh.com/pushobd",
|
||||
"sendBody": true,
|
||||
"contentType": "form-urlencoded",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "token",
|
||||
"value": "={{ $('Webhook').item.json.body.token }}"
|
||||
},
|
||||
{
|
||||
"name": "data_list",
|
||||
"value": "={{ $('Webhook').item.json.body.data_list }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 30000
|
||||
}
|
||||
},
|
||||
"id": "http-pushobd",
|
||||
"name": "Forward to Webhook Receiver",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [700, 300]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Respond to Jimi",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Respond to Jimi": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Forward to Webhook Receiver",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "Jimi Tracksolid"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -1,91 +0,0 @@
|
|||
{
|
||||
"name": "Jimi Trip Report Push",
|
||||
"nodes": [
|
||||
{
|
||||
"parameters": {
|
||||
"httpMethod": "POST",
|
||||
"path": "pushtripreport",
|
||||
"responseMode": "responseNode",
|
||||
"options": {}
|
||||
},
|
||||
"id": "webhook-pushtripreport",
|
||||
"name": "Webhook",
|
||||
"type": "n8n-nodes-base.webhook",
|
||||
"typeVersion": 2,
|
||||
"position": [260, 300],
|
||||
"webhookId": "pushtripreport"
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"respondWith": "json",
|
||||
"responseBody": "={{ JSON.stringify({\"code\": 0, \"msg\": \"success\"}) }}",
|
||||
"options": {}
|
||||
},
|
||||
"id": "respond-pushtripreport",
|
||||
"name": "Respond to Jimi",
|
||||
"type": "n8n-nodes-base.respondToWebhook",
|
||||
"typeVersion": 1.1,
|
||||
"position": [480, 300]
|
||||
},
|
||||
{
|
||||
"parameters": {
|
||||
"method": "POST",
|
||||
"url": "https://tshook.rahamafresh.com/pushtripreport",
|
||||
"sendBody": true,
|
||||
"contentType": "form-urlencoded",
|
||||
"bodyParameters": {
|
||||
"parameters": [
|
||||
{
|
||||
"name": "token",
|
||||
"value": "={{ $('Webhook').item.json.body.token }}"
|
||||
},
|
||||
{
|
||||
"name": "data_list",
|
||||
"value": "={{ $('Webhook').item.json.body.data_list }}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"options": {
|
||||
"timeout": 30000
|
||||
}
|
||||
},
|
||||
"id": "http-pushtripreport",
|
||||
"name": "Forward to Webhook Receiver",
|
||||
"type": "n8n-nodes-base.httpRequest",
|
||||
"typeVersion": 4.2,
|
||||
"position": [700, 300]
|
||||
}
|
||||
],
|
||||
"connections": {
|
||||
"Webhook": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Respond to Jimi",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
},
|
||||
"Respond to Jimi": {
|
||||
"main": [
|
||||
[
|
||||
{
|
||||
"node": "Forward to Webhook Receiver",
|
||||
"type": "main",
|
||||
"index": 0
|
||||
}
|
||||
]
|
||||
]
|
||||
}
|
||||
},
|
||||
"settings": {
|
||||
"executionOrder": "v1"
|
||||
},
|
||||
"tags": [
|
||||
{
|
||||
"name": "Jimi Tracksolid"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -182,24 +182,6 @@ def run_file(path, filename):
|
|||
return True
|
||||
|
||||
|
||||
def sync_role_passwords(conn):
|
||||
"""
|
||||
Keep DB role passwords in sync with env vars on every startup.
|
||||
Safe to run repeatedly — ALTER ROLE is idempotent.
|
||||
This fixes roles created with the placeholder 'SET_PASSWORD_IN_ENV'.
|
||||
"""
|
||||
roles = {
|
||||
"grafana_ro": os.getenv("GRAFANA_DB_RO_PASSWORD"),
|
||||
"pgbouncer": os.getenv("PGBOUNCER_AUTH_PASSWORD"),
|
||||
}
|
||||
with conn.cursor() as cur:
|
||||
for role, password in roles.items():
|
||||
if password:
|
||||
cur.execute(f"ALTER ROLE {role} WITH PASSWORD %s", (password,))
|
||||
print(f" Password synced for role: {role}")
|
||||
conn.commit()
|
||||
|
||||
|
||||
def verify_schema(conn):
|
||||
"""Verify critical tables exist. Exit 1 if missing — blocks service start."""
|
||||
print("Verifying schema...")
|
||||
|
|
@ -252,7 +234,6 @@ def main():
|
|||
|
||||
print(f"\nMigrations: {applied} applied, {skipped} skipped.")
|
||||
|
||||
sync_role_passwords(conn)
|
||||
verify_schema(conn)
|
||||
conn.close()
|
||||
print("Startup checks passed.\n")
|
||||
|
|
|
|||
Loading…
Reference in a new issue