tracksolid_timescale_grafana_prod

kianiadee/tracksolid_timescale_grafana_prod

Fork 0

Commit graph

Author	SHA1	Message	Date
David Kiania	898fd25a5a	feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite Some checks failed Static Analysis / static (push) Has been cancelled Details Tests / test (push) Has been cancelled Details Phase 0 of the three-stakeholder analytics redesign: - 08_analytics_config.sql: ops.cost_rates + ops.kpi_targets with seed fuel rates (KES 195/L NBO+MBA, UGX 5200/L KLA) and 6 seed KPI targets (utilisation_pct, idle_pct global+osp-patrol, fuel_kes_per_100km, mttr_hours, alarms_per_100km). Granted SELECT to grafana_ro. Wired into run_migrations.py MIGRATIONS. - import_drivers_csv.py: full rewrite for the new Mitieng CSV (20260427_FSG_Vehicles_mitieng.csv). Snake_case columns, drops _infer_city() plate-prefix logic in favour of reading assigned_city directly. Adds cost_centre, assigned_route, vehicle_category, vehicle_brand, fuel_100km, depot_address. Treats the literal "NULL" string as missing. Reuses clean(), clean_num(), clean_ts(), get_conn(), get_logger() from ts_shared_rev. Special-cases numeric and timestamptz columns in the UPDATE clause. - audit_device_reconciliation.py: read-only audit comparing the CSV against tracksolid.devices. Reports per-account row counts, IMEIs on one side only, and devices on both sides whose metadata is still NULL. - 260427_device_reconciliation.md + 260427_audit_output.txt: Phase 0.2 reconciliation record. First run: DB has 172 devices, CSV has 162, delta +10 (10 IMEIs in DB-only, mostly fireside-account auto-syncs). Importer run with --only-null --apply filled 154 rows; coverage now assigned_city 152/172, cost_centre 150/172. Applied to stage on 2026-04-27 23:35 UTC. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-27 23:42:37 +03:00
David Kiania	257643cae2	fix: auto-register devices on push + allow CSV import to insert new rows Some checks failed Static Analysis / static (push) Waiting to run Details Tests / test (push) Waiting to run Details Static Analysis / static (pull_request) Has been cancelled Details Tests / test (pull_request) Has been cancelled Details Three changes that together close the FK-violation loop on /pushalarm: 1. import_drivers_csv.py: when an IMEI is in the CSV but not in tracksolid.devices, INSERT a new row instead of skipping. Unblocks the 140 X3/JC400P devices listed as a HIGH open item in CLAUDE.md §10. 2. webhook_receiver_rev.py: new _ensure_device() helper upserts a stub devices row (status='unknown') before inserting an alarm. Handles the third class of devices — not in API sync, not in CSV (e.g. the X3-63282 Kampala device flagged in CLAUDE.md §10). 3. CSV refreshed from Downloads (Apr 21 version, 140 active rows). Also fixes alarm error log previously showing "None" (read deviceImei instead of the integration push's imei field). CSV import already applied live on the instance (63 → 201 devices). Webhook patch requires a Coolify redeploy to pick up _ensure_device(). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-21 12:29:32 +03:00
David Kiania	cebcf74ba2	feat: business analytics expansion + driver CSV import - 01_BusinessAnalytics.md: add §0 usage tags, §2.4 cost-per-ticket, §3.6–3.8 alarm/drift/odometer, §4.4–4.5 dispatch log + SLA metrics, §9 fleet readiness scorecard, §10 service-interval forecaster, Appendix B threshold calibration guide (773 → 1437 lines) - 06_business_analytics_migration.sql: schema support for all new analytics sections — assigned_city column, dispatch_log table, ops schema, service_log, odometer_readings, tickets skeleton, vw_service_forecast view - import_drivers_csv.py: one-shot script to populate driver_name, vehicle_number, vehicle_models, cost_centre, assigned_city, sim, iccid, imsi from 20260414_FS__Logistics - final_fixed.csv (144 rows); dry-run by default, --apply to commit, --only-null for safe additive mode - 20260414_FS__Logistics - final_fixed.csv: source data committed for reproducibility and container exec workflow Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-18 08:30:34 +03:00

Author

SHA1

Message

Date

David Kiania

898fd25a5a

feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite

Static Analysis / static (push) Has been cancelled

Details

Tests / test (push) Has been cancelled

Details

Phase 0 of the three-stakeholder analytics redesign:

- 08_analytics_config.sql: ops.cost_rates + ops.kpi_targets with seed
  fuel rates (KES 195/L NBO+MBA, UGX 5200/L KLA) and 6 seed KPI
  targets (utilisation_pct, idle_pct global+osp-patrol,
  fuel_kes_per_100km, mttr_hours, alarms_per_100km). Granted SELECT to
  grafana_ro. Wired into run_migrations.py MIGRATIONS.

- import_drivers_csv.py: full rewrite for the new Mitieng CSV
  (20260427_FSG_Vehicles_mitieng.csv). Snake_case columns, drops
  _infer_city() plate-prefix logic in favour of reading assigned_city
  directly. Adds cost_centre, assigned_route, vehicle_category,
  vehicle_brand, fuel_100km, depot_address. Treats the literal "NULL"
  string as missing. Reuses clean(), clean_num(), clean_ts(),
  get_conn(), get_logger() from ts_shared_rev. Special-cases numeric
  and timestamptz columns in the UPDATE clause.

- audit_device_reconciliation.py: read-only audit comparing the CSV
  against tracksolid.devices. Reports per-account row counts, IMEIs
  on one side only, and devices on both sides whose metadata is still
  NULL.

- 260427_device_reconciliation.md + 260427_audit_output.txt: Phase 0.2
  reconciliation record. First run: DB has 172 devices, CSV has 162,
  delta +10 (10 IMEIs in DB-only, mostly fireside-account auto-syncs).
  Importer run with --only-null --apply filled 154 rows; coverage now
  assigned_city 152/172, cost_centre 150/172.

Applied to stage on 2026-04-27 23:35 UTC.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-04-27 23:42:37 +03:00

David Kiania

257643cae2

fix: auto-register devices on push + allow CSV import to insert new rows

Static Analysis / static (push) Waiting to run

Details

Tests / test (push) Waiting to run

Details

Static Analysis / static (pull_request) Has been cancelled

Details

Tests / test (pull_request) Has been cancelled

Details

Three changes that together close the FK-violation loop on /pushalarm:

1. import_drivers_csv.py: when an IMEI is in the CSV but not in
   tracksolid.devices, INSERT a new row instead of skipping. Unblocks
   the 140 X3/JC400P devices listed as a HIGH open item in CLAUDE.md §10.

2. webhook_receiver_rev.py: new _ensure_device() helper upserts a stub
   devices row (status='unknown') before inserting an alarm. Handles the
   third class of devices — not in API sync, not in CSV (e.g. the
   X3-63282 Kampala device flagged in CLAUDE.md §10).

3. CSV refreshed from Downloads (Apr 21 version, 140 active rows).

Also fixes alarm error log previously showing "None" (read deviceImei
instead of the integration push's imei field).

CSV import already applied live on the instance (63 → 201 devices).
Webhook patch requires a Coolify redeploy to pick up _ensure_device().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-04-21 12:29:32 +03:00

David Kiania

cebcf74ba2

feat: business analytics expansion + driver CSV import

- 01_BusinessAnalytics.md: add §0 usage tags, §2.4 cost-per-ticket,
  §3.6–3.8 alarm/drift/odometer, §4.4–4.5 dispatch log + SLA metrics,
  §9 fleet readiness scorecard, §10 service-interval forecaster,
  Appendix B threshold calibration guide (773 → 1437 lines)

- 06_business_analytics_migration.sql: schema support for all new
  analytics sections — assigned_city column, dispatch_log table,
  ops schema, service_log, odometer_readings, tickets skeleton,
  vw_service_forecast view

- import_drivers_csv.py: one-shot script to populate driver_name,
  vehicle_number, vehicle_models, cost_centre, assigned_city, sim,
  iccid, imsi from 20260414_FS__Logistics - final_fixed.csv (144 rows);
  dry-run by default, --apply to commit, --only-null for safe additive mode

- 20260414_FS__Logistics - final_fixed.csv: source data committed for
  reproducibility and container exec workflow

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-04-18 08:30:34 +03:00

3 commits