kianiadee/tracksolid_timescale_grafana_prod
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
Tracksolid deployment with timescale & grafana with backup
74 commits 2 branches 0 tags 1 MiB
Python 65.3%
PLpgSQL 29.7%
Shell 4.3%
Dockerfile 0.7%
Find a file
David Kiania 898fd25a5a
Some checks failed
Static Analysis / static (push) Has been cancelled
Details
Tests / test (push) Has been cancelled
Details
feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 
Phase 0 of the three-stakeholder analytics redesign:

- 08_analytics_config.sql: ops.cost_rates + ops.kpi_targets with seed
  fuel rates (KES 195/L NBO+MBA, UGX 5200/L KLA) and 6 seed KPI
  targets (utilisation_pct, idle_pct global+osp-patrol,
  fuel_kes_per_100km, mttr_hours, alarms_per_100km). Granted SELECT to
  grafana_ro. Wired into run_migrations.py MIGRATIONS.

- import_drivers_csv.py: full rewrite for the new Mitieng CSV
  (20260427_FSG_Vehicles_mitieng.csv). Snake_case columns, drops
  _infer_city() plate-prefix logic in favour of reading assigned_city
  directly. Adds cost_centre, assigned_route, vehicle_category,
  vehicle_brand, fuel_100km, depot_address. Treats the literal "NULL"
  string as missing. Reuses clean(), clean_num(), clean_ts(),
  get_conn(), get_logger() from ts_shared_rev. Special-cases numeric
  and timestamptz columns in the UPDATE clause.

- audit_device_reconciliation.py: read-only audit comparing the CSV
  against tracksolid.devices. Reports per-account row counts, IMEIs
  on one side only, and devices on both sides whose metadata is still
  NULL.

- 260427_device_reconciliation.md + 260427_audit_output.txt: Phase 0.2
  reconciliation record. First run: DB has 172 devices, CSV has 162,
  delta +10 (10 IMEIs in DB-only, mostly fireside-account auto-syncs).
  Importer run with --only-null --apply filled 154 rows; coverage now
  assigned_city 152/172, cost_centre 150/172.

Applied to stage on 2026-04-27 23:35 UTC.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-27 23:42:37 +03:00
.claude feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
.forgejo/workflows feat: add db_audit health checks, runner, and scheduled Forgejo workflow 2026-04-12 21:40:29 +03:00
administration Add DB connection string to ops manual, add administration notes, remove stale deploy guide 2026-04-10 22:34:56 +03:00
backup feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
db_audit feat: add db_audit health checks, runner, and scheduled Forgejo workflow 2026-04-12 21:40:29 +03:00
docs feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
dwh feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
grafana fix(grafana): raise geomap maxZoom from 12 to 22 for full-resolution drill-in 2026-04-27 18:32:14 +03:00
n8n-workflows Add n8n workflow templates and change webhook port to 8888 2026-04-08 18:54:42 +03:00
tests perf+fix: SAVEPOINT-per-item pollers, batched GPS inserts, parallel detail fetch 2026-04-18 00:33:55 +03:00
***tracksolid_DB_manual.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**01_BusinessAnalytics.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**02_tracksolid_docker_commands.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**260410_baseline_report.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**OPERATIONS_MANUAL.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
.env fix: point DATABASE_URL at timescale_db container (not legacy 31.97.44.246:5888) 2026-04-18 15:43:49 +03:00
.gitignore Add webhook receiver, consolidate shared utilities, expand telemetry coverage 2026-04-08 16:31:17 +03:00
.python-version chore: align .python-version to 3.12.0 (matches Docker image and pyproject.toml) 2026-04-12 21:41:43 +03:00
02_tracksolid_full_schema_rev.sql Add webhook receiver, consolidate shared utilities, expand telemetry coverage 2026-04-08 16:31:17 +03:00
03_webhook_schema_migration.sql Add webhook receiver, consolidate shared utilities, expand telemetry coverage 2026-04-08 16:31:17 +03:00
04_bug_fix_migration.sql Fix alarm field mapping, distance unit bug, parking params; add schema migrations 2026-04-10 22:18:30 +03:00
05_enhancement_migration.sql Fix alarm field mapping, distance unit bug, parking params; add schema migrations 2026-04-10 22:18:30 +03:00
06_business_analytics_migration.sql feat: business analytics expansion + driver CSV import 2026-04-18 08:30:34 +03:00
07_analytics_views.sql feat: Daily Operations dashboard + tracksolid analytics views 2026-04-19 13:44:18 +03:00
08_analytics_config.sql feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
55_ts_coolify_gemini_prod.code-workspace feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
260412_baseline_report.md Add 260412 baseline report — first trip data, FIX-M16 confirmed 2026-04-12 00:14:27 +03:00
260427_audit_output.txt feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
260427_device_reconciliation.md feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
20260414_FS__Logistics - final_fixed.csv fix: auto-register devices on push + allow CSV import to insert new rows 2026-04-21 12:29:32 +03:00
20260427_FSG_Vehicles_mitieng.csv feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
audit_device_reconciliation.py feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
CLAUDE.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
connecting_python_tracksolid.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
docker-compose.yaml feat(backup): timestamp and schedule in Africa/Nairobi local time 2026-04-24 11:30:20 +03:00
Dockerfile Fix migration failures: switch to full TimescaleDB + use psql runner 2026-04-08 17:17:58 +03:00
documents.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
DWH_Execution_Manual.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
fireside_logistics_cleaned_v2.csv feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
grafanaDeployment.md Add Grafana NOC fleet dashboard with provisioning 2026-04-09 00:01:52 +03:00
grafanaOperationalManual.md Add Grafana NOC operational manual 2026-04-09 00:12:48 +03:00
import_drivers_csv.py feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
ingest_events_rev.py perf+fix: SAVEPOINT-per-item pollers, batched GPS inserts, parallel detail fetch 2026-04-18 00:33:55 +03:00
ingest_movement_rev.py feat: [FIX-M19] multi-account ingest across fireside sub-accounts 2026-04-24 10:43:07 +03:00
new_feature.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
push_webhook.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
pyproject.toml ci: add ruff + mypy static analysis config and Forgejo workflow 2026-04-12 21:32:33 +03:00
README.md first commit 2026-04-07 20:41:16 +03:00
run_migrations.py feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
run_migrations.sh Add idempotent migration runner script 2026-04-10 23:31:57 +03:00
sync_driver_audit.py perf+fix: SAVEPOINT-per-item pollers, batched GPS inserts, parallel detail fetch 2026-04-18 00:33:55 +03:00
tracksolid_analytics_pipeline.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolid_extract.py feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolid_ingestion_pipeline.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolid_update_v2.py fix(api): map new Mitieng CSV columns in tracksolid_update_v2 2026-04-27 23:42:20 +03:00
tracksolid_vehicle_update.py feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolidApiDocumentation.md Update tracksolidApiDocumentation.md with live implementation findings 2026-04-11 07:52:28 +03:00
ts_shared_rev.py feat: [FIX-M19] multi-account ingest across fireside sub-accounts 2026-04-24 10:43:07 +03:00
webhook_receiver_rev.py fix: auto-register devices on push + allow CSV import to insert new rows 2026-04-21 12:29:32 +03:00

README.md