Commit graph

19 commits

Author SHA1 Message Date
david kiania
54a714e0cd docs: finalize staging+FleetOps as-built; phase closed
Some checks failed
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Static Analysis / static (pull_request) Has been cancelled
Tests / test (pull_request) Has been cancelled
- STAGING_FLEETOPS_ARCHITECTURE.md: as-built status banner + phase table marked
  delivered; remaining op follow-up = 2 Forgejo webhooks (FleetOps-prod,
  FleetNow-staging)
- CLAUDE.md §3: document FleetOps (Caddy SPA) + /analytics/*, the fivetitude.com
  staging umbrella, the 8891 staging bridge + dashboard_ro role, /env.js per-env
  injection; refresh migration range to 02-18 + new infra files in the map

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 20:10:14 +03:00
david kiania
2603f0e726 docs: dashboard_ro role + two-stage rollout; mark Phase 1 done
Some checks failed
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Static Analysis / static (pull_request) Has been cancelled
Tests / test (pull_request) Has been cancelled
Updates STAGING_FLEETOPS_ARCHITECTURE.md to reflect the dedicated read-only
dashboard_ro role (replacing the grafana_ro reuse), the explicit v_trips matview
grant, DEFAULT PRIVILEGES, host-only password, and the two-stage plan (staging
now, live prod connection later). Notes migrations 17+18 applied; Phase 0
read-only role complete, webhook deploys still pending.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 12:37:09 +03:00
david kiania
6cf0905b31 feat(dashboard_api): FleetOps analytics endpoints + fuel view (Phase 3)
Adds the read-only /analytics/* surface the FleetOps SPA will consume, plus
the migration that backs the fuel roll-up. All endpoints SELECT the indexed
reporting.* / tracksolid.v_* views and never write, so the forthcoming staging
instance can serve them against the prod DB as grafana_ro.

dashboard_api_rev.py:
  - GET /analytics/fleet-summary     per-vehicle + per-cost-centre roll-up
  - GET /analytics/utilisation       per-vehicle utilisation + daily fleet trend
  - GET /analytics/driver-behaviour  per-driver speeding / harsh index
  - GET /analytics/fuel              actual vs estimated litres (data-gated flags)
  - GET /analytics/filters           dropdown options (alias of GET /webhook/fleet-dashboard)
  - responses run through jsonable_encoder (Decimal->float, date->ISO)
  - VTRIPS_REFRESH_INTERVAL_S<=0 now DISABLES the v_trips refresher, so a
    read-only staging instance never attempts REFRESH (prod still owns it).

migrations/17_fleetops_fuel_view.sql:
  - reporting.v_fuel_daily encapsulates the v_trips->devices join (so the
    read-only role needs SELECT only on the view) and grants it to grafana_ro.

Registered 17 in run_migrations.py. Note: live migration head is 16, not 13
as CLAUDE.md implies. Endpoints are unit-compilable but untested live until
the staging bridge (Phase 1) exists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 12:12:00 +03:00
david kiania
e01b50a8b1 docs: staging environment + FleetOps split architecture
Adds docs/STAGING_FLEETOPS_ARCHITECTURE.md — the project doc for splitting
fleet tracking (FleetNow, frozen prod) from fleet operations (FleetOps, new
SPA) and standing up a staging environment under the fivetitude.com wildcard.

Covers: target topology + env matrix, the two dashboard_api instances
(prod 8890 / staging 8891, read-only role, refresher off), Forgejo->Coolify
webhook deploy + branch promotion model, FleetOps SPA on Caddy (Traefik still
terminates TLS), shared prod read-layer safety model, 6-phase rollout, and a
verification checklist.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 12:05:29 +03:00
david kiania
3015104f5b docs+tooling: OSM POI export workflow + Shell stations data; graded DQ report
Some checks failed
Static Analysis / static (push) Has been cancelled
Tests / test (push) Has been cancelled
- scripts/export_osm_pois.py: reproducible OSM .pbf -> GeoJSON+CSV exporter
  (amenity/brand filter; pyosmium via uv, no system deps).
- docs/OSM_POI_EXPORT.md: runbook (extract -> export -> FleetNow layer) with
  reference counts (1,794 fuel stations; Shell=232).
- shell_stations.geojson/.csv: the Shell export of record (232 pts, kenya-260605).
- docs/reports/260608_fleet_registry_data_quality.*: rewritten as a graded
  (Red/Amber/Yellow) action plan with owners.
- .gitignore: ignore *.osm.pbf (331MB, reproducible). CLAUDE.md: index the new docs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 22:04:49 +03:00
david kiania
45d764a5f1 docs: add FleetNow stakeholder briefings (directors + heads of department)
Some checks failed
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Static Analysis / static (pull_request) Has been cancelled
Tests / test (pull_request) Has been cancelled
Two audience-tailored one-pagers (md + matching PDF) making the case for the
merged live+historical platform, plus a shared demo/talking-points script.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 15:42:37 +03:00
david kiania
347c97ec4c feat(reporting): fleet segmentation + deduped vehicle roster (migration 14)
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Add reporting.fn_fleet_segment() and reporting.v_vehicles, splitting the fleet
into ticket-closing field_service vs specialist plant (crane/pick-up/motorbike)
that does not close immediate customer tickets.

The segment is DERIVED from tracksolid.devices.vehicle_models — itself an
authoritative Tracksolid API field (sync_devices maps jimi.user.device.list ->
vehicleModels) — so it stays API-current with no re-seeding; the manual
vehicle_category column is intentionally unused. v_vehicles collapses the
tracker+dashcam device pairs to one row per vehicle by reusing
reporting.normalize_plate() and the same primary-device precedence as
reporting.v_trips / v_live_positions (auto-merges 'KDS 453Y'/'KDS 453 Y',
resolves within-plate model conflicts via the primary tracker).

Verified live: 80 vehicles (61 field_service / 16 specialist / 3 unassigned),
grafana_ro granted. Includes the supporting data-quality report.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 13:54:47 +03:00
david kiania
8c5a43f3b8 chore(db): purge unused ops + dwh_gold schemas
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Drop the dormant ops (workshop / tickets / dispatch / SLA / odometer)
and dwh_gold (nightly ETL aggregates) schemas plus their dependents —
features never implemented, no live writer or scheduled refresh.

- Prod DB (already applied): DROP SCHEMA ops/dwh_gold CASCADE, plus
  tracksolid.dispatch_log, v_sla_inflight, v_utilisation_daily.
- migrations/12_drop_ops.sql + 13_drop_dwh_gold.sql (forward, all
  IF EXISTS) registered in run_migrations.py for rebuild durability.
- grafana: removed 8 now-broken panels (In-flight SLA, Idle Cost,
  Utilisation Heatmap, Row 7 Field-Service SLAs) from daily_operations;
  panel count 21 -> 13.
- docs: scrubbed CLAUDE.md, PLATFORM_OVERVIEW.html (-19KB), DATA_FLOW.md;
  pre-drop seed snapshot in docs/reports/260605_ops_purge_backup.md.

The separate tracksolid_dwh server (31.97.44.246:5888) is unrelated
and untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 18:11:03 +03:00
david kiania
e060933c55 docs: record fleet-dashboard filter fix + v_trips self-refresh
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Update PLATFORM_OVERVIEW.html (§2 migration, §4 read-API, §5 refresh_log,
§7 ops notes) and CLAUDE.md §7 fix history (FIX-D01, FIX-D02) to reflect
the two 2026-06-05 fixes that closed out the n8n→fleetapi cutover:
form-urlencoded POST body parsing, and moving the reporting.v_trips
matview refresh from the retired n8n job into dashboard_api.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 16:58:20 +03:00
david kiania
26fa1a4dc5 docs: add Grafana dashboards appendix + link PLATFORM_OVERVIEW from CLAUDE.md
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Adds section 6 (Grafana dashboards) to PLATFORM_OVERVIEW.html, generated from
the provisioned dashboard JSON: every panel in the NOC Fleet (9 panels) and
Daily Operations (23 panels) dashboards with type and source view/table.
Renumbers Operational notes to section 7. Links the doc from the CLAUDE.md
codebase map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:50:48 +03:00
david kiania
83a2d06148 docs: add PLATFORM_OVERVIEW.html — current-state platform reference
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Self-contained HTML reference generated from the live DB, documenting the
platform after the maps moved off n8n onto dashboard_api (fleetapi). Covers
architecture/data flow, the n8n→fleetapi migration, deployment topology,
the read-API endpoint reference, and the full database schema — every table
(with columns + row estimates), view, and function across tracksolid /
reporting / ops / dwh_gold / public — plus operational notes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:47:18 +03:00
david kiania
e5b0e192d8 chore(repo): reorganize tree into migrations/ data/ legacy/ docs/
Group root-level files (accreted from incremental changes) by purpose
without moving any deployment entrypoint or breaking imports:

- migrations/  : numbered SQL 02-10
- data/        : source CSVs
- legacy/      : superseded pre-_rev scripts + old pipeline notes (not deployed)
- docs/{manuals,reference,reports}/ : loose manuals, references, reports
- strip stray ** / *** prefixes from 5 doc filenames
- delete empty documents.txt / push_webhook.md

Reference updates so nothing breaks:
- run_migrations.py  -> /app/migrations/<file>
- run_migrations.sh  -> $SCRIPT_DIR/migrations
- import_drivers_csv.py -> data/<csv>
- docker-compose.yaml -> runbook path comment
- CLAUDE.md -> codebase map + inline doc references

Deployed Python (3 services + ts_shared_rev + run_migrations) and the
documented ops one-shots stay at root, preserving the flat-import layout
and all documented commands. Verified: py_compile clean across all modules,
every MIGRATIONS entry resolves under migrations/, CI-referenced paths
(tests/, mypy targets, db_audit) and the grafana build context intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 02:27:30 +03:00
David Kiania
34f5fa1b9c feat(dwh): bronze pipeline migrations, runbook, and execution manual
DWH pipeline (new):
  - dwh/261001_dwh_control.sql — watermarks + per-run audit log schema
  - dwh/261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
  - dwh/261003_dwh_roles.sql — dwh_owner / grafana_ro contract assertion
  - dwh/261004_dwh_observability_views.sql — v_table_freshness,
    v_recent_failures, v_watermark_lag (readable by grafana_ro)
  - docs/DWH_PIPELINE.md — operations runbook (setup, troubleshooting,
    manual re-run, back-fill, rotation)
  - DWH_Execution_Manual.md — reusable playbook for future data
    projects (extract → blob → load pattern, 7 design principles,
    snapshot-vs-incremental matrix, verification gates)
  - docs/superpowers/{specs,plans}/2026-04-24-n8n-dwh-bronze-pipeline-*
    — design spec + 27-task implementation plan

Security:
  - dwh/260423_dwh_ddl_v1.sql — redacted plaintext role passwords to
    'CHANGE_ME_BEFORE_APPLY' placeholders; added SECURITY header
    documenting generation + rotation flow

Docs:
  - CLAUDE.md — §3 adds tracksolid_dwh@31.97.44.246:5888 target,
    §4 adds dwh/ + docs/DWH_PIPELINE.md to codebase map, §5 adds
    bronze + dwh_control schema roll-up, §10 adds deploy task +
    password rotation follow-up

Also includes miscellaneous in-progress files accumulated on this
branch (workspace, analytics notes, vehicle CSVs, extract helpers,
renamed markdown archives).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 01:07:53 +03:00
David Kiania
85d02c81a5 feat: Daily Operations dashboard + tracksolid analytics views
Some checks failed
Static Analysis / static (push) Has been cancelled
Tests / test (push) Has been cancelled
Static Analysis / static (pull_request) Has been cancelled
Tests / test (pull_request) Has been cancelled
Add a second Grafana dashboard focused on daily operational KPIs and live
dispatch, keeping the NOC Live dashboard untouched.

- grafana/provisioning/dashboards-json/daily_operations_dashboard.json
  New dashboard covering §7 Blueprint Panels 3-8 and the §4 dispatch lens:
  freshness banner, today-at-a-glance stat row, active vehicles map,
  currently-idle table, vehicles-not-moved-today, per-vehicle daily KPI
  roll-up, driver behaviour leaderboard, distance trend, alarm frequency,
  idle cost MTD, utilisation heatmap, SLA row (collapsed, data-gated).

- 07_analytics_views.sql
  Nine views in tracksolid.* wrapping the BA-file [DASHBOARD]-tagged
  queries. Each view carries COMMENT ON VIEW with its spec section.
  SELECT granted to grafana_ro. Smoke-tested against live DB.

- run_migrations.py
  Register 06 and 07 in MIGRATIONS list with idempotent seed checks so
  future fresh deploys apply them correctly.

- CLAUDE.md
  Retire the tracksolid_2 schema references (schema no longer exists);
  §9 Fleet State dated 2026-04-19 with correct pipeline status (running,
  875 runs/24h, 0 failures) and accurate position_history row counts
  (hypertable stats don't show in pg_stat_user_tables).

- docs/superpowers/specs/2026-04-19-daily-operations-dashboard-design.md
  Design spec covering architecture, views, panel layout, deployment,
  rollback, and known data gaps.
2026-04-19 13:44:18 +03:00
David Kiania
18e7e668c0 docs: fleet intelligence pitch deck copy and one-pager (25 slides + A4 leave-behind)
Full slide-by-slide copy for elicitation pitch: 6 pain questions, feature
reveal, business case, optional add-ons (RustFS + DuckDB), and one-pager.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 17:05:07 +03:00
David Kiania
c6e4a227c8 docs: add blob storage and data warehouse as optional pitch products
RustFS (S3-compatible blob) and DuckDB (historical analytics) added as
optional add-on tiers with elicitation pain questions and tier model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 17:00:39 +03:00
David Kiania
9f4406d863 docs: fleet intelligence partner pitch design (elicitation method)
6-question pain-first pitch structure targeting regional GPS resellers,
with deck architecture, one-pager layout, and partnership model options.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 16:41:04 +03:00
David Kiania
75d3417a2b docs: add quality program design spec 2026-04-12 21:31:56 +03:00
David Kiania
f9834564ab Add CLAUDE.md and project docs for structured Claude project
CLAUDE.md: cached context file covering project identity, tech stack,
codebase map, schema quick-ref, API gotchas, fix history, working rules,
fleet state, and open items. Structured for maximum cache efficiency —
stable content first, dynamic state at the end.

docs/CONNECTIONS.md: connection parameter shapes (no secrets) for SSH,
DB, API, container resolution, Forgejo, Grafana, n8n.

docs/PROJECT_CONTEXT.md: client business context (telco field service,
3 cities, service types), data quality gaps, KPI framework by domain,
integration roadmap.

docs/KPI_FRAMEWORK.md: living KPI register with status tracking,
thresholds, client feedback log, and review checklist. To be co-developed
with client iteratively.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 20:59:15 +03:00