Commit graph

13 commits

Author SHA1 Message Date
david kiania
347c97ec4c feat(reporting): fleet segmentation + deduped vehicle roster (migration 14)
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Add reporting.fn_fleet_segment() and reporting.v_vehicles, splitting the fleet
into ticket-closing field_service vs specialist plant (crane/pick-up/motorbike)
that does not close immediate customer tickets.

The segment is DERIVED from tracksolid.devices.vehicle_models — itself an
authoritative Tracksolid API field (sync_devices maps jimi.user.device.list ->
vehicleModels) — so it stays API-current with no re-seeding; the manual
vehicle_category column is intentionally unused. v_vehicles collapses the
tracker+dashcam device pairs to one row per vehicle by reusing
reporting.normalize_plate() and the same primary-device precedence as
reporting.v_trips / v_live_positions (auto-merges 'KDS 453Y'/'KDS 453 Y',
resolves within-plate model conflicts via the primary tracker).

Verified live: 80 vehicles (61 field_service / 16 specialist / 3 unassigned),
grafana_ro granted. Includes the supporting data-quality report.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 13:54:47 +03:00
david kiania
8c5a43f3b8 chore(db): purge unused ops + dwh_gold schemas
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Drop the dormant ops (workshop / tickets / dispatch / SLA / odometer)
and dwh_gold (nightly ETL aggregates) schemas plus their dependents —
features never implemented, no live writer or scheduled refresh.

- Prod DB (already applied): DROP SCHEMA ops/dwh_gold CASCADE, plus
  tracksolid.dispatch_log, v_sla_inflight, v_utilisation_daily.
- migrations/12_drop_ops.sql + 13_drop_dwh_gold.sql (forward, all
  IF EXISTS) registered in run_migrations.py for rebuild durability.
- grafana: removed 8 now-broken panels (In-flight SLA, Idle Cost,
  Utilisation Heatmap, Row 7 Field-Service SLAs) from daily_operations;
  panel count 21 -> 13.
- docs: scrubbed CLAUDE.md, PLATFORM_OVERVIEW.html (-19KB), DATA_FLOW.md;
  pre-drop seed snapshot in docs/reports/260605_ops_purge_backup.md.

The separate tracksolid_dwh server (31.97.44.246:5888) is unrelated
and untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 18:11:03 +03:00
david kiania
e060933c55 docs: record fleet-dashboard filter fix + v_trips self-refresh
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Update PLATFORM_OVERVIEW.html (§2 migration, §4 read-API, §5 refresh_log,
§7 ops notes) and CLAUDE.md §7 fix history (FIX-D01, FIX-D02) to reflect
the two 2026-06-05 fixes that closed out the n8n→fleetapi cutover:
form-urlencoded POST body parsing, and moving the reporting.v_trips
matview refresh from the retired n8n job into dashboard_api.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 16:58:20 +03:00
david kiania
26fa1a4dc5 docs: add Grafana dashboards appendix + link PLATFORM_OVERVIEW from CLAUDE.md
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Adds section 6 (Grafana dashboards) to PLATFORM_OVERVIEW.html, generated from
the provisioned dashboard JSON: every panel in the NOC Fleet (9 panels) and
Daily Operations (23 panels) dashboards with type and source view/table.
Renumbers Operational notes to section 7. Links the doc from the CLAUDE.md
codebase map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:50:48 +03:00
david kiania
83a2d06148 docs: add PLATFORM_OVERVIEW.html — current-state platform reference
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Self-contained HTML reference generated from the live DB, documenting the
platform after the maps moved off n8n onto dashboard_api (fleetapi). Covers
architecture/data flow, the n8n→fleetapi migration, deployment topology,
the read-API endpoint reference, and the full database schema — every table
(with columns + row estimates), view, and function across tracksolid /
reporting / ops / dwh_gold / public — plus operational notes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:47:18 +03:00
david kiania
e5b0e192d8 chore(repo): reorganize tree into migrations/ data/ legacy/ docs/
Group root-level files (accreted from incremental changes) by purpose
without moving any deployment entrypoint or breaking imports:

- migrations/  : numbered SQL 02-10
- data/        : source CSVs
- legacy/      : superseded pre-_rev scripts + old pipeline notes (not deployed)
- docs/{manuals,reference,reports}/ : loose manuals, references, reports
- strip stray ** / *** prefixes from 5 doc filenames
- delete empty documents.txt / push_webhook.md

Reference updates so nothing breaks:
- run_migrations.py  -> /app/migrations/<file>
- run_migrations.sh  -> $SCRIPT_DIR/migrations
- import_drivers_csv.py -> data/<csv>
- docker-compose.yaml -> runbook path comment
- CLAUDE.md -> codebase map + inline doc references

Deployed Python (3 services + ts_shared_rev + run_migrations) and the
documented ops one-shots stay at root, preserving the flat-import layout
and all documented commands. Verified: py_compile clean across all modules,
every MIGRATIONS entry resolves under migrations/, CI-referenced paths
(tests/, mypy targets, db_audit) and the grafana build context intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 02:27:30 +03:00
David Kiania
34f5fa1b9c feat(dwh): bronze pipeline migrations, runbook, and execution manual
DWH pipeline (new):
  - dwh/261001_dwh_control.sql — watermarks + per-run audit log schema
  - dwh/261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
  - dwh/261003_dwh_roles.sql — dwh_owner / grafana_ro contract assertion
  - dwh/261004_dwh_observability_views.sql — v_table_freshness,
    v_recent_failures, v_watermark_lag (readable by grafana_ro)
  - docs/DWH_PIPELINE.md — operations runbook (setup, troubleshooting,
    manual re-run, back-fill, rotation)
  - DWH_Execution_Manual.md — reusable playbook for future data
    projects (extract → blob → load pattern, 7 design principles,
    snapshot-vs-incremental matrix, verification gates)
  - docs/superpowers/{specs,plans}/2026-04-24-n8n-dwh-bronze-pipeline-*
    — design spec + 27-task implementation plan

Security:
  - dwh/260423_dwh_ddl_v1.sql — redacted plaintext role passwords to
    'CHANGE_ME_BEFORE_APPLY' placeholders; added SECURITY header
    documenting generation + rotation flow

Docs:
  - CLAUDE.md — §3 adds tracksolid_dwh@31.97.44.246:5888 target,
    §4 adds dwh/ + docs/DWH_PIPELINE.md to codebase map, §5 adds
    bronze + dwh_control schema roll-up, §10 adds deploy task +
    password rotation follow-up

Also includes miscellaneous in-progress files accumulated on this
branch (workspace, analytics notes, vehicle CSVs, extract helpers,
renamed markdown archives).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 01:07:53 +03:00
David Kiania
85d02c81a5 feat: Daily Operations dashboard + tracksolid analytics views
Some checks failed
Static Analysis / static (push) Has been cancelled
Tests / test (push) Has been cancelled
Static Analysis / static (pull_request) Has been cancelled
Tests / test (pull_request) Has been cancelled
Add a second Grafana dashboard focused on daily operational KPIs and live
dispatch, keeping the NOC Live dashboard untouched.

- grafana/provisioning/dashboards-json/daily_operations_dashboard.json
  New dashboard covering §7 Blueprint Panels 3-8 and the §4 dispatch lens:
  freshness banner, today-at-a-glance stat row, active vehicles map,
  currently-idle table, vehicles-not-moved-today, per-vehicle daily KPI
  roll-up, driver behaviour leaderboard, distance trend, alarm frequency,
  idle cost MTD, utilisation heatmap, SLA row (collapsed, data-gated).

- 07_analytics_views.sql
  Nine views in tracksolid.* wrapping the BA-file [DASHBOARD]-tagged
  queries. Each view carries COMMENT ON VIEW with its spec section.
  SELECT granted to grafana_ro. Smoke-tested against live DB.

- run_migrations.py
  Register 06 and 07 in MIGRATIONS list with idempotent seed checks so
  future fresh deploys apply them correctly.

- CLAUDE.md
  Retire the tracksolid_2 schema references (schema no longer exists);
  §9 Fleet State dated 2026-04-19 with correct pipeline status (running,
  875 runs/24h, 0 failures) and accurate position_history row counts
  (hypertable stats don't show in pg_stat_user_tables).

- docs/superpowers/specs/2026-04-19-daily-operations-dashboard-design.md
  Design spec covering architecture, views, panel layout, deployment,
  rollback, and known data gaps.
2026-04-19 13:44:18 +03:00
David Kiania
18e7e668c0 docs: fleet intelligence pitch deck copy and one-pager (25 slides + A4 leave-behind)
Full slide-by-slide copy for elicitation pitch: 6 pain questions, feature
reveal, business case, optional add-ons (RustFS + DuckDB), and one-pager.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 17:05:07 +03:00
David Kiania
c6e4a227c8 docs: add blob storage and data warehouse as optional pitch products
RustFS (S3-compatible blob) and DuckDB (historical analytics) added as
optional add-on tiers with elicitation pain questions and tier model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 17:00:39 +03:00
David Kiania
9f4406d863 docs: fleet intelligence partner pitch design (elicitation method)
6-question pain-first pitch structure targeting regional GPS resellers,
with deck architecture, one-pager layout, and partnership model options.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 16:41:04 +03:00
David Kiania
75d3417a2b docs: add quality program design spec 2026-04-12 21:31:56 +03:00
David Kiania
f9834564ab Add CLAUDE.md and project docs for structured Claude project
CLAUDE.md: cached context file covering project identity, tech stack,
codebase map, schema quick-ref, API gotchas, fix history, working rules,
fleet state, and open items. Structured for maximum cache efficiency —
stable content first, dynamic state at the end.

docs/CONNECTIONS.md: connection parameter shapes (no secrets) for SSH,
DB, API, container resolution, Forgejo, Grafana, n8n.

docs/PROJECT_CONTEXT.md: client business context (telco field service,
3 cities, service types), data quality gaps, KPI framework by domain,
integration roadmap.

docs/KPI_FRAMEWORK.md: living KPI register with status tracking,
thresholds, client feedback log, and review checklist. To be co-developed
with client iteratively.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-12 20:59:15 +03:00