The ingest-health handler returned reporting.v_ingest_health rows straight to
JSONResponse, but last_run_at is a datetime — json.dumps raised TypeError and the
endpoint fell into its except, always returning {"overall":"unknown","endpoints":[]}.
Every other analytics endpoint already routes through jsonable_encoder; this one
didn't. Surfaced when the prod bridge finally got the /health/ingest route.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Parallel to the INC routes, three new endpoints back the FleetOps Tickets tab's CRQ
sub-tab: GET /webhook/crq-dashboard, /webhook/crq-search, /webhook/crq-filter-options
— thin passthroughs over reporting.fn_crq_* (fleettickets migration 16, over
tickets.crq). The INC routes are unchanged. Header route list updated.
Not yet redeployed to the staging/prod bridges (pending go-ahead).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add auto-detecting header-alias layer (_EXPORT_HEADER_ALIASES) so
import_drivers_csv ingests the raw Tracksolid Pro device export
(Title-Case headers: IMEI, Driver Name, LicensePlateNo., Model,
Vehicle Model, …) in addition to the original snake_case schema-mirror
CSV. Add customer_name/device_group to the write set, extend the
driver-placeholder skip list (Unassigned/UG/UG Crane), and exclude
activation_time/expiration/device_group (date-only/casing churn that
would degrade precise API-set values).
Used to apply the 260625 telematics quality-check file: 152 devices
updated (device_name, driver_name, plate, vehicle type, depot_address).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Passthrough over reporting.fn_inc_filter_options (fleettickets migration 14):
engineers, clusters, open ticket ids for the ticket-explorer dropdowns.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Thin passthrough over reporting.fn_inc_search (fleettickets migration 13):
ad-hoc ticket lookup by ticket_id / owner / cluster / status / state / time for
the FleetOps ticket explorer. Mirrors /webhook/inc-dashboard (read-only conn,
jsonb passthrough); validates state in {open,closed,all} and ISO-8601 from/to.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Thin passthrough over reporting.fn_inc_dashboard (fleettickets migration 09),
powering the FleetOps live INC map (open + windowed-closed + metrics). Mirrors
/webhook/tickets for auth/connection and returns the function's jsonb unchanged.
Params: cluster, status, window (today|week|month|custom), from/to (custom).
Validates window enum, custom-requires-from/to, ISO-8601 from/to, and from<to
(400 on failure). Matches docs/dashboard-api-contract.md from the fleettickets repo.
Verified end-to-end against the prod DB (read-only): default/week/cluster
filters return correct payloads; bad window and custom-without-bounds → 400.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_preset_to_range only special-cased today/30d/custom and fell through to 7d for
everything else, so the Fuel Log tab's default "90d" silently returned 7 days.
Parse any positive `Nd` preset into a today-(N-1)..today window. Backward-
compatible: today/30d/custom and the 7d default are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reads reporting.v_fuel_fills / v_fuel_efficiency (owned by the fleetfuel module,
which ingests the rustfs `fuel` bucket). Adds GET /analytics/fuel-fills (totals,
per-vehicle rows incl. km/L, by_department, daily trend, unmatched-plate status)
and /analytics/fuel-fills/recent, reusing _analytics_window + _dim_filters.
Extends /analytics/filters with departments + fuel_types, savepoint-guarded so a
not-yet-migrated v_fuel_fills can't break the existing trips dropdowns.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The `tickets` schema (INC/CRQ map), its migrations, and the ingest/geocode loader
moved to their own repo: repo.rahamafresh.com/kianiadee/fleettickets.git.
- remove migrations 21-23 and tools/import_tickets.py
- run_migrations.py: drop the 21-23 entries (fleettickets owns them now)
- dashboard_api keeps GET /webhook/tickets, calling reporting.fn_tickets_for_map
which fleettickets defines
- move geocoder env-var docs to fleettickets
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migration 11 was applied out of order on 2026-06-10 (it had never been recorded
applied on prod — the reporting objects were hand-created, then migrations 14/15/16
modified them). Re-running 11 recreated reporting.v_live_positions and
reporting.fn_live_positions at their BASE definitions; 15/16 were skipped as
already-applied, so the live map silently lost migration 15's cost-centre exclusion
(personal/management/mtn) and migration 16's vehicle_type/fleet_segment GeoJSON
fields — the live-map vehicle count jumped 74 -> 80.
Migration 20 re-asserts both objects' intended final definitions (verbatim union of
15 + 16). As the highest-numbered migration it always runs last, so the correct
state wins regardless of apply order. Already hot-fixed on prod by re-running 15+16;
this captures it durably. Idempotent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
These subsystems are retired and replaced by better alternatives (FleetNow /
FleetOps SPAs via dashboard_api; in-process pooling; reporting.v_ingest_health).
Remove them so the repo reflects the live stack only. Nothing running depends
on the deleted artifacts.
Deleted (dead artifacts):
- n8n-workflows/ (retired webhook exports), grafana/ (provisioning for the
removed service), dwh/ (migrations for the decommissioned external warehouse)
- runbooks: DWH_PIPELINE.md, DWH_Execution_Manual.md, grafanaDeployment.md,
grafanaOperationalManual.md
Code/config:
- run_migrations.py: drop sync_role_passwords() (its only entries were the now
-dead grafana_ro + pgbouncer syncs; the guard already made it inert)
- .env: remove the two unused GRAFANA_* vars
- ingest_movement_rev.py / db_audit / deploy_dashboard_api_staging.sh: reword
stale Grafana/grafana_ro comments
Docs: scrub n8n/Grafana/DWH from CLAUDE.md, CONNECTIONS, DATA_FLOW,
OPERATIONS_MANUAL, docker_commands, KPI_FRAMEWORK, PLATFORM_OVERVIEW,
STAGING_FLEETOPS, and deprecation-banner the two large SQL libraries
(dwh_gold was already dropped 2026-06-05).
Kept deliberately: the grafana_ro DB role (now an unused read-only login),
applied migration history, dated docs/reports/*, and docs/superpowers/* specs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collapse the backend from 7 Coolify services to 4 app services + the DB.
- Merge ingest_movement + ingest_events into a single ingest_worker:
split each poller's main() into reusable startup_catchup()/register_jobs()
and drive both from one schedule loop in new ingest_worker_rev.py
(standalone entrypoints retained for local debug).
- docker-compose.yaml: replace the two poller services with ingest_worker;
remove the pgbouncer service (dormant; transaction-mode pooling is unsafe
for the advisory-lock'd v_trips refresher) and the grafana service +
grafana-data volume (redundant with the FleetOps SPA).
- Add reporting.v_ingest_health (migration 19) + dashboard_api GET
/health/ingest as the pipeline-freshness surface that replaces Grafana's
health panels.
webhook_receiver stays isolated so a poller fault can't drop inbound pushes.
timescale_db and db_backup are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prod bridge now reads via dashboard_ro with the refresher on REFRESH_DATABASE_URL
(privileged). Both webhooks registered. Updated the as-built banner + §6 safety
note accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enables stage-2: the prod dashboard_api request pool connects as the READ-ONLY
dashboard_ro role (DATABASE_URL) while the v_trips refresher keeps a privileged
connection via REFRESH_DATABASE_URL (falls back to DATABASE_URL when unset, so
single-role/staging deploys are unchanged). Avoids the FIX-D02 trap (a read-only
role cannot REFRESH).
Adds deploy_dashboard_api.sh (the prod bridge deploy, now version-controlled):
strips inherited DATABASE_URL, sets REFRESH_DATABASE_URL=<app role> +
DATABASE_URL=dashboard_ro, CORS incl. fleetops.rahamafresh.com.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- STAGING_FLEETOPS_ARCHITECTURE.md: as-built status banner + phase table marked
delivered; remaining op follow-up = 2 Forgejo webhooks (FleetOps-prod,
FleetNow-staging)
- CLAUDE.md §3: document FleetOps (Caddy SPA) + /analytics/*, the fivetitude.com
staging umbrella, the 8891 staging bridge + dashboard_ro role, /env.js per-env
injection; refresh migration range to 02-18 + new infra files in the map
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updates STAGING_FLEETOPS_ARCHITECTURE.md to reflect the dedicated read-only
dashboard_ro role (replacing the grafana_ro reuse), the explicit v_trips matview
grant, DEFAULT PRIVILEGES, host-only password, and the two-stage plan (staging
now, live prod connection later). Notes migrations 17+18 applied; Phase 0
read-only role complete, webhook deploys still pending.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the grafana_ro reuse with a purpose-built least-privilege login role
that can serve the FULL dashboard_api read surface — so it backs the staging
instance now and can take over the live prod connection later (stage 2).
scripts/dashboard_ro_role.sql (run as postgres, password-free in repo):
- CREATE ROLE dashboard_ro LOGIN, read-only
- SELECT on reporting.* + tracksolid.*; explicit SELECT on the
reporting.v_trips MATERIALIZED VIEW (not covered by GRANT ON ALL TABLES)
- EXECUTE on reporting.fn_* map functions
- ALTER DEFAULT PRIVILEGES so future objects are auto-readable ("dynamic")
scripts/bootstrap_dashboard_ro.sh:
- generates the password into ~/.dashboard_ro.pw (0600), never printed
- applies the DDL via docker exec psql -U postgres -v ro_pw=...
deploy_dashboard_api_staging.sh: build DATABASE_URL from dashboard_ro +
~/.dashboard_ro.pw instead of grafana_ro.
Migrations 17/18 (already applied) are left intact. Not yet executed on host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
migrations/18_grant_reporting_ro.sql — grants USAGE + SELECT on the reporting.*
layer to grafana_ro, with DEFAULT PRIVILEGES so future reporting views are
auto-readable. grafana_ro is the read-only role the staging dashboard_api
connects as; it read tracksolid.* but never reporting.* (the prod dashboard_api
uses the app role), surfacing as "permission denied for view v_filter_drivers /
v_daily_summary" on the staging /analytics/* endpoints. Read-only only — no
write/REFRESH. Registered 18 in run_migrations.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
deploy_dashboard_api_staging.sh — standalone bridge twin of the prod
~/deploy_dashboard_api.sh for the fivetitude.com staging umbrella:
- container dashboard_api_staging on port 8891
- Traefik Host(fleetapi.fivetitude.com), router/service names suffixed -staging
- CORS = fleetnow.fivetitude.com, fleetops.fivetitude.com
- DATABASE_URL derived on-host as a READ-ONLY grafana_ro URL (never printed)
- VTRIPS_REFRESH_INTERVAL_S=0 so the read-only instance never REFRESHes
(prod owns the v_trips materialized-view refresh)
Reuses the webhook_receiver image + app network + WIP bind-mount, exactly like
prod. Not yet executed on the host — awaiting go-ahead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the read-only /analytics/* surface the FleetOps SPA will consume, plus
the migration that backs the fuel roll-up. All endpoints SELECT the indexed
reporting.* / tracksolid.v_* views and never write, so the forthcoming staging
instance can serve them against the prod DB as grafana_ro.
dashboard_api_rev.py:
- GET /analytics/fleet-summary per-vehicle + per-cost-centre roll-up
- GET /analytics/utilisation per-vehicle utilisation + daily fleet trend
- GET /analytics/driver-behaviour per-driver speeding / harsh index
- GET /analytics/fuel actual vs estimated litres (data-gated flags)
- GET /analytics/filters dropdown options (alias of GET /webhook/fleet-dashboard)
- responses run through jsonable_encoder (Decimal->float, date->ISO)
- VTRIPS_REFRESH_INTERVAL_S<=0 now DISABLES the v_trips refresher, so a
read-only staging instance never attempts REFRESH (prod still owns it).
migrations/17_fleetops_fuel_view.sql:
- reporting.v_fuel_daily encapsulates the v_trips->devices join (so the
read-only role needs SELECT only on the view) and grants it to grafana_ro.
Registered 17 in run_migrations.py. Note: live migration head is 16, not 13
as CLAUDE.md implies. Endpoints are unit-compilable but untested live until
the staging bridge (Phase 1) exists.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds docs/STAGING_FLEETOPS_ARCHITECTURE.md — the project doc for splitting
fleet tracking (FleetNow, frozen prod) from fleet operations (FleetOps, new
SPA) and standing up a staging environment under the fivetitude.com wildcard.
Covers: target topology + env matrix, the two dashboard_api instances
(prod 8890 / staging 8891, read-only role, refresher off), Forgejo->Coolify
webhook deploy + branch promotion model, FleetOps SPA on Caddy (Traefik still
terminates TLS), shared prod read-layer safety model, 6-phase rollout, and a
verification checklist.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
