kianiadee/tracksolid_timescale_grafana_prod
1
0
Fork 0
Code Issues Pull requests 4 Projects Releases Packages Wiki Activity Actions 7

Staging environment + FleetOps split #17

Open
kianiadee wants to merge 23 commits from feat/staging-fleetops-architecture into main
pull from: feat/staging-fleetops-architecture
merge into: kianiadee:main
Conversation 0 Commits 23 Files changed 66 +46 -19
2 changed files with 46 additions and 19 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit 54a714e0cd - Show all commits

37
CLAUDE.md
View file

@ -68,22 +68,35 @@ See `docs/CONNECTIONS.md` for the full shape. Summary:
### Map dashboards & read-API
The map UIs read the **`dashboard_api`** service (FastAPI, `dashboard_api_rev.py`) at
The UIs read the **`dashboard_api`** service (FastAPI, `dashboard_api_rev.py`) at
`https://fleetapi.rahamafresh.com` — the stable replacement for the retired n8n webhooks. It serves
GeoJSON from the `reporting.*` functions (`fn_live_positions`, `fn_vehicle_track`, `fn_trips_for_map`)
+ filter options. **`dashboard_api` is a STANDALONE Traefik-labelled bridge container, NOT Coolify-managed** —
it bind-mounts the host file `~/dashboard_api/dashboard_api_rev.py` and is (re)deployed by
`~/deploy_dashboard_api.sh` on the host (an env/CORS change needs a *recreate*, not a restart). Three
single-page apps consume it:
+ filter options, **plus the `/analytics/*` read endpoints** (fleet-summary, utilisation,
driver-behaviour, fuel, filters) that power FleetOps. **`dashboard_api` is a STANDALONE
Traefik-labelled bridge container, NOT Coolify-managed** — it bind-mounts the host file
`~/dashboard_api/dashboard_api_rev.py` and is (re)deployed by `~/deploy_dashboard_api.sh` on the host
(an env/CORS change needs a *recreate*, not a restart). The SPAs that consume it:
| Dashboard | What | Hosting |
|---|---|---|
| `liveposition.rahamafresh.com` | live positions only | `index.html` in rustfs bucket `liveposition` behind an nginx proxy |
| `fleetintelligence.rahamafresh.com` | historical trips only | `index.html` in rustfs bucket `fleetintelligence` behind an nginx proxy |
| `fleetnow.rahamafresh.com` | **merged** live + trips (current best UI) | **own repo** `repo.rahamafresh.com/kianiadee/fleetnow.git`, deployed via **Coolify (Dockerfile → nginx)** |
| `fleetnow.rahamafresh.com` | **merged** live + trips map (fleet *tracking*) | **own repo** `repo.rahamafresh.com/kianiadee/fleetnow.git`, **Coolify (Dockerfile → nginx)** |
| `fleetops.rahamafresh.com` | fuel / analytics / KPIs (fleet *operations*) | **own repo** `repo.rahamafresh.com/kianiadee/fleetops.git` (local `~/Downloads/projects/15_fleetops`), **Coolify (Dockerfile → Caddy)** |
All three origins must be in the API's `DASHBOARD_CORS_ORIGINS` (see FIX-D03). **FleetNow is the
single source of truth for the merged map and lives in its own repo — edit it there, not here.**
All prod origins must be in the API's `DASHBOARD_CORS_ORIGINS` (see FIX-D03). **FleetNow and FleetOps
each live in their own repos — edit them there, not here.**
**Staging (the `fivetitude.com` wildcard umbrella).** A parallel staging stack mirrors the above so the
frozen prod apps are never edited directly: `fleetnow.fivetitude.com`, `fleetops.fivetitude.com`, and a
**second `dashboard_api` bridge** at `fleetapi.fivetitude.com` (port **8891**,
`deploy_dashboard_api_staging.sh` in this repo). The staging bridge reads the **same prod DB** as the
dedicated **read-only `dashboard_ro`** role (`scripts/dashboard_ro_role.sql` + `bootstrap_dashboard_ro.sh`),
with the `v_trips` refresher disabled (`VTRIPS_REFRESH_INTERVAL_S=0`) — prod owns the refresh. Each SPA's
**API base is injected per-environment at container start** (FleetOps via Caddy `templates` → `/env.js`;
FleetNow via an nginx `envsubst` entrypoint → `/env.js`), falling back to the prod API if unset. Deploys
are **Forgejo → Coolify webhooks**. Full topology + runbooks: **`docs/STAGING_FLEETOPS_ARCHITECTURE.md`**
and the fleetops repo's `docs/webhook-auto-deploy.html`.
---
@ -114,11 +127,15 @@ dwh/ # DWH migrations for tracksolid_dwh@31.97.44.246:588
# 261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
# 261003_dwh_roles.sql — role contract assertion
# 261004_dwh_observability_views.sql — freshness/failure views
migrations/ # Numbered SQL migrations 02–13, applied in order by run_migrations.py
migrations/ # Numbered SQL migrations 02–18, applied in order by run_migrations.py
# 02 full schema · 03 webhook · 04 distance fix · 05 enhancements
# 06 ops/analytics · 07 views · 08 config · 09 trips enrichment
# 10_driver_clock_views.sql · 10_pgbouncer_auth.sql · 11 reporting
# 12 drop ops schema · 13 drop dwh_gold schema (both 2026-06-05)
# 14 fleet segment · 15 map exclude cc · 16 live feed vehicle_type
# 17 reporting.v_fuel_daily (FleetOps) · 18 grant reporting.* to grafana_ro
deploy_dashboard_api_staging.sh # Staging dashboard_api bridge (8891, fleetapi.fivetitude.com); see STAGING_FLEETOPS_ARCHITECTURE.md
scripts/dashboard_ro_role.sql + bootstrap_dashboard_ro.sh # Dedicated read-only DB role for the staging bridge
Dockerfile # Custom image for ingest/webhook containers
pyproject.toml # Python project + uv dependency spec
backup/ # pg_dump sidecar scripts and config
@ -148,7 +165,7 @@ tracksolid.alarms -- Alarm events (alarm_type, alarm_name, alarm_time
tracksolid.obd_readings -- OBD diagnostics (push only, awaiting webhook registration)
tracksolid.device_events -- Power on/off tamper events (push only)
tracksolid.ingestion_log -- API call audit trail — 875 runs / 24h, 0 failures at last check (2026-04-19)
tracksolid.schema_migrations -- Applied migrations 02–13
tracksolid.schema_migrations -- Applied migrations 02–18 (17 reporting.v_fuel_daily, 18 grants applied 2026-06-10)
-- PURGED 2026-06-05 (migrations 12 + 13): the dormant `ops` schema (tickets, service_log,
-- odometer_readings, cost_rates, kpi_targets, vw_service_forecast), tracksolid.dispatch_log,
-- and the `dwh_gold` schema (dim_vehicles, fact_daily_fleet_metrics, refresh_daily_metrics).

28
docs/STAGING_FLEETOPS_ARCHITECTURE.md
View file

@ -1,7 +1,17 @@
# Staging Environment & FleetOps Split — Architecture
**Status:** approved 2026-06-10 · **Owner:** kianiadee · **Audience:** both developers (mixed
technical/ops background — readable without prior context).
**Status:** ✅ **delivered 2026-06-10** (phase closed) · **Owner:** kianiadee · **Audience:** both
developers (mixed technical/ops background — readable without prior context).
> **As-built (2026-06-10).** All six surfaces are live: staging `fleetapi` / `fleetops` /
> `fleetnow` `.fivetitude.com`, and prod `fleetops.rahamafresh.com` + `/analytics/*` on
> `fleetapi.rahamafresh.com`. The dedicated read-only `dashboard_ro` role backs the staging
> bridge; the prod bridge still uses the app role (stage-2 migration to `dashboard_ro` is ready
> but deferred). Migrations 17 + 18 applied. The client's live FleetNow map was unaffected
> throughout. **Remaining operational follow-ups** (no code): register the two Forgejo→Coolify
> webhooks for the *FleetOps-prod* and *FleetNow-staging* apps (FleetOps-staging is done) per
> [`webhook-auto-deploy.html`](../../15_fleetops/docs/webhook-auto-deploy.html) in the fleetops
> repo; review/merge tracksolid PR #17.
This document describes how we (a) introduce a **staging environment** under the
`fivetitude.com` umbrella so the production FleetNow map is never edited directly, and (b)
@ -187,14 +197,14 @@ not by a separate DB:
Ordered by dependency and risk — prove the foundation and the deploy pipeline first; touch the
client's production domains **last**.
| Phase | Scope | Exit criterion |
| Phase | Scope | Status (as-built) |
|---|---|---|
| **0 — Foundation** | This document; provision the read-only `dashboard_ro` role (**done**); migrate all Coolify apps to Forgejo webhook deploys (**pending**) | `dashboard_ro` can `SELECT` `reporting.*` + `tracksolid.*` + `v_trips` and nothing else; every Coolify app redeploys via webhook |
| **1 — Staging backbone** | Staging `dashboard_api` bridge (`deploy_dashboard_api_staging.sh`, 8891, `fleetapi.fivetitude.com`, read-only, refresher off, staging CORS) | `curl https://fleetapi.fivetitude.com/health` ok; verifiably read-only; no staging rows in `reporting.refresh_log` |
| **2 — FleetNow staging** | FleetNow repo: `staging` branch + parameterized API base + `fleetnow.fivetitude.com` Coolify app | Renders against staging API; `staging` push deploys staging only, `main` merge deploys prod only; prod FleetNow untouched |
| **3 — FleetOps backend** | `/analytics/*` endpoints in `dashboard_api_rev.py` + `migrations/17_fleetops_fuel_view.sql`; refresher made disable-able (`VTRIPS_REFRESH_INTERVAL_S<=0`); tested on the staging API | Every route returns correct shape on `fleetapi.fivetitude.com`; fuel route returns "needs data" flags |
| **4 — FleetOps SPA** | Scaffold `15_fleetops` (git init + remote + SPA/Dockerfile); `fleetops.fivetitude.com` Coolify app | Renders fuel/analytics/utilisation/driver panels from staging endpoints; CORS clean |
| **5 — Production cutover** | Promote API to prod + prod CORS add; `fleetops.rahamafresh.com` Coolify app; prod DNS record; update `CLAUDE.md` / `CONNECTIONS.md` / `PLATFORM_OVERVIEW.html` | FleetOps live on prod; prod FleetNow/API otherwise unchanged; docs current |
| **0 — Foundation** | This document; provision the read-only `dashboard_ro` role; migrate Coolify apps to Forgejo webhook deploys | ✅ doc + `dashboard_ro` role done; webhook mechanism proven (FleetOps-staging live) — **FleetOps-prod + FleetNow-staging webhook registration is the remaining op follow-up** |
| **1 — Staging backbone** | Staging `dashboard_api` bridge (`deploy_dashboard_api_staging.sh`, 8891, `fleetapi.fivetitude.com`, `dashboard_ro`, refresher off, staging CORS) | ✅ live; verified read-only; no staging rows in `reporting.refresh_log` |
| **2 — FleetNow staging** | FleetNow repo: `staging` branch + `/env.js` nginx `envsubst` API-base injection + `fleetnow.fivetitude.com` Coolify app | ✅ live against staging API; prod FleetNow (`main`) untouched |
| **3 — FleetOps backend** | `/analytics/*` endpoints in `dashboard_api_rev.py` + `migrations/17` (+ `18` grants); refresher disable-able (`VTRIPS_REFRESH_INTERVAL_S<=0`) | ✅ all routes verified on staging then promoted to prod; fuel route data-gated |
| **4 — FleetOps SPA** | `fleetops.git` (`~/Downloads/projects/15_fleetops`), Caddy SPA + `/env.js` injection; `fleetops.fivetitude.com` Coolify app | ✅ live; auto-deploy verified push→live |
| **5 — Production cutover** | Promote API to prod + prod CORS add; `fleetops.rahamafresh.com` Coolify app (`main`); docs | ✅ FleetOps live on prod with real data; prod FleetNow/API otherwise unchanged (one intentional CORS add); `rahamafresh.com` confirmed wildcard (no DNS record needed) |
---