fleetfuel/README.md

fleetfuel

Fleet fuel-spend ingestion and read-schema that powers the Fuel Log tab in FleetOps. Sibling of fleettickets (same self-contained module shape).

The feed is WhatsApp fuel-update messages, extracted by an n8n CDC job from the client's logistics_department.fuel_records table and dropped in the rustfs fuel bucket. Each record is an actual fill: litres, KES amount, odometer, fuel type, driver, department, keyed by number plate (car).

What this owns

Piece	What
migrations/*.sql	The fuel schema: fuel.records (raw-jsonb-first + trigger-derived columns), fuel.norm_plate / fuel.canon_fuel_type / fuel.canon_department normalizers, fuel.ingest_state (CDC watermark), and reporting.v_fuel_fills / reporting.v_fuel_efficiency (the read views). 01 base schema · 02 one-device-per-fill join fix · 03 standardize timestamps to Africa/Nairobi (EAT)
import_fuel.py	Pulls fuel records from the rustfs fuel bucket and upserts them on id
s3util.py	rustfs (S3-compatible) client factory — path-style, custom endpoint
run_migrations.py	Applies migrations/*.sql in order (ledger: fuel.schema_migrations)
shared.py	Minimal DB/logging helpers (self-contained — no tracksolid dependency)

What this does NOT own (stays where it is)

• The DB — the fuel schema lives in the shared tracksolid_db.
• The read-API — dashboard_api (in the tracksolid stack) serves GET /analytics/fuel-fills, which reads reporting.v_fuel_fills (defined here).
• The frontend — the Fuel Log map/panel is a tab in the FleetOps SPA (fleetops repo).

Data model (raw-first)

Each row is id (the source PK) + raw (the full source record as jsonb) + derived, normalized columns the DB trigger fills from raw. The WhatsApp feed is messy (KCA 542Q vs KCA542Q, fuel-type typos like DISIEL, ~30 department spellings), so the normalizers (fuel.norm_plate, fuel.canon_fuel_type, fuel.canon_department) are the single source of truth. Soft-deletes (deleted_at) are kept on the row and excluded by the read views.

The fuel record links to the fleet by plate: reporting.v_fuel_fills joins fuel.norm_plate(car) to fuel.norm_plate(devices.vehicle_number) (LATERAL + LIMIT 1, since a plate can map to more than one device over time) to pick up cost_centre / assigned_city / imei.

Timezone: the source stamps record_datetime in UTC; FleetFuel stores all fuel timestamps as Africa/Nairobi (EAT, UTC+3) wall-clock (timestamp without tz), so record_datetime::date (the daily-trend bucket) is the Kenyan calendar day. See migrations/03_fuel_timezone_eat.sql.

Bucket layout

fuel_records/latest.json              full snapshot   { metadata, records[] }
fuel_records/changes/<ISO-ts>.json    hourly CDC delta (incl. soft-deletes)

Setup

uv sync
cp .env.example .env        # fill in DATABASE_URL, RUSTFS_*
python run_migrations.py    # apply the schema (idempotent)

Run

# full reconcile from the snapshot (self-healing; the default cron job)
python import_fuel.py --snapshot --apply

# incremental, since the stored watermark (lower latency)
python import_fuel.py --changes --apply

# from a local file instead of the bucket
python import_fuel.py --file latest.json --apply

Drop --apply for a dry-run (parses + logs counts, writes nothing).

Deploy

Like fleettickets: a Coolify container/cron in the tracksolid stack runs python run_migrations.py then python import_fuel.py --snapshot --apply hourly (matching the n8n export cadence). Env from the Coolify .env. The dashboard_api and fleetops apps ride their own existing pipelines.