Polling jimi.device.track.mileage does not return start/end coordinates,
fuel, idle, or trip sequence — leaving most trip columns NULL. This change
closes those gaps using data we already have in position_history plus a
best-effort Nominatim lookup.
Migration 09_trips_enrichment.sql adds:
• route_geom (LineString), start_address, end_address, vehicle_plate,
waypoints_count on tracksolid.trips
• GIST indexes on the three geometry columns
• view tracksolid.v_trips_enriched exposing daily_seq + trip_date_eat
(replaces reliance on the device-supplied trip_seq, which is only
populated when /pushtripreport fires)
ingest_movement_rev.py::poll_trips now:
• extracts idleSecond from the poll response (was previously dropped)
• per-trip: SELECTs start fix, end fix, ST_MakeLine route, and waypoint
count from position_history within (start_time, end_time)
• reverse-geocodes start/end via the new ts_shared_rev.reverse_geocode
helper (Nominatim, LRU-cached at ~11m precision, 1 req/sec, never raises)
• caches vehicle_plate from a per-cycle plates dict
• ON CONFLICT preserves webhook-supplied data when /pushtripreport later
delivers native coords/fuel/trip_seq
backfill_trips_enrichment.py is a one-shot script (dry-run by default,
--apply to commit, --imei / --since flags) that runs the same enrichment
against historical NULL rows and COALESCEs only — never overwrites.
DWH bronze mirrors and Grafana panels intentionally not touched (frozen
on this branch until the schema work lands).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Fleet lives across three Tracksolid sub-accounts:
fireside — 63 devices
Fireside@HQ — 52 devices
Fireside_MSA — 41 devices
Previously sync_devices / poll_live_positions / poll_parking only
queried a single TARGET_ACCOUNT, so ~64% of the fleet was invisible to
the pipeline.
Changes:
- ts_shared_rev.py: new TARGETS list (env TRACKSOLID_TARGETS,
comma-separated; falls back to the single TARGET_ACCOUNT).
- ts_shared_rev.py: new get_active_imeis_by_target() helper that
groups active IMEIs by their stored account so parking calls can
pass the right account param per batch.
- ingest_movement_rev.py: sync_devices and poll_live_positions loop
over every target and dedupe by IMEI before upserting. poll_parking
loops over imeis_by_target so each batch carries the matching
account.
- CLAUDE.md: FIX-M19 entry.
Requires new env var TRACKSOLID_TARGETS="fireside,Fireside@HQ,Fireside_MSA"
on the ingest services in Coolify.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
BUG-01: OBD event_time — try unix_to_ts before clean_ts (Jimi sends epoch ints)
BUG-02: push_alarm — guard alarm_type not null (NULL breaks ON CONFLICT dedup)
BUG-03: push_trip_report — _parse_trip_ts handles Jimi BCD format YYMMDDHHmmss
BUG-04: SAVEPOINT per item in all 5 DB endpoints (FK violation on one item no
longer aborts the whole batch; SAVEPOINT now inside try for safety)
BUG-05: Add /pushevent endpoint (log-only; was returning 404 to Jimi)
FIX: push_fault_info — skip null fault_code (NULL != NULL in PG unique index)
FIX: log_ingestion — pass SQL NULL not string "None" when no error occurred
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
