Commit graph

2 commits

Author SHA1 Message Date
david kiania
e5b0e192d8 chore(repo): reorganize tree into migrations/ data/ legacy/ docs/
Group root-level files (accreted from incremental changes) by purpose
without moving any deployment entrypoint or breaking imports:

- migrations/  : numbered SQL 02-10
- data/        : source CSVs
- legacy/      : superseded pre-_rev scripts + old pipeline notes (not deployed)
- docs/{manuals,reference,reports}/ : loose manuals, references, reports
- strip stray ** / *** prefixes from 5 doc filenames
- delete empty documents.txt / push_webhook.md

Reference updates so nothing breaks:
- run_migrations.py  -> /app/migrations/<file>
- run_migrations.sh  -> $SCRIPT_DIR/migrations
- import_drivers_csv.py -> data/<csv>
- docker-compose.yaml -> runbook path comment
- CLAUDE.md -> codebase map + inline doc references

Deployed Python (3 services + ts_shared_rev + run_migrations) and the
documented ops one-shots stay at root, preserving the flat-import layout
and all documented commands. Verified: py_compile clean across all modules,
every MIGRATIONS entry resolves under migrations/, CI-referenced paths
(tests/, mypy targets, db_audit) and the grafana build context intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 02:27:30 +03:00
David Kiania
144dedee90 feat(trips): [FIX-M20] enrich tracksolid.trips with coords, route polyline, addresses, plate
Some checks are pending
Static Analysis / static (push) Waiting to run
Tests / test (push) Waiting to run
Polling jimi.device.track.mileage does not return start/end coordinates,
fuel, idle, or trip sequence — leaving most trip columns NULL. This change
closes those gaps using data we already have in position_history plus a
best-effort Nominatim lookup.

Migration 09_trips_enrichment.sql adds:
  • route_geom (LineString), start_address, end_address, vehicle_plate,
    waypoints_count on tracksolid.trips
  • GIST indexes on the three geometry columns
  • view tracksolid.v_trips_enriched exposing daily_seq + trip_date_eat
    (replaces reliance on the device-supplied trip_seq, which is only
     populated when /pushtripreport fires)

ingest_movement_rev.py::poll_trips now:
  • extracts idleSecond from the poll response (was previously dropped)
  • per-trip: SELECTs start fix, end fix, ST_MakeLine route, and waypoint
    count from position_history within (start_time, end_time)
  • reverse-geocodes start/end via the new ts_shared_rev.reverse_geocode
    helper (Nominatim, LRU-cached at ~11m precision, 1 req/sec, never raises)
  • caches vehicle_plate from a per-cycle plates dict
  • ON CONFLICT preserves webhook-supplied data when /pushtripreport later
    delivers native coords/fuel/trip_seq

backfill_trips_enrichment.py is a one-shot script (dry-run by default,
--apply to commit, --imei / --since flags) that runs the same enrichment
against historical NULL rows and COALESCEs only — never overwrites.

DWH bronze mirrors and Grafana panels intentionally not touched (frozen
on this branch until the schema work lands).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 21:30:20 +03:00