f9834564ab
CLAUDE.md: cached context file covering project identity, tech stack, codebase map, schema quick-ref, API gotchas, fix history, working rules, fleet state, and open items. Structured for maximum cache efficiency — stable content first, dynamic state at the end. docs/CONNECTIONS.md: connection parameter shapes (no secrets) for SSH, DB, API, container resolution, Forgejo, Grafana, n8n. docs/PROJECT_CONTEXT.md: client business context (telco field service, 3 cities, service types), data quality gaps, KPI framework by domain, integration roadmap. docs/KPI_FRAMEWORK.md: living KPI register with status tracking, thresholds, client feedback log, and review checklist. To be co-developed with client iteratively. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
161 lines
8.8 KiB
Markdown
161 lines
8.8 KiB
Markdown
# CLAUDE.md — Fireside Communications · Tracksolid Fleet Intelligence
|
||
|
||
## 1. What This Project Is
|
||
|
||
Fleet telematics ingestion and analytics stack for a **telco first-line support client** operating in Nairobi, Mombasa, and Kampala. The client dispatches field technicians to install, repair, and maintain home and business broadband, handle LOS signal faults, service migrations, and maintain outside plant infrastructure. The fleet is ~80 vehicles across three cities, all tracked via Tracksolid Pro (Jimi IoT API).
|
||
|
||
This repository ingests the Tracksolid Pro API into a TimescaleDB/PostGIS database and visualises fleet and operational KPIs in Grafana. The pipeline is deployed on Coolify at `stage.rahamafresh.com`.
|
||
|
||
**Repository:** `https://repo.rahamafresh.com/kianiadee/tracksolid_timescale_grafana_prod.git`
|
||
|
||
---
|
||
|
||
## 2. Tech Stack
|
||
|
||
| Layer | Technology |
|
||
|---|---|
|
||
| Ingestion | Python 3.12 — `ingest_movement_rev.py`, `ingest_events_rev.py`, `webhook_receiver_rev.py` |
|
||
| Shared utils | `ts_shared_rev.py` — token cache, DB pool, API signing, clean helpers |
|
||
| Database | PostgreSQL 16 + TimescaleDB 2.15 + PostGIS 3 (`tracksolid_db`) |
|
||
| Orchestration | Docker Compose on Coolify |
|
||
| Visualisation | Grafana (provisioned via custom image) |
|
||
| Workflow automation | n8n |
|
||
| API source | Tracksolid Pro / Jimi IoT Open Platform (`eu-open.tracksolidpro.com/route/rest`) |
|
||
| Version control | Forgejo at `repo.rahamafresh.com` |
|
||
|
||
---
|
||
|
||
## 3. Instance & Connection Parameters
|
||
|
||
See `docs/CONNECTIONS.md` for the full shape. Summary:
|
||
|
||
- **SSH:** `ssh -i ~/.ssh/id_ed25519 kianiadee@stage.rahamafresh.com`
|
||
- **DB name:** `tracksolid_db` · **DB user:** `postgres` (internal) · `tracksolid_owner` (app) · `grafana_ro` (read-only)
|
||
- **DB schema:** `tracksolid` (operational) · `infrastructure` · `dwh_gold` (aggregates)
|
||
- **Container naming:** Coolify appends a random suffix. Always resolve with:
|
||
```bash
|
||
docker ps --filter name=<service_name> --format "{{.Names}}" | head -1
|
||
```
|
||
e.g. `docker ps --filter name=timescale_db --format "{{.Names}}" | head -1`
|
||
- **Env vars:** loaded from `.env` via `env_file` in `docker-compose.yaml`. See `docs/CONNECTIONS.md` for variable names. Never hardcode secrets.
|
||
|
||
---
|
||
|
||
## 4. Codebase Map
|
||
|
||
```
|
||
ts_shared_rev.py # Shared: config, signing, DB pool, token cache, clean helpers
|
||
ingest_movement_rev.py # GPS positions, trips, parking, track-list (high-res trail), device sync
|
||
ingest_events_rev.py # Alarm events polling (fallback for webhook push)
|
||
webhook_receiver_rev.py # FastAPI push receiver: /pushobd /pushevent /pushtripreport etc.
|
||
sync_driver_audit.py # One-shot: API↔DB driver/IMEI gap report + full upsert
|
||
run_migrations.py # Applies SQL migrations in order at container startup
|
||
docker-compose.yaml # Services: timescale_db, ingest_movement, ingest_events,
|
||
# webhook_receiver, grafana
|
||
grafana/ # Grafana provisioning (baked into image)
|
||
n8n-workflows/ # n8n workflow exports
|
||
docs/ # Reference docs (connections, API, KPIs, project context)
|
||
02_tracksolid_full_schema_rev.sql # Full schema bootstrap
|
||
03..05_*.sql # Incremental migrations
|
||
01_BusinessAnalytics.md # SQL analytics library (reference before writing queries)
|
||
tracksolidApiDocumentation.md # API endpoint reference
|
||
260412_baseline_report.md # Latest fleet state snapshot
|
||
```
|
||
|
||
---
|
||
|
||
## 5. Database Schema — Key Tables
|
||
|
||
```sql
|
||
tracksolid.devices -- Master device registry (63 devices, imei PK)
|
||
tracksolid.live_positions -- Current position per device (1 row per IMEI, upserted)
|
||
tracksolid.position_history -- All GPS fixes (hypertable, partitioned by gps_time)
|
||
-- source: 'poll' (60s sweep) | 'track_list' (30m high-res)
|
||
tracksolid.trips -- Trip summaries: distance_km, driving_time_s, avg/max speed
|
||
tracksolid.parking_events -- Stop events with duration and address
|
||
tracksolid.alarms -- Alarm events (alarm_type, alarm_name, alarm_time)
|
||
tracksolid.obd_readings -- OBD diagnostics (push only, awaiting webhook registration)
|
||
tracksolid.device_events -- Power on/off tamper events
|
||
tracksolid.ingestion_log -- API call audit trail per endpoint
|
||
dwh_gold.fact_daily_fleet_metrics -- Nightly ETL aggregates per vehicle per day
|
||
```
|
||
|
||
Full DDL: `02_tracksolid_full_schema_rev.sql` + migrations `03`–`05`.
|
||
|
||
---
|
||
|
||
## 6. API Critical Facts
|
||
|
||
**Always read `tracksolidApiDocumentation.md` before adding a new endpoint call.**
|
||
|
||
| Fact | Detail |
|
||
|---|---|
|
||
| Auth | OAuth2 — token cached in `tracksolid.api_token_cache`, refreshed via `jimi.oauth.token.refresh` |
|
||
| Signing | MD5: `secret + sorted(k+v pairs) + secret` — see `build_sign()` in `ts_shared_rev.py` |
|
||
| Batch limit | Max 50 IMEIs per call for most endpoints |
|
||
| `distance` field | **Returns METRES, not km** despite docs. Always divide by 1000. (FIX-M16) |
|
||
| `driverName`/`driverPhone` | From `jimi.user.device.list` — will be NULL if not set in Tracksolid Pro UI |
|
||
| `alarm_type` field | API polling returns `alertTypeId`/`alarmTypeName` — NOT `alarmType`/`alarmName` (FIX-E06) |
|
||
| `durSecond` | Parking endpoint returns `durSecond`, not `seconds` (FIX-M13) |
|
||
| `jimi.device.track.mileage` | `startMileage`/`endMileage` are cumulative odometer in **metres** |
|
||
| Rate limit | Code 1006 — back off and retry with re-sign (handled in `api_post()`) |
|
||
| OBD data | Push only via `/pushobd` webhook — no polling endpoint exists |
|
||
|
||
---
|
||
|
||
## 7. Fix History (do not regress)
|
||
|
||
| Fix ID | File | What it fixed |
|
||
|---|---|---|
|
||
| FIX-M11 | `ingest_movement_rev.py` | Removed erroneous ×1000 on distance (was storing km as mm) |
|
||
| FIX-M13 | `ingest_movement_rev.py` | Parking: added `acc_type=0`, `account`; mapped `durSecond` |
|
||
| FIX-M14 | `ingest_movement_rev.py` | `poll_track_list()` — high-res GPS trail every 30m |
|
||
| FIX-M15 | `ingest_movement_rev.py` | `get_device_locations()` — on-demand precision refresh |
|
||
| FIX-M16 | `ingest_movement_rev.py` | `distance` from API is metres → divide by 1000 before storing |
|
||
| FIX-M17 | `ingest_movement_rev.py` | `sync_devices()` ON CONFLICT now updates all 26 fields (was 5) |
|
||
| FIX-E06 | `ingest_events_rev.py` | Alarm field mapping: `alertTypeId`/`alarmTypeName`/`alertTime` |
|
||
| BUG-02 | Migration 04 | Historical `distance_m` rows ÷1,000,000 → renamed to `distance_km` |
|
||
|
||
---
|
||
|
||
## 8. Working Rules
|
||
|
||
1. **No prod push without explicit user confirmation.** Always state what you are about to push and wait.
|
||
2. **Never rewrite a migration that is already applied.** Check `tracksolid.schema_migrations` first. Add a new numbered migration file for any schema change.
|
||
3. **Read before writing.** Before suggesting any code change, read the relevant source file. Before writing a query, check `01_BusinessAnalytics.md` for an existing pattern.
|
||
4. **Reuse shared utilities.** All DB access via `get_conn()`, all API calls via `api_post()`, all cleaning via `clean()` / `clean_num()` / `clean_int()` / `clean_ts()` in `ts_shared_rev.py`. Do not reinvent these.
|
||
5. **Resolve container names dynamically.** Never hardcode the Coolify suffix. Use `docker ps --filter name=<service>`.
|
||
6. **SSH only when asked.** Default workflow is local code → commit → push. SSH into the instance only when explicitly asked to test or run something live.
|
||
7. **Secrets from env only.** Connection strings, API keys, and passwords live in `.env`. Reference variable names from `docs/CONNECTIONS.md`, never values.
|
||
8. **Two developers, one incoming.** Write code and docs that a second developer (mixed technical/operations background) can follow without prior context.
|
||
|
||
---
|
||
|
||
## 9. Fleet State (as of 2026-04-12 baseline)
|
||
|
||
| Metric | Value |
|
||
|---|---|
|
||
| Total registered devices | 63 (growing to 80) |
|
||
| Devices with GPS fix < 2h | 2 |
|
||
| Devices never reported | 44 |
|
||
| Driver names populated | 0 — must be set in Tracksolid Pro UI first |
|
||
| Cities active | Nairobi (primary), Mombasa (deploying), Kampala (1 device confirmed) |
|
||
| Uganda anomaly | X3-63282 at 0.196, 32.540 — under investigation |
|
||
| Service flags | KDK 829A GP (239,264 km), Belta KCU-647D (235,000 km) |
|
||
|
||
Latest full snapshot: `260412_baseline_report.md`
|
||
|
||
---
|
||
|
||
## 10. Open Items (update as resolved)
|
||
|
||
| Priority | Item |
|
||
|---|---|
|
||
| HIGH | Assign driver names + vehicle numbers in Tracksolid Pro UI |
|
||
| HIGH | Register webhooks: `/pushobd` `/pushoil` `/pushtem` `/pushlbs` `/pushevent` |
|
||
| HIGH | Investigate X3-63282 in Kampala — legitimate or unauthorised? |
|
||
| MEDIUM | Set `fuel_100km` per vehicle type to activate fuel cost calculations |
|
||
| MEDIUM | Investigate 44 silent devices — SIM installed? Activated? |
|
||
| MEDIUM | Co-develop client KPI framework (see `docs/KPI_FRAMEWORK.md`) |
|
||
| LOW | Populate geofences — depot boundaries, city zones |
|
||
| LOW | Run nightly ETL: `SELECT dwh_gold.refresh_daily_metrics(CURRENT_DATE - 1)` |
|