New reporting.fn_inc_search(ticket_id, owner, cluster, status, state, from, to,
limit) over tickets.inc — ad-hoc ticket lookup by id / engineer / cluster /
status / state (closed default / open / all) / time, for historical + current
tracking. owner is case-normalized (initcap(lower(...)), like migration 12);
filters are optional + AND-combined; returns { count, truncated, limit, state,
rows }, capped at limit. Backs GET /webhook/inc-search. Validated in a
rolled-back tx (matheka closed=151, RUIRU open=55, all=22849 truncated).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Building-level names (e.g. 'KAHAWA WENDANI ALVO HOUSE') aren't in OSM, so the
precise forward-geocode 404s and tickets stay on the bare cluster centroid
(observed 0/133 placed). geocode_locations now tries an ordered set of
candidates per location (compose_queries): full precise -> estate (leading 2
tokens) -> leading token, each constrained by the existing cluster viewbox +
25km distance check, accepting the FIRST in-range hit. This places tickets in
the right neighbourhood (e.g. 'KAHAWA WENDANI', 'BAMBURI') instead of the broad
cluster centroid. Wrong-area matches for ambiguous coarse tokens are rejected by
the distance check and fall through; genuinely unmatchable tickets keep the
honest cluster-centroid fallback (no pure-cluster candidate, which would only
mislabel the centroid as geo_source='location'). Verified the cascade finds
hits against live LocationIQ on real samples.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CREATE OR REPLACE reporting.fn_inc_dashboard (supersedes 09) adding
metrics.by_owner — a closures-by-engineer leaderboard over the windowed closed
set: array of { owner, closed, breached, avg_mttr_min } sorted desc. owner is
CASE-NORMALIZED (initcap(lower(...))) to merge the offline closing system's
mixed-case duplicates (observed 102 -> 58 distinct engineers; e.g.
'Elikana Mabonga' + 'ELIKANA MABONGA' -> one). owner on the open/closed GeoJSON
features is normalized the same way so the leaderboard and per-ticket drill-down
agree. Everything else unchanged; the dashboard_api route is a passthrough so no
API change is needed. Validated in a rolled-back tx against prod.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified each finding against the code (+ profiled the 31k-row CSV sample);
implemented only the genuinely valid fixes:
- import_tickets.py: fold _record_meta into the upsert transaction so rows +
snapshot meta commit atomically (BUG 2); guard _ts_from_key against
regex-matching-but-invalid dates so the sort can't crash (BUG 11);
extract_place now splits glued NW prefixes (~1.7k rows, e.g. NWKIAMBU→KIAMBU)
and only drops a trailing '-<seg>' when it's a unit/instruction code, keeping
real-word tails like '-MALL' (BUG 14). Scoped glued-split to NW only —
CO/NE/SE begin real words (COAST/NEW/SEASONS) per the data.
- Dockerfile + pyproject.toml: install from pyproject (single source of truth)
instead of mirroring deps; add build-system + py-modules so `pip install .`
works for the flat-module layout (BUG 9).
- migrations/03_inc_columns.sql: document the eat_ts IMMUTABLE/tzdata footgun
and the manual-recompute path (BUG 6).
- .gitignore: narrow *.json → *.local.json so real fixtures can be versioned;
ignore build/ and *.egg-info/ (BUG 10).
Reclassified/skipped as invalid or by-design: BUG 1, 3, 4, 5, 7, 8, 12, 13.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- tickets.closure_events: append-only observed closures (PK ticket_id, closed_at;
observed_at = first sighting; survives row churn).
- tickets.inc_daily_snapshot: one row per EAT day — open backlog (+ SLA split, by
cluster/status) and created/closed flow; upserted each run.
- tickets.capture_history(): appends new closures + upserts today's snapshot.
- import_tickets calls it after each --apply run (ingest or skip); add
--capture-history CLI flag for standalone runs.
Verified: backfilled 21,282 closures; today's snapshot recorded (open_total 30).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GET /webhook/inc-dashboard wrapper spec: query params (cluster/status/window/from/to)
-> SQL passthrough, full response schema, field semantics (open=live vs closed=window,
mttr minutes, derived vs source SLA, map/metrics geocoding gap), examples, caching/auth.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
One parameterized function returns {window, open GeoJSON, closed GeoJSON, metrics,
freshness} for the FleetOps live INC map:
- open = all is_actionable tickets (live), filtered by cluster/status, with
sla_state/hours_open (from tickets.inc_open_sla)
- closed= closed_at within the selected window (EAT calendar today/week/month or
custom [from,to)), filtered by cluster/status
- metrics= open/closed counts, SLA split (open derived, closed source), by status/
cluster, closure rate + daily series, avg mttr (minutes)
Filters combine with AND; grants to dashboard_ro/grafana_ro. Verified live
(today/month/cluster/status/custom; last-7d closed=913 matches raw).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add tickets.inc_open_sla: open tickets with derived SLA (hours_open, sla_state
vs 48h; clock = created_at_service or first_seen_at fallback) + team/cluster/geog
for dispatch. (One-time legacy region->lowercase backfill applied to live data.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
service_type is always 'inc' (cardinality 1) — zero info, redundant in an INC-only
table. Drop the generated column; stays in raw for audit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mttr generated column is now integer minutes (source raw.mttr is decimal hours),
analytics-friendly. Drop is_alarm/is_auto_created/is_auto_closed generated columns
— all constant `false` in tickets.inc since alarms are filtered at ingest (still
present in raw for audit; loader still filters on raw->>'is_alarm'). is_actionable
kept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
geom is geometry(Point,4326) (planar degrees); add geog = geom::geography (STORED
generated) + GiST index so ST_Distance/ST_DWithin/KNN work in real metres for
nearest-vehicle and radius queries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Feed coords are always empty; redefine the latitude/longitude generated columns to
COALESCE(feed, ST_Y/ST_X(geom)) so they carry the resolved/geocoded position for
every geocoded ticket (precision indicated by geo_source). STORED, recomputes when
geom changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add STORED generated columns derived from raw (text/numeric/bool/double + EAT
timestamptz via an IMMUTABLE tickets.eat_ts() wrapper). Computed for all existing
rows and auto-populated on every future ingest — raw stays the source of truth,
no loader change. Indexes on status/cluster/team/closed_at/is_actionable for the
SLA/team/closure queries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace the aws-CLI subprocess calls with boto3 (list_objects_v2 paginator,
get_object, copy_object+delete_object) using path-style addressing + RUSTFS_*
env. Removes the external aws-CLI dependency so it runs in a slim container.
- Add boto3 to pyproject dependencies.
- Add Dockerfile (python:3.12-slim, deps, TZ=Africa/Nairobi, keep-alive CMD) and
.dockerignore for Coolify; document Coolify Scheduled Task setup in README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rework import_tickets.py from the retired JSON `latest.json` model to the new
hourly full-snapshot CSV export. Strictly INC (CRQ out of scope).
- Ingest the newest automations/inc/<EAT-timestamp>.csv; skip-if-unchanged by
comparing S3 ETag to tickets.import_meta.metadata.source_etag.
- Upsert on ticket_id (PK; no dups, never delete -> closure history accrues).
No truncate. On success, move processed files to automations/inc/processed/.
- Clean at ingest: drop is_alarm=true + the "EXPORT STOPPED..." sentinel; drop
week_*, source_s3_*/source_snapshot_id, department/source_type; lowercase
region, uppercase raw_status; keep service_type + bucket.
- Force path-style S3 addressing; --inc-csv for local dev; --from-bucket for cron.
- Add migrations/02 (import_meta + freshness); refresh README/.env.example/docs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
