kianiadee/fleettickets
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
fleettickets/.env.example

18 lines
804 B
Text
Raw Normal View History

feat: fleettickets — INC/CRQ ticket ingestion, geocoding + read-schema Standalone module extracted from the tracksolid repo (was migrations 21-23 + tools/import_tickets.py). Owns the `tickets` schema in the shared tracksolid_db. - migrations/01_tickets_schema.sql: consolidated final-state schema (tickets.inc/ crq raw-jsonb-first, geo_clusters + geo_locations gazetteers, geom trigger, reporting.fn_tickets_for_map) - import_tickets.py: rustfs bucket ingest + cluster/location geocoding (LocationIQ/OpenCage, viewbox-bounded + cluster-distance guard) - run_migrations.py, shared.py (self-contained), pyproject, .env.example, README The DB stays in tracksolid_db; dashboard_api keeps serving /webhook/tickets; the Tickets map stays a FleetOps tab. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:13:50 +00:00
# fleettickets — copy to .env and fill in. NEVER commit the real .env.
# Shared database (the `tickets` schema lives in tracksolid_db; internal Docker host)
DATABASE_URL=postgresql://tracksolid_owner:<password>@timescale_db:5432/tracksolid_db
feat(crq): add CRQ ingestion via shared engine + thin inc/crq entrypoints Split the INC-only loader into a dataset-agnostic engine (pipeline.py, renamed from import_tickets.py) parameterized by a Dataset config, with thin per-type entrypoints inc/import_inc.py and crq/import_crq.py. CRQ shares INC's identical 32-column source schema and CDC change stream, so the engine is fully shared. - pipeline.py: Dataset config (name/table/prefixes/key_regex/post_apply); INC keeps the capture_history post-apply hook, CRQ has none yet. geocode_locations now unions tickets.crq (geocoding is cross-dataset: one gazetteer/budget). - crq/import_crq.py: drains automations/crq/changes/ from isptickets into tickets.crq (data layer + map; SLA/dashboard/history deferred). - migrations/13_crq_columns.sql: CRQ mirror of 03 — typed STORED generated columns + indexes on tickets.crq (reuses tickets.eat_ts()). - Deployment: Dockerfile/run_ingest.sh run both via `python -m`; pyproject packages inc/crq. Docs (README, implementation, deployment-and-operations, n8n export ref, phase-1) updated for the split + the one-time CRQ seed runbook. tickets.crq already exists (mig 01, LIKE tickets.inc) and is unioned into reporting.fn_tickets_for_map + resolve_ticket_geoms, so CRQ appears on the existing Tickets map once seeded. Verified locally: ruff-clean new files, engine lists/parses both streams against live S3 (crq=52 files, inc unaffected). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 20:16:38 +00:00
# S3 — source ticket CDC streams (isptickets bucket, automations/{inc,crq}/changes/<EAT-ts>.csv)
feat: fleettickets — INC/CRQ ticket ingestion, geocoding + read-schema Standalone module extracted from the tracksolid repo (was migrations 21-23 + tools/import_tickets.py). Owns the `tickets` schema in the shared tracksolid_db. - migrations/01_tickets_schema.sql: consolidated final-state schema (tickets.inc/ crq raw-jsonb-first, geo_clusters + geo_locations gazetteers, geom trigger, reporting.fn_tickets_for_map) - import_tickets.py: rustfs bucket ingest + cluster/location geocoding (LocationIQ/OpenCage, viewbox-bounded + cluster-distance guard) - run_migrations.py, shared.py (self-contained), pyproject, .env.example, README The DB stays in tracksolid_db; dashboard_api keeps serving /webhook/tickets; the Tickets map stays a FleetOps tab. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:13:50 +00:00
RUSTFS_ENDPOINT=https://s3.rahamafresh.com
feat(import_tickets): migrate INC ingest to isptickets bucket + --reseed cutover Provider moved the INC CDC feed to a new bucket (tickets -> isptickets, new per-bucket creds; same s3.rahamafresh.com endpoint, identical 32-col schema). This is config + a one-time reseed, not a rewrite — the loader already drains automations/inc/changes/ oldest->newest with a source_max_key watermark. - default _BUCKET -> isptickets (TICKETS_BUCKET still overrides) - add --reseed: ignore the stored watermark and drain every changes/ file once (the old-bucket watermark may post-date the new bucket's first file). Crash-safe via the existing per-file watermark-advance + archive loop. - refresh stale "newest-file / full-snapshot-per-hour" docstring/comments to the CDC reality; .env.example + README updated (new bucket + reseed runbook). Verified live dry-run: 41/41 files drained (watermark None), alarm/sentinel filter active, exit 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 15:20:04 +00:00
RUSTFS_ACCESS_KEY=isptickets
feat: fleettickets — INC/CRQ ticket ingestion, geocoding + read-schema Standalone module extracted from the tracksolid repo (was migrations 21-23 + tools/import_tickets.py). Owns the `tickets` schema in the shared tracksolid_db. - migrations/01_tickets_schema.sql: consolidated final-state schema (tickets.inc/ crq raw-jsonb-first, geo_clusters + geo_locations gazetteers, geom trigger, reporting.fn_tickets_for_map) - import_tickets.py: rustfs bucket ingest + cluster/location geocoding (LocationIQ/OpenCage, viewbox-bounded + cluster-distance guard) - run_migrations.py, shared.py (self-contained), pyproject, .env.example, README The DB stays in tracksolid_db; dashboard_api keeps serving /webhook/tickets; the Tickets map stays a FleetOps tab. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:13:50 +00:00
RUSTFS_SECRET_KEY=<secret>
RUSTFS_REGION=us-east-1
feat(import_tickets): migrate INC ingest to isptickets bucket + --reseed cutover Provider moved the INC CDC feed to a new bucket (tickets -> isptickets, new per-bucket creds; same s3.rahamafresh.com endpoint, identical 32-col schema). This is config + a one-time reseed, not a rewrite — the loader already drains automations/inc/changes/ oldest->newest with a source_max_key watermark. - default _BUCKET -> isptickets (TICKETS_BUCKET still overrides) - add --reseed: ignore the stored watermark and drain every changes/ file once (the old-bucket watermark may post-date the new bucket's first file). Crash-safe via the existing per-file watermark-advance + archive loop. - refresh stale "newest-file / full-snapshot-per-hour" docstring/comments to the CDC reality; .env.example + README updated (new bucket + reseed runbook). Verified live dry-run: 41/41 files drained (watermark None), alarm/sentinel filter active, exit 0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 15:20:04 +00:00
TICKETS_BUCKET=isptickets
feat: fleettickets — INC/CRQ ticket ingestion, geocoding + read-schema Standalone module extracted from the tracksolid repo (was migrations 21-23 + tools/import_tickets.py). Owns the `tickets` schema in the shared tracksolid_db. - migrations/01_tickets_schema.sql: consolidated final-state schema (tickets.inc/ crq raw-jsonb-first, geo_clusters + geo_locations gazetteers, geom trigger, reporting.fn_tickets_for_map) - import_tickets.py: rustfs bucket ingest + cluster/location geocoding (LocationIQ/OpenCage, viewbox-bounded + cluster-distance guard) - run_migrations.py, shared.py (self-contained), pyproject, .env.example, README The DB stays in tracksolid_db; dashboard_api keeps serving /webhook/tickets; the Tickets map stays a FleetOps tab. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:13:50 +00:00
# Geocoder (keyed — public Nominatim rate-limits bulk)
GEOCODER_PROVIDER=locationiq # locationiq | opencage
GEOCODER_API_KEY=<key>
GEOCODER_MIN_INTERVAL_S=1.1 # throttle to provider TOS
GEOCODER_MAX_KM=25 # reject a location geocode this far from its cluster centroid
Reference in a new issue Copy permalink