5f5d71d500
Split the INC-only loader into a dataset-agnostic engine (pipeline.py, renamed from import_tickets.py) parameterized by a Dataset config, with thin per-type entrypoints inc/import_inc.py and crq/import_crq.py. CRQ shares INC's identical 32-column source schema and CDC change stream, so the engine is fully shared. - pipeline.py: Dataset config (name/table/prefixes/key_regex/post_apply); INC keeps the capture_history post-apply hook, CRQ has none yet. geocode_locations now unions tickets.crq (geocoding is cross-dataset: one gazetteer/budget). - crq/import_crq.py: drains automations/crq/changes/ from isptickets into tickets.crq (data layer + map; SLA/dashboard/history deferred). - migrations/13_crq_columns.sql: CRQ mirror of 03 — typed STORED generated columns + indexes on tickets.crq (reuses tickets.eat_ts()). - Deployment: Dockerfile/run_ingest.sh run both via `python -m`; pyproject packages inc/crq. Docs (README, implementation, deployment-and-operations, n8n export ref, phase-1) updated for the split + the one-time CRQ seed runbook. tickets.crq already exists (mig 01, LIKE tickets.inc) and is unioned into reporting.fn_tickets_for_map + resolve_ticket_geoms, so CRQ appears on the existing Tickets map once seeded. Verified locally: ruff-clean new files, engine lists/parses both streams against live S3 (crq=52 files, inc unaffected). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
34 lines
992 B
TOML
34 lines
992 B
TOML
[build-system]
|
|
requires = ["setuptools>=61"]
|
|
build-backend = "setuptools.build_meta"
|
|
|
|
[project]
|
|
name = "fleettickets"
|
|
version = "0.1.0"
|
|
description = "Field-ops INC/CRQ ticket ingestion, geocoding, and read-schema for the FleetOps Tickets map"
|
|
requires-python = ">=3.12"
|
|
dependencies = [
|
|
"psycopg2-binary>=2.9.9", # DB driver
|
|
"requests>=2.32.3", # geocoder HTTP
|
|
"boto3>=1.34", # S3 (rustfs) access — no aws-CLI dependency
|
|
]
|
|
|
|
[project.optional-dependencies]
|
|
dev = [
|
|
"ruff>=0.4",
|
|
]
|
|
|
|
# Shared engine (pipeline) + helpers as top-level modules, plus the thin per-type
|
|
# entrypoint packages (inc/, crq/). Listed explicitly so `pip install .` works (the
|
|
# Docker image installs the project to pull its deps; runtime runs from /app via -m).
|
|
[tool.setuptools]
|
|
py-modules = ["pipeline", "shared", "run_migrations"]
|
|
packages = ["inc", "crq"]
|
|
|
|
[tool.uv]
|
|
managed = true
|
|
|
|
[tool.ruff]
|
|
target-version = "py312"
|
|
line-length = 100
|
|
lint.select = ["E", "W", "F", "B", "UP"]
|