fleettickets/crq/import_crq.py

"""
crq/import_crq.py — Fireside Communications · CRQ (new-installation) ingestion.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Thin entrypoint over the shared engine (`pipeline.py`) for the CRQ dataset:
    tickets.crq   — new-installation requests  (FleetOps "Tickets" CRQ tab)

CRQ mirrors INC at the data layer — IDENTICAL 32-column CSV schema and the same
incremental CDC change stream automations/crq/changes/<EAT-ts>.csv in the
`isptickets` bucket. This loader upserts on ticket_id, advances the per-dataset
watermark (tickets.import_meta dataset='crq'), and archives each consumed file to
automations/crq/processed/. CRQ flows onto the existing Tickets map via
reporting.fn_tickets_for_map (which already unions tickets.crq).

Scope (current): data layer + map only. CRQ has NO post-apply history capture yet
(installation-lifecycle SLA/backlog semantics differ from incidents — a future
migration). Geocoding is CROSS-DATASET and run from the INC entrypoint
(python -m inc.import_inc --geocode-clusters / --geocode-locations) against the
shared gazetteer, which covers both inc and crq.

Usage (needs DATABASE_URL + RUSTFS_* env; see .env.example):
    python -m crq.import_crq --from-bucket --apply
    python -m crq.import_crq --from-bucket --reseed --apply   # one-time bucket cutover
    python -m crq.import_crq --crq-csv 2026-06-24T12-55-44.csv --apply

Pre-requisite: migrations applied (run_migrations.py) — tickets.crq + its typed
columns (15_crq_table.sql) + geo_clusters/geo_locations + fn_tickets_for_map.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
"""

from __future__ import annotations

import argparse

import pipeline

# CRQ has no post-apply hook yet (history capture is INC-only — see module docstring).
DATASET = pipeline.make_dataset("crq", post_apply=None)


def main() -> None:
    ap = argparse.ArgumentParser(
        description="Ingest CRQ (installation) tickets from CSV (raw-first)")
    ap.add_argument("--apply", action="store_true", help="Write to DB (default: dry-run)")
    ap.add_argument("--from-bucket", action="store_true",
                    help="Drain the incremental CRQ change stream (automations/crq/changes/) "
                         "from the isptickets S3 bucket: every not-yet-processed file "
                         "oldest→newest, upsert on ticket_id, advance the watermark, archive")
    ap.add_argument("--reseed", action="store_true",
                    help="Ignore the stored watermark and drain every file in changes/ once "
                         "(one-time bucket cutover / reseed). Use with --from-bucket --apply")
    ap.add_argument("--crq-csv", dest="local_csv", default=None,
                    help="Local CRQ tickets CSV file (dev)")
    args = ap.parse_args()

    if not (args.from_bucket or args.local_csv):
        ap.error("provide --from-bucket or --crq-csv")
    pipeline.ingest(DATASET, args)


if __name__ == "__main__":
    main()