david kiania
e5b0e192d8
chore(repo): reorganize tree into migrations/ data/ legacy/ docs/
...
Group root-level files (accreted from incremental changes) by purpose
without moving any deployment entrypoint or breaking imports:
- migrations/ : numbered SQL 02-10
- data/ : source CSVs
- legacy/ : superseded pre-_rev scripts + old pipeline notes (not deployed)
- docs/{manuals,reference,reports}/ : loose manuals, references, reports
- strip stray ** / *** prefixes from 5 doc filenames
- delete empty documents.txt / push_webhook.md
Reference updates so nothing breaks:
- run_migrations.py -> /app/migrations/<file>
- run_migrations.sh -> $SCRIPT_DIR/migrations
- import_drivers_csv.py -> data/<csv>
- docker-compose.yaml -> runbook path comment
- CLAUDE.md -> codebase map + inline doc references
Deployed Python (3 services + ts_shared_rev + run_migrations) and the
documented ops one-shots stay at root, preserving the flat-import layout
and all documented commands. Verified: py_compile clean across all modules,
every MIGRATIONS entry resolves under migrations/, CI-referenced paths
(tests/, mypy targets, db_audit) and the grafana build context intact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 02:27:30 +03:00
David Kiania
5418fc48c5
fix(api): map new Mitieng CSV columns in tracksolid_update_v2
...
Switches column references from the old title-case logistics CSV
(IMEI, Device Name, License Plate No., Telephone, Fuel/100km, ...) to
the snake_case Mitieng export shipped on 2026-04-27 (imei, device_name,
vehicle_number, driver_phone, fuel_100km, ...). Without this, the bulk
device-update API tool fails with KeyError: 'IMEI' on the new CSV.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-27 23:42:20 +03:00
David Kiania
34f5fa1b9c
feat(dwh): bronze pipeline migrations, runbook, and execution manual
...
DWH pipeline (new):
- dwh/261001_dwh_control.sql — watermarks + per-run audit log schema
- dwh/261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
- dwh/261003_dwh_roles.sql — dwh_owner / grafana_ro contract assertion
- dwh/261004_dwh_observability_views.sql — v_table_freshness,
v_recent_failures, v_watermark_lag (readable by grafana_ro)
- docs/DWH_PIPELINE.md — operations runbook (setup, troubleshooting,
manual re-run, back-fill, rotation)
- DWH_Execution_Manual.md — reusable playbook for future data
projects (extract → blob → load pattern, 7 design principles,
snapshot-vs-incremental matrix, verification gates)
- docs/superpowers/{specs,plans}/2026-04-24-n8n-dwh-bronze-pipeline-*
— design spec + 27-task implementation plan
Security:
- dwh/260423_dwh_ddl_v1.sql — redacted plaintext role passwords to
'CHANGE_ME_BEFORE_APPLY' placeholders; added SECURITY header
documenting generation + rotation flow
Docs:
- CLAUDE.md — §3 adds tracksolid_dwh@31.97.44.246:5888 target,
§4 adds dwh/ + docs/DWH_PIPELINE.md to codebase map, §5 adds
bronze + dwh_control schema roll-up, §10 adds deploy task +
password rotation follow-up
Also includes miscellaneous in-progress files accumulated on this
branch (workspace, analytics notes, vehicle CSVs, extract helpers,
renamed markdown archives).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 01:07:53 +03:00
