kianiadee/tracksolid_timescale_grafana_prod
1
0
Fork 0
Code Issues Pull requests 4 Projects Releases Packages Wiki Activity Actions 7
Tracksolid deployment with timescale & grafana with backup
78 commits 8 branches 0 tags 2.5 MiB
Python 64.6%
PLpgSQL 32.7%
Shell 2.3%
Dockerfile 0.4%
Find a file
David Kiania e811dd8f34
Some checks are pending
Static Analysis / static (push) Waiting to run
Details
Tests / test (push) Waiting to run
Details
feat(infra): add pgbouncer sidecar to cap tracksolid_db connections 
Phase 1 of the pgbouncer + pgAdmin rollout (runbook:
260507_pgbouncer_deployment.md). pgAdmin4 on the maintainer's laptop has
been exhausting tracksolid_db's max_connections, cascading to pgcli and
operations. Adds an internal-only pgbouncer service in transaction mode
with a small backend pool (default 15) so admin-tool sprawl can no
longer starve the ingest pipeline.

No client cutover this round - ingest, Grafana, webhook, and backup all
keep talking to timescale_db:5432 directly. SCRAM passthrough is wired
via a new pgbouncer role + public.user_lookup() function (migration 10).
The role is created with a placeholder password; sync_role_passwords()
in run_migrations.py replaces it from PGBOUNCER_AUTH_PASSWORD on every
container startup, mirroring the existing grafana_ro convention.

Requires PGBOUNCER_AUTH_PASSWORD to be set in .env before deploy.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 13:21:35 +03:00
.claude feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
.forgejo/workflows feat: add db_audit health checks, runner, and scheduled Forgejo workflow 2026-04-12 21:40:29 +03:00
administration Add DB connection string to ops manual, add administration notes, remove stale deploy guide 2026-04-10 22:34:56 +03:00
backup feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
db_audit feat: add db_audit health checks, runner, and scheduled Forgejo workflow 2026-04-12 21:40:29 +03:00
docs feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
dwh feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
grafana fix(grafana): raise geomap maxZoom from 12 to 22 for full-resolution drill-in 2026-04-27 18:32:14 +03:00
n8n-workflows Add n8n workflow templates and change webhook port to 8888 2026-04-08 18:54:42 +03:00
tests perf+fix: SAVEPOINT-per-item pollers, batched GPS inserts, parallel detail fetch 2026-04-18 00:33:55 +03:00
***tracksolid_DB_manual.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**01_BusinessAnalytics.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**02_tracksolid_docker_commands.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**260410_baseline_report.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
**OPERATIONS_MANUAL.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
.env fix: point DATABASE_URL at timescale_db container (not legacy 31.97.44.246:5888) 2026-04-18 15:43:49 +03:00
.gitignore Add webhook receiver, consolidate shared utilities, expand telemetry coverage 2026-04-08 16:31:17 +03:00
.python-version chore: align .python-version to 3.12.0 (matches Docker image and pyproject.toml) 2026-04-12 21:41:43 +03:00
02_tracksolid_full_schema_rev.sql Add webhook receiver, consolidate shared utilities, expand telemetry coverage 2026-04-08 16:31:17 +03:00
03_webhook_schema_migration.sql Add webhook receiver, consolidate shared utilities, expand telemetry coverage 2026-04-08 16:31:17 +03:00
04_bug_fix_migration.sql Fix alarm field mapping, distance unit bug, parking params; add schema migrations 2026-04-10 22:18:30 +03:00
05_enhancement_migration.sql Fix alarm field mapping, distance unit bug, parking params; add schema migrations 2026-04-10 22:18:30 +03:00
06_business_analytics_migration.sql feat: business analytics expansion + driver CSV import 2026-04-18 08:30:34 +03:00
07_analytics_views.sql feat: Daily Operations dashboard + tracksolid analytics views 2026-04-19 13:44:18 +03:00
08_analytics_config.sql feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
09_trips_enrichment.sql feat(trips): [FIX-M20] enrich tracksolid.trips with coords, route polyline, addresses, plate 2026-05-01 21:30:20 +03:00
10_driver_clock_views.sql feat(analytics): add v_driver_clock_daily/today views for tardiness monitoring 2026-05-04 14:03:40 +03:00
10_pgbouncer_auth.sql feat(infra): add pgbouncer sidecar to cap tracksolid_db connections 2026-05-07 13:21:35 +03:00
55_ts_coolify_gemini_prod.code-workspace feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
260412_baseline_report.md Add 260412 baseline report — first trip data, FIX-M16 confirmed 2026-04-12 00:14:27 +03:00
260427_audit_output.txt feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
260427_device_reconciliation.md feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
260507_pgbouncer_deployment.md feat(infra): add pgbouncer sidecar to cap tracksolid_db connections 2026-05-07 13:21:35 +03:00
20260414_FS__Logistics - final_fixed.csv fix: auto-register devices on push + allow CSV import to insert new rows 2026-04-21 12:29:32 +03:00
20260427_FSG_Vehicles_mitieng.csv feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
audit_device_reconciliation.py feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
backfill_trips_enrichment.py feat(trips): add --skip-geocode flag to backfill script 2026-05-01 22:12:07 +03:00
CLAUDE.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
connecting_python_tracksolid.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
docker-compose.yaml feat(infra): add pgbouncer sidecar to cap tracksolid_db connections 2026-05-07 13:21:35 +03:00
Dockerfile Fix migration failures: switch to full TimescaleDB + use psql runner 2026-04-08 17:17:58 +03:00
documents.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
DWH_Execution_Manual.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
fireside_logistics_cleaned_v2.csv feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
grafanaDeployment.md Add Grafana NOC fleet dashboard with provisioning 2026-04-09 00:01:52 +03:00
grafanaOperationalManual.md Add Grafana NOC operational manual 2026-04-09 00:12:48 +03:00
import_drivers_csv.py feat(analytics): Phase 0 — analytics-config migration and CSV importer rewrite 2026-04-27 23:42:37 +03:00
ingest_events_rev.py perf+fix: SAVEPOINT-per-item pollers, batched GPS inserts, parallel detail fetch 2026-04-18 00:33:55 +03:00
ingest_movement_rev.py feat(trips): [FIX-M20] enrich tracksolid.trips with coords, route polyline, addresses, plate 2026-05-01 21:30:20 +03:00
new_feature.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
push_webhook.md feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
pyproject.toml ci: add ruff + mypy static analysis config and Forgejo workflow 2026-04-12 21:32:33 +03:00
README.md first commit 2026-04-07 20:41:16 +03:00
run_migrations.py feat(infra): add pgbouncer sidecar to cap tracksolid_db connections 2026-05-07 13:21:35 +03:00
run_migrations.sh Add idempotent migration runner script 2026-04-10 23:31:57 +03:00
sync_driver_audit.py perf+fix: SAVEPOINT-per-item pollers, batched GPS inserts, parallel detail fetch 2026-04-18 00:33:55 +03:00
tracksolid_analytics_pipeline.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolid_extract.py feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolid_ingestion_pipeline.txt feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolid_update_v2.py fix(api): map new Mitieng CSV columns in tracksolid_update_v2 2026-04-27 23:42:20 +03:00
tracksolid_vehicle_update.py feat(dwh): bronze pipeline migrations, runbook, and execution manual 2026-04-25 01:07:53 +03:00
tracksolidApiDocumentation.md Update tracksolidApiDocumentation.md with live implementation findings 2026-04-11 07:52:28 +03:00
ts_shared_rev.py feat(trips): [FIX-M20] enrich tracksolid.trips with coords, route polyline, addresses, plate 2026-05-01 21:30:20 +03:00
webhook_receiver_rev.py fix: auto-register devices on push + allow CSV import to insert new rows 2026-04-21 12:29:32 +03:00

README.md