34f5fa1b9c
DWH pipeline (new):
- dwh/261001_dwh_control.sql — watermarks + per-run audit log schema
- dwh/261002_bronze_constraints_audit.sql — ON CONFLICT key assertion
- dwh/261003_dwh_roles.sql — dwh_owner / grafana_ro contract assertion
- dwh/261004_dwh_observability_views.sql — v_table_freshness,
v_recent_failures, v_watermark_lag (readable by grafana_ro)
- docs/DWH_PIPELINE.md — operations runbook (setup, troubleshooting,
manual re-run, back-fill, rotation)
- DWH_Execution_Manual.md — reusable playbook for future data
projects (extract → blob → load pattern, 7 design principles,
snapshot-vs-incremental matrix, verification gates)
- docs/superpowers/{specs,plans}/2026-04-24-n8n-dwh-bronze-pipeline-*
— design spec + 27-task implementation plan
Security:
- dwh/260423_dwh_ddl_v1.sql — redacted plaintext role passwords to
'CHANGE_ME_BEFORE_APPLY' placeholders; added SECURITY header
documenting generation + rotation flow
Docs:
- CLAUDE.md — §3 adds tracksolid_dwh@31.97.44.246:5888 target,
§4 adds dwh/ + docs/DWH_PIPELINE.md to codebase map, §5 adds
bronze + dwh_control schema roll-up, §10 adds deploy task +
password rotation follow-up
Also includes miscellaneous in-progress files accumulated on this
branch (workspace, analytics notes, vehicle CSVs, extract helpers,
renamed markdown archives).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
66 lines
2.7 KiB
PL/PgSQL
66 lines
2.7 KiB
PL/PgSQL
-- =============================================================
|
|
-- DWH ROLES AUDIT
|
|
-- Target Database: tracksolid_dwh
|
|
-- Purpose: Assert that the n8n DWH pipeline's role contract holds:
|
|
-- - dwh_owner exists (writes bronze + dwh_control)
|
|
-- - grafana_ro exists (reads bronze + silver + gold + dwh_control)
|
|
-- - grafana_ro has CONNECT on the database
|
|
-- - grafana_ro has USAGE on every schema it needs
|
|
-- Applies after: 260423_dwh_ddl_v1.sql, 261001_dwh_control.sql
|
|
-- Idempotent: pure assertion, no CREATE ROLE or GRANT statements.
|
|
--
|
|
-- Why this file exists: 260423 creates both roles and grants bronze/silver/gold;
|
|
-- 261001 grants dwh_control. This file is a single checkpoint that verifies
|
|
-- those prior migrations were applied in the right order, and fails loudly
|
|
-- if anything is missing before the pipeline goes live.
|
|
--
|
|
-- Password rotation and sslmode=require enforcement are out-of-band:
|
|
-- rotate via ALTER ROLE ... PASSWORD ... in a psql superuser session,
|
|
-- enforce SSL via the n8n credential (sslmode=require) — not SQL-level.
|
|
-- =============================================================
|
|
|
|
BEGIN;
|
|
|
|
DO $$
|
|
DECLARE
|
|
missing TEXT := '';
|
|
r RECORD;
|
|
BEGIN
|
|
-- 1. Roles exist
|
|
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'dwh_owner') THEN
|
|
missing := missing || E'\n - role dwh_owner missing (expected from 260423)';
|
|
END IF;
|
|
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'grafana_ro') THEN
|
|
missing := missing || E'\n - role grafana_ro missing (expected from 260423)';
|
|
END IF;
|
|
|
|
-- 2. grafana_ro CONNECT on this database
|
|
IF NOT has_database_privilege('grafana_ro', current_database(), 'CONNECT') THEN
|
|
missing := missing
|
|
|| format(E'\n - grafana_ro lacks CONNECT on database %s',
|
|
current_database());
|
|
END IF;
|
|
|
|
-- 3. grafana_ro USAGE on every schema the pipeline / dashboards touch
|
|
FOR r IN
|
|
SELECT unnest(ARRAY['bronze','silver','gold','dwh_control']) AS schema_name
|
|
LOOP
|
|
IF NOT EXISTS (SELECT 1 FROM pg_namespace WHERE nspname = r.schema_name) THEN
|
|
missing := missing
|
|
|| format(E'\n - schema %s missing (expected from 260423/261001)',
|
|
r.schema_name);
|
|
ELSIF NOT has_schema_privilege('grafana_ro', r.schema_name, 'USAGE') THEN
|
|
missing := missing
|
|
|| format(E'\n - grafana_ro lacks USAGE on schema %s',
|
|
r.schema_name);
|
|
END IF;
|
|
END LOOP;
|
|
|
|
IF length(missing) > 0 THEN
|
|
RAISE EXCEPTION E'DWH roles audit FAILED:%s', missing;
|
|
END IF;
|
|
|
|
RAISE NOTICE 'DWH roles audit OK: dwh_owner + grafana_ro present with expected grants.';
|
|
END$$;
|
|
|
|
COMMIT;
|