248 lines
14 KiB
HTML
248 lines
14 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||
<title>FleetFuel — Deployment & Architecture</title>
|
||
<style>
|
||
:root{
|
||
--bg:#15110c; --panel:#1d1812; --panel2:#241d15; --ink:#f3ebdd; --muted:#b8a98f;
|
||
--line:#3a2f22; --accent:#e8913c; --accent2:#f0b454; --green:#7fb069; --blue:#6aa9d8; --code:#120e09;
|
||
}
|
||
*{box-sizing:border-box}
|
||
html{scroll-behavior:smooth}
|
||
body{margin:0; background:var(--bg); color:var(--ink);
|
||
font:16px/1.65 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;
|
||
-webkit-font-smoothing:antialiased;}
|
||
.wrap{max-width:980px; margin:0 auto; padding:48px 24px 96px}
|
||
header.hero{border:1px solid var(--line); border-radius:16px; padding:32px;
|
||
background:linear-gradient(135deg,#221a11,#1a140d); box-shadow:0 10px 40px rgba(0,0,0,.35);}
|
||
.kicker{color:var(--accent2); font-weight:700; letter-spacing:.14em; text-transform:uppercase; font-size:12px}
|
||
h1{margin:.3em 0 .15em; font-size:34px; letter-spacing:-.02em}
|
||
.sub{color:var(--muted); font-size:16px; margin:0}
|
||
.crumbs{margin-top:14px; font-size:13px; color:var(--muted)}
|
||
.crumbs a{color:var(--accent2); text-decoration:none}
|
||
h2{margin:46px 0 14px; font-size:23px; padding-bottom:8px; border-bottom:1px solid var(--line);
|
||
display:flex; align-items:center; gap:12px;}
|
||
h2 .num{flex:0 0 auto; width:30px; height:30px; border-radius:8px; background:var(--accent);
|
||
color:#1a120a; font-size:15px; font-weight:800; display:grid; place-items:center;}
|
||
h3{margin:24px 0 8px; font-size:17px; color:var(--accent2)}
|
||
p{margin:.6em 0}
|
||
code{background:var(--code); color:#f0c992; padding:.12em .42em; border-radius:5px;
|
||
font:13.5px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;}
|
||
pre{background:var(--code); border:1px solid var(--line); border-radius:10px; padding:16px;
|
||
overflow:auto; font:13px/1.6 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; color:#d8c7a6;}
|
||
pre code{background:none; padding:0; color:inherit}
|
||
pre .c{color:#6f6147}
|
||
ul,ol{margin:.5em 0; padding-left:1.3em} li{margin:.34em 0}
|
||
a{color:var(--accent2)}
|
||
.note{border-left:3px solid var(--accent); background:var(--panel2); padding:12px 16px;
|
||
border-radius:0 10px 10px 0; margin:18px 0;}
|
||
.note.warn{border-left-color:#e0683c}
|
||
table{width:100%; border-collapse:collapse; margin:14px 0; font-size:14px}
|
||
th,td{text-align:left; padding:10px 12px; border-bottom:1px solid var(--line); vertical-align:top}
|
||
th{color:var(--accent2); font-size:12px; text-transform:uppercase; letter-spacing:.06em}
|
||
td code{font-size:12.5px}
|
||
.diagram{background:#120e09; border:1px solid var(--line); border-radius:14px; padding:22px 14px; margin:18px 0; overflow-x:auto}
|
||
.diagram svg{display:block; margin:0 auto; min-width:920px}
|
||
.legend{display:flex; gap:18px; flex-wrap:wrap; justify-content:center; margin-top:6px; font-size:12.5px; color:var(--muted)}
|
||
.legend span{display:inline-flex; align-items:center; gap:6px}
|
||
.dot{width:11px; height:11px; border-radius:3px; display:inline-block}
|
||
footer{margin-top:60px; color:var(--muted); font-size:13px; text-align:center; border-top:1px solid var(--line); padding-top:20px}
|
||
.tag{display:inline-block; font-size:11px; font-weight:700; padding:2px 8px; border-radius:6px}
|
||
.tag.new{background:rgba(127,176,105,.18); color:var(--green)}
|
||
.tag.edit{background:rgba(232,145,60,.18); color:var(--accent2)}
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<div class="wrap">
|
||
|
||
<header class="hero">
|
||
<div class="kicker">Deployment & Architecture · 17_fleetfuel</div>
|
||
<h1>FleetFuel — How it runs</h1>
|
||
<p class="sub">From the RustFS <code>fuel</code> bucket to the FleetOps Fuel Log tab — the data flow, where each piece runs on the Coolify VPS, and the runbook to deploy it.</p>
|
||
<div class="crumbs">See also: <a href="plan.html">plan.html</a> (implementation plan) · host: <code>twala.rahamafresh.com</code></div>
|
||
</header>
|
||
|
||
<h2><span class="num">1</span>Solution flow</h2>
|
||
<p>Five stages. The WhatsApp fuel feed is exported to object storage by n8n; FleetFuel pulls it, normalizes
|
||
and stores it, the API reads it back, and the SPA renders it.</p>
|
||
|
||
<div class="diagram">
|
||
<svg viewBox="0 0 940 360" role="img" aria-label="FleetFuel data flow diagram">
|
||
<defs>
|
||
<marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse">
|
||
<path d="M0,0 L10,5 L0,10 z" fill="#e8913c"/>
|
||
</marker>
|
||
<style>
|
||
.box{fill:#1d1812; stroke:#3a2f22; stroke-width:1.5; rx:12}
|
||
.t{fill:#f3ebdd; font:600 14px -apple-system,Segoe UI,Roboto,sans-serif}
|
||
.s{fill:#b8a98f; font:11px ui-monospace,Menlo,monospace}
|
||
.lbl{fill:#f0b454; font:11px ui-monospace,Menlo,monospace}
|
||
.flow{stroke:#e8913c; stroke-width:2; fill:none}
|
||
.own{fill:#7fb069; font:700 10px -apple-system,sans-serif}
|
||
.own2{fill:#6aa9d8; font:700 10px -apple-system,sans-serif}
|
||
</style>
|
||
</defs>
|
||
|
||
<!-- source -->
|
||
<g transform="translate(20,140)">
|
||
<rect class="box" width="160" height="80" rx="12"/>
|
||
<text class="t" x="14" y="28">RustFS · bucket</text>
|
||
<text class="lbl" x="14" y="48"><tspan>fuel</tspan></text>
|
||
<text class="s" x="14" y="66">latest.json + changes/</text>
|
||
</g>
|
||
<text class="own" x="20" y="234">n8n CDC of logistics_department.fuel_records</text>
|
||
|
||
<!-- ingest -->
|
||
<g transform="translate(230,130)">
|
||
<rect class="box" width="170" height="100" rx="12"/>
|
||
<text class="t" x="14" y="26">fleetfuel</text>
|
||
<text class="s" x="14" y="46">run_migrations.py</text>
|
||
<text class="s" x="14" y="64">import_fuel.py</text>
|
||
<text class="s" x="14" y="82">--snapshot --apply</text>
|
||
</g>
|
||
<text class="own" x="230" y="248">Coolify cron · hourly</text>
|
||
|
||
<!-- db -->
|
||
<g transform="translate(450,120)">
|
||
<rect class="box" width="180" height="120" rx="12"/>
|
||
<text class="t" x="14" y="26">tracksolid_db</text>
|
||
<text class="lbl" x="14" y="46">fuel.records</text>
|
||
<text class="s" x="14" y="64">+ normalizers/trigger</text>
|
||
<text class="lbl" x="14" y="86">reporting.v_fuel_fills</text>
|
||
<text class="lbl" x="14" y="104">reporting.v_fuel_efficiency</text>
|
||
</g>
|
||
<text class="own" x="450" y="258">timescale_db container</text>
|
||
|
||
<!-- api -->
|
||
<g transform="translate(680,130)">
|
||
<rect class="box" width="170" height="100" rx="12"/>
|
||
<text class="t" x="14" y="26">dashboard_api</text>
|
||
<text class="s" x="14" y="46">/analytics/fuel-fills</text>
|
||
<text class="s" x="14" y="64"> …/recent</text>
|
||
<text class="s" x="14" y="82">/analytics/filters</text>
|
||
</g>
|
||
<text class="own2" x="680" y="248">fleetapi.rahamafresh.com</text>
|
||
|
||
<!-- spa (second row, under api) -->
|
||
<g transform="translate(680,280)">
|
||
<rect class="box" width="170" height="60" rx="12"/>
|
||
<text class="t" x="14" y="26">FleetOps SPA</text>
|
||
<text class="s" x="14" y="46">“Fuel Log” tab</text>
|
||
</g>
|
||
<text class="own2" x="470" y="318">fleetops.rahamafresh.com →</text>
|
||
|
||
<!-- arrows -->
|
||
<line class="flow" x1="182" y1="180" x2="226" y2="180" marker-end="url(#arrow)"/>
|
||
<text class="lbl" x="170" y="170">S3 GET</text>
|
||
<line class="flow" x1="402" y1="180" x2="446" y2="180" marker-end="url(#arrow)"/>
|
||
<text class="lbl" x="392" y="170">upsert</text>
|
||
<line class="flow" x1="632" y1="180" x2="676" y2="180" marker-end="url(#arrow)"/>
|
||
<text class="lbl" x="628" y="170">SELECT</text>
|
||
<!-- api -> spa -->
|
||
<line class="flow" x1="765" y1="232" x2="765" y2="278" marker-end="url(#arrow)"/>
|
||
<text class="lbl" x="772" y="262">HTTP</text>
|
||
</svg>
|
||
<div class="legend">
|
||
<span><i class="dot" style="background:#7fb069"></i>owned by fleetfuel / data-plane</span>
|
||
<span><i class="dot" style="background:#6aa9d8"></i>read-side (existing repos, edited)</span>
|
||
<span><i class="dot" style="background:#e8913c"></i>data flow</span>
|
||
</div>
|
||
</div>
|
||
|
||
<div class="note">The middle three stages all live in the <b>same shared <code>tracksolid_db</code></b>. FleetFuel
|
||
only <i>owns</i> the <code>fuel</code> schema + the two <code>reporting.*</code> views; the API and SPA are existing
|
||
repos that gained a fuel surface. This is the same module shape as <code>fleettickets</code>.</div>
|
||
|
||
<h2><span class="num">2</span>Where it runs (Coolify VPS)</h2>
|
||
<p>Host <code>twala.rahamafresh.com</code> runs everything under <b>Coolify</b>. The tracksolid stack (Coolify
|
||
app <code>bo3nov2ija7g8wn9b1g2paxs</code>) and the RustFS server are co-located:</p>
|
||
<table>
|
||
<thead><tr><th>Container</th><th>Role</th></tr></thead>
|
||
<tbody>
|
||
<tr><td><code>timescale_db-…</code></td><td>Postgres 16 + TimescaleDB — the shared <code>tracksolid_db</code> (target of the migration + ingest)</td></tr>
|
||
<tr><td><code>dashboard_api-…</code> / <code>dashboard_api_staging</code></td><td>FastAPI read-API (<code>fleetapi.rahamafresh.com</code> / staging)</td></tr>
|
||
<tr><td><code>ingest_worker-…</code>, <code>webhook_receiver-…</code></td><td>Tracksolid telematics ingestion (poll + push)</td></tr>
|
||
<tr><td><code>db_backup-…</code></td><td>Nightly DB dump → RustFS (already holds <code>RUSTFS_*</code> env)</td></tr>
|
||
<tr><td><code>rustfs-…</code></td><td>The S3-compatible object store serving the <code>fuel</code> bucket</td></tr>
|
||
<tr><td><code>forgejo-runner</code></td><td>CI runner for <code>repo.rahamafresh.com</code> (auto-deploys on push)</td></tr>
|
||
</tbody>
|
||
</table>
|
||
<p>FleetFuel deploys as a <b>new Coolify resource</b> in the same project so it shares the internal Docker
|
||
network and can reach <code>timescale_db:5432</code> and the RustFS endpoint.</p>
|
||
|
||
<h2><span class="num">3</span>Deployment runbook</h2>
|
||
|
||
<h3>3a · First-time: schema + backfill</h3>
|
||
<p>One-off, to create the <code>fuel</code> schema and load the full history.</p>
|
||
<pre><code><span class="c"># on the VPS, in the fleetfuel checkout (pulled from repo.rahamafresh.com)</span>
|
||
uv sync <span class="c"># installs psycopg2-binary + boto3</span>
|
||
cp .env.example .env && $EDITOR .env <span class="c"># DATABASE_URL + RUSTFS_* (see §4)</span>
|
||
|
||
python run_migrations.py <span class="c"># creates fuel.* + reporting.v_fuel_fills/efficiency (idempotent)</span>
|
||
python import_fuel.py --snapshot <span class="c"># DRY-RUN: parse + log counts, writes nothing</span>
|
||
python import_fuel.py --snapshot --apply <span class="c"># full reconcile from fuel_records/latest.json</span></code></pre>
|
||
|
||
<h3>3b · Recurring: the ingest cron</h3>
|
||
<p>A scheduled container that keeps the DB current. The <code>--snapshot</code> full-reconcile is idempotent and
|
||
self-healing (picks up edits + soft-deletes), so a simple hourly run is safe:</p>
|
||
<pre><code><span class="c"># hourly, matching the n8n export cadence</span>
|
||
python run_migrations.py && python import_fuel.py --snapshot --apply</code></pre>
|
||
<p>Lower-latency alternative: <code>import_fuel.py --changes --apply</code> processes only new
|
||
<code>fuel_records/changes/*.json</code> since the watermark in <code>fuel.ingest_state</code>.</p>
|
||
|
||
<h3>3c · Read-side: API + SPA <span class="tag edit">edit</span></h3>
|
||
<ol>
|
||
<li>Commit + push the <code>dashboard_api_rev.py</code> change (new <code>/analytics/fuel-fills</code> endpoints) on the
|
||
tracksolid repo → Coolify auto-deploys via the Forgejo webhook.</li>
|
||
<li>Commit + push the <code>15_fleetops/src/index.html</code> change (Fuel Log tab) → Coolify rebuilds the SPA image.</li>
|
||
<li>Promotion path per repo: feature → <code>staging</code> → <code>main</code>.</li>
|
||
</ol>
|
||
<div class="note">Order matters: the migration (3a) must run <b>before</b> the API deploy hits
|
||
<code>/analytics/fuel-fills</code>. The extended <code>/analytics/filters</code> is savepoint-guarded, so deploying the
|
||
API before the migration won’t break the existing Logistics dropdowns — the fuel dropdowns just stay empty
|
||
until the view exists.</div>
|
||
|
||
<h2><span class="num">4</span>Configuration</h2>
|
||
<table>
|
||
<thead><tr><th>Env var</th><th>Value / source</th></tr></thead>
|
||
<tbody>
|
||
<tr><td><code>DATABASE_URL</code></td><td><code>postgresql://tracksolid_owner:<pw>@timescale_db:5432/tracksolid_db</code> (internal Docker host)</td></tr>
|
||
<tr><td><code>RUSTFS_ENDPOINT</code></td><td><code>https://s3.rahamafresh.com</code></td></tr>
|
||
<tr><td><code>RUSTFS_ACCESS_KEY</code> / <code>RUSTFS_SECRET_KEY</code></td><td>RustFS keypair (same store the <code>db_backup</code> service uses)</td></tr>
|
||
<tr><td><code>RUSTFS_REGION</code></td><td><code>us-east-1</code></td></tr>
|
||
<tr><td><code>FUEL_BUCKET</code></td><td><code>fuel</code></td></tr>
|
||
</tbody>
|
||
</table>
|
||
<div class="note warn"><b>Secret hygiene:</b> the real <code>.env</code> is gitignored and never committed. The RustFS key
|
||
used during development was shared in plaintext and should be <b>rotated</b>.</div>
|
||
|
||
<h2><span class="num">5</span>Verification</h2>
|
||
<ol>
|
||
<li><b>Schema:</b> <code>\dt fuel.*</code> shows <code>fuel.records</code> / <code>fuel.ingest_state</code>;
|
||
<code>\dv reporting.v_fuel_*</code> shows the two views.</li>
|
||
<li><b>Data:</b> <code>SELECT count(*), count(*) FILTER (WHERE deleted_at IS NULL) FROM fuel.records;</code>
|
||
(≈1922 / ≈1888 at first load).</li>
|
||
<li><b>Join health:</b> <code>SELECT count(*) FILTER (WHERE vehicle_number IS NOT NULL) FROM reporting.v_fuel_fills;</code>
|
||
— the plate-match rate against <code>tracksolid.devices</code>.</li>
|
||
<li><b>API:</b> <code>curl "https://fleetapi.rahamafresh.com/analytics/fuel-fills?period=90d"</code> →
|
||
<code>totals</code> / <code>rows</code> / <code>by_department</code> / <code>trend</code> populated;
|
||
<code>/analytics/filters</code> includes <code>departments</code>.</li>
|
||
<li><b>SPA:</b> open FleetOps → <b>Fuel Log</b> tab → KPI strip, spend/litres chart, per-vehicle & recent tables render.</li>
|
||
</ol>
|
||
|
||
<h2><span class="num">6</span>Rollback</h2>
|
||
<ul>
|
||
<li><b>Read-side:</b> revert the two commits (dashboard_api + fleetops); Coolify redeploys the prior image.</li>
|
||
<li><b>Data:</b> the <code>fuel</code> schema is additive and isolated — nothing else reads it, so it can be left
|
||
in place or dropped (<code>DROP SCHEMA fuel CASCADE;</code> + <code>DROP VIEW reporting.v_fuel_fills, reporting.v_fuel_efficiency;</code>)
|
||
with no impact on the existing Logistics/Tickets surfaces.</li>
|
||
<li><b>Cron:</b> pause/stop the FleetFuel Coolify resource — ingestion halts, existing rows remain.</li>
|
||
</ul>
|
||
|
||
<footer>FleetFuel deployment & architecture · generated 2026-06-11 · companion to <a href="plan.html">plan.html</a></footer>
|
||
|
||
</div>
|
||
</body>
|
||
</html>
|