tracksolid_timescale_grafana_prod/grafanaDeployment.md

# Grafana NOC Fleet Dashboard — Deployment Guide

## Overview

Single-region NOC dashboard monitoring 80 vehicles (Nairobi, Mombasa, Kampala) via Tracksolid/Jimi GPS.
Live location, direction arrows, speed, and driver info. Auto-refreshes every 30 seconds.

---

## Prerequisites

- Docker Compose stack running (`timescale_db`, `grafana`)
- `.env` file with all existing variables
- Add one new variable to `.env`:
  ```
  GRAFANA_DB_RO_PASSWORD=<password for grafana_ro postgres user>
  ```

---

## File Structure

```
grafana/
└── provisioning/
    ├── datasources/
    │   └── tracksolid_postgres.yaml
    ├── dashboards/
    │   └── noc_fleet.yaml
    └── dashboards-json/
        └── noc_fleet_dashboard.json
```

---

## Step 1 — docker-compose.yaml (grafana service)

Replace the existing `grafana` service block with:

```yaml
grafana:
  image: grafana/grafana:11.0.0
  restart: always
  depends_on:
    timescale_db:
      condition: service_healthy
  env_file: .env
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD}
    - GF_USERS_DEFAULT_THEME=dark
    - GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH=/etc/grafana/provisioning/dashboards-json/noc_fleet_dashboard.json
  volumes:
    - grafana-data:/var/lib/grafana
    - ./grafana/provisioning:/etc/grafana/provisioning:ro
```

---

## Step 2 — Datasource: `grafana/provisioning/datasources/tracksolid_postgres.yaml`

```yaml
apiVersion: 1

datasources:
  - name: TracksolidDB
    type: postgres
    uid: tracksolid_pg
    url: timescale_db:5432
    database: tracksolid_db
    user: grafana_ro
    secureJsonData:
      password: ${GRAFANA_DB_RO_PASSWORD}
    jsonData:
      sslmode: disable
      maxOpenConns: 5
      maxIdleConns: 2
      connMaxLifetime: 14400
      postgresVersion: 1600
      timescaledb: true
    editable: false
    isDefault: true
```

> **Note:** `uid: tracksolid_pg` is referenced by the dashboard JSON. Never rename it after deployment.

---

## Step 3 — Dashboard Provider: `grafana/provisioning/dashboards/noc_fleet.yaml`

```yaml
apiVersion: 1

providers:
  - name: NOC Fleet Dashboards
    orgId: 1
    folder: NOC
    type: file
    disableDeletion: true
    updateIntervalSeconds: 30
    allowUiUpdates: false
    options:
      path: /etc/grafana/provisioning/dashboards-json
```

---

## Step 4 — Dashboard JSON: `grafana/provisioning/dashboards-json/noc_fleet_dashboard.json`

### Dashboard Settings

| Setting | Value |
|---------|-------|
| UID | `noc-fleet-live` |
| Auto-refresh | 30s |
| Theme | Dark |
| Schema version | 39 (Grafana 11.0.0) |
| Editable | false |
| Bookmark URL | `/d/noc-fleet-live` |

---

### Panel Layout

| Row | Panel | Type | Notes |
|-----|-------|------|-------|
| 1 | Total Vehicles | Stat | Count all enabled devices |
| 1 | Online Now | Stat | GPS fix < 5 min — green |
| 1 | Recent (5-30m) | Stat | GPS fix 5-30 min — amber |
| 1 | Offline | Stat | GPS fix > 30 min — red |
| 1 | Moving Now | Stat | speed > 0 AND acc on |
| 1 | Avg Speed (km/h) | Stat | Moving vehicles only |
| 2 | Live Vehicle Locations | Geomap | Direction arrows, color by plate |
| 3 | Vehicle Status Table | Table | All vehicles, sorted Online first |
| 4 | Ingestion Health | Table | Collapsed by default |

---

### SQL Queries

**Total Vehicles**
```sql
SELECT COUNT(*) AS "Total Vehicles"
FROM tracksolid.devices WHERE enabled_flag = 1;
```

**Online Now**
```sql
SELECT COUNT(*) AS "Online"
FROM tracksolid.v_fleet_status
WHERE connectivity_status = 'online';
```

**Recent (5-30 min)**
```sql
SELECT COUNT(*) AS "Recent"
FROM tracksolid.v_fleet_status
WHERE connectivity_status = 'recent';
```

**Offline**
```sql
SELECT COUNT(*) AS "Offline"
FROM tracksolid.v_fleet_status
WHERE connectivity_status = 'offline';
```

**Moving Now**
```sql
SELECT COUNT(*) AS "Moving"
FROM tracksolid.v_fleet_status
WHERE speed > 0 AND acc_status = '1' AND connectivity_status = 'online';
```

**Avg Speed**
```sql
SELECT ROUND(AVG(speed)::numeric, 1) AS "Avg Speed km/h"
FROM tracksolid.v_fleet_status
WHERE speed > 0 AND acc_status = '1' AND connectivity_status = 'online';
```

**Geomap — Live Vehicle Locations**
```sql
SELECT
    d.imei,
    d.vehicle_number,
    d.vehicle_name,
    d.driver_name,
    d.driver_phone,
    d.city,
    d.device_group,
    lp.lat,
    lp.lng,
    lp.speed,
    lp.direction,
    lp.acc_status,
    lp.loc_desc,
    lp.gps_time,
    CASE
        WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes'  THEN 'online'
        WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 'recent'
        ELSE 'offline'
    END AS connectivity_status
FROM tracksolid.devices d
INNER JOIN tracksolid.live_positions lp USING (imei)
WHERE d.enabled_flag = 1
  AND lp.lat IS NOT NULL
  AND lp.lng IS NOT NULL
ORDER BY d.vehicle_number;
```

> INNER JOIN — only vehicles with a valid GPS fix appear on the map.
> `acc_status` is TEXT ('0'/'1') in the schema — all queries use string comparisons.

**Vehicle Status Table**
```sql
SELECT
    d.vehicle_number                                       AS "Plate",
    d.vehicle_name                                         AS "Vehicle",
    d.driver_name                                          AS "Driver",
    d.driver_phone                                         AS "Driver Phone",
    d.city                                                 AS "City",
    ROUND(lp.speed::numeric, 0)                           AS "Speed (km/h)",
    lp.loc_desc                                            AS "Last Location",
    lp.gps_time                                            AS "Last Fix",
    CASE
        WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes'  THEN 'Online'
        WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 'Recent'
        ELSE 'Offline'
    END AS "Status",
    EXTRACT(EPOCH FROM (NOW() - lp.gps_time))::int / 60  AS "Min Since Fix"
FROM tracksolid.devices d
LEFT JOIN tracksolid.live_positions lp USING (imei)
WHERE d.enabled_flag = 1
ORDER BY
    CASE
        WHEN lp.gps_time >= NOW() - INTERVAL '5 minutes'  THEN 0
        WHEN lp.gps_time >= NOW() - INTERVAL '30 minutes' THEN 1
        ELSE 2
    END,
    d.vehicle_number;
```

> LEFT JOIN — offline vehicles (no GPS fix) still appear in the table showing their null status.

**Ingestion Health**
```sql
SELECT
    endpoint                                AS "Endpoint",
    TO_CHAR(run_at, 'HH24:MI DD-Mon')      AS "Last Run",
    CASE WHEN success THEN 'OK' ELSE 'FAIL' END AS "Result",
    error_message                           AS "Error",
    seconds_ago                             AS "Lag (s)"
FROM tracksolid.v_ingestion_health
ORDER BY endpoint;
```

---

### Geomap Configuration

| Property | Value |
|----------|-------|
| Basemap | Carto Dark |
| Location mode | `coords` — fields `lat` / `lng` |
| Marker symbol | `img/icons/marker/arrow-up.svg` (built-in Grafana 11) |
| Rotation | field: `direction` (0° = North, clockwise) |
| Color | field: `vehicle_number`, scheme: `palette-classic-by-name` |
| Tooltip hidden fields | `lat`, `lng`, `imei` |
| Direction unit override | `degree` (shows `245°` in tooltip) |
| Initial view | East Africa center — auto-fits to vehicle positions |

### Connectivity Status Color Mapping

| Value | Color |
|-------|-------|
| Online / online | Green |
| Recent / recent | Amber |
| Offline / offline | Red |

### Stat Panel Thresholds

| Panel | Green | Amber | Red |
|-------|-------|-------|-----|
| Online Now | > 0 | — | = 0 |
| Recent | — | > 0 | — |
| Offline | = 0 | — | > 0 |
| Moving Now | > 0 | — | — |
| Avg Speed | < 80 km/h | 80-120 | > 120 |

All stat panels: `colorMode: background`, `graphMode: none`.

---

## Step 5 — Deploy

```bash
docker compose up -d grafana
```

---

## Step 6 — Verify

1. Open `http://localhost:3000/d/noc-fleet-live`
2. Map renders with arrow markers on vehicle positions
3. Arrows rotate per heading (northbound vehicle = arrow pointing up)
4. Tooltip shows: plate, driver name, speed, city, connectivity status
5. Table sorts: Online → Recent → Offline
6. 30s auto-refresh fires (watch "Last Fix" timestamps)
7. Check provisioning logs:
   ```bash
   docker compose logs grafana | grep -i provision
   ```

---

## Notes

- The `grafana/` directory must be committed to git — Coolify's clone must include it
- `:ro` mounts prevent Grafana from writing back into the repo
- `allowUiUpdates: false` in the provider YAML means UI edits are not persisted — the JSON file is the source of truth
- `acc_status` in `live_positions` is TEXT (`'0'`/`'1'`), not integer — all queries use string comparisons
- `uid: tracksolid_pg` in the datasource YAML is referenced by the dashboard JSON — never rename it after deployment
- ServiceNow ticket integration is deferred to a future phase