fleettickets/n8n-s3-export-workflows.md

n8n S3 Export Workflows

Overview

Both workflows run in the Africa/Nairobi timezone and use the Rahamafresh Tickets S3 credential.

Workflow	n8n ID	Source	Bucket
Fuel Records S3 Export	IP2KNAfFazAjTesh	logistics_department.fuel_records	fuel
FTTH Automation Ticket S3 Export	JI3QkcJeHk9eYRsY	isp_department_crq.automations and isp_department_osp.automations	tickets

Schedules

• Hourly delta export: 10 minutes after each hour, from 01:10 through 23:10 (0 10 1-23 * * *).
• Daily full export: 00:05 (0 5 0 * * *).
• The 00:05 run exports rows up to the end of the previous local day. For example, Wednesday's run exports the snapshot through Tuesday 23:59:59.
• The full export has its own state and does not read or advance the hourly delta pointer.

Fuel Records S3 Export

Source and change tracking

The source is logistics_department.fuel_records. The table has an indexed updated_at column and an update trigger that refreshes it whenever a row is changed.

Hourly runs select rows where:

updated_at > last_successful_delta_export_at
AND updated_at <= requested_at

The delta pointer advances to the maximum exported updated_at only after all S3 uploads complete and the downloaded latest.json passes validation. A failed or empty run does not move the pointer incorrectly.

The full run selects all rows created before local midnight, independently of the delta pointer.

Fuel object keys

Every successful run updates:

• fuel_records/latest.json
• fuel_records/latest.csv

Delta runs with changed rows also write:

• fuel_records/changes/YYYY-MM-DDTHH-mm-ss.json
• fuel_records/changes/YYYY-MM-DDTHH-mm-ss.csv

Full runs write:

• fuel_records/full/YYYY-MM-DD.json
• fuel_records/full/YYYY-MM-DD.csv

Exports larger than 5,000 rows additionally produce numbered JSON and CSV parts such as -part-0001.

Fuel state

Fuel state is stored in n8n workflow static data and is updated only after S3 read-back validation:

• last_successful_delta_export_at
• last_successful_full_export_date
• rows_exported
• destination_key and destination_keys
• n8n_execution_id
• success
• error_message
• completed_at

FTTH Automation Ticket S3 Export

Source and change tracking

The workflow requests an export package from the active SCOREBOARD service:

POST /api/v1/ftth/automation-export/package

Datasets:

• CRQ: isp_department_crq.automations
• INC: isp_department_osp.automations

Hourly delta packages select records changed after the last successful delta export. The SCOREBOARD service creates an export run before returning the package. n8n uploads every returned object and then calls:

POST /api/v1/ftth/automation-export/mark-sent

Only a SUCCESS acknowledgement advances the delta pointer. Upload failure marks the run FAILED and preserves the previous pointer.

Full packages use the previous local date as snapshot_date, select the complete current-state dataset through the previous day, and update only the full-export date after successful upload.

FTTH object keys

CRQ:

• automations/crq/latest.json
• automations/crq/latest.csv
• automations/crq/changes/YYYY-MM-DDTHH-mm-ss.json
• automations/crq/changes/YYYY-MM-DDTHH-mm-ss.csv
• automations/crq/full/YYYY-MM-DD.json
• automations/crq/full/YYYY-MM-DD.csv

INC:

• automations/inc/latest.json
• automations/inc/latest.csv
• automations/inc/changes/YYYY-MM-DDTHH-mm-ss.json
• automations/inc/changes/YYYY-MM-DDTHH-mm-ss.csv
• automations/inc/full/YYYY-MM-DD.json
• automations/inc/full/YYYY-MM-DD.csv

Exports larger than 5,000 rows additionally produce numbered JSON and CSV parts.

FTTH state

State and audit history are stored in ftth_automation.automation_export_runs. Each run records:

• export type and requested timestamp
• last successful delta timestamp
• last successful full export date
• snapshot date
• row count
• destination object keys
• n8n execution ID
• status (PENDING, SUCCESS, or FAILED)
• completion timestamp and error summary

File Contents

JSON files contain:

{
  "metadata": {
    "exported_at": "...",
    "export_type": "delta or full",
    "source_schema": "...",
    "source_table": "...",
    "dataset": "crq, inc, or omitted for fuel",
    "row_count": 0,
    "last_successful_delta_export_at": "...",
    "last_successful_full_export_date": "...",
    "snapshot_date": "...",
    "n8n_execution_id": "..."
  },
  "records": []
}

Fuel metadata names the previous and candidate delta pointers explicitly. CSV files contain the same exported records as tabular rows with a header line. CSV files do not contain the JSON metadata envelope.

Manual Tests

Both workflows expose production POST webhooks.

Fuel:

curl -X POST \
  -H 'Content-Type: application/json' \
  -d '{"export_type":"delta"}' \
  https://n8n.firesideafrica.cloud/webhook/fuel-records-s3-export

curl -X POST \
  -H 'Content-Type: application/json' \
  -d '{"export_type":"full"}' \
  https://n8n.firesideafrica.cloud/webhook/fuel-records-s3-export

FTTH:

curl -X POST \
  -H 'Content-Type: application/json' \
  -d '{"export_type":"delta"}' \
  https://n8n.firesideafrica.cloud/webhook/ftth-automation-s3-export

curl -X POST \
  -H 'Content-Type: application/json' \
  -d '{"export_type":"full","force":true}' \
  https://n8n.firesideafrica.cloud/webhook/ftth-automation-s3-export

After a test:

1. Confirm the webhook response has success: true.
2. Open the execution ID in n8n and confirm every upload succeeded.
3. Confirm the response lists the expected bucket and destination keys.
4. Check the S3 object timestamps and inspect the JSON metadata and row count.
5. For FTTH, confirm the matching export run is SUCCESS.

Production test record

Tests run on June 15, 2026:

Workflow	Type	Execution	Result
Fuel Records S3 Export	Delta	402524	Success; 0 changed rows; latest JSON and CSV validated
Fuel Records S3 Export	Full	402527	Success; 1,965 rows; snapshot date 2026-06-14
FTTH Automation Ticket S3 Export	Delta	402530	Success; CRQ and INC latest/change JSON and CSV written
FTTH Automation Ticket S3 Export	Full	402536	Success; 44,114 rows; snapshot date 2026-06-14; 28 objects including batch parts

Troubleshooting

1. Check the n8n execution and identify whether the source query/package, upload, read-back validation, or mark-sent step failed.
2. Confirm the Rahamafresh Tickets S3 credential can write to the configured bucket.
3. For fuel, inspect workflow static data. Do not manually advance last_successful_delta_export_at after a failed run.
4. Verify fuel_records.updated_at is populated and its update trigger exists if fuel changes are missing.
5. For FTTH, inspect ftth_automation.automation_export_runs, including status, destination_object_keys, n8n_execution_id, and error_summary.
6. Confirm the SCOREBOARD health endpoint is healthy and that the configured export token and base URL are correct.
7. Re-run the appropriate manual webhook after fixing the failure. A failed run leaves the last successful pointer unchanged, so the rows are retried.

Published Version Check

In n8n, open each workflow and confirm:

• Active is enabled.
• The saved versionId equals activeVersionId.
• The trigger list contains the hourly schedule, daily 00:05 schedule, and manual webhook.
• A new production webhook execution uses the same active version and returns the expected destination keys.

Current published versions as of June 15, 2026:

• Fuel Records S3 Export: 6833e5e5-97a0-41be-8f82-9ec612de92ce
• FTTH Automation Ticket S3 Export: b2171088-eac2-439b-97e8-83dfa8117783