FIELD REPORT 2026 · §2

Pipeline — MQTT → Pi → Aurora

How a sample becomes a row, and how the same sample also becomes a Pi-local SQLite write

IN DEVELOPMENT FILED · 2026-05-07 TAGS · PIPELINE · MQTT · AURORA · LAMBDA · MOSQUITTO
ABSTRACT
ON FILE DRAFT · 2026-05-07

End-to-end path from sensor publish to durable row. Two parallel sinks (edge SQLite + cloud Aurora) consume the same MQTT events, which is what makes the §3 paired-measurement methodology possible. Skeleton; final prose by 2026-08.

§2.1 · Stages

A reading takes the same path on both sides up to the broker, then diverges:

  1. Sample. ESP32 reads its I²C / ADC sensors at the configured cadence.
  2. Publish. Sample is published over Wi-Fi to the Pi-local Mosquitto broker on a topic of the form home/{node_id}/{metric}.
  3. Edge sink. A Pi-local subscriber writes to a SQLite ring buffer and computes derived metrics in-process. No cloud dependency.
  4. Cloud sink. The Pi-local mqtt-bridge subscriber forwards the same MQTT topics to AWS Aurora Serverless v2 via a thin Lambda ingest (POST /home/sensor-data with x-api-key).
  5. Durability. Aurora row is written; daily archival job (separate Lambda, archival-broker) freezes a day’s rows into NDJSON.gz on S3.

DRAFT — sequence diagram pending §2 finalisation.

§2.2 · MQTT contract

TopicDirectionPayload
home/{node_id}/tempESP32 → broker{"value": 22.4, "ts": "..."}
home/{node_id}/humidityESP32 → broker{"value": 47.1, "ts": "..."}
home/{node_id}/moistureESP32 → broker{"value": 41.0, "ts": "..."}
home/{node_id}/cmd/waterbroker → ESP32{"duration_sec": 15}
home/{node_id}/heartbeatESP32 → broker{"uptime": 12345, "rssi": -56}

DRAFT — full topic catalogue + retained-message policy by 2026-07.

§2.3 · Aurora schema (read-only excerpt)

The cloud path’s relevant table is home_sensor_readings. Schema is stable and versioned via @scraper/db’s schema.ts. Public exporter (PLN-003) reads from the same table and emits gap-aware JSON to S3 for the live dashboard.

CREATE TABLE home_sensor_readings (
  id            BIGSERIAL PRIMARY KEY,
  recorded_at   TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  node_id       TEXT NOT NULL,
  stable_id     TEXT,                              -- IDENT-001 phase
  temp_c        DOUBLE PRECISION,
  humidity_pct  DOUBLE PRECISION,
  moisture_pct  DOUBLE PRECISION,
  raw           JSONB                              -- full sensor envelope
);

raw::jsonb is intentionally not included in the public exporter output (per ADR-010). The methodology dataset uses the same scrubbed shape as the public dashboard, with the same gaps treated the same way.

DRAFT — full DDL + retention policy detail by 2026-07.

§2.4 · Cadences

Default sampling cadences, all configurable per-node via Aurora’s scraper_configs table:

NodeSensorsDefault cadence
esp32-1BME280 + soil + valve60s
esp32-2BME280 + BH1750 + CO₂30s
esp32-3BME280 + soil60s

Lambda ingest is asynchronous; cloud-write latency does not back up the ESP32. If the hub is offline, samples queue at the broker (volatile, non-persistent for v1).

DRAFT — quantified back-pressure behaviour during simulated WAN flap pending §3.5 confounder analysis.

§2.5 · Why two sinks?

The methodology question (§3.1) is answerable only if both paths see the same MQTT events. Edge and cloud are not alternatives in the production rig — they are parallel processors of the same publish stream during the measurement window. After the measurement window ends, the chosen path stays in production and the other is decommissioned for that variable.


Status: structural draft v0.1, 2026-05-07. Citation: https://plantir.garden/thesis/2026/pipeline is locked per ADR-011. Related ADRs: docs/adr/006-aurora-serverless.md, docs/adr/008-node-architecture.md.

RELATED DECISIONS
NOMINAL DISCLOSURE SCHEDULE
  • ADR-006 — see docs/adr/ in source repo
  • ADR-008 — see docs/adr/ in source repo