mqtt_bus / adapter_implementation_examples.md
1 contributor
322 lines | 9.045kb

Adapter Implementation Examples

Purpose

This document captures practical implementation conventions that proved useful while building the ZG-204ZV adapters:

  • Zigbee2MQTT -> HomeBus adapter
  • HomeBus -> HomeKit adapter

It complements addapters.md.

addapters.md defines the normative adapter role and contract boundaries.

This document focuses on practical Node-RED integration patterns, cold-start behavior, flow topology, and known failure modes.


Two Distinct Adapter Roles

In practice, two different adapter classes appear in the system:

  1. ingress adapters

These consume vendor topics and publish canonical bus topics.

Example:

zigbee2mqtt/ZG-204ZV/... -> vad/home/...

  1. consumer adapters

These consume canonical bus topics and project them into another consumer model.

Example:

vad/home/... -> HomeKit

Rule:

  • keep ingress normalization separate from consumer-specific projection
  • do not mix HomeKit, dashboard, or automation constraints into the ingress adapter
  • treat HomeKit adapters as bus consumers, not as protocol translators

Pipeline:

Device -> Ingress Adapter -> MQTT Bus -> Consumer Adapter -> Consumer System


Recommended Node-RED Flow Topology

Ingress Adapter Pattern

Recommended structure:

  1. dynamic or static mqtt in on vendor broker
  2. adapter normalization node
  3. mqtt out on canonical broker
  4. optional debug nodes on raw input and normalized output

If the adapter controls vendor subscription dynamically:

  • reserve one output for mqtt in control messages
  • reserve one output for MQTT-ready publish messages

Practical convention:

  • when an adapter has multiple semantic outputs, keep the mqtt in control output as the last output
  • this keeps semantic outputs stable and makes flow rewiring easier

Consumer Adapter Pattern

Recommended structure:

  1. dynamic mqtt in on canonical broker
  2. consumer adapter
  3. consumer-specific nodes or services
  4. optional debug nodes for last, value, and control messages

Typical example:

  • subscribe to .../last and .../value
  • use retained last for cold start
  • continue on live value
  • optionally unsubscribe from .../last after bootstrap completes

Cold-Start Pattern with last

For stateful consumers such as HomeKit, a practical pattern is:

  1. subscribe to .../last
  2. subscribe to .../value
  3. initialize consumer state from retained last
  4. keep consuming live value
  5. optionally unsubscribe from .../last after bootstrap completes

Example:

  • subscribe vad/home/balcon/+/south/last
  • subscribe vad/home/balcon/+/south/value
  • wait until all required capabilities are satisfied
  • unsubscribe vad/home/balcon/+/south/last

Bootstrap completion SHOULD be defined explicitly.

Good rules:

  • all required capabilities have arrived from either last or value
  • or the consumer has entered a known good state for its required capability set

Bad rule:

  • unsubscribe on first live message only

That rule is incomplete because a single live message does not imply that the consumer has received all required initial state.


Dynamic mqtt in Control Messages

For Node-RED dynamic mqtt in, control messages SHOULD remain simple.

Preferred forms:

{
  "action": "subscribe",
  "topic": "vad/home/balcon/+/south/last",
  "qos": 2,
  "rh": 0,
  "rap": true
}
{
  "action": "unsubscribe",
  "topic": "vad/home/balcon/+/south/last"
}

Practical recommendation:

  • prefer one control message per topic
  • do not rely on a single multi-topic control message unless runtime behavior is already verified in the target Node-RED version

This keeps behavior easier to debug and makes sidebar traces clearer.


Dedicated MQTT Session Requirement for Retained Bootstrap

This was a critical lesson from the HomeKit consumer adapter.

If a dynamic consumer depends on retained last as its cold-start mechanism, it SHOULD use its own MQTT client session.

In Node-RED terms:

  • give that dynamic mqtt in its own mqtt-broker config node
  • do not share the same broker config with unrelated static or dynamic subscribers when deterministic bootstrap matters

Reason:

  • broker config nodes in Node-RED represent shared MQTT client sessions
  • retained replay behavior is tied to subscribe operations in that session
  • when static and dynamic subscribers share the same session, retained bootstrap can become non-deterministic from the perspective of one consumer

Observed failure mode:

  • a static debug subscriber on .../last receives retained messages
  • a dynamic consumer on the same broker config subscribes later
  • the consumer does not reliably observe the cold-start retained replay it expects
  • later live updates re-publish last, creating the false impression that only live traffic works

Rule:

  • consumers that require deterministic retained bootstrap SHOULD have a dedicated broker config

This is especially important for:

  • HomeKit adapters
  • dashboard cold-start consumers
  • control nodes that reconstruct state from retained last

It is usually not necessary for:

  • simple live-only debug subscribers
  • low-value telemetry observers that do not depend on retained bootstrap semantics

Practical Semantics: Raw vs Derived State

Adapters must be careful not to publish device-generated derived semantics as if they were raw truth.

Example from ZG-204ZV:

  • raw Zigbee payload contains presence
  • with fading_time = 0, that field is effectively raw motion detection
  • canonical output should therefore be motion/value
  • canonical presence should remain reserved for held or higher-level derived occupancy semantics

Rule:

  • when a device field is operationally a raw signal, publish it as the raw canonical capability
  • keep higher-level semantics for fusion, aggregation, or dedicated higher-level adapters

This prevents single-device vendor quirks from leaking into the semantic bus.


Status, Errors, and Flow Observability

Adapter nodes SHOULD expose useful runtime status in the Node-RED editor.

Useful status content:

  • detected devices count
  • processed input count
  • translated output count
  • invalid topic count
  • invalid payload count
  • dead-letter count

Adapters SHOULD also surface problems through:

  • node.warn for malformed input and validation failures
  • node.error for internal exceptions
  • <site>/sys/adapter/<adapter_id>/error
  • <site>/sys/adapter/<adapter_id>/dlq
  • <site>/sys/adapter/<adapter_id>/stats

Rule:

  • errors should be visible both in operational topics and in Node-RED flow messages

That combination makes debugging much faster than relying on only one channel.


Recommended Debug Setup During Implementation

While implementing a new adapter, use three debug perspectives:

  1. raw ingress debug

What the source mqtt in actually receives.

  1. control-path debug

What control messages are sent to a dynamic mqtt in.

  1. semantic output debug

What canonical bus publications are emitted.

For consumers using last bootstrap, it is useful to have:

  • one static debug subscriber on .../last
  • one static debug subscriber on .../value
  • one debug node on the dynamic mqtt in output
  • one debug node on control messages

This makes it possible to distinguish:

  • no retained message exists
  • retained exists but the dynamic subscription did not activate
  • retained exists but the consumer path is using the wrong MQTT session
  • parsing or capability mapping is discarding the message

Example Integration: ZG-204ZV

Ingress Adapter

Source:

zigbee2mqtt/ZG-204ZV/<site>/<location>/<device_id>

Output:

  • .../value for live semantic samples
  • .../last retained for latest timestamped sample
  • .../meta retained
  • .../availability retained
  • .../sys/adapter/... for operational topics

Consumer Adapter

Source:

  • vad/home/<location>/<capability>/<device_id>/last
  • vad/home/<location>/<capability>/<device_id>/value

Output:

  • HomeKit service updates
  • dynamic mqtt in control for bootstrap subscribe/unsubscribe

Lessons learned:

  • occupancy may need to be synthesized locally from motion and a fading timer
  • numeric values may arrive as strings and should be normalized defensively
  • bootstrap completeness must be defined per required capability set
  • deterministic last bootstrap requires a dedicated MQTT session for the dynamic consumer

Summary Rules

  • ingress adapters and consumer adapters are different roles and should remain separate
  • prefer one control message per dynamic subscription topic
  • keep dynamic mqtt in control output as the last output when practical
  • define bootstrap completeness explicitly
  • do not unsubscribe from last on first live message alone
  • use dedicated MQTT broker config nodes for dynamic consumers that depend on retained bootstrap
  • expose operational errors both in sys and in Node-RED flow messages
  • treat raw device-generated signals as raw semantics, not as already-derived business meaning