[Dev] Inheritance in MQTT templates?

toggledbits

Turn debug level 5 on for MQTTController. It will show you how it's loading the templates and building the final entity configuration. The message fragment you'll want to look for is merged template for sovrumsdorr... then work backwards in the log from there to configuring entity sovrumsdorr. Everything in between is the setup.

Also examine the debug messages at startup for where it's finding templates and loading them... make sure everything it's loading is what's intended, and you don't have a stray old file sitting around somewhere that it's sucking in.

Edit: your logs are showing a lot of duplicate definitions in the startup messages...

Crille

It seems to merge the events: section of string_sensor and power_switch in ...common.yaml and wrap value_sensor.units in string_sensor capability.
I've looked for strange characters that could cause it but found nothing, validated all files OK in yamllint.

[latest-24152]2024-06-06T06:29:11.699Z <MQTTController:5:MQTTController.js:146> MQTTController#mosquitto-mqtt configuring entity sovrumsdorr
[latest-24152]2024-06-06T06:29:11.699Z <MQTTController:5:MQTTController.js:205> MQTTController#mosquitto-mqtt merged template for sovrumsdorr (from /home/homebridge/reactor/config/mqtt_templates/zigbee2mqtt_common.yaml/zigbee2mqtt_common.yaml):
{
    "capabilities": [
        "door_sensor",
        "x_mqtt_device",
        "value_sensor",
        "battery_power"
    ],
    "primary_attribute": "door_sensor.state",
    "requires": [
        "topic"
    ],
    "include": [
        "zigbee2mqtt_availability_no_get",
        "zigbee2mqtt_lqi",
        "zigbee2mqtt_battery_power",
        "zigbee2mqtt_tuya_door_sensor"
    ],
    "events": {
        "%prefix:zigbee2mqtt%/%topic%": {
            "door_sensor.state": {
                "json_payload": true,
                "expr": "payload?.contact"
            },
            "value_sensor.value": {
                "json_payload": true,
                "if_expr": "! isnull(payload)",
                "expr": "payload.linkquality",
                "attribute": "value_sensor.value"
            },
            "value_sensor.units": "lqi",
            "string_sensor.value": {
                "json_payload": true,
                "if_expr": "! isnull(payload)",
                "expr": "payload?.update_available ? \"OTA firmware update available!\" : \"no update available\"",
                "attribute": "string_sensor.value"
            },
            "power_switch.state": {
                "json_payload": true,
                "expr": "payload?.state == \"ON\"",
                "attribute": "power_switch.state"
            },
            "battery_power.level": {
                "json_payload": true,
                "expr": "payload?.battery / 100",
                "attribute": "battery_power.level"
            },
            "battery_power.since": {
                "expr": "time()",
                "attribute": "battery_power.since"
            }
        },
        "%prefix:zigbee2mqtt%/%topic%/availability": {
            "x_mqtt_device.online": {
                "if_expr": "! isnull(payload)",
                "expr": "lower(payload) == \"online\"",
                "attribute": "x_mqtt_device.online"
            }
        }
    },
    "__source": "/home/homebridge/reactor/config/mqtt_templates/zigbee2mqtt_common.yaml/zigbee2mqtt_common.yaml",
    "actions": {
        "x_mqtt_device": {
            "poll": false
        }
    },
    "name": "Sovrumsdörr",
    "topic": "Sovrumsdörr"
}
[latest-24152]2024-06-06T06:29:11.699Z <MQTTController:5:MQTTController.js:264> MQTTController#mosquitto-mqtt set primary attribute of door_sensor.state to Entity#mosquitto-mqtt>sovrumsdorr
[latest-24152]2024-06-06T06:29:11.700Z <MQTTController:5:MQTTController.js:289> MQTTController#mosquitto-mqtt registering event handler %prefix:zigbee2mqtt%/%topic% for sovrumsdorr
[latest-24152]2024-06-06T06:29:11.700Z <MQTTController:5:MQTTController.js:289> MQTTController#mosquitto-mqtt registering event handler %prefix:zigbee2mqtt%/%topic%/availability for sovrumsdorr
[latest-24152]2024-06-06T06:29:11.700Z <MQTTController:5:MQTTController.js:368> MQTTController#mosquitto-mqtt registering action x_mqtt_device.poll for sovrumsdorr

Regarding the duplicates, it could be the templates with different primary_attribute: and it is on my roadmap to bring it up for discussion if attributes also could be parameterized with defaults like %primary_attribute:[temperature / humidity / power_switch / dimming / motion / light]% or %temperature_sensor.units:[C / F]%.

EDIT: Or is caused by the mystery of this issue? e.g. string_sensor and power_switch is somehow included in all templates.

therealdb

@Crille at some point I had similar problems, I fixed them and I reported here. Now I cannot repro it again, but it seems very similar and related to primary attributes in base templates. It seems like the first one is used and that’s causing troubles along the hierarchy, but, as I said, I’m not able to repro it again.

toggledbits

It also looks like something may be getting a shallow copy when it needs a deep copy, and it's being modified later and polluting the other references. I'm really tied up today, but I'll look at this as soon as I can.

Did you address the template redefinitions that were being logged at startup? That has nothing to do with primary attributes; the entire template is trying to be redefined.

Crille

No rush!

@toggledbits said in [Dev] Inheritance in MQTT templates?:

Did you address the template redefinitions that were being logged at startup?

You mean this?

<MQTTController:WARN> MQTTController#mqtt file zigbee2mqtt_tuya_door_sensor.yaml in /home/homebridge/reactor/config/mqtt_templates/zigbee2mqtt_tuya_door_sensor.yaml overrides existing template definition zigbee2mqtt_tuya_door_sensor

If so I don't know what to do about it, it's just the template provided in my earlier post and there is no reference/copy of it in the reactor folder/subfolders that I know of.

toggledbits

With debug level 5, that message will have another message that follows it to tell you what file originally defined it.

Crille

This?

[latest-24152]2024-06-06T19:29:35.079Z <MQTTController:5:MQTTController.js:941> MQTTController#mqtt loading template defs, considering zigbee2mqtt_tuya_door_sensor.yaml
[latest-24152]2024-06-06T19:29:35.079Z <MQTTController:5:MQTTController.js:987> MQTTController#mqtt loading YAML template data from /home/homebridge/reactor/config/mqtt_templates/zigbee2mqt
t_tuya_door_sensor.yaml
[latest-24152]2024-06-06T19:29:35.080Z <MQTTController:WARN> MQTTController#mqtt file zigbee2mqtt_tuya_door_sensor.yaml in /home/homebridge/reactor/config/mqtt_templates/zigbee2mqtt_tuya_do
or_sensor.yaml overrides existing template definition zigbee2mqtt_tuya_door_sensor
[latest-24152]2024-06-06T19:29:35.080Z <MQTTController:5:MQTTController.js:918> MQTTController#mqtt previously defined in /home/homebridge/reactor/config/mqtt_templates/zigbee2mqtt_tuya_doo
r_sensor.yaml/zigbee2mqtt_tuya_door_sensor.yaml

toggledbits

Uh yes...

Digging in now...

toggledbits

I need the config for all of your zigbee2mqtt-related entities. Just looking at the one sovrumsdorr, I'm not able to reproduce the problem.

Crille

Config as in reactor.yaml?

toggledbits

Yes, but just the entities that use the zigbee2mqtt templates. I don't need all of it.

Crille

entities:
  "zigbee2mqtt_sensor":
    name: "Zigbee2MQTT"
    topic: "zigbee2mqtt"
    include: my_zigbee2mqtt_sensor
  "datarummet":
    name: "Datarummet"
    topic: "Datarummet"
    include: zigbee2mqtt_tradfri_light_bulb
  "sovrummet":
    name: "Sovrummet"
    topic: "Sovrummet"
    include: zigbee2mqtt_tradfri_light_bulb_ws
  "milia":
    name: "Milia"
    topic: "Milia"
    include: zigbee2mqtt_tradfri_light_bulb_ws
  "hallen1":
    name: "Hallen 1"
    topic: "Hallen 1"
    include: zigbee2mqtt_tradfri_light_bulb
  "hallen2":
    name: "Hallen 2"
    topic: "Hallen 2"
    include: zigbee2mqtt_tradfri_light_bulb
  "hallen3":
    name: "Hallen 3"
    topic: "Hallen 3"
    include: zigbee2mqtt_tradfri_light_bulb
  "fonster_koket":
    name: "Fönster köket"
    topic: "Fönster köket"
    include: zigbee2mqtt_tradfri_light_bulb_ws
  "soffan":
    name: "Soffan"
    topic: "Soffan"
    include: zigbee2mqtt_tradfri_light_bulb_ws
  "framsida":
    name: "Framsida"
    topic: "Framsida"
    include: zigbee2mqtt_tradfri_light_bulb
  "skank":
    name: "Skänk"
    topic: "Skänk"
    include: zigbee2mqtt_tradfri_light_bulb
  "baksida":
    name: "Baksida"
    topic: "Baksida"
    include: zigbee2mqtt_tradfri_light_bulb
  "lampa_spel":
    name: "Lampa spel"
    topic: "Lampa spel"
    include: zigbee2mqtt_tradfri_light_bulb
  "lampa_ella":
    name: "Lampa Ella"
    topic: "Ellas rum"
    include: zigbee2mqtt_tradfri_light_bulb
  "lampa_gym":
    name: "Lampa gym"
    topic: "Lampa gym"
    include: zigbee2mqtt_tradfri_light_bulb
  "lampa_tv":
    name: "Lampa TV"
    topic: "Lampa TV"
    include: zigbee2mqtt_tradfri_control_outlet
  "vitrinskap":
    name: "Vitrinskåp"
    topic: "Vitrinskåp"
    include: zigbee2mqtt_tradfri_control_outlet
  "klimat_sovrum":
    name: "Klimat sovrum"
    topic: "Klimat sovrum"
    include: zigbee2mqtt_aqara_sensor_temperature_humidity
  "klimat_kaminen":
    name: "Klimat kaminen"
    topic: "Klimat kaminen"
    include: zigbee2mqtt_aqara_sensor_temperature_humidity
  "klimat_dusch":
    name: "Klimat dusch"
    topic: "Klimat dusch"
    include: zigbee2mqtt_aqara_sensor_humidity_temperature
  "klimat_tvattstugan":
    name: "Klimat tvättstugan"
    topic: "Klimat tvättstugan"
    include: zigbee2mqtt_aqara_sensor_temperature_humidity
  "rorelse_hallen":
    name: "Rörelse hallen"
    topic: "Rörelse hallen"
    include: zigbee2mqtt_aqara_light_motion_sensor
  "rorelse_kallaren":
    name: "Rörelse källaren"
    topic: "Rörelse källaren"
    include: zigbee2mqtt_aqara_light_motion_sensor
  "rorelse_forradet":
    name: "Rörelse förrådet"
    topic: "Rörelse förrådet"
    include: zigbee2mqtt_tuya_motion_light_sensor
  "mini_switch_1":
    name: "Mini Switch källaren"
    topic: "Mini Switch källaren"
    include: zigbee2mqtt_aqara_mini_switch
  "jul_framsida":
    name: "Jul Framsida"
    topic: "Jul Framsida"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "jul_koket":
    name: "Jul Köket"
    topic: "Jul Köket"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "jul_sovrummet":
    name: "Jul Sovrummet"
    topic: "Jul Sovrummet"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "jul_datarummet":
    name: "Jul Datarummet"
    topic: "Jul Datarummet"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "element_gillestugan":
    name: "Element gillestugan"
    topic: "Element gillestugan"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "vaxtbelysning_kallare":
    name: "Växtbelysning källare"
    topic: "Växtbelysning källare"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "kompressor":
    name: "Kompressor"
    topic: "Kompressor"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "luftrenare":
    name: "Luftrenare Sovrum"
    topic: "Luftrenare"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "uttag_uterummet":
    name: "Uttag uterummet"
    topic: "Uttag uterummet"
    include: zigbee2mqtt_tuya_smart_plug_pm
  "sovrumsdorr":
    name: "Sovrumsdörr"
    topic: "Sovrumsdörr"
    include: zigbee2mqtt_tuya_door_sensor
  "entrebelysning":
    name: "Entrébelysning"
    topic: "Entrebelysning"
    include: zigbee2mqtt_tuya_light_bulb_rgbcw
  "lampa_uterummet":
    name: "Lampa uterummet"
    topic: "Lampa uterummet"
    include: zigbee2mqtt_tuya_light_bulb_rgbcw

EDIT: unrelated, found entity id "jul_köket" and renamed it to "jul_koket" in post and Reactor.

toggledbits

OK. I also need the definitions for zigbee2mqtt_tradfri_light_bulb and all of the other templates not in the common file you previously posted.

toggledbits

Wait... are those just available on Github? Are your current versions published in any branch there? If so, I'll just grab them from there.

Crille

No I haven't published them due to this issue but I can put them in the test branch, hold on.

Crille

Available in zigbee2mqtt_test.zip

Crille

Looks like MQTTController 24159 solved it. I don't see any errors or warnings in the log after a restart, except all of these but I quess that's normal?

<MQTTController:WARN> MQTTController#mqtt file /home/homebridge/reactor/config/ext/MQTTController/templates/tasmota.yaml overrides existing template definition tasmota_generic_relay from /home/homebridge/reactor/config/ext/MQTTController/templates/tasmota.yaml

toggledbits

No not normal, and I don't get those.

Crille

Strange! I wonder what’s causing that in my setup then as it’s the integrated templates as well.

toggledbits

Are you running more than one instance of MQTTController?

How to upgrade from an old version of MSR?

This trigger no longer working - complaining about the operator needing changing

Can you run MSR on Home Assistant OS ?

Self test

Access control - allowing anonymous user to dashboard

VEC Virtual Switch Auto Off

Upcoming Storage Change -- Got Back-ups?

Oddness in Copy/Move of Reactions

[Solved] function isRuleEnabled() issue

[Reactor] Problem with Global Reactions and groups

Possible feature request 2?

Reactor (Multi-System/Multi-Hub) Announcements

Copying a global reaction

[Reactor] Bug when sending MQTT boolean payloads

Difficulty defining repeating annual period

Need help with sequence

Possible feature request?

Logic Assistance: Exterior Lights on when Illuminance Below Threshold

Time series documentation

MQTT templates for ZIgbee scene controller, or a better way?

[Dev] Inheritance in MQTT templates?

Recent Topics