MQTTController configuration with multiple topics per entity

[mgvra]

Hi,
This post is a kind of feature request and a question at the same time.
I've just recently started to play with the MSR, thanks to friend of mine @tunnus
who recommended to try it out. So bear with me if I'm asking trivial or stupid questions.

Some background information of my setup

I have a KNX bus as a hardware backbone which I'm accessing through my own KNXNet/IP protocol stack implementation. On top of that I have NodeJS based dockerized microservices which are communicating using the MQTT. Since I'm still lacking a good rule engine, I've decided to try the MSR, and so far everything is looking pretty promising.

MSR version: latest-22274-24dfd72c (tested on Ubuntu but will eventually run on RPi 4)
MQTTController version: 22260

So what is the problem then?

Having well over 100 entities (62 relays alone) definitely means that I have to go with the entity template configuration. I've already succesfully POCed some simple binary and value sensors, e.g. entities that provide only state changes and are so to speak read-only. These are working just fine, but the problem lies within the switches and generally speaking
entities that have actions.

But let me first explain how I do the MQTT topic mapping for the KNX group addresses, which are basically unique identifiers for the attributes and functions of the KNX devices.

For example a motion sensor has a single group address, let's say 0/6/0,
for which we the following MQTT topics:

For status updates: /knx/0-6-0/status
And for the read operation: /knx/0-6-0/read

The group address 0/6/0 is formatted as 0-6-0 since it would clash with the MQTT topic.

The problem arises with swiches and gets even worse with dimmable lights.

In the KNX world we have separate group addresses for the statuses and for the write operations.

For example a simple switch would have a setup like this:

0/0/55 write (toggle on/off)
0/0/56 status (current status, can be also used to read the status)

And for the dimmable light we may have:

0/0/3 write (toggle on/off)
0/0/4 write (absolute dimming value)
0/0/5 write (relative dimming value)
0/0/6 status (current status, can be also used to read the status)

Now these would map to MQTT topics like this:

knx/0-0-55/write
knx/0-0-56/read
knx/0-0-56/status

knx/0-0-3/write
knx/0-0-4/write
knx/0-0-5/write
knx/0-0-6/status
knx/0-0-6/read

And there lies the problem. Since the MQTTController configration is only interpolating the %topic% variable, it's not possible to pass those additional 'write' topics into the template.

I've tried to get my head around this and didn't find any easy solution. One that crossed mind was the possibility to interpolate all the entity's config keys into the template.

I'm open to all suggestions since I would love get on writing the rules and logic with the MSR. Anyway, keep up the good work @toggledbits !

br,
mgvra

[toggledbits]

Is there a reason in the actions you don't just:

    actions:
      power_switch:
        "on":
          topic: "knx/%topic%/write"
          payload:
            expr: true
            type: json
        "off":
          topic: "knx/%topic%/write"
          payload:
            expr: false
            type: json

Also, the %% substitutions will happen with any simple name from the top level on your entity configuration. So, for example:

    mh3-ceiling-light:
      name: "MH3 Ceiling Light"
      topic: "0-0-56"
      type: Switch
      flavor: raspberry
      uses_template: knx-switch

The substitution %raspberry% would be valid in your configuration and template. As would %name% and %type%, if those had any use/meaning. So if you have different construction for read and write, you could just create readtopic and writetopic for your template to use instead of plain topic. Using topic isn't required, and it isn't the only thing you can use.

I think it's much more flexible than you are imagining it is, and I'm hoping this key piece of information helps you discover that.

Edit: Please don't screen shot code and formatted data like YAML. Using the fenced code block formatting available and pasting the actual text is preferable (at least, to me, so I can copy-paste from your code to modify it if necessary, rather than having to retype it).

[mgvra]

Thanks for your quick response!

@toggledbits said in MQTTController configuration with multiple topics per entity:

Is there a reason in the actions you don't just:

    actions:
      power_switch:
        "on":
          topic: "knx/%topic%/write"
          payload:
            expr: true
            type: json
        "off":
          topic: "knx/%topic%/write"
          payload:
            expr: false
            type: json

Well the problem with KNX is that you need at least two separate group addresses for read/updates and write. And the way the topics are defined in my configuration, there is no common section like %topic% in the middle. So I must do it like: /knx/%status%/read and /knx/%toggle%/write (e.g. /knx/0-0-1/read and /knx/0-0-2/write). Hopefully this makes sense in the config examples below.

I think it's much more flexible than you are imagining it is, and I'm hoping this key piece of information helps you discover that.

It definitely helped a lot and I've managed to get it working. Maybe I've just missed this in the docs. But thanks anyway!

Entity definitions look like this:

      # Lights (on/off)
        mh3-ceiling-light:
          name: "MH3 ceiling light"
          toggle: 0-0-55
          status: 0-0-56          
          type: Switch
          uses_template: knx-switch

      # Lights (dimmable)
        living-room-wall-lights:
          name: "Living room wall lights"
          toggle: 0-1-5
          status: 0-1-16
          absolute_dimming: 0-1-28        
          type: Switch
          uses_template: knx-dimmable

And their respective template definitions:

  knx-switch:
    query: "knx/%status%/read"
    capabilities:
      - power_switch
      - toggle
    primary_attribute: power_switch.state
    events:
      "knx/%status%/status":
        "power_switch.state":
          json_payload: true
          expr: "payload.data.payload.value"
    actions:
      power_switch:
        "on":
          topic: "knx/%toggle%/write"
          payload:
            expr: "true"
            type: json
        "off":
          topic: "knx/%toggle%/write"
          payload:
            expr: "false"
            type: json
        set:
          topic: "knx/%toggle%/write"
          payload:
            expr: "parameters.state"
            type: json

  knx-dimmable:
    query: "knx/%status%/read"
    capabilities:
      - power_switch
      - toggle
      - dimming
    primary_attribute: power_switch.state
    events:
      "knx/%status%/status":
        "power_switch.state":
          json_payload: true
          expr: "bool(payload.data.payload.value)"
        "dimming.level":
          json_payload: true
          expr: "payload.data.payload.value / 100"
    actions:
      power_switch:
        "on":
          topic: "knx/%toggle%/write"
          payload:
            expr: "true"
            type: json
        "off":
          topic: "knx/%toggle%/write"
          payload:
            expr: "false"
            type: json
        set:
          topic: "knx/%toggle%/write"
          payload:
            expr: "parameters.state"
            type: json
      dimming:
        set:
          topic: "knx/%absolute_dimming%/write"
          payload:
            expr: "parameters.level * 100"
            type: json

Edit: Please don't screen shot code and formatted data like YAML. Using the fenced code block formatting available and pasting the actual text is preferable (at least, to me, so I can copy-paste from your code to modify it if necessary, rather than having to retype it).

My bad, should have used the formatted code section.

[toggledbits]

@mgvra said in MQTTController configuration with multiple topics per entity:

Maybe I've just missed this in the docs.

More likely I did, but I'll add that to the list.

[toggledbits]

Just checked... it is mentioned in the docs... in the "tip" right before the "Echo Capability" heading:

All topic strings will be put through substitution, and the keywords in the substititions are values from the device configuration (like topic). If you have devices with multiple relays, add keywords like channel or unit, mirror whatever the device manufacturer tends to use, so that you effectively parameterize your template and let it work the same for multiple instances of a device's components. To help clarify this, look at the implementation of the "real" shelly_relay template in MQTTController's mqtt_devices.yaml file.

Probably needs some embellishment to be really clear.

[toggledbits]

By the way, you can also access the configuration values in the expressions you use, by access through the config object in your expressions. For example, you could refer to config.toggle or config.status to access those values from your entity configuration.

This was not mentioned in the docs and I'm updating it now.