@toggledbits

@toggledbits said in Custom capabilities in MQTT templates:

Showing off all your cool toys, eh?

Well I wouldn't boast with employer provided montitor, and definitely wouldn't call it cool, just damn wide.

But speaking of showing off cool toys, this is something I bet you've never seen before, one of my weird projects:
A custom made flow through heater for my summer cottage's outdoor shower. Based on old used Webasto ThermoTop EVO 5kW car engine diesel
heater and controlled by ESP32 which provides WiFi AP and web UI. Fully modular, so that the heating unit can be removed before the winter.
Wouldn't call this cool either (no pun intented) but unique. Works pretty well, at least @tunnus agrees.

Old WIP photo

Thanks for letting me know. I'll see what I can do to make it look more presentable.

np

br,
mgvra

@toggledbits

Thank you for your reply! Well this is what I mean, the answer is right in your face (local_mqtt_capabilities.yaml) and you just can't see it.
Now it works like charm and I can even refactor my existing KNX templates using the custom capabilities.

For anyone else who might be struggling with the same problem, here's my config:

local_mqtt_capabilities.yaml

capabilities:
  x_mqtt_sonos_announcement:
    actions:
      speak:         
        arguments:
          text:
            type: string
          volume:
            type: int
            default: 50
          delay:
            type: int
            default: 500

local_mqtt_devices.yaml

templates:
  sonos-announcement: 
    capabilities:
     - x_mqtt_sonos_announcement
    actions:
      x_mqtt_sonos_announcement:
        speak:          
          topic: "sonos/cmd/speak"
          payload:
            expr: >
              { "text": parameters.text, "volume": parameters.volume, "delayMs": parameters.delay, "onlyWhenPlaying": false, "engine": "neural" }
            type: json

UI looks like this:

FYI there's a subtle layout issue with the login screen background image when using ultra wide monitors, just to let you know.

@therealdb

Well thanks for your reply, but didn't get it working.
This is my config now:

capabilities:
  x_sonos_announcement:
    actions:
      speak:         
        arguments:
          text:
            type: string
          volume:
            type: int
          delay:
            type: int
templates:

  sonos-announcement: 
    capabilities:
     - x_sonos_announcement
    actions:
      x_sonos_announcement:
        speak:          
          topic: "sonos/cmd/speak"
          payload:
            expr: >
              { "text": parameters.text, "volume": parameters.volume, "delayMs": parameters.delay, "onlyWhenPlaying": false, "engine": "neural" }
            type: json

I just fail to see what is the problem here.

br,
mgvra

Hi,
I'm trying to integrate the sonos-mqtt (https://sonos2mqtt.svrooij.io/) with the MSR and it's coming along nicely so far.
But cannot wrap my head around how to define custom capabilities in MQTT templates. I need this for the TTS announcements and similarly for the notification sounds where I would pass the sound file as parameter.

So this is what I have in the local_mqtt_devices.yaml

capabilities:
  x_sonos_announcement:
    attributes:
    actions:
      speak:
        arguments:
          text:
            type: string
          volume:
            type: int
          delay:
            type: int

And this is the template:

templates:
  sonos-announcement: 
    capabilities: 
      - x_sonos_announcement
    actions:
      x_sonos_announcement:
        speak:
          topic: "sonos/cmd/speak"
          payload:
            expr: >
              { "text": parameters.text, "volume": parameters.volume, "delayMs": parameters.delay, "onlyWhenPlaying": false, "engine": "neural" }
            type: json

So the speak action should send something like this to topic sonos/cmd/speak

{
  "text": "message goes here",
  "volume": 50,
  "delayMs": 100,
  "onlyWhenPlaying": false,
  "engine": "neural"
}

At startup the MSR seems to be quite unhappy with my configuration:

reactor  | [latest-25016]2025-02-09T08:19:59.029Z <MQTTController:WARN> MQTTController#mqtt entity Entity#mqtt>sonos-announcement unable to configure capabilities [Array][ "x_sonos_announcement" ]
reactor  | i18n: missing fi-FI language string: Configuration for {0:q} is incomplete because the following requested capabilities are undefined: {1}
reactor  | i18n: missing fi-FI language string: Configuration for {0:q} has unrecognized capability {1:q} in actions
reactor  | Trace: Configuration for {0:q} is incomplete because the following requested capabilities are undefined: {1}
reactor  |     at _T (/opt/reactor/server/lib/i18n.js:611:28)
reactor  |     at AlertManager.addAlert (/opt/reactor/server/lib/AlertManager.js:125:25)
reactor  |     at MQTTController.sendWarning (/opt/reactor/server/lib/Controller.js:627:30)
reactor  |     at MQTTController.start (/var/reactor/ext/MQTTController/MQTTController.js:268:26)
reactor  |     at async Promise.allSettled (index 0)

Configuration for "sonos-announcement" has unrecognized capability "x_sonos_announcement" in actions
Controller: MQTTController#mqtt
Last 10:21:37 AM

Configuration for "sonos-announcement" is incomplete because the following requested capabilities are undefined: x_sonos_announcement
Controller: MQTTController#mqtt
Last 10:21:37 AM

This is probably a pretty stupid question and the approach may not even work at all, but maybe someone or @toggledbits for sure, could point me to the right direction.
Basically the idea is to be able to send TTS messages from reactions using entity actions. I've previously used HTTP requests to Sonos HTTP API (https://hub.docker.com/r/chrisns/docker-node-sonos-http-api/) for the same functionality, but since moving to sonos-mqtt, I need a way to send the TTS notifications using MQTTController. Along with the actual message, volume and delay must also be parameterizable.

br,
mgvra

MSR latest-25016-d47fea38 / MQTTController [0.2.24293]

A case with only a single when (and possibly an else) is an if statement, so why not use that? This is the workaround/recommendation. Using case in this way just seems unnecessarily complex

Well that's true and I don't think anyone uses the case-when structure with a single when. But for example in my case, I was testing the expressions, had written only one section and was getting this mysterious syntax error. As a software engineer myself, coming from C / Java / JS background, I naturally associated the case-when structure to switch-case where it's perfectly fine to have a single case, although as you mentioned, does not make much sense.

This is not a bug that needs to be necessarily fixed ASAP, but it would be nice if you could add a clarification to the manual, stating that the case-when structure must have at least two when sections.

Well I stumbled across exactly the same bug and was able to resolve the problem only because @tunnus knew what was the issue.

This would be nice a improvement.

Thanks for your quick reply!

You can use expressions in condition operands by using the substitution syntax: ${{ your-expression }}

I'll use that for the brightness threshold.

The workaround is to group conditions and use additional conditions as gating criteria. For example, if you would change the "sustained for" delay based on time of day, say 120 for day and 600 for night, then you would have conditions structured like this:

Group A - Daytime -- AND group
    Rule Condition: Rule "Is Day" is TRUE
    Entity Condition: some entity test that tests a value
        condition option: Sustained for 120 seconds
Group B - Night -- AND group
    Rule Condition: Rule "Is Day" is FALSE
    Entity Condition: the same test as used in group A
        condition option: Sustained for 600 seconds

Well unfortunately this doesn't help much if I need to set an arbitrary delay value from my custom UI.

You cannot use expression substitution in the values of condition options. These are required to be constants (literal values).

Could this be a new feature request to support expression substitution in timeout values (sustain / delay reset etc) if it's feasible to implement?

br,
mgvra

Some background
I have two simple rules that will toggle my carport's down lights depending on outdoor brightness when I open the motorized sliding gate. I.e. if the brightness is 1500 lux or less and the gate is opened, the down lights will be switched on, and when the gate is closed, the lights will switch off after 3 minutes. Pretty simple 'coming home' functionality so to speak.

Here's the switch off rule showing the 'hard coded' thresholds

The problem
I would like to parameterize the lux and switch off delay using entities. I bet the lux threshold can be done with local expressions the same way I do my WC background music scheduling:

weekdays_enabled = (time() >= time(getEntity( "mqtt>wc-music-weekdays-enable" ).attributes.string_sensor.value )) && (time() <= time(getEntity( "mqtt>wc-music-weekdays-disable" ).attributes.string_sensor.value ))

But the since I'm using the Condition must be sustained for functionality for the switch off rule, I have no idea how to parameterize that with some entity value.

So my question is that is it even possible to do this using entities or could there some other way to accomplish the same functionality? In this example it's the sustain -value but I might need same kind of parameterization for the delay reset also.

I have to go with entities, since my own HA system will provide the threshold values for the MSR through the MQTTController.

Help would be appreciated @toggledbits
br,
mgvra

MSR latest-23302-b7def56a and MQTTController [0.1.23254]

Thanks for yet another quick reply! I'll start playing with the expressions and JSON payloads now that I know the proper syntax.

          expr: >
            { "brightness": parameters.level * 255 }

So it is this implicit parameters object that contains the value (level) to be set.
In the case of step dimming, is this the notation?

          expr: >
            { "brightness": parameters.step * 255 }

Don't forget you can set MQTTController's log level to 5 while doing this work to troubleshoot what you are doing and see more clearly what is ultimately being sent.

Good point!

I do have functions for converting between HS(L), RGB, and XY. Currently they are only available to HubitatController, but I can easily extend them to MQTTController as well. The trick there, however, is that your device's implementation is the same (e.g. for HSL 0° is red, 120° is green, 240° is blue, etc.). There's a de facto standard, but some devices have their own way.

Well that would be really nice if it doesn't require too much effort. A new release of the MSR or just the MQTTController?

The trick there, however, is that your device's implementation is the same (e.g. for HSL 0° is red, 120° is green, 240° is blue, etc.). There's a de facto standard, but some devices have their own way.

Well I think those functions should exist even if they don't work with all possible devices. It's sure better than nothing and chances are that they work just fine.

br,
mgvra

Some background
I'm trying to integrate a Zigbee device into the MSR using zigbee2mqtt bridge and MQTTController. The device in question is a cheap mood light that has following properties that I'd like to control:

• switch (on/off)
• brightness
• color

I'v already managed to get the switch part working and can toggle the light on/off. Also the brightness value is mapped back to MSR. In zigbee2mqtt it has a value range from 0 to 254, so this the reason for the expression:

expr: 'payload.brightness / 254'

Here's the entity definition (don't know whether the type should be something else than the Switch)

zigbee-lidl-mood-light:
  name: 'Lidl Mood Light'
  friendly_name: 'Mood Light'
  type: Switch
  uses_template: lidl-moodlight      

And the corresponding template (NOTE: rgb_color has not been defined in this example):

lidl-moodlight:
  init: "zigbee2mqtt/%friendly_name%/get/state"
  query: "zigbee2mqtt/%friendly_name%/get/state"
  capabilities:
    - power_switch
    - toggle
    - dimming
  primary_attribute: power_switch.state
  events:
    "zigbee2mqtt/%friendly_name%":
      "power_switch.state":
        json_payload: true
        expr: 'upper(payload.state) == "ON"'
      "dimming.level":
        json_payload: true
        expr: 'payload.brightness / 254'          
  actions:
    power_switch:
      "on":
        topic: "zigbee2mqtt/%friendly_name%/set/state"
        payload: 'ON'
      "off":
        topic: "zigbee2mqtt/%friendly_name%/set/state"
        payload: 'OFF'
      set:
        topic: "zigbee2mqtt/%friendly_name%/set/state"
        payload: 
          expr: "parameters.state ? 'ON' : 'OFF'"
          type: raw
      toggle:
        topic: "zigbee2mqtt/%friendly_name%/set/state"
        payload: 'TOGGLE'

The problem
In order to control the brightness or the RGB color values, I would have send a JSON payload in corresponding actions. But I have no idea how to define it in the template. The reason why the switch part is working is that the zigbee2mqtt accepts also plain ON / OFF / TOGGLE string payloads in that case.

But the brightness should be controlled with the following payload:

{"brightness": 196}

And the RGB color like:

{"color":{"rgb":"46,102,150"}}

Here's the link for the documentation (the Exposes part defines the messages).

So how should I define the JSON payload for example for the dimming action? It definitely should be some sort of expressions since I have to map the MSR real value (0...1) to (0...254) for the zigbee2mqtt.

actions:
    dimming:
      set:
        topic: "zigbee2mqtt/%friendly_name%/set"
        payload: 
          expr: ?????
          type: json

Another problem is the RGB value. I could use the rgb_color capability for the setting but the problem is that the zigbee2mqtt only reports the current color in hue/saturation or xy coordinates.

Here's an example of published message after setting the color:

Topic: zigbee2mqtt/Mood Light QoS: 0
{
   "brightness":254,
   "color":{
      "hue":240,
      "saturation":100,
      "x":0.1355,
      "y":0.0399
   },
   "color_mode":"xy",
   "color_temp":574,
   "linkquality":96,
   "state":"ON"
}

I would have to map those values back to RGB, but is it even possible with existing constructs in MQTTController's templates?

Help would be appreciated @toggledbits
br,
mgvra

Thanks for your quick reply!
With added NUL group and interval it seems to be working just fine.

Well don't know about great post, just asking 'stupid' questions and hoping to learn something along the way.
But it's nice if it can be used as an example of how tackle that kind of problem.

br,
mgvra

Some background

I have MQTTController based entities that I want to use in time calculations in my rules. These entities represent two separate time ranges, one for the weekdays and the other for the weekends. In this particular example they are used to control my doorbell activity (whether it's enabled or disabled).

MQTT message which is used to update the entities. NOTE: time is in 24h format

{
  "data": {
    "weekdays": {
      "enable": "09:28",
      "disable": "20:00"
    },
    "weekends": {
      "enable": "09:00",
      "disable": "21:00"
    }
  }
}

Entity definitions (here's only the definition for the weekdays for brevity)

        doorbell-weekdays-enable:
          name: 'Ovikello päälle (arkisin)'
          topic: 'doorbell/settings'
          capabilities:
            - string_sensor
          primary_attribute: string_sensor.value
          events: 
            "%topic%":
              "string_sensor.value":
                json_payload: true
                expr: "payload.data.weekdays.enable"

        doorbell-weekdays-disable:
          name: 'Ovikello pois päältä (arkisin)'
          topic: 'doorbell/settings'
          capabilities:
            - string_sensor
          primary_attribute: string_sensor.value
          events: 
            "%topic%":
              "string_sensor.value":
                json_payload: true
                expr: "payload.data.weekdays.disable"

I have verified that the entities are populated correctly and the values are working in local expressions.

The entity (only the range start for brevity):

The rule:

And the local expression within the rule:

weekdays_enabled = (time() >= time(getEntity( "mqtt>doorbell-weekdays-enable" ).attributes.string_sensor.value )) &&
(time() <= time(getEntity( "mqtt>doorbell-weekdays-disable" ).attributes.string_sensor.value ))

The problem
Local expression doesn't seem to evaluate to true even if the current time gets within the range. But if the related entities are updated, then the expression is evaluated and the rule is triggered.

As seen from the screenshots, the current time is 09:28:12 and the start time in corresponding entity is set to 09:28. But still the rule is not triggered. However, if I update the entity e.g. to 09:27, the local expression will get evaluated and the rule is triggered.

My understanding is that the local expression is working correctly but since it's not evaluated, it will not trigger the rule.

Is this a bug or have I misunderstood something? If so, how should I implement this kind of functionality?

br,
mgvra

Reactor (Multi-hub) latest-23078-d592d400 + MQTTController [0.1.23010]

@toggledbits any comments on this?

I'm a little bit confused how to configure extended capabilities for the entities in the MQTTController.

mqtt_cababilities.yaml states that: "...You can add your own capabilities by adding a "capabilities" section to your own local_mqtt_capabilities.yaml file in your config directory."

So I did this in my local_mqtt_capabilities.yaml in the config directory:

version: 22353
revision: 1
format: 1

capabilities:
  x_my_config:
    attributes:
      partyMode:
        type: bool
    actions:
      some_action:
        topic: '%topic/write'
        payload: "some value"

And in my reactor.yaml I have following configuration entry:

        my-config:          
          name: 'My custom configuration'
          topic: 'my-config'
          query: "%topic%/read"
          init: "%topic%/read"
          capabilities: 
            - x_my_config
          primary_attribute: x_my_config.partyMode
          events:
            "%topic%/status":
              "x_my_config.partyMode":
                json_payload: true
                expr: "payload.partyMode"

This sort of works but not quite. The partyMode attribute gets updated when messages arrive for that specific topic, so that's fine. But I don't see x_my_config capability in the Capabilities list in the UI and also the some_action is missing.

Addition to that the primary value is not set and I'm getting the following error:

Have I completely misunderstood the idea behind the capability extension and the attributes? If so, could someone point me to the right direction with few explanatory example configs.

What I'm trying to achieve is a set of configuration options that could be updated thru the MQTT, and also provide some actions that the rules could invoke.

Reactor (Multi-hub) latest-23063-c464b685 + MQTTController [0.1.23010]

br,
mgvra

@toggledbits said in REST API implementation:

@mgvra said in REST API implementation:

I would have to be able to disable that particular rule without the MSR UI for the reasons I've explained in my previous post.

Your logic could include, for example, examination of a virtual switch that determines if the rule should be applied at all. This could be part of the triggers for that rule, or a constraint on that rule. This is a very common model. I myself use it for "Party Mode" in my house, to prevent the automatic locking of doors and the "Night mode" from dousing of all the lights that would cause an unwelcome interruption of the merriment.

Thanks for your quick response. Well it seems that I have to forget the REST API approach if it's going the get scrapped anyway. I just wanted to know the 'proper' way to go before I have too many rules to refactor. But as you pointed out, it seems that I have to go with the virtual switch / config object while integrating MSR to my existing backend/UI.

@gwp1 said in REST API implementation:

@mgvra said in REST API implementation:

I kind of think toggling the rules as a core functionality since it's already on the UI.

I see this opening up the proverbial can-o-worms wherein another obscure rule turns off the one you're using to drive something completely unrelated to the first.

Well I don't see it that way. You can already enable/disable the rules from the UI. Adding that same functionality to a different interface (REST API) woudn't open can-o-worms.

Please correct me if I'm wrong, but if you enable the rule I doesn't mean that it will trigger to true. It just evaluates the rule and if the conditions / constraints are met, it will trigger, otherwise not.

Of course if at the exact same moment you enable the rule and the conditions are, it will trigger, but that should happen anyway. So if you could do enabling / disabling via the REST API wouldn't it be just like hitting that red/green switch 'remotely'?

I think of rules as true/false vs on/off (knowing I could be miles away from how Patrick intended lol)

Well I think too. Maybe I was a bit unclear what I meant by toggling the rules. I didn't mean altering the rules evaluation outcome (true/false), just enabling/disabling the evaluation.

@toggledbits said in REST API implementation:

Enabling and disabling rules is not a way to add additional conditional logic to your automations, IMO, and I've stuck pretty firmly to that opinion (at least, I haven't been convinced otherwise to date).

Actually I don't see enabling / disabling rules as way to add additional conditional logic but merely as a user preference. For example I have a single rule that triggers lounge music playing through Sonos in my toilet whenever the PIR toggles the lights. But since I would like provide other users a possibility to switch off that feature, I would have to be able to disable that particular rule without the MSR UI for the reasons I've explained in my previous post.
And when I saw the REST API specs, that felt like a proper way to do it.

See also this topic: https://smarthome.community/post/13111

I think this post is more about the additional logic or maybe I'm just seeing it in a different light. I kind of think toggling the rules as a core functionality since it's already on the UI. You could do it with additional rules or entity attributes but it just feels more of hack. But maybe it's just me.

Hi,
@toggledbits do you have any plans to implement the missing endpoints of the REST API in near future? Especially the rule and ruleset functionality (list/enable/disable etc. as documented) would be really useful. I know there must tons of feature requests and the REST API isn't probably on the top of the backlog, so I'm not holding my breath. But if it's right around the corner on your roadmap, it would be nice to know.

I'm asking this because I need a clean way to enable and disable certain rules from my custom home automation UI. I don't want to expose the MSR UI to the rest of my family to make sure they don't accidentally mess up with the rules and settings.

I could use a custom MQTT entity as a dynamic config object containing flags for different rules but it would be a lot cleaner to be toggle the rules thru the REST API.

br,
mgvra

Reactor (Multi-hub) latest-22310-a6da8097
MQTTController [0.1.22306]

Thanks for your quick response!

@toggledbits said in MQTTController configuration with multiple topics per entity:

Is there a reason in the actions you don't just:

    actions:
      power_switch:
        "on":
          topic: "knx/%topic%/write"
          payload:
            expr: true
            type: json
        "off":
          topic: "knx/%topic%/write"
          payload:
            expr: false
            type: json

Well the problem with KNX is that you need at least two separate group addresses for read/updates and write. And the way the topics are defined in my configuration, there is no common section like %topic% in the middle. So I must do it like: /knx/%status%/read and /knx/%toggle%/write (e.g. /knx/0-0-1/read and /knx/0-0-2/write). Hopefully this makes sense in the config examples below.

I think it's much more flexible than you are imagining it is, and I'm hoping this key piece of information helps you discover that.

It definitely helped a lot and I've managed to get it working. Maybe I've just missed this in the docs. But thanks anyway!

Entity definitions look like this:

      # Lights (on/off)
        mh3-ceiling-light:
          name: "MH3 ceiling light"
          toggle: 0-0-55
          status: 0-0-56          
          type: Switch
          uses_template: knx-switch

      # Lights (dimmable)
        living-room-wall-lights:
          name: "Living room wall lights"
          toggle: 0-1-5
          status: 0-1-16
          absolute_dimming: 0-1-28        
          type: Switch
          uses_template: knx-dimmable

And their respective template definitions:

  knx-switch:
    query: "knx/%status%/read"
    capabilities:
      - power_switch
      - toggle
    primary_attribute: power_switch.state
    events:
      "knx/%status%/status":
        "power_switch.state":
          json_payload: true
          expr: "payload.data.payload.value"
    actions:
      power_switch:
        "on":
          topic: "knx/%toggle%/write"
          payload:
            expr: "true"
            type: json
        "off":
          topic: "knx/%toggle%/write"
          payload:
            expr: "false"
            type: json
        set:
          topic: "knx/%toggle%/write"
          payload:
            expr: "parameters.state"
            type: json

  knx-dimmable:
    query: "knx/%status%/read"
    capabilities:
      - power_switch
      - toggle
      - dimming
    primary_attribute: power_switch.state
    events:
      "knx/%status%/status":
        "power_switch.state":
          json_payload: true
          expr: "bool(payload.data.payload.value)"
        "dimming.level":
          json_payload: true
          expr: "payload.data.payload.value / 100"
    actions:
      power_switch:
        "on":
          topic: "knx/%toggle%/write"
          payload:
            expr: "true"
            type: json
        "off":
          topic: "knx/%toggle%/write"
          payload:
            expr: "false"
            type: json
        set:
          topic: "knx/%toggle%/write"
          payload:
            expr: "parameters.state"
            type: json
      dimming:
        set:
          topic: "knx/%absolute_dimming%/write"
          payload:
            expr: "parameters.level * 100"
            type: json

Edit: Please don't screen shot code and formatted data like YAML. Using the fenced code block formatting available and pasting the actual text is preferable (at least, to me, so I can copy-paste from your code to modify it if necessary, rather than having to retype it).

My bad, should have used the formatted code section.