Good morning,
I'm trying to figure out if there is a way to evaluate if a command was completed and retry if it did not complete.
I have 14 iBlinds 3.1 z-wave controllers in my home. 95% of the time, they work just fine. Occasionally, I'll get a blind that does not open on the first attempt. When I go into Home Assistant, and manually open or close the blind, it works.
I have 3 reactions set up for each room. One to open, one to close, and one partial open for sun glare. Each of them is set up as below.
5afc9924-4300-4718-9e23-8855c4a3a9fd-image.png
The reactions are set up to wait for 5 seconds before going onto the next blind, so the z-wave network doesn't get overwhelmed.
In addition, the set command to run the reactions has "Wait for completion" checked.
3919fc06-c1f1-4c49-bf95-716028d18a27-image.png
I also have a routine set up whenever a z-wave device reports as non-functional (dead), it'll get pinged to wake it up. This usually works to wake them up.
16df4bff-c733-4ec2-a55c-c964238ada3b-image.png
Appreciate any ideas to make this more reliable.
I'm running:
Reactor latest-24190-bd310acc, Bare-metal on Fedora WaveJSController [0.1.23326] Home Assistant: 2024.7.0I think this feature request could be accomplished with the use of two or more rules, but it would be great if there was a way to wait for an event or trigger to occur before continuing on in the reactions.
For example, I have a rule that will turn on some exterior lights if you arrive home after the porch lights have been turned off. Right now this rule randomly will turn off between 5-10 minutes after the person has entered the geofence. On some occasions this 5-10 minutes isn't long enough, say if you are unloading the car or something. I would like to kick off the reaction, but pause it part way through and wait for the door to close and lock, then continue it on. Hubitat Rule Machine has a "Wait for event" option, but I really want to keep all my logic within MSR.
Good morning,
I'm going through my ruleset this morning, trying to get away from haas>blahblablah entries and completely migrate them all to zwavejs>xx-0 entries where possible.
I have 3 Aeon MultiSensor 6 devices in my home, all USB powered.
When using Haas entries, I see an entry for hass>binary_sensor_guest_bedroom_multisensor_home_security_motion_detection, and motion_sensor.state (primary)
Screenshot 2024-07-25 at 8.25.53 AM.png
Under ZwaveJS, this entry appears to be missing.
Screenshot 2024-07-25 at 8.26.51 AM.png
From the Entities page:
battery_power.level=1
battery_power.since=1721910337433
binary_sensor.state=false
humidity_sensor.units="%"
humidity_sensor.value=46
light_sensor.units="Lux"
light_sensor.value=5
tamper.state=false
temperature_sensor.units="°F"
temperature_sensor.value=73.8
x_debug.dt={}
x_zwave_values.Basic_currentValue=0
x_zwave_values.Battery_isLow=false
x_zwave_values.Battery_level=100
x_zwave_values.Binary_Sensor_Any=false
x_zwave_values.Configuration_Automatic_Report_Group_1_Battery_1=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Humidity_64=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Luminance_128=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Temperature_32=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Ultraviolet_16=1
x_zwave_values.Configuration_Automatic_Report_Group_2_Battery_1=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Humidity_64=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Luminance_128=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Temperature_32=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Ultraviolet_16=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Battery_1=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Humidity_64=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Luminance_128=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Temperature_32=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Ultraviolet_16=0
x_zwave_values.Configuration_Automatic_Reporting_Interval_Group_1=3600
x_zwave_values.Configuration_Automatic_Reporting_Interval_Group_2=3600
x_zwave_values.Configuration_Automatic_Reporting_Interval_Group_3=3600
x_zwave_values.Configuration_Automatic_Temperature_Reporting_Unit=2
x_zwave_values.Configuration_Battery_Level_Threshold=10
x_zwave_values.Configuration_Current_Power_Mode_65280=0
x_zwave_values.Configuration_Humidity_Above_Lower_Limit_32=0
x_zwave_values.Configuration_Humidity_Below_Lower_Limit_2=0
x_zwave_values.Configuration_Humidity_Change_Threshold=10
x_zwave_values.Configuration_Humidity_Recover_Limit=5
x_zwave_values.Configuration_Humidity_Sensor_Calibration=0
x_zwave_values.Configuration_LED_Blinking=0
x_zwave_values.Configuration_Lighting_Recover_Limit=10
x_zwave_values.Configuration_Lock_Configuration=0
x_zwave_values.Configuration_Low_Battery_Threshold=20
x_zwave_values.Configuration_Low_Temperature_Alarm_15_C=0
x_zwave_values.Configuration_Lower_Humidity_Limit=50
x_zwave_values.Configuration_Lower_Lighting_Limit=100
x_zwave_values.Configuration_Lower_Temperature_Limit_4294901760=320
x_zwave_values.Configuration_Lower_Temperature_Limit_Unit_3840=2
x_zwave_values.Configuration_Lower_Ultraviolet_Limit=4
x_zwave_values.Configuration_Luminance_Above_Lower_Limit_64=0
x_zwave_values.Configuration_Luminance_Below_Lower_Limit_4=0
x_zwave_values.Configuration_Luminance_Change_Threshold=100
x_zwave_values.Configuration_Luminance_Sensor_Calibration=0
x_zwave_values.Configuration_Motion_Sensor_Report_Type_to_Send=2
x_zwave_values.Configuration_PIR_Sensitivity=3
x_zwave_values.Configuration_PIR_Sensor_Timeout=20
x_zwave_values.Configuration_Recover_Limit_Temperature_Unit_255=2
x_zwave_values.Configuration_Report_Above_Humidity_Threshold_32=0
x_zwave_values.Configuration_Report_Above_Luminance_Threshold_64=0
x_zwave_values.Configuration_Report_Above_Temperature_Threshold_16=0
x_zwave_values.Configuration_Report_Above_Ultraviolet_Threshold_128=0
x_zwave_values.Configuration_Report_Below_Humidity_Threshold_2=0
x_zwave_values.Configuration_Report_Below_Luminance_Threshold_4=0
x_zwave_values.Configuration_Report_Below_Temperature_Threshold_1=0
x_zwave_values.Configuration_Report_Below_Ultraviolet_Threshold_8=0
x_zwave_values.Configuration_Reset_Parameters_101_103_to_Default_Values=null
x_zwave_values.Configuration_Reset_Parameters_111_113_to_Default_Values=null
x_zwave_values.Configuration_Reset_to_Factory_Default_Setting=null
x_zwave_values.Configuration_Selective_Reporting=0
x_zwave_values.Configuration_Sleep_State_255=2
x_zwave_values.Configuration_Temperature_Above_Lower_Limit_16=0
x_zwave_values.Configuration_Temperature_Below_Lower_Limit_1=0
x_zwave_values.Configuration_Temperature_Calibration_Offset_65280=0
x_zwave_values.Configuration_Temperature_Calibration_Unit_255=2
x_zwave_values.Configuration_Temperature_Change_Threshold_4294901760=20
x_zwave_values.Configuration_Temperature_Recover_Limit_65280=20
x_zwave_values.Configuration_Temperature_Threshold_Unit_3840=2
x_zwave_values.Configuration_Ultraviolet_Above_Lower_Limit_128=0
x_zwave_values.Configuration_Ultraviolet_Below_Lower_Limit_8=0
x_zwave_values.Configuration_Ultraviolet_Change_Threshold=2
x_zwave_values.Configuration_Ultraviolet_Recover_Limit=2
x_zwave_values.Configuration_Ultraviolet_Sensor_Calibration=0
x_zwave_values.Configuration_Upper_Humidity_Limit=60
x_zwave_values.Configuration_Upper_Lighting_Limit=1000
x_zwave_values.Configuration_Upper_Temperature_Limit_4294901760=824
x_zwave_values.Configuration_Upper_Temperature_Limit_Unit_3840=2
x_zwave_values.Configuration_Upper_Ultraviolet_Limit=8
x_zwave_values.Configuration_Wake_Device_for_10_minutes_After_Power_On=1
x_zwave_values.Configuration_Wake_Up_Timeout=15
x_zwave_values.Manufacturer_Specific_manufacturerId=134
x_zwave_values.Manufacturer_Specific_productId=100
x_zwave_values.Manufacturer_Specific_productType=258
x_zwave_values.Multilevel_Sensor_Air_temperature=73.8
x_zwave_values.Multilevel_Sensor_Humidity=46
x_zwave_values.Multilevel_Sensor_Illuminance=5
x_zwave_values.Multilevel_Sensor_Ultraviolet=0
x_zwave_values.Notification_Home_Security_Cover_status=0
x_zwave_values.Notification_Home_Security_Motion_sensor_status=0
x_zwave_values.Notification_alarmLevel=0
x_zwave_values.Notification_alarmType=0
x_zwave_values.Version_firmwareVersions=["1.17"]
x_zwave_values.Version_hardwareVersion=100
x_zwave_values.Version_libraryType=3
x_zwave_values.Version_protocolVersion="4.54"
x_zwave_values.Wake_Up_controllerNodeId=1
x_zwave_values.Wake_Up_wakeUpInterval=3600
zwave_device.capabilities=[32,48,49,112,113,114,128,132,134]
zwave_device.endpoint=0
zwave_device.failed=false
zwave_device.generic_class="Multilevel Sensor"
zwave_device.impl_sig="23326:1:22315:1"
zwave_device.is_beaming=false
zwave_device.is_listening=true
zwave_device.is_routing=true
zwave_device.is_secure=false
zwave_device.manufacturer_info=[134,258,100]
zwave_device.max_data_rate=null
zwave_device.node_id=53
zwave_device.specific_class="Routing Multilevel Sensor"
zwave_device.status=4
zwave_device.status_text="alive"
zwave_device.version_info=[null,"1.17"]
zwave_device.wakeup_interval=3600
I'm running:
Reactor latest-24190-bd310acc, Bare-metal on Fedora
WaveJSController [0.1.23326]
Home Assistant: 2024.7.0
I'm fetching certain data five past every hour, but I would like to do it closer to the hour, e.g. 1 or 2 past (but not at the hour).
I experimented with the following rule that almost works (triggers also at the hour which is not the intention). Any advice for a solution?
It would be nice to have an ability to bookmark a direct link to a dashboard item. In my case I would use this feature to access a virtual switch directly.
Hi @toggledbits
Would you please consider adding an extra sublevel in the rulesets?
I have grouped my rules in rooms/ areas. This works great for me, but I would also like to group rules for the same functionality (in a room). This would make the rules easier to find and name.
Please let me know if this is an option. Thanks!
@togglebits I am curious as to why the tilt_sensor.state (primary) = NULL. I believe it should show true or false. I have to use binary_sensor.state instead in my rules.
Again, not sure if this is related to Reactor/ZwaveJSController implementation or the actual Z-Wave JS UI docker version. I have copied, below, the attributes of the tilt sensor in hopes it can help.
Thanks in advance.
Reactor version 23302
ZWaveJSController version 23254
Z-Wave JS UI version 9.3.0.724519f
zwave-js version 12.2.3
@toggledbits,
I have this device attached to my system, but use a DSC panel. If you need testers to move forward, I'm happy to help.
I'm curious what your thinking the use case is for this. I currently have it integrated into HomeAssistant, and it works fine for the most part. The one thing I can't do is bypass zones, which I would like to have the ability to do.
Are you looking at more direct control for the panel, as opposed to having to jump through HA (or another system) first?
Build 21228 has been released. Docker images available from DockerHub as usual, and bare-metal packages here.
Home Assistant up to version 2021.8.6 supported; the online version of the manual will now state the current supported versions; Fix an error in OWMWeatherController that could cause it to stop updating; Unify the approach to entity filtering on all hub interface classes (controllers); this works for device entities only; it may be extended to other entities later; Improve error detail in messages for EzloController during auth phase; Add isRuleSet() and isRuleEnabled() functions to expressions extensions; Implement set action for lock and passage capabilities (makes them more easily scriptable in some cases); Fix a place in the UI where 24-hour time was not being displayed.I have the following ruleset which I though had been working well until this morning when I noticed it's not.
I've put four weather conditions in an in array and one of them is the current weather condition - but the rule is not true. Now the cloud cover percentage is not yet met but this is an or rule so as long as the "Current Conditions" are met then it should go true.
What's the obvious thing I'm missing here? (I've tried spaces/no spaces in the in and no difference.)
Hi,
Running the latest MSR latest-24152-3455578a with the latest HA 2024.6.1. When trying to call a service I get the following in the MSR logs. Is this a version mismatch? I am not seeing anything in the HA logs.
[latest-24152]2024-06-11T10:29:56.162Z <Rule:INFO> rule-Monitor-DHW (rule-lsvq5k3x in Central Heating) started [latest-24152]2024-06-11T10:29:58.625Z <HassController:WARN> HassController#hass unknown service opentherm_gw.set_hot_water_setpoint in x_hass.call_service action on Thermostat#hass>climate_living_room_otgw [latest-24152]2024-06-11T10:29:58.626Z <HassController:INFO> HassController#hass: sending payload for x_hass.call_service on Thermostat#hass>climate_living_room_otgw action: [Object]{ "type": "call_service", "service_data": { "gateway_id": "living_room_otgw", "temperature": 65 }, "domain": "opentherm_gw", "service": "set_hot_water_setpoint", "target": { "entity_id": "climate.living_room_otgw" } } [latest-24152]2024-06-11T10:29:58.627Z <HassController:ERR> HassController#hass request 1718101798626<6/11/2024, 12:29:58 PM> (call_service) failed: [Object]{ "id": 1718101798626, "type": "result", "success": false, "error": { "code": "invalid_format", "message": "extra keys not allowed @ data['entity_id']" } } [latest-24152]2024-06-11T10:29:58.627Z <HassController:WARN> HassController#hass action x_hass.call_service([Object]{ "service": "opentherm_gw.set_hot_water_setpoint", "data": "{ \"gateway_id\": \"living_room_otgw\", \"temperature\": 65 }" }) on Thermostat#hass>climate_living_room_otgw failed!Cheers Rene
Some background
I'm trying to integrate a Zigbee device into the MSR using zigbee2mqtt bridge and MQTTController. The device in question is a cheap mood light that has following properties that I'd like to control:
I'v already managed to get the switch part working and can toggle the light on/off. Also the brightness value is mapped back to MSR. In zigbee2mqtt it has a value range from 0 to 254, so this the reason for the expression:
expr: 'payload.brightness / 254'Here's the entity definition (don't know whether the type should be something else than the Switch)
zigbee-lidl-mood-light: name: 'Lidl Mood Light' friendly_name: 'Mood Light' type: Switch uses_template: lidl-moodlightAnd the corresponding template (NOTE: rgb_color has not been defined in this example):
lidl-moodlight: init: "zigbee2mqtt/%friendly_name%/get/state" query: "zigbee2mqtt/%friendly_name%/get/state" capabilities: - power_switch - toggle - dimming primary_attribute: power_switch.state events: "zigbee2mqtt/%friendly_name%": "power_switch.state": json_payload: true expr: 'upper(payload.state) == "ON"' "dimming.level": json_payload: true expr: 'payload.brightness / 254' actions: power_switch: "on": topic: "zigbee2mqtt/%friendly_name%/set/state" payload: 'ON' "off": topic: "zigbee2mqtt/%friendly_name%/set/state" payload: 'OFF' set: topic: "zigbee2mqtt/%friendly_name%/set/state" payload: expr: "parameters.state ? 'ON' : 'OFF'" type: raw toggle: topic: "zigbee2mqtt/%friendly_name%/set/state" payload: 'TOGGLE'The problem
In order to control the brightness or the RGB color values, I would have send a JSON payload in corresponding actions. But I have no idea how to define it in the template. The reason why the switch part is working is that the zigbee2mqtt accepts also plain ON / OFF / TOGGLE string payloads in that case.
But the brightness should be controlled with the following payload:
{"brightness": 196}And the RGB color like:
{"color":{"rgb":"46,102,150"}}Here's the link for the documentation (the Exposes part defines the messages).
So how should I define the JSON payload for example for the dimming action? It definitely should be some sort of expressions since I have to map the MSR real value (0...1) to (0...254) for the zigbee2mqtt.
actions: dimming: set: topic: "zigbee2mqtt/%friendly_name%/set" payload: expr: ????? type: jsonAnother problem is the RGB value. I could use the rgb_color capability for the setting but the problem is that the zigbee2mqtt only reports the current color in hue/saturation or xy coordinates.
Here's an example of published message after setting the color:
Topic: zigbee2mqtt/Mood Light QoS: 0 { "brightness":254, "color":{ "hue":240, "saturation":100, "x":0.1355, "y":0.0399 }, "color_mode":"xy", "color_temp":574, "linkquality":96, "state":"ON" }I would have to map those values back to RGB, but is it even possible with existing constructs in MQTTController's templates?
Help would be appreciated @toggledbits
br,
mgvra
That's probably more appropriate to post on Mantis for @toggledbits, but since I know there's at least @Crille publishing templates, my intent with this post is to open a broader discussion.
Long story short: I'm starting to slowly add new template for Shelly Plus and I noticed I'll end up with a dozen more templates, all similar but simply different in trivial details, all sharing a large amount of code and all needing special cares when fixing bugs/adding features (as the latest wifi_status addition).
So, I'm wondering if it's time to start thinking of some sort of inheritance in templates, where I could just create a generic shelly_gen1 and use it as a base for shelly_relay, and this be used as the base for shelly_relay_power and so on.
I could probably achieve this with some sort of scripting on my side to generate templates via code, but maybe there's a better way of doing this, or it's already on the radar.
I need a handful of victims volunteers to help test previews of the next build of Reactor. A long-standing request was for "a simple login mechanism," but in practice, adding user authentication and competent access control turned out to be a pretty big project with a lot of big changes on both server and client sides. It's a bit more than I'm comfortable testing myself and springing out to everyone at once, so I'd like to work with a small group to put it through "sea trials."
Major changes/features include:
User authentication with hashed password storage; User group configuration with application restriction (admin, dashboard, API); Detailed control over API access, with user- and token-based authentication/authorization; Improvements to the HTTPS service; Improvements to UI coordination with the core for Rules and Reactions.If this sounds like something you'd like to help with, drop me a reply here in this thread or privately.
[Solution: Reactor 24115 is not compatible with MQTTController > 24120]
Reactor latest-24115 bare metal.
All MQTT entities stop working after updating MQTTController to 24142, downgrade to 24120 and they are back. Templates and configured entities has not been changed between versions.
I'm not sure if uses_template should be replaced by ìnclude in entity configuration in reactor.yaml but I guess not, I've tried it and it did not do any difference.
I have not tried to update Reactor to userauth version.
Example entity in reactor.yaml that uses MQTTController included template:
entities: "takflakt_kallare": name: "Takfläkt källare" topic: "Källartemp" unit: "" uses_template: tasmota_generic_relay init: "cmnd/%topic%/POWER%unit%"Any hints? Do you need more info, please let me know.
Log from startup:
I'm slowly migrating all my stuff to MQTT under MSR, so I have a central place to integrate everything (and, in a not-so-distant future, to remove virtual devices from my Vera and leave it running zwave only).
Anyway, here's my reactor-mqtt-contrib package:
![](https://github.com/fluidicon.png)
Contrib MQTT templates for Reactor. Contribute to dbochicchio/reactor-mqtt-contrib development by creating an account on GitHub.
Simply download yaml files (everything or just the ones you need) and you're good to go.
I have mapped my most useful devices, but I'll add others soon. Feel free to ask for specific templates, since I've worked a lot in the last weeks to understand and operate them.
The templates are supporting both init and query, so you have always up-to-date devices at startup, and the ability to poll them. Online status is supported as well, so you can get disconnected devices with a simple expression.
Many-many thanks to @toggledbits for its dedication, support, and patience with me and my requests 🙂
Is the following config correct?
- id: time_series name: "Out temp" capabilities: temperature_sensor: attributes: value: model: time series entity: "hass>sensor_outdoor_temperature" attribute: "temperature_sensor.value" interval: 5 # minutes retention: 20 # minutes aggregate: wma primary_attribute: temperature_sensor.value type: ValueSensorSpecifically, is "depth" directive needed/mandatory here? Reason I'm asking is that I'm not getting a "final" value in MSR, only debug values are shown:
temperature_sensor.units=null temperature_sensor.value=null x_virtualentity.last_request_time=null x_virtualentity.request_failures=null x_virtualentity.template=null x_virtualentity.timeseries_debug=[{"time":1716537360000,"value":22.1},{"time":1716537660000,"value":22},{"time":1716537960000,"value":22},{"time":1716538260000,"value":21.9},{"time":1716538560000,"value":22}]Good morning,
I apologize if this subject has been covered. I did try the search, but I'm not coming up with any topics on my issue.
I'm running userauth-24137-57b41335, bare metal installation on Fedora 39 Server.
I have a rule set up to turn the Eco mode off on my Nest Thermostat when the thermostat is set to Away (Rule State: Away Mode), the user (Driver) presence in my car changes to true, and the destination is set to home.
93804f7c-62d6-42c0-ae04-ff602011a6fd-image.png
This works fine for most days, where I'm headed home from work (commute is about 45 minutes). What I don't want it to do is set change it to Eco mode if my ETA is more than an hour.
There is a sensor/entity for Time to Arrival when the Destination is set. What it appears to provide is the Time OF arrival/ETA, not time until arrival. If it was Time until Arrival, and it was a numeric value, I could simply test if the value is less than 60 and be done with it.
I pulled up the history through Home Assistant for my morning commute and this appears to be what it is providing.
c2a32739-c84f-4a05-95d9-73793ed818f5-image.png
So what I need to do is to do a calculation of the the ETA from the sensor value and subtract the current time, and get a value in minutes that I can determine if it's less than 60.
I believe I can do this with the local expression, but I don't see anything for the system time, or local time. Also, do the local expressions update themselves if the sensors do?
Good morning,
I'm running userauth-24137-57b41335 on Fedora 39, bare metal installation.
ZWaveJSController 0.1.23254
Home Assistant:
Core, 2024.5.3 Supervisor, 2024.05.1 Operating System, 12.3 Frontend, 20240501.1I'm trying to troubleshoot a Dynamic Group Controller and notification alert that I've set up for low battery level.
In my Reactor.config, I have the following lines:
name: "Dynamic Group Controller" implementation: DynamicGroupController config: groups: "zwavejs_dead": select: - include_group: "zwavejs" filter_expression: "entity?.attributes?.zwave_device?.status == 3" group_actions: true "low_battery": select: - include_capability: battery_power filter_expression: > entity.attributes.battery_power.level < 0.35The idea here is that I should only have members of this group that have a battery level below 35%. When I go into Entities, I show a whole slew of devices, none of which have a battery level below the threshold.
a77e445b-ab78-4752-a624-3c4117f34f90-image.png
I also tried setting up a rule to generate a push notification once a day, but with all of the group members, I've had to disable the rule. I believe I have it set up correctly, but I'm not 100% sure. I want the notification to tell me the battery level for that device as well.
289b4f68-03ba-49c0-8275-f0f197d13a3a-image.png
ce24a76e-6865-40bd-bd85-632e54d315a8-image.png dc837424-deb5-4ef7-8f0d-3676f1769535-image.png
Can anyone point to me what I may have misconfigured to get these results?
I should also note I'm only interested in ZWaveJS devices. It's showing me battery status for my iPad and car as well, which I don't need it to send me.
[Solved] latest-22328 restart fails
-
Solution: An update to openLuups MQTT implementation on handling acknowledge packets for QoS > 0 solved this issue.
EDIT: Seems related to any restart without any configuration changes. If I revert to 22310 I can restart Reactor from UI and with systemd within seconds, but with 22328 it fails to restart., both from UI and systemd. I have to stop the service and then start again.
I tested to comment out my http (not https)?baseurl
in the config on my bare metal Ubuntu install latest-22328 and triggered a restart from the UI but Reactor would not start after that.
If I uncomment the key and restart the service, Reactor comes back to life.
Is my setup an exceptional circumstance or is this only applicable on new installs -
It's working for me, and I've done some fresh installs of 22328 while testing alternatives to Raspberry Pi (a couple of promising boards so far). I hate to say it, but your post is right on the line of "I tried X and it didn't work for me," so without more detail, I can't really guide you.
-
@toggledbits said in latest-22328 and baseurl [EDIT]: restarts fail:
It's working for me
This is at first all I wanted to know, if anyone else was having the same issue or it's just my setup.
Sorry for the lack of details, I did not know what else to provide as the log is dead silent after shutting down...
I will debug further.
-
Looks like it's related to openLuups MQTT server. The shutdown process hangs after sending LWT to openLuup (MQTTController#mqtt in log).
Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.775Z <app:NOTICE> Closing Structure... Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.776Z <Structure:INFO> Structure#1 Stopping controllers... Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.776Z <Controller:NOTICE> VeraController#vera stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.778Z <Controller:ERR> Controller VeraController#vera is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.799Z <EzloController:NOTICE> EzloController#ezlo stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.800Z <wsapi:WARN> client close from unknown connection? "192.168.1.2#5" Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.800Z <wsapi:WARN> client close from unknown connection? "192.168.1.238#4" Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.800Z <wsapi:WARN> client close from unknown connection? "192.168.1.238#3" Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.800Z <wsapi:WARN> client close from unknown connection? "192.168.1.238#2" Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.800Z <wsapi:WARN> client close from unknown connection? "192.168.1.2#1" Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.802Z <EzloController:NOTICE> EzloController#ezlo connection closed: 1000 closing Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.804Z <Controller:ERR> Controller EzloController#ezlo is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.804Z <Controller:NOTICE> EzloController#ezlo stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.806Z <DynamicGroupController:null> DynamicGroupController#groups stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.806Z <Controller:NOTICE> DynamicGroupController#groups stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.808Z <Controller:ERR> Controller DynamicGroupController#groups is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.809Z <HassController:NOTICE> HassController#hass stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.810Z <HassController:NOTICE> HassController#hass websocket closing, 1000 Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.810Z <Controller:NOTICE> HassController#hass stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.811Z <Controller:ERR> Controller HassController#hass is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.823Z <Controller:NOTICE> OWMWeatherController#weather stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.824Z <Controller:ERR> Controller OWMWeatherController#weather is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.825Z <Controller:NOTICE> SystemController#reactor_system stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.826Z <Controller:ERR> Controller SystemController#reactor_system is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.827Z <MQTTController:NOTICE> MQTTController#mosquitto-mqtt stopping, sending LWT Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.870Z <MQTTController:NOTICE> LWT sent; closing broker connection Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.870Z <Controller:NOTICE> MQTTController#mosquitto-mqtt stopping Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.871Z <Controller:ERR> Controller MQTTController#mosquitto-mqtt is off-line! Nov 26 10:45:58 homebridge node[686832]: [latest-22328]2022-11-26T09:45:58.874Z <MQTTController:NOTICE> MQTTController#mqtt stopping, sending LWT Nov 26 10:46:03 homebridge node[686832]: [latest-22328]2022-11-26T09:46:03.452Z <httpapi:INFO> HTTP server closed.
If I disable that controller Reactor restarts fine as usual.
@toggledbits Is this the moment openLuup MQTT support ends in Reactor or can I change something to make it work again?
@akbooer I've updated openLuup from 22.9.3 to 22.11.22 but no success to this issue.The thing is it's running great when it finally comes up again after a long wait of a forced restart with systemd but it doesn't feel right to force kill the process even though the shutdown process is almost finished.
-
@crille said in latest-22328 restart fails:
Looks like it's related to openLuups MQTT server. The shutdown process hangs after sending LWT to openLuup (MQTTController#mqtt in log).
I think we've seen an issue like that previously. I don't think I see that line in the log you posted?
There may be a problem with retained messages (ie. LWT) and wildcard subscriptions in the openLuup server... I'd have to check.
-
I use a popular package to handle the MQTT broker connection, so I can't see the innards of communications and confirm, but from the rhythm of the log output, it appears that the broker is not sending an ACK to the publish of the LWT; the
publish()
call appears to be sitting there waiting for it. -
Does this mean 22328 require an ACK but 22310 does not? or has something else changed?
-
22310 still required an ACK, in a sense. It didn't wait for it. But because the ACK never arrived, the task in the
mqtt
package also never cleared (they don't have a time-out mechanism), so it just stayed in the queue forever... as did every other topic sent with a non-zero QoS. This actually causes a memory leak that could lead to exhaustion and a crash, because the ACK never comes so the task is never removed from the queue, and those tasks remain and proliferate in the queue.IMO, I think it's fine if @akbooer doesn't truly support QoS levels 1 and 2 in his MQTT implementation just for basic use, but not sending an ACK regardless isn't the right choice, in my view, and it's going to cause problems for a lot of clients that may not be immediately evident (like memory leaks/exhaustion).
-
@toggledbits said in latest-22328 restart fails:
I think it's fine if @akbooer doesn't truly support QoS levels 1 and 2 in his MQTT implementation just for basic use
It only supports QoS 0 .
Nevertheless, the protocol should be respected. AFAIK all PUBLISH requests receive an ACK unless the connection goes down in between times.
-
This is the end of
parse.PUBLISH()
:-- ACKNOWLEDGEMENT -- The receiver of a PUBLISH Packet MUST respond according to Table 3.4 - Expected Publish Packet -- response as determined by the QoS in the PUBLISH packet [MQTT-3.3.4-1] --[[ Table 3.4 - Expected Publish Packet response QoS Level Expected Response QoS 0 None QoS 1 PUBACK Packet QoS 2 PUBREC Packet --]] local ack -- None, because we only handle QoS 0 return ack, nil, TopicName, ApplicationMessage, RETAIN end
Comments to the contrary, it appears it returns
ack
, which is declared butnil
... so... no ACK? -
@toggledbits does Reactor publish it's LWT message with QoS > 0 even though the MQTTController config is at
qos: 0
? otherwise the expected response would be none. -
Yes, it uses QoS 1 (and retain true) because it's a "vital" message. The
qos
you can set in config is for the echo/entity publish functionality; it does not affect other messages. Still, this only requires that the broker acknowledge its receipt (3.3.4), not any delivery, and does not even enforce that QoS on subscribers (3.8.3) -
"The receiver of a PUBLISH Packet MUST respond according to Table 3.4 - Expected Publish Packet response as determined by the QoS in the PUBLISH Packet."
So even though the server only supports QoS 0 it's obligated to send a PUBACK on a QoS 1 level packet as described in table 3.4, correct?
-
Yes, PUBACK for QoS 1, and PUBREC for QoS 2. I don't think that would be a big problem for @akbooer when he gets the time, because all of the information contained in the response can be sourced from the request (i.e. topic, packet identifier, etc.). And otherwise the treatment of the PUBLISH packet can be the same (no further changes beyond sending those ACKs). Not actually having guaranteed delivery behind that is, in my view, an acceptable variance. On the sending side (repeat to subscribers), even though a subscriber may request QoS 1 or 2 for packets from the broker, it still must accept QoS 0 packets (because the requested QoS is a maximum, not an absolute), so everything outbound at QoS 0 isn't likely going to cause problems, especially in this world.
To summarize: if he just provides the PUBACK and PUBREC responses to fix that layer of the protocol, that's good enough. No further actions required above that layer. Not fully compliant (no guaranteed delivery), but at that point, few if any would ever notice.
The other question I don't have an answer for (or I've forgotten; and haven't dug through code to figure out)... @akbooer, does it support retain? If so, what's the storage mechanism, and is it persistent?
-
Well, I must be reading the MQTT 3.1.1 spec all wrong.
When I read:
"The SUBACK Packet sent by the Server to the Client MUST contain a return code for each Topic Filter/QoS pair. This return code MUST either show the maximum QoS that was granted for that Subscription or indicate that the subscription failed [MQTT-3.8.4-5]. The Server might grant a lower maximum QoS than the subscriber requested. The QoS of Payload Messages sent in response to a Subscription MUST be the minimum of the QoS of the originally published message and the maximum QoS granted by the Server. The server is permitted to send duplicate copies of a message to a subscriber in the case where the original message was published with QoS 1 and the maximum QoS granted was QoS 0 [MQTT-3.8.4-6]."
...I understood it to mean that, since I only ever grant QoS 0, then no message would require a PUBACK or a PUBREC.
I realise that this is my bad for writing my own MQTT broker, but it made sense in the context of openLuup, especially in support of Shelly devices which was the reason I did it in the first place. I apologise if this has led to some difficulties, and I'm starting to look at an MQTT validation suite to check out my implementation further. I also realize that Mosquitto is the de-facto standard, but it turns out that having an internal server confers some significant benefits in terms of the internal openLuup architecture.
However, if anyone can clarify the about QoS response issue further, I'm very happy to comply. If a simple fix is to send PUBACK or PUBREC, then I'll do it, but I want to know the reason why. I do understand that the original CONNECT request contains a LWT QoS, per this paragraph:
3.1.2.6 Will QoS
"Position: bits 4 and 3 of the Connect Flags."
"These two bits specify the QoS level to be used when publishing the Will Message."
"If the Will Flag is set to 0, then the Will QoS MUST be set to 0 (0x00) [MQTT-3.1.2-13]."
"If the Will Flag is set to 1, the value of Will QoS can be 0 (0x00), 1 (0x01), or 2 (0x02). It MUST NOT be 3 (0x03) [MQTT-3.1.2-14]."
...but I had assumed that QoS to be overridden by the actual level established in SUBSCRIBE / SUBACK. However, now that I write that, it seems the LWT actually may have a separate life from standard messages?
-
@toggledbits said in latest-22328 restart fails:
The other question I don't have an answer for (or I've forgotten; and haven't dug through code to figure out)... @akbooer, does it support retain? If so, what's the storage mechanism, and is it persistent?
Yes, it supports retained messages.
Is it persistent? Not across openLuup restarts.
The openLuup console page: openLuupIP:3480/console?page=mqtt shows all current subscriptions and also (at the bottom) retained messages.
-
@akbooer said in latest-22328 restart fails:
...I understood it to mean that, since I only ever grant QoS 0, then no message would require a PUBACK or a PUBREC.
This section is about your response to a SUBSCRIBE (SUBACK)and what QoS you use to PUBLISH. If you only grant QoS 0 on subscribe, then you will never expect a PUBACK or PUBREC because you never publish anything to a subscriber other than QoS 0.
It in no way limits what a client may publish to the broker, which could include any QoS.