Good morning,
I'm trying to figure out if there is a way to evaluate if a command was completed and retry if it did not complete.
I have 14 iBlinds 3.1 z-wave controllers in my home. 95% of the time, they work just fine. Occasionally, I'll get a blind that does not open on the first attempt. When I go into Home Assistant, and manually open or close the blind, it works.
I have 3 reactions set up for each room. One to open, one to close, and one partial open for sun glare. Each of them is set up as below.
5afc9924-4300-4718-9e23-8855c4a3a9fd-image.png
The reactions are set up to wait for 5 seconds before going onto the next blind, so the z-wave network doesn't get overwhelmed.
In addition, the set command to run the reactions has "Wait for completion" checked.
3919fc06-c1f1-4c49-bf95-716028d18a27-image.png
I also have a routine set up whenever a z-wave device reports as non-functional (dead), it'll get pinged to wake it up. This usually works to wake them up.
16df4bff-c733-4ec2-a55c-c964238ada3b-image.png
Appreciate any ideas to make this more reliable.
I'm running:
Reactor latest-24190-bd310acc, Bare-metal on Fedora WaveJSController [0.1.23326] Home Assistant: 2024.7.0I think this feature request could be accomplished with the use of two or more rules, but it would be great if there was a way to wait for an event or trigger to occur before continuing on in the reactions.
For example, I have a rule that will turn on some exterior lights if you arrive home after the porch lights have been turned off. Right now this rule randomly will turn off between 5-10 minutes after the person has entered the geofence. On some occasions this 5-10 minutes isn't long enough, say if you are unloading the car or something. I would like to kick off the reaction, but pause it part way through and wait for the door to close and lock, then continue it on. Hubitat Rule Machine has a "Wait for event" option, but I really want to keep all my logic within MSR.
Good morning,
I'm going through my ruleset this morning, trying to get away from haas>blahblablah entries and completely migrate them all to zwavejs>xx-0 entries where possible.
I have 3 Aeon MultiSensor 6 devices in my home, all USB powered.
When using Haas entries, I see an entry for hass>binary_sensor_guest_bedroom_multisensor_home_security_motion_detection, and motion_sensor.state (primary)
Screenshot 2024-07-25 at 8.25.53 AM.png
Under ZwaveJS, this entry appears to be missing.
Screenshot 2024-07-25 at 8.26.51 AM.png
From the Entities page:
battery_power.level=1
battery_power.since=1721910337433
binary_sensor.state=false
humidity_sensor.units="%"
humidity_sensor.value=46
light_sensor.units="Lux"
light_sensor.value=5
tamper.state=false
temperature_sensor.units="°F"
temperature_sensor.value=73.8
x_debug.dt={}
x_zwave_values.Basic_currentValue=0
x_zwave_values.Battery_isLow=false
x_zwave_values.Battery_level=100
x_zwave_values.Binary_Sensor_Any=false
x_zwave_values.Configuration_Automatic_Report_Group_1_Battery_1=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Humidity_64=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Luminance_128=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Temperature_32=1
x_zwave_values.Configuration_Automatic_Report_Group_1_Ultraviolet_16=1
x_zwave_values.Configuration_Automatic_Report_Group_2_Battery_1=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Humidity_64=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Luminance_128=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Temperature_32=0
x_zwave_values.Configuration_Automatic_Report_Group_2_Ultraviolet_16=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Battery_1=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Humidity_64=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Luminance_128=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Temperature_32=0
x_zwave_values.Configuration_Automatic_Report_Group_3_Ultraviolet_16=0
x_zwave_values.Configuration_Automatic_Reporting_Interval_Group_1=3600
x_zwave_values.Configuration_Automatic_Reporting_Interval_Group_2=3600
x_zwave_values.Configuration_Automatic_Reporting_Interval_Group_3=3600
x_zwave_values.Configuration_Automatic_Temperature_Reporting_Unit=2
x_zwave_values.Configuration_Battery_Level_Threshold=10
x_zwave_values.Configuration_Current_Power_Mode_65280=0
x_zwave_values.Configuration_Humidity_Above_Lower_Limit_32=0
x_zwave_values.Configuration_Humidity_Below_Lower_Limit_2=0
x_zwave_values.Configuration_Humidity_Change_Threshold=10
x_zwave_values.Configuration_Humidity_Recover_Limit=5
x_zwave_values.Configuration_Humidity_Sensor_Calibration=0
x_zwave_values.Configuration_LED_Blinking=0
x_zwave_values.Configuration_Lighting_Recover_Limit=10
x_zwave_values.Configuration_Lock_Configuration=0
x_zwave_values.Configuration_Low_Battery_Threshold=20
x_zwave_values.Configuration_Low_Temperature_Alarm_15_C=0
x_zwave_values.Configuration_Lower_Humidity_Limit=50
x_zwave_values.Configuration_Lower_Lighting_Limit=100
x_zwave_values.Configuration_Lower_Temperature_Limit_4294901760=320
x_zwave_values.Configuration_Lower_Temperature_Limit_Unit_3840=2
x_zwave_values.Configuration_Lower_Ultraviolet_Limit=4
x_zwave_values.Configuration_Luminance_Above_Lower_Limit_64=0
x_zwave_values.Configuration_Luminance_Below_Lower_Limit_4=0
x_zwave_values.Configuration_Luminance_Change_Threshold=100
x_zwave_values.Configuration_Luminance_Sensor_Calibration=0
x_zwave_values.Configuration_Motion_Sensor_Report_Type_to_Send=2
x_zwave_values.Configuration_PIR_Sensitivity=3
x_zwave_values.Configuration_PIR_Sensor_Timeout=20
x_zwave_values.Configuration_Recover_Limit_Temperature_Unit_255=2
x_zwave_values.Configuration_Report_Above_Humidity_Threshold_32=0
x_zwave_values.Configuration_Report_Above_Luminance_Threshold_64=0
x_zwave_values.Configuration_Report_Above_Temperature_Threshold_16=0
x_zwave_values.Configuration_Report_Above_Ultraviolet_Threshold_128=0
x_zwave_values.Configuration_Report_Below_Humidity_Threshold_2=0
x_zwave_values.Configuration_Report_Below_Luminance_Threshold_4=0
x_zwave_values.Configuration_Report_Below_Temperature_Threshold_1=0
x_zwave_values.Configuration_Report_Below_Ultraviolet_Threshold_8=0
x_zwave_values.Configuration_Reset_Parameters_101_103_to_Default_Values=null
x_zwave_values.Configuration_Reset_Parameters_111_113_to_Default_Values=null
x_zwave_values.Configuration_Reset_to_Factory_Default_Setting=null
x_zwave_values.Configuration_Selective_Reporting=0
x_zwave_values.Configuration_Sleep_State_255=2
x_zwave_values.Configuration_Temperature_Above_Lower_Limit_16=0
x_zwave_values.Configuration_Temperature_Below_Lower_Limit_1=0
x_zwave_values.Configuration_Temperature_Calibration_Offset_65280=0
x_zwave_values.Configuration_Temperature_Calibration_Unit_255=2
x_zwave_values.Configuration_Temperature_Change_Threshold_4294901760=20
x_zwave_values.Configuration_Temperature_Recover_Limit_65280=20
x_zwave_values.Configuration_Temperature_Threshold_Unit_3840=2
x_zwave_values.Configuration_Ultraviolet_Above_Lower_Limit_128=0
x_zwave_values.Configuration_Ultraviolet_Below_Lower_Limit_8=0
x_zwave_values.Configuration_Ultraviolet_Change_Threshold=2
x_zwave_values.Configuration_Ultraviolet_Recover_Limit=2
x_zwave_values.Configuration_Ultraviolet_Sensor_Calibration=0
x_zwave_values.Configuration_Upper_Humidity_Limit=60
x_zwave_values.Configuration_Upper_Lighting_Limit=1000
x_zwave_values.Configuration_Upper_Temperature_Limit_4294901760=824
x_zwave_values.Configuration_Upper_Temperature_Limit_Unit_3840=2
x_zwave_values.Configuration_Upper_Ultraviolet_Limit=8
x_zwave_values.Configuration_Wake_Device_for_10_minutes_After_Power_On=1
x_zwave_values.Configuration_Wake_Up_Timeout=15
x_zwave_values.Manufacturer_Specific_manufacturerId=134
x_zwave_values.Manufacturer_Specific_productId=100
x_zwave_values.Manufacturer_Specific_productType=258
x_zwave_values.Multilevel_Sensor_Air_temperature=73.8
x_zwave_values.Multilevel_Sensor_Humidity=46
x_zwave_values.Multilevel_Sensor_Illuminance=5
x_zwave_values.Multilevel_Sensor_Ultraviolet=0
x_zwave_values.Notification_Home_Security_Cover_status=0
x_zwave_values.Notification_Home_Security_Motion_sensor_status=0
x_zwave_values.Notification_alarmLevel=0
x_zwave_values.Notification_alarmType=0
x_zwave_values.Version_firmwareVersions=["1.17"]
x_zwave_values.Version_hardwareVersion=100
x_zwave_values.Version_libraryType=3
x_zwave_values.Version_protocolVersion="4.54"
x_zwave_values.Wake_Up_controllerNodeId=1
x_zwave_values.Wake_Up_wakeUpInterval=3600
zwave_device.capabilities=[32,48,49,112,113,114,128,132,134]
zwave_device.endpoint=0
zwave_device.failed=false
zwave_device.generic_class="Multilevel Sensor"
zwave_device.impl_sig="23326:1:22315:1"
zwave_device.is_beaming=false
zwave_device.is_listening=true
zwave_device.is_routing=true
zwave_device.is_secure=false
zwave_device.manufacturer_info=[134,258,100]
zwave_device.max_data_rate=null
zwave_device.node_id=53
zwave_device.specific_class="Routing Multilevel Sensor"
zwave_device.status=4
zwave_device.status_text="alive"
zwave_device.version_info=[null,"1.17"]
zwave_device.wakeup_interval=3600
I'm running:
Reactor latest-24190-bd310acc, Bare-metal on Fedora
WaveJSController [0.1.23326]
Home Assistant: 2024.7.0
I'm fetching certain data five past every hour, but I would like to do it closer to the hour, e.g. 1 or 2 past (but not at the hour).
I experimented with the following rule that almost works (triggers also at the hour which is not the intention). Any advice for a solution?
It would be nice to have an ability to bookmark a direct link to a dashboard item. In my case I would use this feature to access a virtual switch directly.
Hi @toggledbits
Would you please consider adding an extra sublevel in the rulesets?
I have grouped my rules in rooms/ areas. This works great for me, but I would also like to group rules for the same functionality (in a room). This would make the rules easier to find and name.
Please let me know if this is an option. Thanks!
@togglebits I am curious as to why the tilt_sensor.state (primary) = NULL. I believe it should show true or false. I have to use binary_sensor.state instead in my rules.
Again, not sure if this is related to Reactor/ZwaveJSController implementation or the actual Z-Wave JS UI docker version. I have copied, below, the attributes of the tilt sensor in hopes it can help.
Thanks in advance.
Reactor version 23302
ZWaveJSController version 23254
Z-Wave JS UI version 9.3.0.724519f
zwave-js version 12.2.3
@toggledbits,
I have this device attached to my system, but use a DSC panel. If you need testers to move forward, I'm happy to help.
I'm curious what your thinking the use case is for this. I currently have it integrated into HomeAssistant, and it works fine for the most part. The one thing I can't do is bypass zones, which I would like to have the ability to do.
Are you looking at more direct control for the panel, as opposed to having to jump through HA (or another system) first?
Build 21228 has been released. Docker images available from DockerHub as usual, and bare-metal packages here.
Home Assistant up to version 2021.8.6 supported; the online version of the manual will now state the current supported versions; Fix an error in OWMWeatherController that could cause it to stop updating; Unify the approach to entity filtering on all hub interface classes (controllers); this works for device entities only; it may be extended to other entities later; Improve error detail in messages for EzloController during auth phase; Add isRuleSet() and isRuleEnabled() functions to expressions extensions; Implement set action for lock and passage capabilities (makes them more easily scriptable in some cases); Fix a place in the UI where 24-hour time was not being displayed.I have the following ruleset which I though had been working well until this morning when I noticed it's not.
I've put four weather conditions in an in array and one of them is the current weather condition - but the rule is not true. Now the cloud cover percentage is not yet met but this is an or rule so as long as the "Current Conditions" are met then it should go true.
What's the obvious thing I'm missing here? (I've tried spaces/no spaces in the in and no difference.)
Hi,
Running the latest MSR latest-24152-3455578a with the latest HA 2024.6.1. When trying to call a service I get the following in the MSR logs. Is this a version mismatch? I am not seeing anything in the HA logs.
[latest-24152]2024-06-11T10:29:56.162Z <Rule:INFO> rule-Monitor-DHW (rule-lsvq5k3x in Central Heating) started [latest-24152]2024-06-11T10:29:58.625Z <HassController:WARN> HassController#hass unknown service opentherm_gw.set_hot_water_setpoint in x_hass.call_service action on Thermostat#hass>climate_living_room_otgw [latest-24152]2024-06-11T10:29:58.626Z <HassController:INFO> HassController#hass: sending payload for x_hass.call_service on Thermostat#hass>climate_living_room_otgw action: [Object]{ "type": "call_service", "service_data": { "gateway_id": "living_room_otgw", "temperature": 65 }, "domain": "opentherm_gw", "service": "set_hot_water_setpoint", "target": { "entity_id": "climate.living_room_otgw" } } [latest-24152]2024-06-11T10:29:58.627Z <HassController:ERR> HassController#hass request 1718101798626<6/11/2024, 12:29:58 PM> (call_service) failed: [Object]{ "id": 1718101798626, "type": "result", "success": false, "error": { "code": "invalid_format", "message": "extra keys not allowed @ data['entity_id']" } } [latest-24152]2024-06-11T10:29:58.627Z <HassController:WARN> HassController#hass action x_hass.call_service([Object]{ "service": "opentherm_gw.set_hot_water_setpoint", "data": "{ \"gateway_id\": \"living_room_otgw\", \"temperature\": 65 }" }) on Thermostat#hass>climate_living_room_otgw failed!Cheers Rene
Some background
I'm trying to integrate a Zigbee device into the MSR using zigbee2mqtt bridge and MQTTController. The device in question is a cheap mood light that has following properties that I'd like to control:
I'v already managed to get the switch part working and can toggle the light on/off. Also the brightness value is mapped back to MSR. In zigbee2mqtt it has a value range from 0 to 254, so this the reason for the expression:
expr: 'payload.brightness / 254'Here's the entity definition (don't know whether the type should be something else than the Switch)
zigbee-lidl-mood-light: name: 'Lidl Mood Light' friendly_name: 'Mood Light' type: Switch uses_template: lidl-moodlightAnd the corresponding template (NOTE: rgb_color has not been defined in this example):
lidl-moodlight: init: "zigbee2mqtt/%friendly_name%/get/state" query: "zigbee2mqtt/%friendly_name%/get/state" capabilities: - power_switch - toggle - dimming primary_attribute: power_switch.state events: "zigbee2mqtt/%friendly_name%": "power_switch.state": json_payload: true expr: 'upper(payload.state) == "ON"' "dimming.level": json_payload: true expr: 'payload.brightness / 254' actions: power_switch: "on": topic: "zigbee2mqtt/%friendly_name%/set/state" payload: 'ON' "off": topic: "zigbee2mqtt/%friendly_name%/set/state" payload: 'OFF' set: topic: "zigbee2mqtt/%friendly_name%/set/state" payload: expr: "parameters.state ? 'ON' : 'OFF'" type: raw toggle: topic: "zigbee2mqtt/%friendly_name%/set/state" payload: 'TOGGLE'The problem
In order to control the brightness or the RGB color values, I would have send a JSON payload in corresponding actions. But I have no idea how to define it in the template. The reason why the switch part is working is that the zigbee2mqtt accepts also plain ON / OFF / TOGGLE string payloads in that case.
But the brightness should be controlled with the following payload:
{"brightness": 196}And the RGB color like:
{"color":{"rgb":"46,102,150"}}Here's the link for the documentation (the Exposes part defines the messages).
So how should I define the JSON payload for example for the dimming action? It definitely should be some sort of expressions since I have to map the MSR real value (0...1) to (0...254) for the zigbee2mqtt.
actions: dimming: set: topic: "zigbee2mqtt/%friendly_name%/set" payload: expr: ????? type: jsonAnother problem is the RGB value. I could use the rgb_color capability for the setting but the problem is that the zigbee2mqtt only reports the current color in hue/saturation or xy coordinates.
Here's an example of published message after setting the color:
Topic: zigbee2mqtt/Mood Light QoS: 0 { "brightness":254, "color":{ "hue":240, "saturation":100, "x":0.1355, "y":0.0399 }, "color_mode":"xy", "color_temp":574, "linkquality":96, "state":"ON" }I would have to map those values back to RGB, but is it even possible with existing constructs in MQTTController's templates?
Help would be appreciated @toggledbits
br,
mgvra
That's probably more appropriate to post on Mantis for @toggledbits, but since I know there's at least @Crille publishing templates, my intent with this post is to open a broader discussion.
Long story short: I'm starting to slowly add new template for Shelly Plus and I noticed I'll end up with a dozen more templates, all similar but simply different in trivial details, all sharing a large amount of code and all needing special cares when fixing bugs/adding features (as the latest wifi_status addition).
So, I'm wondering if it's time to start thinking of some sort of inheritance in templates, where I could just create a generic shelly_gen1 and use it as a base for shelly_relay, and this be used as the base for shelly_relay_power and so on.
I could probably achieve this with some sort of scripting on my side to generate templates via code, but maybe there's a better way of doing this, or it's already on the radar.
I need a handful of victims volunteers to help test previews of the next build of Reactor. A long-standing request was for "a simple login mechanism," but in practice, adding user authentication and competent access control turned out to be a pretty big project with a lot of big changes on both server and client sides. It's a bit more than I'm comfortable testing myself and springing out to everyone at once, so I'd like to work with a small group to put it through "sea trials."
Major changes/features include:
User authentication with hashed password storage; User group configuration with application restriction (admin, dashboard, API); Detailed control over API access, with user- and token-based authentication/authorization; Improvements to the HTTPS service; Improvements to UI coordination with the core for Rules and Reactions.If this sounds like something you'd like to help with, drop me a reply here in this thread or privately.
[Solution: Reactor 24115 is not compatible with MQTTController > 24120]
Reactor latest-24115 bare metal.
All MQTT entities stop working after updating MQTTController to 24142, downgrade to 24120 and they are back. Templates and configured entities has not been changed between versions.
I'm not sure if uses_template should be replaced by ìnclude in entity configuration in reactor.yaml but I guess not, I've tried it and it did not do any difference.
I have not tried to update Reactor to userauth version.
Example entity in reactor.yaml that uses MQTTController included template:
entities: "takflakt_kallare": name: "Takfläkt källare" topic: "Källartemp" unit: "" uses_template: tasmota_generic_relay init: "cmnd/%topic%/POWER%unit%"Any hints? Do you need more info, please let me know.
Log from startup:
I'm slowly migrating all my stuff to MQTT under MSR, so I have a central place to integrate everything (and, in a not-so-distant future, to remove virtual devices from my Vera and leave it running zwave only).
Anyway, here's my reactor-mqtt-contrib package:
![](https://github.com/fluidicon.png)
Contrib MQTT templates for Reactor. Contribute to dbochicchio/reactor-mqtt-contrib development by creating an account on GitHub.
Simply download yaml files (everything or just the ones you need) and you're good to go.
I have mapped my most useful devices, but I'll add others soon. Feel free to ask for specific templates, since I've worked a lot in the last weeks to understand and operate them.
The templates are supporting both init and query, so you have always up-to-date devices at startup, and the ability to poll them. Online status is supported as well, so you can get disconnected devices with a simple expression.
Many-many thanks to @toggledbits for its dedication, support, and patience with me and my requests 🙂
Is the following config correct?
- id: time_series name: "Out temp" capabilities: temperature_sensor: attributes: value: model: time series entity: "hass>sensor_outdoor_temperature" attribute: "temperature_sensor.value" interval: 5 # minutes retention: 20 # minutes aggregate: wma primary_attribute: temperature_sensor.value type: ValueSensorSpecifically, is "depth" directive needed/mandatory here? Reason I'm asking is that I'm not getting a "final" value in MSR, only debug values are shown:
temperature_sensor.units=null temperature_sensor.value=null x_virtualentity.last_request_time=null x_virtualentity.request_failures=null x_virtualentity.template=null x_virtualentity.timeseries_debug=[{"time":1716537360000,"value":22.1},{"time":1716537660000,"value":22},{"time":1716537960000,"value":22},{"time":1716538260000,"value":21.9},{"time":1716538560000,"value":22}]Good morning,
I apologize if this subject has been covered. I did try the search, but I'm not coming up with any topics on my issue.
I'm running userauth-24137-57b41335, bare metal installation on Fedora 39 Server.
I have a rule set up to turn the Eco mode off on my Nest Thermostat when the thermostat is set to Away (Rule State: Away Mode), the user (Driver) presence in my car changes to true, and the destination is set to home.
93804f7c-62d6-42c0-ae04-ff602011a6fd-image.png
This works fine for most days, where I'm headed home from work (commute is about 45 minutes). What I don't want it to do is set change it to Eco mode if my ETA is more than an hour.
There is a sensor/entity for Time to Arrival when the Destination is set. What it appears to provide is the Time OF arrival/ETA, not time until arrival. If it was Time until Arrival, and it was a numeric value, I could simply test if the value is less than 60 and be done with it.
I pulled up the history through Home Assistant for my morning commute and this appears to be what it is providing.
c2a32739-c84f-4a05-95d9-73793ed818f5-image.png
So what I need to do is to do a calculation of the the ETA from the sensor value and subtract the current time, and get a value in minutes that I can determine if it's less than 60.
I believe I can do this with the local expression, but I don't see anything for the system time, or local time. Also, do the local expressions update themselves if the sensors do?
Good morning,
I'm running userauth-24137-57b41335 on Fedora 39, bare metal installation.
ZWaveJSController 0.1.23254
Home Assistant:
Core, 2024.5.3 Supervisor, 2024.05.1 Operating System, 12.3 Frontend, 20240501.1I'm trying to troubleshoot a Dynamic Group Controller and notification alert that I've set up for low battery level.
In my Reactor.config, I have the following lines:
name: "Dynamic Group Controller" implementation: DynamicGroupController config: groups: "zwavejs_dead": select: - include_group: "zwavejs" filter_expression: "entity?.attributes?.zwave_device?.status == 3" group_actions: true "low_battery": select: - include_capability: battery_power filter_expression: > entity.attributes.battery_power.level < 0.35The idea here is that I should only have members of this group that have a battery level below 35%. When I go into Entities, I show a whole slew of devices, none of which have a battery level below the threshold.
a77e445b-ab78-4752-a624-3c4117f34f90-image.png
I also tried setting up a rule to generate a push notification once a day, but with all of the group members, I've had to disable the rule. I believe I have it set up correctly, but I'm not 100% sure. I want the notification to tell me the battery level for that device as well.
289b4f68-03ba-49c0-8275-f0f197d13a3a-image.png
ce24a76e-6865-40bd-bd85-632e54d315a8-image.png dc837424-deb5-4ef7-8f0d-3676f1769535-image.png
Can anyone point to me what I may have misconfigured to get these results?
I should also note I'm only interested in ZWaveJS devices. It's showing me battery status for my iPad and car as well, which I don't need it to send me.
Quality of Life Request: Update Button
-
Massive upvote on all 3. Just like back in the old Vera days
(Unsure how easy update would be with the various ways of implementing MSR inside/outside docker etc) - unless a standard install method is specified, and one of the features of that is 'update' capability.
Docker-Compose please -
@Cadwizzard said:
Just like back in the old Vera days
How soon we forget the tales of bricked Veras. Who among us didn't have a little sense that they were playing Russian roulette every time we hit that button?
Unsure how easy update would be with the various ways of implementing MSR inside/outside docker etc
OK. He hits it on the head here. Let me explain some of the complications and my reservations around this.
The biggest pitfall is for docker users, IMO; that's the majority of you. The first thing you need to understand about docker is that the image and the container are separate objects in the system. The container is created from the image, but it's basically a copy, not linked in any meaningful way. The container can change, so that's good — I can download a release package and apply it to the container, restart, and the container will now be running the new release files. Unfortunately, that has no bearing on what happens to the image. Changing files in the container does nothing to the image. So let's take a scenario... @tunnus (Docker, Synology) downloads the image for Reactor 22274 and creates a container for it, so he's now running 22274. A little later, 22291 is released, so he hits the handy, flashy new "Upgrade" button and the container is upgraded in place. Perfect. Except, not... his image is still 22274. Stay with me now... In all likelihood, because of the "ease" of the automated upgrade, tunnus never needs to download a new image again (so he thinks), so he never bothers (it's a pain anyway on Synology, I'll agree). So build 22293 comes along, and then 22302, and then 22305, and then 22308, and he upgrades to all of them using the automated process, but the image is still sitting there on his NAS at 22274. The problem strikes if, for any reason (DSM major upgrade?), he decides to reset and rebuild the container, or delete it. He will get.... 22274. Because that's the image he has.
Can I make docker download the newer image as part of the upgrade process? No. Reactor is running inside the container, and the container, by definition, contains Reactor and keeps it from doing anything external to the container (except the limited data volume that's specifically created for the single purpose). So the running Reactor instance has no ability whatsoever to cause docker/DSM to pull later images. Pulling a new image and rebuilding the container is the real "right" way to upgrade, but it's not possible to automate it from within the container itself (and it's darned clunky in Synology's UI, unfortunately).
It's not hard to imagine that this problem would not bite him for months or years. But when it bites, it has the potential to bite hard. Imagine along Reactor's evolutionary path from 22274 to a future 24107 (released in 2024, all automated updates between, no image refreshes), there are changes that needed to be made to the data structures of rules, reactions, stored states, etc. (not at all hard to imagine, it actually happens all the time). It is easy, although sometimes a bit cumbersome, for me to provide forward compatibility: to make sure that newer versions of code read the old data and upgrade the structures, and the mechanisms for those upgrades remain in the code for some time. But there is no way under the sun for 22274, now running once again unexpectedly, to know what to do with data from the future 24107 build, and there's a chance it could do something really bad to it. Now tunnus has an old version running in his container with corrupt data. I hope he has a backup.
I'll take the opportunity to say that this is a cautionary tale for all of you who stay on older builds. I keep the code that reads and upgrades the data, when needed, for a while, so that people who skip a few upgrades can safely do so and "jump in time" when they are ready to apply a new build, but I don't and won't keep that code forever; it becomes a maintenance nightmare and it's beyond my available time and sensibilities to test every possible combination of upgrades between versions. If you're running on a Reactor that's more than a year out (21307 or earlier), you're playing with fire as far as I'm concerned, and you should not expect a smooth upgrade when you get around to it. You may need to upgrade to an interim build still available, which works for bare-metal, but isn't an option for docker users. And before the "I can't have something like that in my home" people start in here, please know that I'm sorry that the free software I offer you and for which I provide ready, quick, and free ongoing support (and upgrades) isn't perfectly to your liking. If you don't like the way it works, you have alternatives, and I fully support your freedom to choose them.
To continue with @Cadwizzard's point: this is equally or more egregious, unfortunately, for docker-compose users, because up to this point, the recommended way for stopping Reactor when using docker-compose is to run
docker-compose down
. This causes Reactor to stop, but also deletes the container. Any upgrades applied to the container are lost in that instant, because the container is discarded. When you later rundocker-compose up -d
, the container is re-created from the (old) image, and will be whatever version that image is. Maybe not a disaster, maybe it is. This could be addressed by retraining docker-compose users to usedocker-compose stop
rather thandocker-compose down
, but the distinction would need to be taught (and learned) as both are useful, and the infrequency of use of these commands would likely suffer from brain-drain over time (i.e. when to use which and what their side effects could be/will be lost on the user a few months from now). But it's such a subtle distinction that people will shoot themselves in the foot easily and regularly, I fear.Bare-metal is somewhat easier, because at least the process can be assured it's writing on the one and only (relevant) image, in the install directory, so that's a bit of relief. Unfortunately, a lot of people really don't understand Linux file permissions, their relations to users and groups, etc., and routinely goof up the permissions of files all over their system, including in the Reactor install directory. This isn't a problem for them after the first "fix," because thereafter they do the manual upgrades the same way, logged in as the same user (in some cases, even as root, which is a serious no-no), and so it works for them as that user in that case, good enough. But for an automated process running in an unprivileged environment, it can mean that some files aren't writable, and the upgrade only half-happens... the upgrade process crashes, some files are new and some are old, and the Reactor install is basically dead and broken. I can't fix the permissions from the running instance, because it's running as an unprivileged user (well, hopefully; woe unto those who run anything as root). The user then manually applies the upgrade to recover the system, which goes fine because of course he's running the privileged user with the right permissions. A bug/post for the upgrade process then gets reported, and I then spend hours or days going back and forth, digging through the user's 3,000+ files in a typical Reactor install, looking for the broken ones and teaching the user how to fix them. (Permissions and their potential brokenness is also an issue for doing automated backups/restores, since that was mentioned as well.)
Oh, and then there's Windows. I won't even start. I've already written a book (again).
With regard to the suggestion of a standard install method: (a) there is no "one size fits all" — what works for Ubuntu doesn't work for Synology DSM or QNAP, and certainly not for Windows; and (b) the install methods that are recommended are all carefully documented; experience shows that I can write out every detail I can think of, and what actually happens on the user's system is 100% of that or some amount less, or the user has some condition in their system/environment that I could not/did not anticipate that causes problems. My preferred method for most users is docker (and specifically, with docker-compose), because the container strategy removes some of these risks, but that's not always the easiest for their environment (e.g. Synology has docker but no -compose), and the accepted mechanisms for upgrading containers in the docker world in general are ironically exactly the subject of complaints by OP and others here, despite the relative ubiquity and ease of these mechanisms.
The point is, there is no panacaea here. You run these systems, not me. You do things I have no knowledge of, and sometimes those things bite back. The majority of my time supporting this product is troubleshooting your environments, not my code (I'm not saying I'm perfect — I make mistakes and bugs are a reality, but they're not the majority of support issues here in terms of time spent). Anything and everything I do in the system is looked at not just through the lens of whether it's convenient for the user, but very much through the lens of supportability. There are lots of features I get asked to do, and as you've seen (even recently), there are some that I refuse to consider simply because it would make the system less supportable, in my view. As features get added, not only is the usability of the system required to improve, but its quality is expected to improve as well (fewer bugs, better support, etc.) — those are my expectations, which I'm sure you share. If one doesn't consider supportability (and that means both in user support and code maintenance/reliability/scalability), one ends up with a lot of features that nobody asked for, don't work, and aren't usable (I can point you in the direction of such a product as an alternative if you're really interested in that).
There is a running, hidden upgrade process in the current build. I've been experimenting with this for a bit, getting to learn it, and discovering these issues. It's not that I won't consider making it available; I'm still studying it, and pondering the wisdom of it. Maybe sometimes I worry too much about things like this, I don't know. But when it goes wrong, there's nobody but you and me to fix it, and there's a lot of you and only one of me, so as I said in another recent conversation, handing out something that feels like a grenade with no pin sometimes doesn't seem like the best idea to me, and there are probably other things this system needs to do that I can better spend my time on. Maybe this is one of those things.
I'll leave this one up to you guys. If you can tolerate these side effects, I'll release the feature. But know that if you break your install because (docker) you somehow delete the container and recreate it from an old image, or (bare-metal) your install has broken permissions or other issues that the upgrade process can't work through, my answer will be short: that's a risk you accepted, do a clean reinstall from a current image, restore your config/state backup, and start over.
-
One more class of knowledge! Really the desire to have an automatic process would be very good, but your explanation makes clear the difficulty and risks, and I do not want to have it. As you said, the errors we generate are enough, I don't want to implement more risks. I think almost everyone here has a wife, so better to stay in the safety of the system working.
Well let's remove this from wishlist request, and could you share this list so we know everything that's coming in the future? Also put an item, display the status widget, in a window/iframe inside the HE dashboard
-
@toggledbits this makes tons of sense why anyone should want an update button mainly Docker users.
In terms of bare metal users, say if a user messed with their files permissions enough that it would cause issues when updating Reactor, wouldn't they run into the same errors even if they manually updated or used the update button? I wouldn't mind an update button for bare metal users, since from your explanation seems like a possible issue with won't come up with the actually update process itself, it can come with something else (like file permissions etc). Meaning that they'd run into these errors even if they manually updated Reactor like we do now.
Not arguing though, its a fairly low level request from me. I can clearly see why an update button for Docker users could be a slow and silent death. As @CatmanV2 for bare metal the update process really only takes 90 seconds ahah.
-
@pabla said in Quality of Life Request: Update Button:
wouldn't they run into the same errors even if they manually updated or used the update button?
Not necessarily... some users... I've seen it... will run into permission problems and their answer, not understanding the problem or how to fix it, is to use
sudo tar xvf
to just lay tracks over everything. This would eliminate the permissions problem unpackaging the archive, but new files may become root-owned, which isn't right but the code doesn't care as long as its readable. If their umask allows world-readable files (and 022 is a common default that does exactly that), the Reactor runtime will never know permissions are broken, because every file it needs is readable without consideration of ownership. The un-tar'ing doesn't touchlogs
,config
, etc. so any permissions there aren't relevant and aren't changed. And because some of the files are now root-owned that shouldn't be, the permissions problem has been made worse and again, unless they are truly fixed the right way, thensudo
will continue to be the only way upgrades will succeed. It perpetuates and exacerbates.I really get how painful the docker upgrades are on Synology. I'm guessing QNAP is probably not much different, and I think several people have been bitten by Portainer oddities regardless of platform.
The process just needs more thought. I could, for example, from the next build onward, prevent the system from starting if the config and data are from a newer version. The problem there is that it needs to be detected early in startup, and if the system can't use the data, it has to exit hard, because it can't run without any data at all, and it can't touch what it has. There would be no UI feedback other than "DISCONNECTED" (i.e. the behavior when Reactor can't start). A "click-to-upgrade" to fix it wouldn't be an option because the UI would not be running, so a manual upgrade would be required at that point. And maybe that's OK? Maybe that's such an extreme/infrequent circumstance that it should be that way? A manual upgrade once in a blue moon may not be so bad... I don't know... looking for feedback... trying to figure it out...
-
I personally do not think the update process on Synology docker is that bad. A few more clicks than an easy button but not horrible. All my other docker containers are updated the same way. I like the docker image though. I am not familiar with the other platforms so I can’t comment on those update processes.
-
C CatmanV2 referenced this topic on
-
I too am happy with the current process. Super fast for me. I run everything under Synology/Docker. I no longer have issues upgrading containers such as Reactor, HA, etc. since I switched to Portainer several months ago. So not sure what those "Portainer" oddities are/were. Something I should keep an eye on? Or I have just been lucky?
-
Some input from a Windows user.
An update button would of course be a nice to have feature, but I also agree with several other here. A "normal" update, aka don't need new dependencies, just take a short moment to install.
Were I usually stumble is when an update of dependencies are needed. That have taken me hours of search-try--error-tryagain before getting that to work sometimes.My dream would be to have a "windows installer" for MSR, that checks dependencies, install a systemservice etc.
Over time I think that would be a safer/more stable way, with fewer user errors.With this said, I can understand really understand that @toggledbits need to handle this "his way" to be able to support differen't enviroments (and users
)
-
I don't know if this helps for other Docker users, but not long after I got started with Docker I found Portainer, and I've been running it alongside Reactor and my other containers on my Raspberry Pi 4. With Portainer, there may not be a one-step update button, but I find it makes updates much easier.
I just updated Reactor to the latest. All I had to do was go to the Portainer URL in my browser, then
- Click on the Reactor container in the Containers list
-
Click 'Recreate"
-
Toggle "Always pull new image" on the window that pops up
- Click "Recreate"
It isn't one click, but it can be done in a browser tab from any machine with network access to the Docker host. No VNC/SSH into the machine, no Docker commands to run from the command line.
Portainer also has links to view the container logs and to open a command window in the container, which I use all the time. You can also use the "duplicate/edit" button to change or add environment variables while updating, which is how I added the NODE_PATH a few updates back.
-
Thanks for the Portainer explanation, I'm certain I've had a spell cast on me. I'll try again once the Pi400 become available once more.
Lastly, I think the point has been lost, it's about QOL, not about how easy it is to do in another way.
From my perspective if it isn't easy to use by 98% of the public then it's too much trouble and they might look at it then discard it for another solution.
The comments so far are from users who are in the 2% and are happy to tinker. I'm happy for you.
If anyone wants to see how Consumer friendly software should be to set up, then have a look at Homeseer4. Update ...no problem with 1 click. -
@black-cat Isn't homeseer a walled garden like Hubitat, Ezlo, etc.? You buy their hub and live within their infrastructure.
That's not MSR. MSR works on various OS/hardware and communicates with multiple hubs.
Whilst I appreciate your POV, it's not apples>apples comparison you're making here.
-
@gwp1 said in Quality of Life Request: Update Button:
You buy their hub and live within their infrastructure.
Nup, you can use any old Laptop or RasPi. Runs on Windows or Lynx. i'd love to promote MSR to Homeseer users but it lacks the simplicity hence the backing of the request.
Realistically, I'm not going to see it happen which is a shame as Patrick has put a lot of time into development for the 2%. -
@sweetgenius I agree, Synology Docker container upgrade process is not too bad.
I frequently keep both MSR and Synology UI open on separate browser tabs and either do a quick upgrade using "reset" or a bit careful upgrade using "duplicate settings" and retaining old container as a backup/rollback option.
Originally I favored a simple update button for MSR, but after Patrick's explanations I realized it's not that simple after all.