I need a handful of victims volunteers to help test previews of the next build of Reactor. A long-standing request was for "a simple login mechanism," but in practice, adding user authentication and competent access control turned out to be a pretty big project with a lot of big changes on both server and client sides. It's a bit more than I'm comfortable testing myself and springing out to everyone at once, so I'd like to work with a small group to put it through "sea trials."
Major changes/features include:
User authentication with hashed password storage; User group configuration with application restriction (admin, dashboard, API); Detailed control over API access, with user- and token-based authentication/authorization; Improvements to the HTTPS service; Improvements to UI coordination with the core for Rules and Reactions.If this sounds like something you'd like to help with, drop me a reply here in this thread or privately.
(Using userauth-24120-7745fb8d build in Docker)
There's a filtering capability for entities in reactor.yaml, but I have a case where I don't want to filter an entity altogether, but would like to "throttle" it, as this sensor updates every 1-2 seconds (and therefore unnecessarily takes database space).
Sensor data comes through home assistant, and seems that there's no way to control update interval at that end.
So I'm asking if plugin configuration could support limiting/throttling updates for certain entities?
Good morning,
Hopefully this is a simple request. I believe the title should be self explanatory, but just in case, I'll elaborate.
On the status tab, we all get alerts if a device state has changed (i.e., been removed). This is great, but when I go into the entities tab, I have to either type the name (or a portion thereof) of the device that has been removed, or I need to scroll all the way through my list of devices. This is infrequent, however, yesterday I replaced a failed device in my HAAS environment. It was a Z-Wave switch that is added using the Smart Scan QR code, which normally makes it pretty easy. However, some devices don't get fully added the first time around, so it'll add multiple entries into HAAS until it get's the S2 authentication correct and the device fully included. It did this to me yesterday, and I had to delete the incomplete device from my installation. MSR still saw the entities of that failed/incomplete switch entity, and I was left with 8 alerts and entities that I needed to removed.
It's not a huge problem, but this example was just one switch. If I were to add replace multiple devices at once, this could be a bit more annoying to remove. It would be helpful to be able to filter by removed entities, so I can find them all quickly and delete them. Continuing that train of thought, it would also be useful to have check boxes next to those lines, and perhaps do a select all type of thing so they could be deleted in one mouse click.
@toggledbits I have finally finished up the SSL using Let's Encrypt and am getting this from my local browser:
f3d0ac22-272e-46c1-b7e3-57b08bdd1555-image.png
21c04fe1-1760-4ce6-a4de-2285d3349940-image.png
3a7022db-5add-40a1-b9a2-0c0b97fa211b-image.png
I know you said in the docs that using a self-signed could lead to this but this is LE.
Hi @toggledbits,
I don't know if I'm the only one, so I'm reporting here first instead of opening a bug.
Basically, with the latest 2-3 updates of Reactor and MQTTController, after a restart previous statuses are lost (for both Virtual and MQTT entities), until they're restored.
It's particularly annoying for Virtual Entities, because I have to set them all over again (I've coded some defaults at startup if the values are empty, but sometimes these are not the correct values before the update).
Not easy to reproduce, and logs are gone, but the first time I tought it was me hallucinating, the second one didn't bother too much, after the third I realized it's something not coming from me.
the behavior could be seen in this screenshot:
1c007fc2-4dbc-4476-8dca-e5aa111e4642-image.png
Any hint is appreciated.
Build 21228 has been released. Docker images available from DockerHub as usual, and bare-metal packages here.
Home Assistant up to version 2021.8.6 supported; the online version of the manual will now state the current supported versions; Fix an error in OWMWeatherController that could cause it to stop updating; Unify the approach to entity filtering on all hub interface classes (controllers); this works for device entities only; it may be extended to other entities later; Improve error detail in messages for EzloController during auth phase; Add isRuleSet() and isRuleEnabled() functions to expressions extensions; Implement set action for lock and passage capabilities (makes them more easily scriptable in some cases); Fix a place in the UI where 24-hour time was not being displayed.I'm slowly migrating all my stuff to MQTT under MSR, so I have a central place to integrate everything (and, in a not-so-distant future, to remove virtual devices from my Vera and leave it running zwave only).
Anyway, here's my reactor-mqtt-contrib package:
Contrib MQTT templates for Reactor. Contribute to dbochicchio/reactor-mqtt-contrib development by creating an account on GitHub.
Simply download yaml files (everything or just the ones you need) and you're good to go.
I have mapped my most useful devices, but I'll add others soon. Feel free to ask for specific templates, since I've worked a lot in the last weeks to understand and operate them.
The templates are supporting both init and query, so you have always up-to-date devices at startup, and the ability to poll them. Online status is supported as well, so you can get disconnected devices with a simple expression.
Many-many thanks to @toggledbits for its dedication, support, and patience with me and my requests 🙂
Good morning,
So Home Assistant decided to change the default weather home format that I've been using for the past year and a half. I had two Global Expressions set up to pull the high and low temp forecast for the day. Now it's pulling null values.
094c9205-cc9e-4fcc-ac4f-1bf54acea299-image.png
In the dev tools, it now uses a new service (Weather. get forecasts), plural, where the old Weather.get forecast is depreciated and now longer functions.
8c7a1fcc-dd3f-4268-a0b7-29d542f86adc-image.png
It shows a templow field, and a temperature field, which I presume is the forecast high.
When I head back over to MSR, I'm having a hard time finding those values in the Entities tab.
c5ea1048-a72e-4647-9c50-9d0c5fd20767-image.png
wx.asoftime=null wx.ceiling=null wx.ceiling_unit=null wx.cloud_cover=null wx.condition_code=null wx.description="partlycloudy" wx.feels_like=null wx.humidity=57 wx.humidity_unit="%" wx.icon=null wx.location=null wx.precipitation_1hr=null wx.precipitation_24hr=null wx.precipitation_other=null wx.precipitation_type=null wx.precipitation_unit="in" wx.pressure=30 wx.pressure_unit="inHg" wx.temperature=55 wx.temperature_unit="°F" wx.visibility=null wx.visibility_unit="mi" wx.wind_compass=210.3 wx.wind_conditions=null wx.wind_direction="SSW" wx.wind_gust=null wx.wind_speed=6.28 wx.wind_speed_unit="mph" x_hass.domain="weather" x_hass.entity_id="weather.forecast_home" x_hass.services=["weather"] x_hass.state="partlycloudy" x_hass_attr.attribution="Weather forecast from met.no, delivered by the Norwegian Meteorological Institute." x_hass_attr.cloud_coverage=85.9 x_hass_attr.dew_point=40 x_hass_attr.friendly_name="New Windsor Weather" x_hass_attr.humidity=57 x_hass_attr.precipitation_unit="in" x_hass_attr.pressure=30 x_hass_attr.pressure_unit="inHg" x_hass_attr.supported_features=3 x_hass_attr.temperature=55 x_hass_attr.temperature_unit="°F" x_hass_attr.visibility_unit="mi" x_hass_attr.wind_bearing=210.3 x_hass_attr.wind_speed=6.28 x_hass_attr.wind_speed_unit="mph"There is a x_hass_attr.temperature, but that appears to be the current temperature, not the high that I found on the dev tools screenshot.
Any ideas?
Running:
Core
2024.4.3
Supervisor
2024.04.0
Operating System
12.2
Frontend
20240404.2
MSR: latest-24057-e9add9f5
Hey Patrick, I recently have been noticing that MSR has been acting up ie. it's been needing restarts and has been slow. I began trouble shooting by looking at the logs and have noticed the following errors for a lot of entities. I thought maybe a simple reboot of RPi was needed and I kept seeing the same errors in the system logs. I am oddly enough not seeing these same errors in the MSR logs. Where things started getting weird is whenever I rebooted MSR it wouldn't come back online .I would have to restart the RPi then it would come back online. I just restarted MSR again to capture logs and it restarted fine, so I guess its good for now? I think this is more or so a corrupted SD card issue rather a MSR issue but well being troubleshooting from here. The SD card is about 1-2 years old.
Apologies if this post is everywhere, I cannot consistently recreate any oddities that are happening, that's what is leading me to believe my SD is going bad.
PS: If anyone knows how to diagnose a corrupt SD card please chime in.
MSR latest-24057-e9add9f5
Home Assistant 2024.4.3
Raspberry Pi 3b+
This system has been running flawlessly year after year for the time changes twice a year literally since MSR came out so I was caught off-guard when this happened this morning.
Time in MSR browser is EST, time on RPi is local time (DST).
76ed5313-b9b9-46d4-b0f9-462c40e99750-image.png
195e61c5-58a7-4453-b96a-18cebae75550-image.png
I've rebooted the RPi I've restarted MSR after double-checking the time on the RPi. Used a completely different browser to eliminate any caching concerns. Double-checked MSR reactor.yamla5f23151-d691-4343-8499-8e77a55528e5-image.png
What am I missing here @toggledbits ?
Hi,
For the standard capabilities MSR sends both a value record and a units record to InfluxDB. The latter I would like not to send as they are not really any use for me and it will reduce the number of records send to my InfluxDB.
Is there a quick way to do this with a filter_entities line like: *>units?
Or do I have to update all capabilities to read like this:
power_sensor:
attributes:
value: true
Cheers Rene
I'm trying to replicate this
wallbox_set_number.PNG
into a MQTT entity where I could set a number with a min and max value.
I can't find a standard capability that fits or any documentation on local MQTT capabilities and the only post on the forum mentioning local MQTT capabilities is this post, is it even possible in current release?
My trial and error work in local_mqtt_capabilities.yaml isn't much to show as it's just a copy of mqtt_capabilities.yaml with changed names and then I got stuck.
Any guidance, examples, documentation, future feature request or denial would be much appreciated, thanks!
Reactor 24057-e9add9f5 bare metal
MQTTController 24050
Hi guys,
I've recently bought a new Govee outdoor permanent lights set, and I love it. WAF is pretty high, and the product is good quality. I hope to never run lights in the front of the house.
This new addition has found me searching for something to control these lights, locally. Govee has officials remote and LAN APIs and Home Assistant has it supported, but some undocumented stuff that's integrated into an Homebridge plugin that seems very promising. Without this plugin, my playlist is orchestrated via the cloud and that makes zero sense.
In the past I got some inspiration from plugins running on other platforms and Homebridge seems one of the most active. I could map its devices via HomeKit-local on HA, but I've decommissioned Homebridge years ago when we settled to Alexa (and I want to stay simple), so I had an idea: why get inspiration and rewrite things, when you could write an Homebridge adapter that could load any Homebridge plugin and run them natively under Reactor (MSR)?
I'm not sure if that's viable or made any sense, so I'm posting here to get feedback, encouragement and your thoughts. Anyone could be potentially interested in such a thing?
Hi- looking for a hint in where to start. My goal is to set a PIN code in a zwave kwikset lock triggered in a rule.
The device isn’t exposing methods to help. The x-hass.call-service looks promising, but what would the service name be?
Plan b would be send the zwave controller a config command- I don’t see any way to explicitly send a command through JS Zwave in my environment.
Running reactor bare metal. JS Zwave is running as an add on inside HASS OS.
Any tips are appreciated.
Hey crew, I'm trying to use MSR to control the RGB values of a Z-Wave bulb in Home Assistant.
Problem I'm running into - I would like to use 'rgb_color.set' to control this, but it doesn't work, instead it always passes the values '255,255,255' to HA no matter what values I enter within MSR.
More notes and examples below - I'm wondering if this is a formatting issue that I'm missing? Thanks for any help!
NOTES FROM TROUBLESHOOTING:
'rgb_color.set_rgb' works successfully, which seems strange. You'd think they would both be affected I've tried a couple different formats, like adding quotes, adding/removing spaces between the RGB values, nothing has fixed it.EXAMPLES:
When I use 'rgb_color.set_rgb', the values successfully carry over to Home Assistant:
f0f4befc-a642-428e-8923-e5f856ca7e2b-image.png
0af0a4f8-50b9-4100-b1e8-52a0de4cbcbb-image.png
But when I use 'rgb_color.set', the values DO NOT successfully carry over to Home Assistant:
9e2d7004-8085-4b70-bb3e-45614b7260a0-image.png 0d630228-c74b-4db8-89bd-2572a08608a3-image.png
DETAILS:
Bulb is LZW42 by Inovelli MSR version: stable-23242-5ee8e1d4HA DETAILS
Core 2024.2.5 Supervisor 2024.02.1 Operating System 12.0Hi,
I’m running MSR in a docker container on my Synology Nas. The container is automatically updated using watchtower weekly.
It was working. Now, after the update, Reactor webpage is able to load, and all indications on the webpage suggests that it is working fine. However, the updated statuses from Home Assistant and Vera are not being detected.
The container logs show the following error
Reactor stable-23344-5aad7754 app 23344 configuration from /var/reactor/config NODE_PATH /opt/reactor:/opt/reactor/node_modules [stable-23344]2024-02-28T21:57:49.516Z <app:null> Reactor build stable-23344-5aad7754 starting on v16.15.1 [stable-23344]2024-02-28T21:57:49.517Z <app:null> Process ID 1 user/group 0/0; docker; platform linux/x64 #69057 SMP Fri Jan 12 17:02:28 CST 2024; locale (undefined) [stable-23344]2024-02-28T21:57:49.517Z <app:null> Basedir /opt/reactor; data in /var/reactor/storage [stable-23344]2024-02-28T21:57:49.517Z <app:null> NODE_PATH=/opt/reactor:/opt/reactor/node_modules [stable-23344]2024-02-28T21:57:49.696Z <Structure:null> Module Structure v23172 [stable-23344]2024-02-28T21:57:49.698Z <Capabilities:null> Module Capabilities v23331 [stable-23344]2024-02-28T21:57:49.780Z <Plugin:null> Module Plugin v22300 [stable-23344]2024-02-28T21:57:49.790Z <TimerBroker:null> Module TimerBroker v22283 [stable-23344]2024-02-28T21:57:49.794Z <Entity:null> Module Entity v22353 [stable-23344]2024-02-28T21:57:49.866Z <Controller:null> Module Controller v23069 [stable-23344]2024-02-28T21:57:50.030Z <default:null> Module Ruleset v22293 [stable-23344]2024-02-28T21:57:50.031Z <default:null> Module Rulesets v22146 [stable-23344]2024-02-28T21:57:50.066Z <GlobalExpression:null> Module GlobalExpression v23211 [stable-23344]2024-02-28T21:57:50.581Z <Predicate:null> Module Predicate v23093 [stable-23344]2024-02-28T21:57:50.595Z <AlertManager:null> Module AlertManager v22283 [stable-23344]2024-02-28T21:57:50.600Z <Rule:null> Module Rule v23107 [stable-23344]2024-02-28T21:57:50.614Z <GlobalReaction:null> Module GlobalReaction v22324 [stable-23344]2024-02-28T21:57:50.617Z <Engine:null> Module Engine v23339 [stable-23344]2024-02-28T21:57:50.635Z <httpapi:null> Module httpapi v23058 [stable-23344]2024-02-28T21:57:50.680Z <wsapi:null> Module wsapi v23172 [stable-23344]2024-02-28T21:57:50.789Z <TaskQueue:null> Module TaskQueue 21351 [stable-23344]2024-02-28T21:57:50.790Z <VeraController:null> Module VeraController v23109 [stable-23344]2024-02-28T21:57:50.971Z <HassController:null> Module HassController v23344 [stable-23344]2024-02-28T21:57:51.716Z <DynamicGroupController:null> Module DynamicGroupController v22313 [stable-23344]2024-02-28T21:57:52.253Z <SystemController:null> Module SystemController v23331 i18n: missing en-US language string: The version of nodejs you are using ({0}) is now end-of-life, and so is deprecated for use with Reactor. Please upgrade nodejs to {2}.{3} or higher as soon as possible; the current LTS version is recommended. Releases of Reactor produced after {1} will not run under this version of nodejs at all. [stable-23344]2024-02-28T21:57:52.256Z <Controller:CRIT> SyntaxError: Unexpected end of JSON input [-] SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at /opt/reactor/server/lib/Controller.js:464:51 at Array.forEach (<anonymous>) at SystemController._restoreEntities (/opt/reactor/server/lib/Controller.js:458:36) at new Controller (/opt/reactor/server/lib/Controller.js:45:42) at new SystemController (/opt/reactor/server/lib/SystemController.js:29:9) at /opt/reactor/server/lib/Controller.js:101:37 Trace: The version of nodejs you are using ({0}) is now end-of-life, and so is deprecated for use with Reactor. Please upgrade nodejs to {2}.{3} or higher as soon as possible; the current LTS version is recommended. Releases of Reactor produced after {1} will not run under this version of nodejs at all. at _T (/opt/reactor/server/lib/i18n.js:468:37) at AlertManager.addAlert (/opt/reactor/server/lib/AlertManager.js:126:25) at /opt/reactor/app.js:381:140 [stable-23344]2024-02-28T21:57:59.313Z <app:CRIT> SyntaxError: Unexpected end of JSON input [-] SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at IndividualFileStrategy.getDataObject (/opt/reactor/server/lib/IndividualFileStrategy.js:114:54) at DelayWriteCacheStrategy.getDataObject (/opt/reactor/server/lib/DelayWriteCacheStrategy.js:87:50) at Container.getDataObject (/opt/reactor/server/lib/Container.js:69:67) at Function.getInstance (/opt/reactor/server/lib/Data.js:37:179) at Rule.getRuleStates (/opt/reactor/server/lib/Rule.js:507:100) at Rule.getConditionState (/opt/reactor/server/lib/Rule.js:538:47) at new Rule (/opt/reactor/server/lib/Rule.js:378:47) at Function.getInstance (/opt/reactor/server/lib/Rule.js:387:36) at /opt/reactor/server/lib/Engine.js:263:53 i18n: missing en-US language string: HomeAssistant on {0:q} may be an unsupported version. The reported version ({1}) has not been certified/tested with this version of Reactor and may cause errors. You must either modify your HomeAssistant install, or see if an update to Reactor has been made available. Trace: HomeAssistant on {0:q} may be an unsupported version. The reported version ({1}) has not been certified/tested with this version of Reactor and may cause errors. You must either modify your HomeAssistant install, or see if an update to Reactor has been made available. at _T (/opt/reactor/server/lib/i18n.js:468:37) at AlertManager.addAlert (/opt/reactor/server/lib/AlertManager.js:126:25) at HassController.sendWarning (/opt/reactor/server/lib/Controller.js:197:36) at /opt/reactor/server/lib/HassController.js:1117:370 at processTicksAndRejections (node:internal/process/task_queues:96:5)I’ve tried using “latest-amd64” and it does not work either. The logs show similar json input error.
Reactor latest-24057-e9add9f5 app 24052 configuration from /var/reactor/config NODE_PATH /opt/reactor:/opt/reactor/node_modules [latest-24057]2024-02-28T22:27:30.466Z <app:null> Reactor build latest-24057-e9add9f5 starting on v20.10.0 [latest-24057]2024-02-28T22:27:30.522Z <app:null> Process ID 1 user/group 0/0; docker; platform linux/x64 #69057 SMP Fri Jan 12 17:02:28 CST 2024; locale (undefined) [latest-24057]2024-02-28T22:27:30.522Z <app:null> Basedir /opt/reactor; data in /var/reactor/storage [latest-24057]2024-02-28T22:27:30.522Z <app:null> NODE_PATH=/opt/reactor:/opt/reactor/node_modules [latest-24057]2024-02-28T22:27:30.667Z <Structure:null> Module Structure v23172 [latest-24057]2024-02-28T22:27:30.673Z <Capabilities:null> Module Capabilities v23331 [latest-24057]2024-02-28T22:27:30.780Z <Plugin:null> Module Plugin v22300 [latest-24057]2024-02-28T22:27:30.787Z <TimerBroker:null> Module TimerBroker v22283 [latest-24057]2024-02-28T22:27:30.791Z <Entity:null> Module Entity v22353 [latest-24057]2024-02-28T22:27:30.796Z <Controller:null> Module Controller v23069 [latest-24057]2024-02-28T22:27:30.811Z <default:null> Module Ruleset v22293 [latest-24057]2024-02-28T22:27:30.811Z <default:null> Module Rulesets v22146 [latest-24057]2024-02-28T22:27:30.821Z <GlobalExpression:null> Module GlobalExpression v23211 [latest-24057]2024-02-28T22:27:30.890Z <Predicate:null> Module Predicate v23093 [latest-24057]2024-02-28T22:27:30.958Z <AlertManager:null> Module AlertManager v22283 [latest-24057]2024-02-28T22:27:31.027Z <Rule:null> Module Rule v24057 [latest-24057]2024-02-28T22:27:31.033Z <GlobalReaction:null> Module GlobalReaction v22324 [latest-24057]2024-02-28T22:27:31.036Z <Engine:null> Module Engine v24023 [latest-24057]2024-02-28T22:27:31.042Z <httpapi:null> Module httpapi v24057 [latest-24057]2024-02-28T22:27:31.216Z <wsapi:null> Module wsapi v24057 [latest-24057]2024-02-28T22:27:31.296Z <TaskQueue:null> Module TaskQueue 21351 [latest-24057]2024-02-28T22:27:31.297Z <VeraController:null> Module VeraController v24050 [latest-24057]2024-02-28T22:27:31.365Z <HassController:null> Module HassController v24048 [latest-24057]2024-02-28T22:27:31.659Z <DynamicGroupController:null> Module DynamicGroupController v22313 [latest-24057]2024-02-28T22:27:31.668Z <SystemController:null> Module SystemController v23331 [latest-24057]2024-02-28T22:27:31.673Z <Controller:CRIT> SyntaxError: Unexpected end of JSON input [-] SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at /opt/reactor/server/lib/Controller.js:464:51 at Array.forEach (<anonymous>) at SystemController._restoreEntities (/opt/reactor/server/lib/Controller.js:458:36) at new Controller (/opt/reactor/server/lib/Controller.js:45:43) at new SystemController (/opt/reactor/server/lib/SystemController.js:237:9) at /opt/reactor/server/lib/Controller.js:101:37 [latest-24057]2024-02-28T22:27:38.845Z <app:CRIT> SyntaxError: Unexpected end of JSON input [-] SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at IndividualFileStrategy.getDataObject (/opt/reactor/server/lib/IndividualFileStrategy.js:51:45) at DelayWriteCacheStrategy.getDataObject (/opt/reactor/server/lib/DelayWriteCacheStrategy.js:89:49) at Container.getDataObject (/opt/reactor/server/lib/Container.js:69:65) at Data.getInstance (/opt/reactor/server/lib/Data.js:45:179) at Rule.getRuleStates (/opt/reactor/server/lib/Rule.js:515:101) at Rule.getConditionState (/opt/reactor/server/lib/Rule.js:546:47) at new Rule (/opt/reactor/server/lib/Rule.js:371:47) at Rule.getInstance (/opt/reactor/server/lib/Rule.js:380:36) at /opt/reactor/server/lib/Engine.js:828:53 i18n: missing en-US language string: HomeAssistant on {0:q} may be an unsupported version. The reported version ({1}) has not been certified/tested with this version of Reactor and may cause errors. You must either modify your HomeAssistant install, or see if an update to Reactor has been made available. Trace: HomeAssistant on {0:q} may be an unsupported version. The reported version ({1}) has not been certified/tested with this version of Reactor and may cause errors. You must either modify your HomeAssistant install, or see if an update to Reactor has been made available. at _T (/opt/reactor/server/lib/i18n.js:614:37) at AlertManager.addAlert (/opt/reactor/server/lib/AlertManager.js:128:25) at HassController.sendWarning (/opt/reactor/server/lib/Controller.js:197:36) at /opt/reactor/server/lib/HassController.js:1133:374 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)How do I fix this?
Noticed right away last night at closing time that the open/close had become inverted with the update to v23326.
It wasn't an awful bit of lift to change all Reactions to reflect the change but it was jarring initially when everything flipped.
Are there release notes on that version, @toggledbits ?
Installing latest-24052-d039d560 and I did a dumb thing: moving too quickly, I deleted package.json instead of package-lock.json because the latter didn't exist.
Ran the installer script - but in the wrong directory so I moved package.json manually.
Everything has come back EXCEPT for my MQTT Cloud OwnTracks entities. They're erroring because of this:
eb6f35dc-1981-432a-993d-aefae54cba02-image.png
(Don't heckle my folder structure - it's run this way for three years)
reactor.yaml is fine, all MQTT configs exactly as they should be.
The folder mqtt_templates is present:
f57fa07d-516b-4c8e-8cb0-eeeeaf34de14-image.png
The file it says is "unreadable" is present and hasn't changed (today's date is the Changed date because I did a quick nonsense edit to see that it was not corrupted in some way).
b8fc46a1-f66f-417d-b095-c8e3744ccbe8-image.png
The logs show:
[latest-24052]2024-02-22T01:04:40.320Z <ZWaveJSController:WARN> zwavejs_capabilities defines x_zwave_device_cfg, which is not consistent with the recommended naming [latest-24052]2024-02-22T01:04:40.321Z <ZWaveJSController:WARN> zwavejs_capabilities defines x_zwave_mfg_spec, which is not consistent with the recommended naming [latest-24052]2024-02-22T01:04:40.321Z <ZWaveJSController:WARN> zwavejs_capabilities defines x_zwave_device_ver, which is not consistent with the recommended naming [latest-24052]2024-02-22T01:04:40.322Z <ZWaveJSController:WARN> zwavejs_capabilities defines x_zwave_values, which is not consistent with the recommended naming [latest-24052]2024-02-22T01:04:40.323Z <Controller:INFO> ZWaveJSController#zwavejs loaded zwavejs capabilities ver 23194 rev 1 format 1 [latest-24052]2024-02-22T01:04:40.338Z <Controller:INFO> ZWaveJSController#zwavejs loaded implementation data ver 23254 rev 1 format 1 [latest-24052]2024-02-22T01:04:40.339Z <Structure:INFO> Starting controller SystemController#reactor_system [latest-24052]2024-02-22T01:04:40.345Z <Controller:NOTICE> Controller SystemController#reactor_system is now online. [latest-24052]2024-02-22T01:04:40.572Z <HassController:INFO> HassController#hass device mapping data loaded; checking... [latest-24052]2024-02-22T01:04:40.574Z <HassController:WARN> HassController: implementation of capability battery_maintenance does not provide attribute state [latest-24052]2024-02-22T01:04:40.577Z <HassController:WARN> HassController: implementation of capability input_select.selector does not provide attribute values [latest-24052]2024-02-22T01:04:40.586Z <MQTTController:ERR> MQTTController#mqtt can't read/load templates from /home/reactor/Documents/reactor/config/mqtt_templates/owntracks_sensor.yaml: [Error] Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default. [-] [latest-24052]2024-02-22T01:04:40.595Z <MQTTController:CRIT> Error: Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default. [-] Error: Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default. at Object.safeLoad (/home/reactor/Documents/reactor/node_modules/js-yaml/index.js:10:11) at MQTTController._load_implementation (/home/reactor/Documents/reactor/ext/MQTTController/MQTTController.js:807:38) at async MQTTController.start (/home/reactor/Documents/reactor/ext/MQTTController/MQTTController.js:95:13) at async Promise.allSettled (index 3) [latest-24052]2024-02-22T01:04:40.597Z <MQTTController:ERR> MQTTController#mqtt2 can't read/load templates from /home/reactor/Documents/reactor/config/mqtt_templates/owntracks_sensor.yaml: [Error] Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default. [-] [latest-24052]2024-02-22T01:04:40.603Z <MQTTController:CRIT> Error: Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default. [-] Error: Function yaml.safeLoad is removed in js-yaml 4. Use yaml.load instead, which is now safe by default. at Object.safeLoad (/home/reactor/Documents/reactor/node_modules/js-yaml/index.js:10:11) at MQTTController._load_implementation (/home/reactor/Documents/reactor/ext/MQTTController/MQTTController.js:807:38) at async MQTTController.start (/home/reactor/Documents/reactor/ext/MQTTController/MQTTController.js:95:13) at async Promise.allSettled (index 4) [latest-24052]2024-02-22T01:04:40.608Z <HassController:NOTICE> HassController#hass connecting to ws://192.168.1.198:8123/api/websocket [latest-24052]2024-02-22T01:04:40.616Z <MQTTController:INFO> MQTTController#mqtt instance topic ident is mqtt [latest-24052]2024-02-22T01:04:40.617Z <Controller:INFO> MQTTController#mqtt 0 dead entities older than 86400000s purged [latest-24052]2024-02-22T01:04:40.619Z <MQTTController:NOTICE> MQTTController#mqtt connecting to broker at mqtt://192.168.1.198:1883/ [latest-24052]2024-02-22T01:04:40.691Z <MQTTController:INFO> MQTTController#mqtt2 instance topic ident is mqtt2 [latest-24052]2024-02-22T01:04:40.692Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_iphone14p uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.697Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_iphone8 uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.726Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_iphone14jay uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.730Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_iphone14jen uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.735Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_iphone14addie uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.768Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_iphone14bella uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.773Z <MQTTController:ERR> MQTTController#mqtt2 entity owntracks_androidaa uses_template=owntracks_sensor, template not defined [latest-24052]2024-02-22T01:04:40.777Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_androidaa no longer available, marking BinarySensor#mqtt2>owntracks_androidaa for removal [latest-24052]2024-02-22T01:04:40.781Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_iphone14addie no longer available, marking BinarySensor#mqtt2>owntracks_iphone14addie for removal [latest-24052]2024-02-22T01:04:40.786Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_iphone14bella no longer available, marking BinarySensor#mqtt2>owntracks_iphone14bella for removal [latest-24052]2024-02-22T01:04:40.790Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_iphone14jay no longer available, marking BinarySensor#mqtt2>owntracks_iphone14jay for removal [latest-24052]2024-02-22T01:04:40.794Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_iphone14jen no longer available, marking BinarySensor#mqtt2>owntracks_iphone14jen for removal [latest-24052]2024-02-22T01:04:40.798Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_iphone14p no longer available, marking BinarySensor#mqtt2>owntracks_iphone14p for removal [latest-24052]2024-02-22T01:04:40.802Z <MQTTController:NOTICE> MQTTController#mqtt2 device owntracks_iphone8 no longer available, marking BinarySensor#mqtt2>owntracks_iphone8 for removal [latest-24052]2024-02-22T01:04:40.807Z <Controller:INFO> MQTTController#mqtt2 0 dead entities older than 86400000s purged [latest-24052]2024-02-22T01:04:40.808Z <MQTTController:NOTICE> MQTTController#mqtt2 connecting to broker at mqtts://7ef71f0117fe40d28c2df12205c4a0d0.s2.eu.hivemq.cloud:8883 [latest-24052]2024-02-22T01:04:40.816Z <VirtualEntityController:INFO> VirtualEntityController#virtual configuring virtual Driving Flag - 2 (virt1) [latest-24052]2024-02-22T01:04:40.826Z <VirtualEntityController:INFO> VirtualEntityController#virtual configuring virtual Far Away Flag - 2 (virt2) [latest-24052]2024-02-22T01:04:40.827Z <VirtualEntityController:INFO> VirtualEntityController#virtual configuring virtual Garage Door Momentary Contact vSwitch (virt3) [latest-24052]2024-02-22T01:04:40.828Z <Controller:INFO> VirtualEntityController#virtual 0 dead entities older than 86400000s purged [latest-24052]2024-02-22T01:04:40.829Z <ZWaveJSController:INFO> ZWaveJSController#zwavejs connecting to ws://192.168.1.198:3000 [latest-24052]2024-02-22T01:04:40.836Z <Controller:NOTICE> Controller VirtualEntityController#virtual is now online. [latest-24052]2024-02-22T01:04:40.839Z <app:NOTICE> Starting HTTP server and API... [latest-24052]2024-02-22T01:04:40.845Z <httpapi:NOTICE> httpapi: starting HTTP service on port 8111 [latest-24052]2024-02-22T01:04:40.854Z <app:NOTICE> Starting Reaction Engine... [latest-24052]2024-02-22T01:04:40.856Z <Engine:INFO> Reaction Engine starting [latest-24052]2024-02-22T01:04:40.857Z <Engine:INFO> Checking rule sets... [latest-24052]2024-02-22T01:04:40.880Z <Engine:INFO> Checking rules... [latest-24052]2024-02-22T01:04:41.421Z <Engine:INFO> Data check complete; no corrections. [latest-24052]2024-02-22T01:04:41.571Z <Rule:NOTICE> Driving Flag (rule-l97r2umw in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.572Z <Rule:NOTICE> Driving Flag - Far Away (rule-l97r814k in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.573Z <Rule:NOTICE> DST (rule-l118lzpd in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.573Z <Rule:NOTICE> HVAC Cooling Master (Local APIs) (rule-ktinsq18 in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.574Z <Rule:NOTICE> HVAC Heating before Alarm times (rule-l7hirm3d in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.574Z <Rule:NOTICE> HVAC Heating Master (Local APIs) (rule-ktio3vc1 in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.575Z <Rule:NOTICE> HVAC Neutral Master (rule-ktint8c3 in Shared Rules) starting [latest-24052]2024-02-22T01:04:41.575Z <Rule:NOTICE> It's Raining (rule-lfs6r1lq in Shared Rules) starting [latest-24I'm flummoxed.
[Solved] Is there a cap or max number of devices a Global Reaction should not exceed?
-
@wmarcolin said in [Solved] Is there a cap or max number of devices a Global Reaction should not exceed?:
Is it possible to have something in the configuration that creates the delay when sending each command to Hubitat? Like something like 0.5 seconds between actions?
As stated in this thread, He added pacing to Hubitat.
From the docs:
action_pace — sets the minimum delay between actions sent to the hub (i.e. when a Reaction includes many); sending large numbers of requests can overwhelm the hub, it has been found, so this attempts to slow the pace to avoid this issue. The value must be an integer greater than 0, and the units are milliseconds; the default is 25."the nightmare I had with Vera now happens on Hubitat" If the problem was present before then I doubt that this will solve it.
-
@sweetgenius per your history above, by adding this delay to all actions, did it solve the problem? What time frame are you using? Did you leave the default at 25 milliseconds, or did you add more?
-
@wmarcolin said in [Solved] Is there a cap or max number of devices a Global Reaction should not exceed?:
per your history above,
I do not have any history above, Its not my thread nor did I comment until I read your post. I just read the thread and release notes and pointed out that both mention pacing.
-
@sweetgenius ops, thanks!
-
@toggledbits hi master!
I am going into a state of despair with Hubitat, and thinking that I have made a bad switch from VeraPlus to Hubitat.
Well, as you always comment, look at the log, as I already commented my suspicion that the MSR sent all commands to Hubitat, and this one that failed was confirmed, as you mention in your message.
Routines below.
Looking at the log, I don't understand what sequence the MSR performed, but I see that all of the above actions were sent to Hubitat without fail.
[latest-21349]2021-12-17T01:46:57.319Z <Rule:NOTICE> Rule#rule-kx9oxcss configuration changed; reloading [latest-21349]2021-12-17T01:46:57.321Z <Rule:NOTICE> Rule#rule-kx9oxcss stopping rule [latest-21349]2021-12-17T01:46:57.324Z <Rule:NOTICE> Rule Rule#rule-kx9oxcss stopped [latest-21349]2021-12-17T01:46:57.325Z <Rule:INFO> Rule#rule-kx9oxcss (nGarden) started [latest-21349]2021-12-17T01:46:57.327Z <Rule:INFO> nGarden (Rule#rule-kx9oxcss) SET! [latest-21349]2021-12-17T01:46:57.331Z <Engine:INFO> Enqueueing "nGarden<SET>" (rule-kx9oxcss:S) [latest-21349]2021-12-17T01:46:57.345Z <Engine:NOTICE> Starting reaction nGarden<SET> (rule-kx9oxcss:S) [latest-21349]2021-12-17T01:46:57.346Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>298: http://192.168.33.22/apps/api/67/devices/298/on [latest-21349]2021-12-17T01:46:57.348Z <Engine:INFO> Enqueueing "nLight Garden ON" (re-kx65h5u7) [latest-21349]2021-12-17T01:46:57.350Z <Engine:INFO> Enqueueing "nLight Security ON" (re-kx67amal) [latest-21349]2021-12-17T01:46:57.352Z <Engine:INFO> Enqueueing "nLight Corredor Evening" (re-kx659j8a) [latest-21349]2021-12-17T01:46:57.377Z <Engine:NOTICE> Resuming reaction nGarden<SET> (rule-kx9oxcss:S) from step 4 [latest-21349]2021-12-17T01:46:57.378Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>296: http://192.168.33.22/apps/api/67/devices/296/on [latest-21349]2021-12-17T01:46:57.379Z <Engine:NOTICE> Starting reaction nLight Garden ON (re-kx65h5u7) [latest-21349]2021-12-17T01:46:57.380Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>97: http://192.168.33.22/apps/api/67/devices/97/on [latest-21349]2021-12-17T01:46:57.381Z <Engine:NOTICE> Starting reaction nLight Security ON (re-kx67amal) [latest-21349]2021-12-17T01:46:57.382Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>197: http://192.168.33.22/apps/api/67/devices/197/on [latest-21349]2021-12-17T01:46:57.383Z <Engine:NOTICE> Starting reaction nLight Corredor Evening (re-kx659j8a) [latest-21349]2021-12-17T01:46:57.384Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>419: http://192.168.33.22/apps/api/67/devices/419/on [latest-21349]2021-12-17T01:46:57.456Z <Engine:NOTICE> Resuming reaction nGarden<SET> (rule-kx9oxcss:S) from step 5 [latest-21349]2021-12-17T01:46:57.457Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>297: http://192.168.33.22/apps/api/67/devices/297/on [latest-21349]2021-12-17T01:46:57.563Z <Engine:NOTICE> Resuming reaction nLight Garden ON (re-kx65h5u7) from step 1 [latest-21349]2021-12-17T01:46:57.564Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>162: http://192.168.33.22/apps/api/67/devices/162/on [latest-21349]2021-12-17T01:46:57.673Z <Engine:NOTICE> Resuming reaction nLight Security ON (re-kx67amal) from step 1 [latest-21349]2021-12-17T01:46:57.674Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>229: http://192.168.33.22/apps/api/67/devices/229/on [latest-21349]2021-12-17T01:46:57.782Z <Engine:NOTICE> Resuming reaction nLight Corredor Evening (re-kx659j8a) from step 1 [latest-21349]2021-12-17T01:46:57.784Z <Engine:NOTICE> nLight Corredor Evening delaying until 1639705619783<16/12/2021 20:46:59> [latest-21349]2021-12-17T01:46:57.889Z <Engine:NOTICE> Resuming reaction nGarden<SET> (rule-kx9oxcss:S) from step 6 [latest-21349]2021-12-17T01:46:57.890Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>449: http://192.168.33.22/apps/api/67/devices/449/on [latest-21349]2021-12-17T01:46:57.995Z <Engine:NOTICE> Resuming reaction nLight Garden ON (re-kx65h5u7) from step 2 [latest-21349]2021-12-17T01:46:57.996Z <HubitatController:null> HubitatController#hubitat final action path for power_switch.on on Entity#hubitat>385: http://192.168.33.22/apps/api/67/devices/385/on [latest-21349]2021-12-17T01:46:58.106Z <Engine:NOTICE> Resuming reaction nLight Security ON (re-kx67amal) from step 2 [latest-21349]2021-12-17T01:46:58.106Z <Engine:INFO> nLight Security ON all actions completed. [latest-21349]2021-12-17T01:46:58.214Z <Engine:NOTICE> Resuming reaction nGarden<SET> (rule-kx9oxcss:S) from step 7 [latest-21349]2021-12-17T01:46:58.215Z <Engine:INFO> nGarden<SET> all actions completed. [latest-21349]2021-12-17T01:46:58.321Z <Engine:NOTICE> Resuming reaction nLight Garden ON (re-kx65h5u7) from step 3 [latest-21349]2021-12-17T01:46:58.322Z <Engine:INFO> nLight Garden ON all actions completed. [latest-21349]2021-12-17T01:46:59.795Z <Engine:NOTICE> Resuming reaction nLight Corredor Evening (re-kx659j8a) from step 2 [latest-21349]2021-12-17T01:46:59.796Z <HubitatController:null> HubitatController#hubitat final action path for color_temperature.set on Entity#hubitat>419: http://192.168.33.22/apps/api/67/devices/419/setColorTemperature/2850 [latest-21349]2021-12-17T01:46:59.800Z <Engine:NOTICE> Resuming reaction nLight Corredor Evening (re-kx659j8a) from step 3 [latest-21349]2021-12-17T01:46:59.800Z <Engine:INFO> nLight Corredor Evening all actions completed.
I have checked each of the devices, all are in the log. I am even using the action_pace: 100 setting and I see that the interval was obeyed.
But out of 10 lights that should be on, as you can see on the Hubitat's panel only 2 were.
Sorry @toggledbits to ask, and I will understand if you don't answer because I see that the MSR is perfect.
Any recommendations for Hubitat? Reset the whole Z-wave radio and pair it again? Any APP that can check radio occupancy or Hubitat processing? Maybe you with your experience can give me some guidance, and again sorry, I know it is not MSR, but I'm almost back to the Vera with all its problems of drive and evolution.
-
@wmarcolin I don't have a Hubitat myself but have you looked at the logs on the hub per https://docs.hubitat.com/index.php?title=Logs for hints of what is happening when set reaction fires?
-
@wmarcolin Something you've not surfaced, wireless interference. How close is your Hubitat to your WiFi router (they should not be near each other as they will interfere due to the frequencies in use), how close are your z-wave hubs to each other, etc.
Are all devices the kind that are plugged into electricity? I've discovered with some battery-operated devices that they sleep a lot to conserve battery and that delays actions. My iblind controllers are a perfect example: sending a refresh first and then the command seems to make them much happier regarding responding to commands.
As @toggledbits noted in a response to me previously in the thread (and you've supported), the mesh plays a huge role, too.
I had a Veralite and then moved to a VeraSecure. I've done direct comparisons between my VeraSecure and Hubitat using MSR to trigger the rules and the Hubitat was much faster. That, amongst other reasons, is why my VeraSecure is now completely offline.
-
@wmarcolin said in [Solved] Is there a cap or max number of devices a Global Reaction should not exceed?:
Looking at the log, I don't understand what sequence the MSR performed, but I see that all of the above actions were sent to Hubitat
Hmmm. I can't answer for the Hubitat part, but I explain the order. The first three actions in nGarden<SET> are Run Reaction, so these enqueue those reactions with the executive -- they are not run in-line. That's why you see the three "Enqueueing" lines, followed by a resume of nGarden<Set> from step 4. We see the action output for device 298, which is step 3 (numbered from 0), before the enqueue messages because enqueueing itself is an asynchronous operation, so the executive quickly started the three Run Reaction enqueue requests, then ran the device 298 Entity Action. Running an entity action is asynchronous, so the executive had to wait for that operation to finish. Since it went into a wait state, the tasks for the three Run Reaction enqueues could run, so they did. When they were done and the 298 device action was finished sending, nGarden<SET> could then resume from step 4 (numbered from 0, so 5 as we look at it). That's device 296 so we see that on the next line. Again, device actions have to wait for the send, so execution paused of nGarden<Set> paused there, which allowed nLight Garden ON, the first of the three enqueued reactions, to start and send its first command to device 97. That blocked that reaction, so nLight Security ON was next in the queue and it started and sent its first device action to 197. That blocked that reaction, so nLight Corredor Evening started and ran its first action against 419. It blocked, of course, so everything paused about 70ms until nGarden<Set> became the first ready task, so it resumed at step 5 (from 0, or 6 as we count from 1). And so on, until all were sent.
I'm not sure what your pacing configuration was at this point, but overall it appears about right for the number of tasks sent. It's hard to tell without more debug on, and maybe I'll add some "standard" messages about device queueing while we're looking at this (since debug on a Controller instance can be very large and a bit like sipping from a firehouse).
One thing to note also is that each Entity Action blocks while sending -- the reaction waits for the hub to acknowledge the request. For that to happen, the request must be sent, and the hub has to give an HTTP 200 (OK) response to the request (if it gives an error, that would be logged, and there are no errors logged in this snippet). So at the least, the hub has acknowledged the request, but that doesn't mean it has completed the request, let alone that the request was successful in its overall execution (e.g. manipulating the device). That's a different and much bigger problem.
I'm not done looking at this. I want to study the timing more carefully as well. There's something about it that doesn't seem right to me. As I said, I'm going to add some more standard (non-debug) diagnostic output to this while we're looking at it, and roll a new release later today, for you to try and send me new logs.
-
Ok, as you can see from the picture, my Hubitat is next to my Asus router, where the Vera Plus used to be. In theory radio waves from the router should interfere with the Zigbee because both frequencies are in 2.4, and should not in any way interfere with the Hubitat that works at a frequency of 900 Mhz. But I'm trying everything, and I've already moved the Hubitat 2 meters away from the router.
The vera was already disconnected and I have also removed it and put it away, who knows, maybe in the future I will find some use for it.
With respect to your comment of battery devices, I understand, but it is not the case, as I mentioned above I tried to make a sequence of lights, the 6 devices are 4 Aeontec Micro Switch G2 (DSC26103), 1 Zipato Bulb 2, and another Everspring AN145, that is all connected to the electricity all the time. Your battery point is very valid, but it is not what has caused me panic.
Finally, your comment about Hubitat being faster than Vera is what I read everywhere, but unfortunately, this is not what is happening to me at the moment. Slow reactions, even using Hubitat's own dashboard commands it takes a while for the action to happen, so I go back to the mesh network issue above. Maybe now moving it can help.
But still remains the question of the lack of response to MSR at the same speed.
I will now read the post of @toggledbits to see what he comments.
Thank you very much for your attention.
-
Build 21351 just posted. This has a fix for a problem in the setup of the task queue for HubitatController that is likely causing it to not pace as expected. Let's give this a try, and in addition, try a few different values for
action_pace
. I would start from 25 and work up. -
@toggledbits once again thank you for your kind attention.
What depends on me to send log, do tests I am at your disposal. Just tell me what I should do that I will be promptly attending.
@gwp1 in a previous message I had informed that I had moved Hubitat 2 meters away from my central office, which has my Asus router, no-break, the modems of the internet providers. I have just radicalized, and now the Hubitat is more than 20 meters away and open space, well away from all this magnetic field and radio frequency of a possible interference, let's see if this helps communication.
So now it remains to investigate why the very fast MSR running on a dedicated notebook (Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz 8Gb RAM, 300Gb SSD), maybe running over the Hubitat.
Thanks
-
@wmarcolin said in [Solved] Is there a cap or max number of devices a Global Reaction should not exceed?:
Just tell me what I should do that I will be promptly attending.
You are always helpful, and that's appreciated. For the moment, just run it as it comes. Let's see what happens. Start with a 25 for
action_pace
, and if you continue to have issues, move it up to 50. If 50 isn't enough, then I'll ask for logs. Let me know how it goes. -
OK, changed action_pace to 25 (it was 100), and upgraded to build 21351.
-
@toggledbits in this last upgrade process, I got these zombie processes. How is it possible to kill them? Thanks.
-
Stop Reactor. Then grab the
reactor.log
file and upload it to me. I'm going to DM you a link.After you've uploaded the log file, remove the
storage/states/reaction_queue.json
file and restart Reactor. -
@toggledbits done boss!!
-
I think that in another track you have also received this message, which is occurring exactly when running the above event of turning on several lights.
-
@wmarcolin Very good. That means the more aggressive checks are working. It appears that Hubitat's event socket is a good bit more fragile than its Hass equal. I will add an option to the next release to silence this warning (although the reconnect will still be logged in the log file). You should also be able to see the effect of the reconnects on the system entity's
x_hubitat_sys.reconnects
counter.For comparison, I don't get these errors unless I force them. The one restart shown below was because I upgraded the hub to 2.3.0.120.
-
TL;DR: Hubitat needs more aggressive WebSocket connection health tests and recovery, and that's been added as of 21351. When recovery is needed (which should be very rare), device states may be delayed up to 120 seconds. Don't use WiFi for your Reactor host or hub in production. If your Reactor host and hub aren't on the same network segment (LAN), you may see more reconnects. If you see reconnects when your Reactor host and hub are on the same network segment, you likely have a network quality issue.
I want to explain how I understand the problem reported, and how the fix (which is more of a workaround) works.
The events websocket is one of two channels used by HubitatController to get data from the hub. When HubitatController connects to the hub, it begins with a query to MakerAPI to fetch the bulk data for all devices -- "give me everything". Thereafter, the events socket provides (only) updates (changes). Being a WebSocket connection, it has a standard-required implementation of ping-pong that both serves to keep the connection alive and test the health of the connection. In Reactor, I use a standard library to provide the WebSocket implementation, and this library is in wide use, so while it's almost certainly not bug free (nothing is), it has sufficient exposure to be considered trustworthy. I imagine Hubitat does the same thing, but since it's a closed system, I don't know for sure; they may use something common, or they may have rolled their own, or they may have chosen some black sheep from among many choices for whatever reason. In any case, neither Hubitat nor Reactor implement the WebSocket protocol itself, we just use our respective WebSocket libraries to open and manage the connections and send/receive data.
Apparently there is a failure mode for the connection, and we don't know if it's on the Hubitat (Java) side or in the nodejs package, where the events can stop coming, but apparently the ping-pong mechanism continues to work for the connection, otherwise it would be torn down/flagged as closed/error by the libraries on both ends. There's no easy way to tell if Hubitat has stopped sending messages or the nodejs library has stopped receiving or passing them, and since the libraries/packages on both ends are black boxes as far as I'm concerned, I don't really care, I just want it to work better. So...
HubitatController versions prior to 21351 relied solely on the WebSocket's native ping-pong mechanism to describe connection health, as Reactor does for Hass and even its own UI-to-Engine connection (lending credence to the theory that the nodejs library is not the cause). But for Hubitat it appears the WebSocket ping-pong alone is not enough, so 21351 has introduced some additional tests at the application layer. If any of these fails, the connections are closed and re-opened. When reopened, a full device/state inventory is done again as usual, so the current state of all devices is reestablished. Any missed device updates during the "dead time" would be corrected by this inventory.
By the way, one of the things that exacerbates the problem with the Hubitat events WebSocket is that it's a one-way connection at its application later: Hubitat only transmits. There is no message I can send over the WebSocket for which I could expect a speedy reply as proof of health. I have to find other things to do through MakerAPI in an attempt to force Hubitat to send me data over the WebSocket, and this takes more time as well. If there was two-way communication, it would be a lot easier and faster to know if the connection was healthy.
So the question that remains, then, is what is that timing? By default, HubitatController will start its aggressive recovery at 60 seconds of channel silence. If the channel then remains silent for an additional 60 seconds, the close/re-open recovery occurs. So even if the connection fails, the maximum time to recovery and correct state of all devices will be just over 120 seconds. So even in worst-case conditions, entity states should not lag more than that. Given that these stalls are the exception rather than the rule for most users, these pauses should be rare.
There is one tuning parameter that may be useful to set on VPN connections or any other "distanced" connection (i.e. any connection where the Reactor host and the hub are not on the same network segment, and in particular may traverse connection-managing software or hardware like proxies, stateful packet inspection and intrusion detection systems, load balancers, etc.). That is
websocket_ping_interval
, which will be added to the next build. This will set the interval, in milliseconds, between pings (default 60000). This should be sufficiently narrow to prevent some VPNs from aborting the socket in some cases (see the WebSocket missive at the end), but if not, smaller values can be tried, at the expense of additional network traffic and a slight touch on CPU. If the reconnects don't improve significantly, a different VPN option should be chosen.And this brings me to two recommendations:
- You should not use a WiFi connection for either the Reactor host or hub in production use. These are fine for testing and experimentation, but are an inappropriate choice in production for both reliability and performance reasons.
- If you use a VPN between the Reactor host and the hub, "subscription VPNs" are probably best avoided, as these will be the most aggressive in connection management and cause the most disconnects and failures. That's because they are tuned for surfing web traffic and checking email, basically, where the connections are open-query-response-close — connections don't stay open very long, typically. There are optimizations of HTTP where connections are kept open after a response to allow for a follow-up query (e.g. request an embedded image after requesting a document), but these are generally much shorter than the expected infinite open of a WebSocket connection (more on this at the bottom). Point-to-point VPNs that you set up and manage yourself are likely to provide better stability and performance (e.g. PPTP, SSH tunnels, etc.).
I will also add this: in my network, I do not get Hubitat WebSocket stalls and reconnects. I have had to force them through various devious means to test the behavior I've just implemented. I owned a commercial data center in the San Francisco Bay Area, with a managed network offered to clients with 100% uptime service level agreements. I built and maintained that network. My home network is a reflection of that — good quality equipment, meticulous cabling, sensible architecture (scaled down appropriately for the lesser scope and demands), and active data collection and monitoring. My network runs clean, and when there are problems, I know it (and where). If your Reactor host and Hubitat hub are on the same network segment, hardwired and not WiFi, and you are getting reconnects, I think you should audit your network quality. Something isn't happy. It only takes one bad cable, or one bad connector on one end of one cable, to cause a lot of problems.
---
For anyone interested, one reason why WebSockets can be troublesome in network environments where connection management may be done between the endpoints is that a WebSocket typically begins its life as an HTTP request. The client makes an HTTP request to the server with specific headers that ask that the connection to be "converted" (they call it "upgraded") from HTTP to WebSocket. If the server agrees, the connection becomes persistent and a new session layer is introduced. But because the connection starts as HTTP, any interstitial proxy or device that is managing the connection as it passes through may mistake it for a plain HTTP (web page) request, and when the connection doesn't tear itself down after a short period the proxy/device thinks is reasonable for HTTP requests, it forces the issue and sends a disconnect to both ends. This is necessary because tracking open connections consumes memory and CPU on these devices, and in a commercial ISP environment this could mean tens of thousands or hundreds of thousands of open connections at a single interface/gateway. So to keep from being overwhelmed, these devices may just time out those connections (on a predictable schedule/timeout, or just due to load), but because it's not really an HTTP connection at that point (it's been upgraded to a WebSocket), the proxy/device is breaking a connection that both the server and client expect to be persistent, and that can then cause all kinds of problems, the most benign of which is forcing the two endpoints to reconnect frequently and waste a lot of time and bandwidth doing it. On a LAN, you typically don't have these problems, because the two endpoints have no stateful management between them (network switches, if present between, just pass traffic, not manage connections), so barring network problems, there's no reason for them to be disconnected until either asks to close.