Navigation

    Discussion Forum to share and further the development of home control and automation, independent of platforms.

    SmartHome Community

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Unsolved
    (Last Updated: April 30, 2022)
    • MSR No longer Setting Reactions

      T

      Hello Everyone -

      I have a peculiar situation that I'm hoping to get some help with.

      Running Vera which is still handling my Zwave, MySensors, Zigbee, and still a balance of my Reactors.

      However, I've also been running MSR for several months in preparation for an eventual move off of Vera.

      Vera had one of its meltdowns this morning related to my Opensprinkler plug in.

      After getting that straightened out, I added the OpenSprinkler plugin to HAAS and that's when I started having issues.

      My existing MSR reactors (even simple ones) don't seem to fire their reactions. I can see the conditions go "true". If I manually fire the Reaction, it works fine.

      I do see this a lot in the logs:

      [1.0.2-21350]2022-05-16T19:02:44.518Z <Rule:INFO> Rule#rule-460 evaluation in progress; waiting for completion

      (Various rules - 460 is just an example)

      Is there anything I can do to further troubleshoot?

      Multi-System Reactor
    • Reactor (Multi-System/Multi-Hub) Announcements

      toggledbits

      Build 21228 has been released. Docker images available from DockerHub as usual, and bare-metal packages here.

      Home Assistant up to version 2021.8.6 supported; the online version of the manual will now state the current supported versions; Fix an error in OWMWeatherController that could cause it to stop updating; Unify the approach to entity filtering on all hub interface classes (controllers); this works for device entities only; it may be extended to other entities later; Improve error detail in messages for EzloController during auth phase; Add isRuleSet() and isRuleEnabled() functions to expressions extensions; Implement set action for lock and passage capabilities (makes them more easily scriptable in some cases); Fix a place in the UI where 24-hour time was not being displayed.
      Multi-System Reactor
    • MSR Telegram Messages Stopped Working

      T

      My MSR has been working flawlessly until a couple of days ago when it stopped sending Telegram Messages (or should I say they stopped getting through!). I have updated to MSR version 22004. The reactor.log is updating and is showing no errors:

      [latest-22004]2022-05-15T17:52:31.417Z Engine:NOTICE Starting reaction Message Test (re-l37h2fkj)
      [latest-22004]2022-05-15T17:52:31.417Z Engine:INFO Message Test all actions completed.
      [latest-22004]2022-05-15T17:52:31.704Z Engine:INFO Enqueueing "Message Test" (re-l37h2fkj)
      [latest-22004]2022-05-15T17:52:31.735Z Engine:NOTICE Starting reaction Message Test (re-l37h2fkj)
      [latest-22004]2022-05-15T17:52:31.736Z Engine:INFO Message Test all actions completed.
      [latest-22004]2022-05-15T17:52:32.007Z Engine:INFO Enqueueing "Message Test" (re-l37h2fkj)
      [latest-22004]2022-05-15T17:52:32.035Z Engine:NOTICE Starting reaction Message Test (re-l37h2fkj)
      [latest-22004]2022-05-15T17:52:32.036Z Engine:INFO Message Test all actions completed.
      [latest-22004]2022-05-15T17:52:41.324Z Engine:INFO Enqueueing "Variable Telegram Test" (re-kvnm57c3)

      If I use the following in my web browser the telegram message gets through:

      https://api.telegram.org/botxxxxx......./sendMessage?chat_id=xxxxx......&text=Test Message HTTPS

      I am running MSR on my QNAP network attached storage. Any assistance would be appreciated.

      Just another question...... I would like to post in the HOME HARDWARE VERA section of Smarthome, but I do not have the option to set up a new post. Do I need to request access for this?

      Multi-System Reactor
    • SSL Certificate for Multi System Reactor via LetsEncrypt

      W

      Details of runtime environment:

      Reactor (Multi-hub) stable-22055-9b40e12 Running as a service on Windows Server 2022 Std 21H2 Connecting to HASS (HASSOS) Is a migration WIP from OpenLuup/Reactor

      A complete description of your objective:

      Using LetsEncrypt, application "certifyTheWeb", retrieve and apply SSL certificate to Multi System Reactor

      A description of your approach/solution/implementation so far:

      Reviewed Config files, can't seen mention of certificate files in there

      Actual Question:

      How do I apply a certificate to whatever web server Reactor uses? I'm familiar with managing certificates on IIS and Apache but Reactor doesn't use either of these and I have not been able to locat ethe config file that Reactor uses for this. Which config file needs editing to apply a certificate? Does it use the computer certificate store or it's own store (e.g. pem files)

      The reason I want to achieve this, other than the general goodness of having everything I can in TLS is that I have Reactor as an IFRAME in my HASS console and as HASS is HTTPS and Reactor is HTTP then the iframe refuses to open.

      Search for "certificate" or "ssl" did not reveal anything relevant in this forum.

      Multi-System Reactor
    • MSR ZEN32 weird behavior

      R

      @toggledbits
      Hi Patrick,
      Running HASS 2022.5.3
      MSR zwavejs-22126-834c3e9

      Seeing weird behavior in MSR with regards to my Zen32's. Not sure how to capture info that might be relevent but will describe what I am seeing.
      In the example below I pressed scene 001 4 times. It flashed and changed the primary value to "4". The remaining buttons were previously showing the PV as "double" I pushed each button one time and they flashed and displayed the PV as 4.
      zen32.JPG

      In the zwavejs2mqtt debug screen that sequence looks like this.

      2022-05-10 09:38:44.078 INFO ZWAVE: Node 45: value notification: 91-0-scene-001 5 2022-05-10T16:38:44.081Z CNTRLR « [Node 045] received CentralScene notification { "nodeId": 45, "ccId": "Central Scene", "ccCommand": "0x03", "payload": "0x898501" } 2022-05-10 09:38:45.982 INFO ZWAVE: Node 45: value notification: 91-0-scene-002 0 2022-05-10T16:38:45.984Z CNTRLR « [Node 045] received CentralScene notification { "nodeId": 45, "ccId": "Central Scene", "ccCommand": "0x03", "payload": "0x8a8002" } 2022-05-10 09:38:47.807 INFO ZWAVE: Node 45: value notification: 91-0-scene-003 0 2022-05-10T16:38:47.810Z CNTRLR « [Node 045] received CentralScene notification { "nodeId": 45, "ccId": "Central Scene", "ccCommand": "0x03", "payload": "0x8b8003" } 2022-05-10 09:38:48.858 INFO ZWAVE: Node 45: value notification: 91-0-scene-004 0 2022-05-10T16:38:48.861Z CNTRLR « [Node 045] received CentralScene notification { "nodeId": 45, "ccId": "Central Scene", "ccCommand": "0x03", "payload": "0x8c8004" }

      Let me know what other info you would like me to supply.
      I have played around with a bunch of other key stroke combo's and it looks like buttons 2-5 display what ever button 1 is set to.
      Have tested this on 4 different ZEN32's and it is same on all.

      Multi-System Reactor
    • HASS 2022.5 issue with ZwaveJS in MSR

      R

      Running zwavejs-22118-baab576 in MSR, HASSOS 2022.5.1

      Due to an unreported issue in HASS, when I upgraded the 2022.5 yesterday, it dropped ZwaveJS2MQTT from my HASSOS build. Finally got it reinstalled and it appears to be working the way it was previously.
      MSR is now showing zwavejs down and in entities, nothing is updating.

      Seeing this in the logs:
      ZWaveJSController#zwavejs ws_open caught [Error]Error: connect ECONNREFUSED 192.168.0.16:3000

      Are there additional steps needed that are not mentioned in the manual?
      At a loss to figure out what is causing issue.

      Multi-System Reactor
    • Global Expression – Reference Error

      wmarcolin

      Hi!

      After several tests, restarting the computer where the MSR is, and the MSR itself by Tools > Restart, I have a persistent error in the logs.

      32d45bc3-51f8-4718-a113-3381e9d7edf2-image.png

      Looking at the global expression (screen below), I see that it is highlighted as an error, however, if I look at two other expressions above that have the same structure, they do not present an error, so I do not think the problem is with the expression's code.

      ca8a7867-4ac7-4f9a-9913-15a80da18f05-image.png

      This expression uses a group, which validates 4 devices of type door sensor, whether the status is true or false, door open or closed.

      "dgc_entrance_open": name: DGC Entrance Open select: - include_entity: - "hubitat>37" - "hubitat>38" - "hubitat>99" - "hubitat>130" filter_expression: > entity.attributes?.door_sensor?.state == true

      I also compared it with other groups, the structure is identical.

      Looking at the 4 devices in this group, I see that all are active in the MSR, have similar variables, are used in other groups and rules that do not present a problem.

      52b099f4-b9c4-46cb-ad3d-c873b09b7e13-image.png

      Looking at the log, I understand that it refers to some object that would be null, at which point I am lost on what to actually look at.

      [latest-22118]2022-05-02T22:33:11.711Z <Engine:INFO> Reaction Engine starting [latest-22118]2022-05-02T22:33:11.719Z <Engine:INFO> Checking rule sets... [latest-22118]2022-05-02T22:33:11.746Z <Engine:INFO> Checking rules... [latest-22118]2022-05-02T22:33:11.748Z <Engine:INFO> Data check complete; no corrections. [latest-22118]2022-05-02T22:33:11.892Z <Engine:ERR> [Engine]Engine#1: error evaluating expression EntranceOPEN: [ReferenceError]ReferenceError: Invalid scope in reference to member attributes of (object)null [latest-22118]2022-05-02T22:33:11.893Z <Engine:ERR> [Engine]Engine#1: expression: sort(each id in ( getEntity('groups>dgc_entrance_open').attributes.sys_group.members ): getEntity(id).name) [latest-22118]2022-05-02T22:33:11.894Z <Engine:CRIT> ReferenceError: Invalid scope in reference to member attributes of (object)null ReferenceError: Invalid scope in reference to member attributes of (object)null at _run (C:\MSR\reactor\common\lexp.js:1449:31) at _run (C:\MSR\reactor\common\lexp.js:1443:33) at _run (C:\MSR\reactor\common\lexp.js:1443:33) at _run (C:\MSR\reactor\common\lexp.js:1536:38) at _run (C:\MSR\reactor\common\lexp.js:1480:33) at C:\MSR\reactor\common\lexp.js:1283:29 at Array.forEach (<anonymous>) at _run (C:\MSR\reactor\common\lexp.js:1282:28) at run (C:\MSR\reactor\common\lexp.js:1663:22) at Object.evaluate (C:\MSR\reactor\common\lexp.js:1699:20) [latest-22118]2022-05-02T22:33:12.232Z <Engine:NOTICE> Reaction Engine running! [latest-22118]2022-05-02T22:33:12.233Z <Rule:INFO> Rule#rule-kxjp2el2 (Mode DAY) started [latest-22118]2022-05-02T22:33:12.234Z <Rule:INFO> Rule#rule-kxjp7kc8 (Mode EVENING) started [latest-22118]2022-05-02T22:33:12.234Z <Rule:INFO> Rule#rule-kxjparz7 (Mode NIGHT) started [latest-22118]2022-05-02T22:33:12.234Z <Rule:INFO> Rule#rule-kxjpd0d6 (Mode AWAY ∆) started [latest-22118]2022-05-02T22:33:12.235Z <Rule:INFO> Rule#rule-kxjqrcr9 (Mode PARTY) started [latest-22118]2022-05-02T22:33:12.235Z <Rule:INFO> Rule#rule-kxjqyhnu (Mode VACATION ∆) started

      Finally, if I return the expression and press the Try This Expression button, it is executed and the error disappears.

      102e5013-b9aa-4c26-98f5-1ae545f8897f-image.png

      This situation started to occur on the latest version of Reactor 22118, on a Windows 10 Bare Metal installation. Does anyone have a similar scenario?

      Sorry for the long post, I tried to document as best as possible to try to identify the problem.

      Thanks.

      Multi-System Reactor
    • Cheapest platform on which to run MSR

      T

      Please can anyone give me advice on a cheap way to run MSR (both initial cost and ongoing electricity cost). I currently run it on my QNAP network drive (which works very well incidentally). However, this draws a constant 35Watts (£90/yr @ 30p/kWh) and given the recent rise in fuel costs I am evaluating the best way forwards. Note, I only really need to turn my QNAP on for an hour a week to back-up my computer data. I understand that MSR can run on a Raspberry PI, but I am not competent with Linux! Thanks.

      Multi-System Reactor
    • Build 22118 - entity updates

      G

      I just noticed that the new hotness picked up the entities from when I added an integration temporarily to my Home Assistant and popped a lovely info notice into the Alerts pane.

      I know the info is new - but when did MSR suddenly become more real-time with devices being added to hubs? This is great!

      Multi-System Reactor
    • [Solved] Is there a cap or max number of devices a Global Reaction should not exceed?

      G

      Noticed in the last few nights that my Global Reaction to turn on a mere six lights in the evenings the same two devices no longer turn on. One is a dimming smart bulb, the other is a wall outlet.

      This has been running flawlessly for months - just in the last few nights I've noticed these two devices being left behind and yes, it's always these two devices only.

      "What changed?"

      No Hubitat firmware updates No new devices added to Hubitat list itemTwo MSR updates (the last two, I keep up with the latest) list itemHome Assistant releases (should have zero bearing, HA is nothing more than pretty dashboards) list itemAdded MQTT to MSR

      I did do a z-wave "repair" last night and am waiting to see if this helps.

      Is there any maximum or cap on number of devices that should be included in a Global Reaction?

      *Marked as solved as the question asked was answered. The issue persists but the question was answered.

      Multi-System Reactor
    • [Solved] Which version of npm should we be running for reactor-latest-22118-0f0772c.tar.gz

      G

      I've borked my install of reactor-latest-22118-0f0772c.tar.gz for some reason and am retracing steps. Running npm i --no-save --omit dev prompted to update npm and I did (current version now 8.6.0). Currently, attempts to load via browser/s (Brave and Safari) result in "This site can’t be reached192.168.1.xxx refused to connect." I am still able to SSH into the RPi.

      I haven't panicked yet but did just pull my config and storage directories - I believe I read where that's enough for a back-up if a clean install ends up being needed.

      So the question of the day: which version of nmp should we be running on for the latest Reactor?

      Multi-System Reactor
    • Is a global timestamp expression possible ?

      B

      I’ve looked and can’t find the answer which concerns me that the answer is obvious and thats why I can't find the answer to this.

      I’m new to MSR and this is my first post, so please bare with me 🙂

      What I’m trying to do is have a global expression that contains the current time so I can use it in Global reactions as well as Rulesets as part of my syslog messages.

      Example of what I’m experiencing:
      In Expressions, create a global expression: gMyTOD = strftime("%T %D”)
      In Rulesets, in a rule, create a local expression: MyTOD = strftime("%T %D”)

      In that same rule create a Notify type of Syslog.
      Set syslog message to: gMyTOD time is ${{gMyTOD}} and MyTOD time is ${{MyTOD}}
      Trigger the rule.

      The result is that the local expression (MyTOD) contains the correct time but the global expression (gMyTOD) does not update and reflects the wrong time in the final syslog message.

      Is there a different approach I should be using?

      Thank you in advance for any insight.

      -bh

      Multi-System Reactor
    • Restart router when it's gone offline?

      F

      Hi!
      Once a month or so, my internet access is lost, and I have to cut the power to the router, to restart it. When I'm away this become a problem, so I thought that if I got a zigbee switch (in Home Assistant), problem is solved. But how do I (MSR) know if it's offline?

      What's the easiest and most effective way to make this work? Do I ping a few websites every ten minutes or so, or is there a better approch? If I'm going to ping, what URL is recommended?

      For information, I have a set up with Asus routers, Home Assistent with zigbee-, z-wave and wifi devices (a total of appr. 150) and every of my 60-70 automation is done by MSR.
      I run MSR ver. 22080-ae7212f (docker on my Home Assistant blue). HA ver 2022.4.6.
      Thanks in advance.
      Take care, you all!
      /Fanan

      Multi-System Reactor
    • MSR for Scene Controllers

      therealdb

      I still have some code running on my Vera that I want to port and it's related to scene controllers.

      I'm mostly using Fibaro's switch/roller shutter and on the Vera side I'm getting sl_CentralScene/sl_CentralSceneAttr, but it seems too fast to be intercepted by MSR.

      What's anyone using? Thanks.

      Multi-System Reactor
    • Best strategy to get the difference between pulses in MSR

      therealdb

      So, I have a rule set that it's triggered when a local variable changes. Since it's tied to my solar production, it could change very frequently, so I've just added a reset delay to calm it down, but it's not working as intended.

      What I really need is a way to, let's say, define a percentage and define a variable to understand if the new value has changed +/- 10% before going true.

      I've tried a couple of ways, but I'm posting here anyway, in case anyone has found a better way to tackle this. Thanks!

      Multi-System Reactor
    • Status of MSR in Home Assistant

      Matohl

      Perhaps a dumb question. Can I check from Home Assistant that MSR is up and running? I would like to see that in a status page in HA. I can of course ping the other Pi but it doesn't say anything about the MSR status.

      Skärmavbild 2022-04-23 kl. 18.23.03.png

      Multi-System Reactor
    • Notifications from Alerts

      Crille

      @toggledbits Is there a way to get Current Alerts as an entity so I could monitor those and send notifications if needed.
      My usecase is sometimes openLuup crashes and MSR picks that up pretty quick, sending a pushover notification to me would get my attention to restart it.
      Of course there are many ways to monitor connectivity but since MSR already is doing that it would be neat to take advantage of it.

      Multi-System Reactor
    • Invocable rules via MQTT

      therealdb

      Hey @toggledbits.
      From the docs:

      You cannot control the state of rules via MQTT. Rule state is driven exclusively by the result of its conditions.

      And I'm OK when rules are triggered by something else, but I've built a couple of rules to be used by other rules, to streamline the logic, and it'll be useful to invoke a rule via MQTT. It's probably close to what rule/:id/restart is doing in the HTTP api.

      Thanks!

      Multi-System Reactor
    • Home Assistant Script, Reaction requires seclection that isnt there

      S

      I'm trying to create some automation so I can control my pool variable speed pump from reactor rather than iAqualink (which sucks). My controller is MQTT enabled so I created several scripts for my desired speeds. (I know reactor has MQTT ability but honestly I read the docs and could not make much sense of it and since I already have it set up in Home Assistant, I figured I would just leverage that instead)

      So I created all of my scripts and tested them, they worked perfectly. The I created a rule in reactor and set running the script as the action. The problem is that reactor is telling me that I have to make a selection. I try to click the dropdown and there is nothing to select. I cant save the rule, the only option is to exit.

      9e0a3147-5058-4893-9860-2f0d759dfe3c-image.png

      Here is my script in Home Assistant. Any advice on how to get this to work? If I am looking at this the wrong way I am open to suggestions (I would rather be able to call a specific action and specify a speed value but I cannot figure out how to do that). Thanks in advance.

      bfc7ba89-70a5-47d3-a111-0c2927ac3c73-image.png

      Multi-System Reactor
    • Installed latest MSR container and host time is wrong

      M

      Hi, I've been using msr for a while now and haven't had timezone issues until today when I updated to latest docker image (latest-22080-ae7212f)

      both docker date command and my local date command show the same time, same timezone, but for some reason MSR shows the host time as 2 hours in the past as a result all my timed rules run 2 hours late.

      my locale is set correctly in the host machine and is bound into the container as well.

      One possible hint I get is maybe this from the container startup log, but other than timezone stuff everything else seems to be working fine and all my settings loaded up just fine.

      [latest-22080]2022-04-07T07:52:52.423Z <SystemController:null> Module SystemController v22080 root {} CommonJS loading [latest-22080]2022-04-07T07:52:57.236Z <Engine:CRIT> SyntaxError: Unexpected end of JSON input SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at IndividualFileStrategy.getDataObject (/opt/reactor/server/lib/IndividualFileStrategy.js:121:46) at Container.getDataObject (/opt/reactor/server/lib/Container.js:102:53) at new GlobalExpression (/opt/reactor/server/lib/GlobalExpression.js:105:179) at Function.getInstance (/opt/reactor/server/lib/GlobalExpression.js:123:89) at /opt/reactor/server/lib/Engine.js:1349:67 at Array.forEach (<anonymous>) at Engine.initializeGlobalExpressions (/opt/reactor/server/lib/Engine.js:1345:145) at Engine.start (/opt/reactor/server/lib/Engine.js:590:285) at /opt/reactor/app.js:362:103

      Let me know if I can provide further info to help debug this.

      Thanks

      Multi-System Reactor
    For those who registered but didn't received the confirmation email, please send an email to support@smarthome.community with the email you used

    [Solved] Is there a cap or max number of devices a Global Reaction should not exceed?

    Multi-System Reactor
    5
    40
    415
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • toggledbits
      toggledbits @wmarcolin last edited by

      @wmarcolin Very good. That means the more aggressive checks are working. It appears that Hubitat's event socket is a good bit more fragile than its Hass equal. I will add an option to the next release to silence this warning (although the reconnect will still be logged in the log file). You should also be able to see the effect of the reconnects on the system entity's x_hubitat_sys.reconnects counter.

      For comparison, I don't get these errors unless I force them. The one restart shown below was because I upgraded the hub to 2.3.0.120.

      b66090a7-5f2d-42a7-81a6-6934f71f378f-image.png

      Author of Multi-system Reactor and Reactor, DelayLight, Switchboard, and about a dozen other plugins that run on Vera and openLuup.

      1 Reply Last reply Reply Quote 0
      • toggledbits
        toggledbits last edited by

        TL;DR: Hubitat needs more aggressive WebSocket connection health tests and recovery, and that's been added as of 21351. When recovery is needed (which should be very rare), device states may be delayed up to 120 seconds. Don't use WiFi for your Reactor host or hub in production. If your Reactor host and hub aren't on the same network segment (LAN), you may see more reconnects. If you see reconnects when your Reactor host and hub are on the same network segment, you likely have a network quality issue.

        I want to explain how I understand the problem reported, and how the fix (which is more of a workaround) works.

        The events websocket is one of two channels used by HubitatController to get data from the hub. When HubitatController connects to the hub, it begins with a query to MakerAPI to fetch the bulk data for all devices -- "give me everything". Thereafter, the events socket provides (only) updates (changes). Being a WebSocket connection, it has a standard-required implementation of ping-pong that both serves to keep the connection alive and test the health of the connection. In Reactor, I use a standard library to provide the WebSocket implementation, and this library is in wide use, so while it's almost certainly not bug free (nothing is), it has sufficient exposure to be considered trustworthy. I imagine Hubitat does the same thing, but since it's a closed system, I don't know for sure; they may use something common, or they may have rolled their own, or they may have chosen some black sheep from among many choices for whatever reason. In any case, neither Hubitat nor Reactor implement the WebSocket protocol itself, we just use our respective WebSocket libraries to open and manage the connections and send/receive data.

        Apparently there is a failure mode for the connection, and we don't know if it's on the Hubitat (Java) side or in the nodejs package, where the events can stop coming, but apparently the ping-pong mechanism continues to work for the connection, otherwise it would be torn down/flagged as closed/error by the libraries on both ends. There's no easy way to tell if Hubitat has stopped sending messages or the nodejs library has stopped receiving or passing them, and since the libraries/packages on both ends are black boxes as far as I'm concerned, I don't really care, I just want it to work better. So...

        HubitatController versions prior to 21351 relied solely on the WebSocket's native ping-pong mechanism to describe connection health, as Reactor does for Hass and even its own UI-to-Engine connection (lending credence to the theory that the nodejs library is not the cause). But for Hubitat it appears the WebSocket ping-pong alone is not enough, so 21351 has introduced some additional tests at the application layer. If any of these fails, the connections are closed and re-opened. When reopened, a full device/state inventory is done again as usual, so the current state of all devices is reestablished. Any missed device updates during the "dead time" would be corrected by this inventory.

        By the way, one of the things that exacerbates the problem with the Hubitat events WebSocket is that it's a one-way connection at its application later: Hubitat only transmits. There is no message I can send over the WebSocket for which I could expect a speedy reply as proof of health. I have to find other things to do through MakerAPI in an attempt to force Hubitat to send me data over the WebSocket, and this takes more time as well. If there was two-way communication, it would be a lot easier and faster to know if the connection was healthy.

        So the question that remains, then, is what is that timing? By default, HubitatController will start its aggressive recovery at 60 seconds of channel silence. If the channel then remains silent for an additional 60 seconds, the close/re-open recovery occurs. So even if the connection fails, the maximum time to recovery and correct state of all devices will be just over 120 seconds. So even in worst-case conditions, entity states should not lag more than that. Given that these stalls are the exception rather than the rule for most users, these pauses should be rare.

        There is one tuning parameter that may be useful to set on VPN connections or any other "distanced" connection (i.e. any connection where the Reactor host and the hub are not on the same network segment, and in particular may traverse connection-managing software or hardware like proxies, stateful packet inspection and intrusion detection systems, load balancers, etc.). That is websocket_ping_interval, which will be added to the next build. This will set the interval, in milliseconds, between pings (default 60000). This should be sufficiently narrow to prevent some VPNs from aborting the socket in some cases (see the WebSocket missive at the end), but if not, smaller values can be tried, at the expense of additional network traffic and a slight touch on CPU. If the reconnects don't improve significantly, a different VPN option should be chosen.

        And this brings me to two recommendations:

        1. You should not use a WiFi connection for either the Reactor host or hub in production use. These are fine for testing and experimentation, but are an inappropriate choice in production for both reliability and performance reasons.
        2. If you use a VPN between the Reactor host and the hub, "subscription VPNs" are probably best avoided, as these will be the most aggressive in connection management and cause the most disconnects and failures. That's because they are tuned for surfing web traffic and checking email, basically, where the connections are open-query-response-close — connections don't stay open very long, typically. There are optimizations of HTTP where connections are kept open after a response to allow for a follow-up query (e.g. request an embedded image after requesting a document), but these are generally much shorter than the expected infinite open of a WebSocket connection (more on this at the bottom). Point-to-point VPNs that you set up and manage yourself are likely to provide better stability and performance (e.g. PPTP, SSH tunnels, etc.).

        I will also add this: in my network, I do not get Hubitat WebSocket stalls and reconnects. I have had to force them through various devious means to test the behavior I've just implemented. I owned a commercial data center in the San Francisco Bay Area, with a managed network offered to clients with 100% uptime service level agreements. I built and maintained that network. My home network is a reflection of that — good quality equipment, meticulous cabling, sensible architecture (scaled down appropriately for the lesser scope and demands), and active data collection and monitoring. My network runs clean, and when there are problems, I know it (and where). If your Reactor host and Hubitat hub are on the same network segment, hardwired and not WiFi, and you are getting reconnects, I think you should audit your network quality. Something isn't happy. It only takes one bad cable, or one bad connector on one end of one cable, to cause a lot of problems.

        ---

        For anyone interested, one reason why WebSockets can be troublesome in network environments where connection management may be done between the endpoints is that a WebSocket typically begins its life as an HTTP request. The client makes an HTTP request to the server with specific headers that ask that the connection to be "converted" (they call it "upgraded") from HTTP to WebSocket. If the server agrees, the connection becomes persistent and a new session layer is introduced. But because the connection starts as HTTP, any interstitial proxy or device that is managing the connection as it passes through may mistake it for a plain HTTP (web page) request, and when the connection doesn't tear itself down after a short period the proxy/device thinks is reasonable for HTTP requests, it forces the issue and sends a disconnect to both ends. This is necessary because tracking open connections consumes memory and CPU on these devices, and in a commercial ISP environment this could mean tens of thousands or hundreds of thousands of open connections at a single interface/gateway. So to keep from being overwhelmed, these devices may just time out those connections (on a predictable schedule/timeout, or just due to load), but because it's not really an HTTP connection at that point (it's been upgraded to a WebSocket), the proxy/device is breaking a connection that both the server and client expect to be persistent, and that can then cause all kinds of problems, the most benign of which is forcing the two endpoints to reconnect frequently and waste a lot of time and bandwidth doing it. On a LAN, you typically don't have these problems, because the two endpoints have no stateful management between them (network switches, if present between, just pass traffic, not manage connections), so barring network problems, there's no reason for them to be disconnected until either asks to close.

        Author of Multi-system Reactor and Reactor, DelayLight, Switchboard, and about a dozen other plugins that run on Vera and openLuup.

        wmarcolin 1 Reply Last reply Reply Quote 1
        • wmarcolin
          wmarcolin @toggledbits last edited by

          @toggledbits master!

          Another lesson in knowledge, and dedication to understanding and solving problems.

          Well, I spent two days reading your post before trying to answer anything.

          First starting from the end, in my case my network is all Giga, CAT7 cables with industrial connectors. All the cabling of the house came with cables ready to not run the risk of redoing connectors. Then I even hired a company to certify the network, which also uses management switches that I can validate the network. So ping inside my house between any equipment is < 1ms.

          8531f96f-e697-48f6-9691-8e74ebacf9d1-image.png

          From the technical side what I see is that the connection between MSR and Hubitat is fragile, and it becomes even more so when MSR is much faster in its actions without Hubitat responses.

          What you would be doing for a future version is strengthening the validation of this communication, in particular, to return the status of given orders. That is, if you tell the MSR to turn on a light, make sure that the return state says that the light is on.

          I was in doubt, in case of saying that it was not turned on, would there be a reset? Could this be a summary?

          toggledbits 1 Reply Last reply Reply Quote 0
          • toggledbits
            toggledbits @wmarcolin last edited by toggledbits

            Ping (and traceroute) are not network quality tools. They are path tools. To measure link quality, since you said you had managed switches, you'd need to look at the error counts on each port. That may be worth a squiz.

            As @Alan_F has observed, you can get restarts from a quiet channel that is just naturally quiet. If you don't have a lot of devices, and things aren't changing often, it's very likely to see a quiet channel. HubitatController probes by picking a device and making it refresh, which usually causes some event activity on the WebSocket channel. But it's possible that the device it picks (randomly) may not do much when refreshed. You can set probe_device to the device ID (number) of a device that consistently causes events when asked to refresh to mitigate that random effect. That may take some experimentation, using the Hubitat UI to ask devices to refresh, and watching for green highlights in the Reactor Entities list. When you find a device that consistent "lights" when you refresh it on Hubitat, you've probably got a reliable probe device.

            I have noted in my own network that certain devices (that shall be unnamed) are sufficiently chatty that I'm at no risk of a quiet channel. Let's just say anything that monitors energy is a good canary in the mine.

            Author of Multi-system Reactor and Reactor, DelayLight, Switchboard, and about a dozen other plugins that run on Vera and openLuup.

            wmarcolin 1 Reply Last reply Reply Quote 0
            • wmarcolin
              wmarcolin @toggledbits last edited by

              @toggledbits

              The ping tests were to validate the connection, speed, below is the screen of the switch that the MSR and Hubitat are on, no errors. I have a weekly reset programmed, so this information is from last Friday until now.

              ca085548-f8da-408f-b57c-9535eda260ab-image.png

              Ok, so what I should set now is a device connection probe, something like the APP Device Watchdog. I'm going this way.

              Anyway, I think I mentioned before, I ended up moving Hubitat far away from my wifi router on Saturday, and since last night after deactivating the devices again one by one, I started rebuilding my mesh. I already see good results with the move away, several devices that previously had trouble responding, now seem to work better. Hopefully, by Wednesday I will have finished the rebuild, and will be able to check how the network behavior and ping-pong between MSR and Hubitat is now with the latest version of MSR.

              Thanks.

              1 Reply Last reply Reply Quote 0
              • toggledbits
                toggledbits last edited by

                Very good. In my limited experience with this new heuristic, Z-Wave devices seem to be the most predictable for probes among the basic devices. In the current build, whatever you choose must support the Refresh (Hubitat native) capability (aka x_hubitat_Refresh in Reactor). In the next build, the config value probe_action will be available to let you use a different action, if you need to, and probe_parameters (an object) containing any key/value parameter pairs that the command may need (optional).

                I've also found that the "Hub Information" app (or more correctly, the device that this app manages) makes a good probe target. It has some useful data, and also very reliably updates fields when commanded to do so, even at a high frequency, so the next build will have specific support for this app/device if it finds it among the hub's inventory.

                Author of Multi-system Reactor and Reactor, DelayLight, Switchboard, and about a dozen other plugins that run on Vera and openLuup.

                wmarcolin 1 Reply Last reply Reply Quote 0
                • wmarcolin
                  wmarcolin @toggledbits last edited by

                  @toggledbits hi!

                  I have been operating for two days with version 23360, and the chaos described in the messages above is no longer happening. The recurrent failures of leaving an action without executing as a whole are no longer oberved.

                  Thanks for all your efforts!

                  A question, in version 21353 we started seeing the message below, which you made possible after deactivating the alert.

                  b2ee2739-e903-4f3a-875a-b1158c2fa53d-image.png

                  Is it possible to make some kind of query, or trigger some action when this message happens? I would like for example to send a Telegram to notify me.

                  Thanks.

                  1 Reply Last reply Reply Quote 0
                  • toggledbits
                    toggledbits last edited by toggledbits

                    Yes, there are at least two ways to do that now. Any thoughts on where you might look?

                    Edit: Same question as this thread: https://smarthome.community/topic/832/identifying-when-msr-cannot-connect-to-home-assistant

                    And this thread is wandering, so maybe let's leave it alone.

                    Author of Multi-system Reactor and Reactor, DelayLight, Switchboard, and about a dozen other plugins that run on Vera and openLuup.

                    wmarcolin 2 Replies Last reply Reply Quote 0
                    • Unlocked by  toggledbits toggledbits 
                    • wmarcolin
                      wmarcolin @toggledbits last edited by wmarcolin

                      Hi, I asked @toggledbits to reopen this very long thread, to give a testimony of a situation that I believe can help others.

                      I apologize for the long message, let's recapitulate the history.

                      The discussion was based on possible device limits on MSR actions, which Patrick explained there would not be, but the situation described was very similar to the scenario I posted in my message, of numerous failures on actions sent from MSR to Hubitat (December 16, 12:28am). That MSR was much faster than Hubitat could process.

                      I posted an example where I had to execute a series of Reactions that would turn on lights, and also turn on outlets, it always failed, I demonstrated that the MSR would activate everything, but Hubitat would not execute.

                      Then came the topic and orientation that my Hubitat should not be next to my WiFi router that could be interfering with the Hubitat signal, I even sent a picture on December 17 and provided the change (TIP 1).

                      Well obviously, as the change of position was big, I had to redo 3 times the entire mesh network, adding and removing devices. I followed the instruction of many masters, do the network from the center, i.e., from Hubitat to the outside, including first devices that use electric power because they are repeaters, and then those with exclusive battery power (TIP 2).

                      Well, our friend Patrick releases version 21351 and then 23360 where he adds much more aggressive management in the communication MSR x Hubitat, it improved a lot. But unfortunately, I kept having problems.

                      Then I asked for help again to go forward and see what to do to improve, we entered in the theme that several posts from the Hubitat community mentioned devices that use the S0 security, that this creates problems in the mesh network by high traffic of unnecessary information, new action remove and include again the devices that had S0 (TIP 3), another action that helped the network. What was not possible, we reactivated the Vera hub and put these devices back in, removing them from the Hubitat network.

                      We were evolving, but the situation persisted, actions that triggered many actions to Hubitat could still have failures, and the worst actions using Hubitat's own dashboard were also not being executed.

                      Well, I returned to the discussion of when I changed the Vera to Hubitat, which highlighted several points of change, but one very bothered me, the Hubitat Z-Wave signal, much weaker than the vera (https://smarthome.community/topic/776/switching-from-vera-to-hubitat/9?_=1644189698709).

                      Well 4 days ago (2/2), moved by the courage I opened my Hubitat and followed the post (https://community.hubitat.com/t/external-antenna/81396/28) (TIP 4), and installed an external antenna for z-wave, here I show that I bought and installed (https://community.hubitat.com/t/elevation-c7-possible-faulty-z-wave-radio/52977/91). In this post the discussion started with the theme S0 and S2, and went into the antenna theme.

                      MY TESTIMONY OF WHAT HAPPENED

                      A revolution, my Hubitat got a new life, it is another equipment:

                      • Before I had 15 direct devices in the hub, today after 4 days there are already 36 of 64, and I see that every day is increasing as the network is being restructured. There was the absurdity of equipment in the same environment as the HE, but behind a column, using two other devices to reach the HE that was less than 4 meters away, now communicates directly;
                      • There was almost no equipment that communicated at 100kbps, most were between 9.6 and 40, now most are already 100, and a small number, 5/64 are at 9.6 kbps;
                      • Remember the thing where I had to turn on several lights and power outlets all together? it didn't fail anymore, as the devices speak better and faster like the HE, I don't see this failure anymore;
                      • I also talked about actions commanded by the Dashboard that the device did not respond to, it is not happening anymore either.

                      In summary, in my case that 3/4 of the devices are not repeaters, they use batteries, my z-wave mesh network had a lack of repeaters, and this generated a generalized degradation. Now, with this better signal, if not eliminated the problem, I reduced it to almost zero.

                      Now pay attention, the operation of putting up the external antenna seems simple, but it is not. The antenna connector that is soldered to the board is very small and difficult to handle. So if you go this way, look for a cell phone repair shop, they will surely have the best technique for this change.

                      Thank you, and sorry again for the long message.

                      @gwp1 maybe this can help if you still have a problem. Your tip to move the HE away from the Wifi was also precious, thank you.

                      @SweetGenius your comment that there might be an overwhelming in the hub was correct, the action_pace action was a help, but the signal improvement I describe was the solution when I have a more fast response of the devices, reducing the overwhelm. Thanks.

                      @toggledbits our last messages, before I wanted to incinerate Hubitat, also helped a lot on the way. Thank you for all your dedication.

                      wmarcolin 1 Reply Last reply Reply Quote 3
                      • wmarcolin
                        wmarcolin @wmarcolin last edited by

                        Well, after such a good result with Z-Wave, now also external antenna for Zigbee with the same benefit.

                        cf21e073-f854-436f-a3da-b1ceab53b231-image.png

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post

                        Welcome. If you’d like to participate in the discussion, rather than just read, then you can join the forum. As a member, you can interact with others here to share your experience and ask the questions you need answered.

                        Powered by NodeBB | Contributors
                        Hosted freely by PointPub Media Communications Inc. | Contact us