Gradually turn on lights.

[toggledbits]

Have you tried restarting MSR and specifically NOT refreshing the device on Hubitat? That would be useful to figure out if Maker API handing back correct info consistently, but for that check to be valid, nothing on the Hubitat side can change (i.e. don't do Refresh or anything else to the device).

[Alan_F]

I think I have... but I can't remember if that also fixed it. I'll wait for it to happen again and try the MSR restart and then report back.

[toggledbits]

OK. The handling of the WebSocket (events) connection is pretty vanilla, but that may not be enough to detect every possible failure mode, so I've made some changes to make it pretty aggressive (and tunable), downright ruthless by default. I'll be curious to see if that picks up anything on your local hub. I'm sure it will give you notices on the VPN connection. Look for that later this evening.

Edit: by the way, not sure what the devices are in question here, but this popped up for me today: https://community.hubitat.com/t/2-3-0-119-c7-sending-off-to-z-wave-outlet-doing-nothing/85487

I have a ton of the affected device. I guess I have some extra work coming.

[toggledbits]

Build 21351 just posted. I'd like to see what happens for you on this release.

[Alan_F]

Since the device that got out of sync last night is a virtual switch, all the z-wave communication, polling, and refreshing isn't in play. Usually flipping the virtual device would register on MSR. Last night it didn't.

Possibly (or likely) a factor is that Google Assistant turns off every indoor light at the same time is turns on this virtual switch. While all the commands seemed to execute correctly (some of the lights are already off, so I can't tell if the off command for those failed), MSR didn't see the virtual switch change state.

This works most nights, so it may take a while to catch it failing again. Maybe I should move the 'all off' to a Hubutat or MSR rule instead of having the Google routine dump so many commands to Hubitat all at once.

I tried to update to the new version (using the recreate / pull latest image function in Portainer) and it still seems to be running 21349. I'll try again in the morning to see if it pulls the latest.

[Alan_F]

I got the new version. I'm getting the orange bell icon at the top with this alert:

Hubitat "hubitat_xxxxx" is reconnecting because the events feed seems to have become unresponsive.
Last 6:28:59 PM; this alert has repeated 56 times.

I am only getting this from the Hubitat that is connected via VPN. I have not seen any alerts for my hub.

[toggledbits]

Also read here: https://smarthome.community/post/10887

I'm not surprised that the VPN connection dies frequently. The more aggressive check will help it recover more quickly. For your own hub, let it run a while. It seems like your misses are sporadic, and so may be the zombification of the connection.

[Alan_F]

I'm running OpenVPN client and server on two DD-WRT routers. They are less than a mile apart and are both attached to 1 Mbps fiber internet. The other network is basically another subnet on my LAN. Ping times are ~5ms between the Pi4 running Reactor and the Hubitat, and I'm recording video on my Blue Iris server from a camera at the other end of the tunnel without any issues. The connection seems to be pretty solid.

The firewall should be allowing everything through between the two machines. Is there any particular traffic that might be getting blocked or dropped that would result in MSR detecting a loss of connection to the Hubitat? I can check the router logs to see if they show anything that might be relevant.

[toggledbits]

It's a WebSocket connection. The application layer protocol is report only, meaning that Hubitat sends data to Reactor, but there is nothing going from Reactor to Hubitat at the application layer. In the session layer, there is a protocol-required ping-pong. That will occur every 60 seconds. You might make sure there are no HTTP connection time lengths being enforced (WebSockets connections ride within HTTP). If that's controllable in your configuration, it should be indefinite/infinite. You might look at the reactor.log file (always) to see what it says about its unhappiness with the connection; that could give you a clue.

[Alan_F]

So I think in my case it really was just quiet on the entity updates, resulting in a lot of warnings and restarts. I added the Hubitat Ping app to my hub and set it to ping it's gateway every 60 seconds. I know from running several ping jobs on my own hub that they show up as recently updated entities on MSR every time they fire. Sure enough, since starting the polling on their hub there have been no more websocket restarts or warnings.I also haven't had any out-of-sync devices yet, so I'll continue to wait until I see that again and try the MSR restart instead of the Hubitat refresh to see what the result is.