[Reactor] Bug when sending MQTT boolean payloads

[LibraSun]

I vote that the expanded list of Rule Sets be (a) indented slightly, so as to distinguish them from the primary UI elements (e.g. "Entities" et seq), and (b) perhaps display a vertical rule | alongside to further differentiate them as a "submenu".

Something like:

Right now, the bold-vs-not-bold sometimes throws me, because my list of Rule Sets (greatly truncated here, for clarity) can be longer than the scroll window itself.

P.S. THANKS for the "PUT" and other verbs now available with HTTP Request!

[LibraSun]

@toggledbits would it be possible to augment vera>housemode with its own timestamp, so that repeated setting to the same mode can be detected? Currently, I have no obvious way to trigger on "HOME" if Vera is already set to "HOME", etc.

I wasn't even certain that Vera herself reacts internally in any way when I click "HOME" on the Dashboard while she's already in "HOME" mode, so I peeked in her LuaUPnP log and saw:

08	04/06/21 8:56:08.257	JobHandler_LuaUPnP::HandleActionRequest device: 0 service: urn:micasaverde-com:serviceId:HomeAutomationGateway1 action: SetHouseMode <0x70a0e520>
08	04/06/21 8:56:08.257	JobHandler_LuaUPnP::HandleActionRequest argument serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1 <0x70a0e520>
08	04/06/21 8:56:08.258	JobHandler_LuaUPnP::HandleActionRequest argument action=SetHouseMode <0x70a0e520>
08	04/06/21 8:56:08.258	JobHandler_LuaUPnP::HandleActionRequest argument Mode=1 <0x70a0e520>
08	04/06/21 8:56:08.258	JobHandler_LuaUPnP::HandleActionRequest argument rand=0.7538517467636494 <0x70a0e520>
06	04/06/21 8:56:08.261	Device_Variable::m_szValue_set device: 224 service: urn:micasaverde-com:serviceId:SecuritySensor1 variable: Armed was: 0 now: 0 #hooks: 0 upnp: 0 skip: 0 v:0x1057770/NONE duplicate:1 <0x70a0e520>
06	04/06/21 8:56:08.282	Device_Variable::m_szValue_set device: 264 service: urn:micasaverde-com:serviceId:SecuritySensor1 variable: Armed was: 0 now: 0 #hooks: 0 upnp: 0 skip: 0 v:0x1057770/NONE duplicate:1 <0x

Short of creating a virtual switch or sensor that Vera "arms" on every mode change, I'm unsure how to detect the logged reactions back on MSR. Any thoughts?

P.S. Funny, I'm also just discovering that Vera cannot trigger her own Scenes based on House Mode changing. Who knew?

EDIT: So, I went with creating a (binary on/off) Virtual Switch using Switchboard plug-in. By having Vera turn this switch "On" for every House Mode, MSR can now detect "updates" to House Mode, not merely "changes". This is useful to my workflow. I'll simply create an additional action in SET REACTION which turns the new VS back "Off" automatically.

[toggledbits]

I would not be able to make this work on RFV or MSR. There is no timestamp for house mode on Vera because it isn't a state variable, it's an attribute of (virtual) device 0, the HomeAutomationGateway1 engine (which is why it appears as an attribute of the root in user_data). There's nothing to put a watch callback on, because you can only watch state variables. This is why the HouseModes plugin has to poll for changes. Reactor (for Vera) takes a different approach (to eliminate polling and the response delay it causes--RFV reacts immediately), but nonetheless can't see when house mode is set to what it already is because Vera itself won't take any action when that happens. And therefore MSR cannot see it either (although MSR uses yet another approach and can respond immediately on changes like RFV).

[toggledbits]

By the way, this could also segue into a treatise about why there's no updates operator in MSR, because it's related, so let's do that:

On Vera, every state variable also has a timestamp. The general behavior on Vera is that when a state variable value changes, its timestamp is updated as well. If a state variable is set to the same value it already has (write without change), the timestamp does not change. There are, however, exceptions to this rule: state variables whose name starts with sl_ will get a timestamp update when their value is set, even if it is set to the same value it already has. This specifically allows scene controllers, locks, and keypads to work. These device types all use sl_-named variables for those states/data that can be "active" when set to the same value (e.g. the same lock code is used twice on a keypad or lock, or the same button is pressed back-to-back on a scene controller).

This also manifests in Vera as a call to the watch callback for the variable, if any. The watch callback is typically only called when the value of the state variable changes, unless it's an sl_-named variable, in which case the watch callback is called unconditionally. This is how Reactor for Vera (RFV) detects and implements the updates operator.

Given that information, one might think that you can, from outside Luup (i.e. in MSR using HTTP requests for interface), detect same-value updates by tracking the timestamp, but there are two problems:

1. The timestamp is not exported in the user_data or status requests (or any that I'm aware of), so you can't access it from outside the system. The only place it is visible/available is within Luup itself by calling luup.variable_get();
2. Even if we got the timestamps, the timestamps are not stable, they are volatile: Luup does not store the timestamps in user_data.json, it only keeps them in RAM (which is probably also why they are not exported). Every time Vera reloads, every state variable's timestamp is updated/rewritten to the current time during startup. So it's actually not usable as a persistent value to compare to prior values across reloads, it's only usable within the context of the current Luup reload. That raises the complexity of determining if a timestamp change is the result of an actual change, or a reload with no change in value, or a reload with a change in value during downtime.
3. If we stored timestamps with values, it would double the memory consumption of attributes (because then it has to store two values for every attribute rather than one, and there are a lot of attributes in memory), and do so only for the benefit of one operator applicable in a very small number of cases.

Aside from this, MSR isn't a Vera plugin, or a logic engine just for Vera. Neither Hubitat nor Hass, nor any other system I've yet seen, has timestamps with which an equivalent comparison could be made, or a replacement mechanism by which updates could be reliably implemented for entities originating in any HAC. So in all, updates hasn't (yet) shown a use case/demand that overwhelms the costs in the face of the available workarounds.

[LibraSun]

But I'd argue this also gives a solid rationale for instantiating a "Reactor Helper Service" back on the host controller, expressly for detecting and propagating these otherwise unavailable "Updates". I know you've considered it, and while not a top priority by any stretch, something to debate with MSR's most ardent users.

[toggledbits]

Definitely doable, and some controllers may require it, but as yet, I'm still holding out. Having a no-dependency install on the controller side I think is a big win.

[LibraSun]

@toggledbits while testing the random() function, I created a Rule whose Set Reaction simply waited a period defined by ${{timerD}} where:

timerD := random()*5 // (number) 3.7140944926129995 since 16:59:40

then turned on a light, performed another identically-defined Delay, turned off the light, and then re-ran itself with [Run Set Reaction].

Seemed to run as intended, namely with the light going on and off at visibly divergent intervals (0-5 seconds each). However, I notice that the displayed Expression value in Rule Sets (with the Rule expanded, so I could watch its status) never seemed to change. It sat fixed at 3.7140944926129995 since 16:59:40 throughout, despite running for well over a minute (fun to watch!).

Should I expect the variable in this instance to update, i.e. each time it is referenced? Or is it truly updating internally and just not showing me the new values with each run?

[toggledbits]

If you are running the same reaction (which I probably should not allow, eh?), it does not re-evaluate the rule, so the expression value never changes. You're getting the same delay every time. Any variance you are seeing is caused by device communication.

[LibraSun]

@toggledbits said in Low-priority GUI feedback:

...the expression value never changes. You're getting the same delay every time.

So:
(a) if I indeed WANT different values each cycle, I assume I should be placing random() nakedly inside the ${{ }}.
(b) up to you whether [ Run ] should permit self-iteration or not, although aside from the risk of cycling too quickly, I'm loathe to say "Save users from themselves" (unless you envision abuses of the "Use Pulse Instead, Dammit" variety looming ahead).

On a separate note, might you consider placing an "Open Eye" / "Closed Eye" icon next to "Enabled"/"Disabled" rules so that color-blind users can benefit? I'm sure the current %(#cc0000)[red font] means nothing to them.

[toggledbits]

@librasun Yes, (a) would work.

WRT (b), reactions aren't throttled, and it's not something I want to go to right now because I don't need to maintain state for reactions, they just run. There is also an unexposed mode of reactions where you can run them inline, rather than as separate processes, so the step after Run Reaction would not start until the reaction finishes (in the current/default mode, the reaction is started asynchronously). That would cause infinite recursion and eventually take things down. Even if I block direct self-reference, I would not go so far as to block A runs B, B runs A. Caveat user, but possible.

I don't like that icon anyway. But I'm trying to stay in one icon family, because using multiple is painful (different sizes, spacing, alignments, etc.), and the library I'm using is big, but still I haven't found anything in it I'm happier with. But I agree, I need to stay away from color alone as a cue (and in this case, it's not just color, but the icon change is very subtle). Here are the eyes in that library:

If you want to look through others, go here: https://icons.getbootstrap.com/

[LibraSun]

@toggledbits said in Low-priority GUI feedback:

Here are the eyes in that library:

Though I don't "love" the eyes, I'm accustomed to them being used with Switchboard app (I believe), so I know they do the job.

We don't need "fancy".

[LibraSun]

@toggledbits here on the Scope screen, one marvels at MSR's extreme precision! :-0

[LibraSun]

@toggledbits I just happened to catch a glimpse, in Rule Sets with one of my Rules expanded, of the countdown displaying:
false as of 13:23:58 waiting 0-0:00:03 // then 0-0:00:02, then 0-0:00:01 I believe...
instead of just:
false as of 13:23:58 waiting 0:00:01 02, 03, etc.
The extra '0-` bit disappeared after a few seconds, and only happened at the beginning of the count. (My constraint condition is supposed to be true for 60 seconds, and I was specifically watching it to monitor progress.)

[LibraSun]

@toggledbits Nice touch allowing the Expression definition cells to be persistently resized. Really helps with large "Objects".
Also just noticing the A-Z sort button atop Rule Sets, which I haven't used yet, but that's a nice touch! New to 21096 or just me not noticing before?

[toggledbits]

They're not really persistently resized, it's just taking a guess at the size needed based on the number of newlines, but wrapping of very long lines will probably goof it up. But, better than not doing it.

A-Z has been around for a very long time...

[LibraSun]

@toggledbits said in Low-priority GUI feedback:

A-Z has been around for a very long time...

One thing that I do believe changed with the release of 21096 is that Rule Sets now resets to the topmost ruleset on each visit. Not sure that's a positive development...?

With about 20 Rule Sets, while editing a particular group (say "HVAC"), I may jump back and forth to Entities, Expressions, Scope, etc. then back to "HVAC".

Now, Rule Sets always snaps back to "Assistants" (that is, label nearest "A" -- I keep them alphabetically sorted). Do you have any user feedback to support that behavior? (I can live with it; it's just extra mouse clicks, lol) If not, I vote "go back to 'LastSelectedRuleset'".

[LibraSun]

IMPORTANT @toggledbits -- I wanted you to bask in this PR-free day, so am asking in the Forum instead:
If Rule A has nothing in the Set Reaction, I've noticed that it does not "trip" or "trigger" despite the Conditions being met. Consequently, another Rule B failed to execute with its otherwise automatic [Rule State] [Rule A Set] being true.

Any immediate thoughts on this? And would my immediate "fix" be to include at least a "Comment" in Rule A's Set Reaction?

I know this is an edge case, but I have several rules like Rule A all grouped together, half with nothing in Set (almost as "placeholders" for when I think of actual stuff I want them to do).

[toggledbits]

I have tons of rules with no reactions. The setting and saving of the rule state is also quite separate in the code from the point where the reaction is selected and started (or not).

Anything logged?

[LibraSun]

Nope, nothing logged, leaving me to believe it's the underlying trigger mechanism - a Yale Assure deadbolt - is letting me down. Thanks for confirming, though, so I know which rabbit hole to go down tomorrow.

[toggledbits]

If your Rule logging level is 5, there should be sufficient info in the logs for you to see "entity-changed" entries like this:

2021-04-09T01:55:23.325Z <Rule:5:Rule.js:466> Rule#rule-grpwswz9u9 dependency notification entity-changed Entity#house>device_6 from Entity#house>device_6
2021-04-09T01:55:23.325Z <Rule:5:Rule.js:479> Rule#rule-grpwswz9u9 requesting eval; entity-changed Entity#house>device_6

That will be followed by (possibly with stuff between):

2021-04-09T01:55:23.325Z <Rule:5:Rule.js:814> Rule#rule-grpwswz9u9 (Auto Fan On) evaluate() acquiring mutex
2021-04-09T01:55:23.326Z <Rule:5:Rule.js:820> Rule#rule-grpwswz9u9._evaluate() mutex acquired, evaluating

This indicates that the rule is setting up for evaluation (making sure it's not already doing an evaluation). Then:

2021-04-09T01:55:23.326Z <Rule:5:Rule.js:668> Rule#rule-grpwswz9u9 evaluateExpressions() with 0 expressions
2021-04-09T01:55:23.326Z <Rule:5:Rule.js:863> Rule#rule-grpwswz9u9._evaluate() trigger state now false (was false)
2021-04-09T01:55:23.326Z <Rule:5:Rule.js:871> Rule#rule-grpwswz9u9._evaluate() constraints state true
2021-04-09T01:55:23.326Z <Rule:null> Rule#rule-grpwswz9u9 rule state now false, changed no

The first of these lines indicates the number of expressions being evaluated, which is the first phase of rule evaluation. The next line follows the evaluation of the trigger conditions and gives the new trigger state and the prior state. The line that follows is the evaluation result of the constraints. The fourth line states the overall rule state, and if that's different from the prior rule state. It is after this last line that the reaction to run is chosen (it is only run if the rule state changed as shown in the last log entry).