Organizing/ structuring rule sets and rules

[tunnus]

I'm trying to migrate a couple of SiteSensors to MSR, and implementing those as rules with HTTP request reactions. MSR build in use is 21243.

Some challenges though, as I'm capturing an array and because I need those values in the array in multiple rules, I have to use global expressions. Seems that these global expressions/variables are not updating after initialization, but if I try local expressions, they update normally.

(for testing purposes I'm switching between "g_UplinkResponse" and "temp_local_array" in capture response)

Global expressions:

So "g_temp_var" and "temp_local_var" should yield the same result, but in this case the values are different (85.9 vs. 36.8, the latter being the most current & correct value)

I also have a test rule using "g_temp_var" just to induce change, but the value stays the same.

@toggledbits, have I understood something wrong or could this be a bug?

[toggledbits]

From the docs:

"Global variables are visible to all rules in the system, and can be referred to by any rule-based variable or any other global variable. Global variables are evaluated when a rule-based variable calls for the value. It is therefore normal and expected that, when looking at the Global Expressions list, no values are changing, unless rule variables that reference those global variables are also being evaluated. For example, a global variable consisting of simply time() will not continuously update and display the current time in the Global Expression list. It will only be updated if a rule asks for it.

Ref: https://reactor.toggledbits.com/docs/Expressions-%26-Variables/

[tunnus]

@toggledbits yep, I had read that part of the manual, that’s why I have that rule with ”changes”, still those values are not updating?

[toggledbits]

If g_temp_var is not changing, changes is never going to fire. But g_UplinkResponse is changing.

Edit: to try to clarify more, it's a chicken and egg problem. g_temp_var will not be updated until it is referenced by a running rule, but the rule only runs when g_temp_var is changed. So without something external causing g_temp_var to change, the rule will never run and call for g_temp_var to be updated. If you check g_UplinkResponse for changes, but use g_temp_var everywhere else in the rule, you should get the behavior you are looking for, because g_UplinkResponse is changed externally by another rule on an interval, so that will stimulate the run of the rule to evaluate everything else needed.

[gwp1]

This is a very timely thread as I'm starting to explore how to migrate two Site Sensors (one pulls weather data from Ambient API, the other pulls weather data from OpenWxMap API) to feed my HVAC needs.

Disclaimer: I have near zero KB regarding expressions let along how they work in MSR. I'll be watching this thread closely.

[tunnus]

@gwp1 you are probably better off with this post than what's here, but feel free to follow also this discussion

[tunnus]

@toggledbits okay, now I have this test rule set up:

Situation remains the same, global expression(s) is not changing

[toggledbits]

I can't tell from the timestamps, but given that they're similar, did the value actually change? No hit if the value doesn't actually change.

By the way, how are you determining if there was a change or not? I mean, how do you know it's not changing. Does something happen when the conditions are met (is there a Reaction that does something?)?

[tunnus]

@toggledbits I know it's supposed to be something else as I can monitor the same values via a couple of different means (an app for instance).

But as I wrote before, if I use local expressions, values are as they should be:

[toggledbits]

@tunnus Well, I'm not getting a clear picture here from the little bits and pieces you're posting, so I set up a little test environment of my own where I have an HTTP request run, set a global variable, and have a rule with a condition that goes true when the global changes with a reaction that sends an alert notification that the variable has changed. I get the notification every time I run the HTTP request.

Here's the rule:

And the variables:

Here's the reaction I can run manually to make a request and store the result in the global variable. The API function I'm calling returns a timestamp (with milliseconds), so every request is pretty much guaranteed to give me a different response from the last.

This seems to model what you're telling me you want to do, and it works fine.

[tunnus]

@toggledbits sorry, I tried to depict the problem in the first post as well as I could.

But yes, your test case models my case and now I'm baffled. I'll try to setup my test from a scratch and see what happens...

EDIT: replicated this case, no change. Local variables work nicely, but global expressions won’t update. Patrick, any logs I could provide to solve this?

Only difference I could think of (between my and your test setup) is that if response structures (array etc) differ just enough and code handles local & global expressions a bit differently.

[toggledbits]

The key here is that the response to the HTTP request has to change. If it doesn't, then the changes isn't going to work, and nothing will trigger the rule to evaluate and cause the other global variables to be updated. Remember that changes requires an actual change of the value, not just a fetch of the value and setting it (again) to what's already there... no change, no changes.

[tunnus]

@toggledbits what's strange here is that I only change this:

to this:

or vice versa, URL & other settings remain the same, local expression works, changes are immediately detected. And yes, response is really changing.

Here's the most current test rule:

(at the time of the screenshot, "-111.0" was the correct value)

The only time I get an updated/correct value (for the global expression) is when a global variable/expression is initialized (when it is empty), after that it remains the same.

Maybe this problem is unique to my setup.

[tunnus]

@LibraSun, @cw-kid & others, could you test if you can replicate this error I'm experiencing?

EDIT: I have another rule querying the same API, for a bit different values and that works fine even with global expressions, so reproducing the same error might prove to be difficult...

[wmarcolin]

@toggledbits, in your test proposal, when the HTTP Request fails, an error is generated and thus an Alert is displayed on the dashboard. Since this is an under-control situation, is it possible to inhibit this alert?

[toggledbits]

Depends on how you're doing the implementation. What's the error that's generated?

[wmarcolin]

@toggledbits I am referring to this error alert, generated when I interrupt the internet access, automatically the HTTP GET will fail, which is expected and I will give the necessary treatment. But I would like to not have this alert in the status dashboard, because as I mention it would be an expected problem, I don't need the alert.

Now of course this is no big problem, just exploring more possibilities.

[tunnus]

@toggledbits if you'd be willing to troubleshoot this issue further, I could provide you the whole response to this http request? But where is that response stored on the filesystem?

What seems to happen is that if my global expression/variable is empty, fresh data is initially stored to that, but subsequent queries are "ignored", data is not replaced.

In the log I can see new/fresh values, but for some reason this new data is not used:

2021-10-01T14:04:00.321Z <Rule:5:Rule.js:997> Rule#rule-ktc09w2v._evaluate() trigger state now false (was false)
2021-10-01T14:04:00.323Z <Rule:5:Rule.js:999> Rule#rule-ktc09w2v._evaluate() constraints state true
2021-10-01T14:04:00.325Z <Rule:5:Rule.js:1008> Rule#rule-ktc09w2v rule state now false, changed no
2021-10-01T14:04:00.548Z <Engine:INFO> Engine#1 reaction Nibe uplink SiteSensor<SET> step 0 HTTP request to https://192.168.XXXX:8443/rest/items?tags=nibe2vera&recursive=false succeeded (200 OK)
2021-10-01T14:04:00.552Z <Engine:5:Engine.js:1547> Engine#1 response type application/json
2021-10-01T14:04:00.556Z <Engine:5:Engine.js:1169> Engine#1 global set variable g_UplinkTest = (object) [ { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Control_HotWaterBoost", "state": "0", "stateDescription": { "pattern": "%.0f", "readOnly": false, "options": [  ] }, "commandDescription": { "commandOptions": [ { "command": "0", "label": "Off" }, { "command": "1", "label": "3h" }, { "command": "2", "label": "6h" }, { "command": "3", "label": "12h" }, { "command": "4", "label": "One time" } ] }, "editable": true, "type": "Number", "name": "NibeRESTAPI_Control_HotWaterBoost", "label": "Hot water boost", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Status_DegreeMinutes", "state": "-54.3", "stateDescription": { "pattern": "%.0f", "readOnly": true, "options": [  ] }, "editable": true, "type": "Number", "name": "NibeRESTAPI_Status_DegreeMinutes", "label": "Degree Minutes", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Ventilation_FanSpeed", "state": "72.0", "stateDescription": { "pattern": "%.0f", "readOnly": true, "options": [  ] }, "editable": true, "type": "Number", "name": "NibeRESTAPI_Ventilation_FanSpeed", "label": "Fan Speed", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Addition_Blocked", "state": "ON", "stateDescription": { "readOnly": true, "options": [  ] }, "editable": true, "type": "Switch", "name": "NibeRESTAPI_Addition_Blocked", "label": "Blocked", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_CprInfoEp14_CurrentComprFrequency", "state": "20.0 Hz", "stateDescription": { "pattern": "%.0f", "readOnly": true, "options": [  ] }, "editable": true, "type": "Number:Frequency", "name": "NibeRESTAPI_CprInfoEp14_CurrentComprFrequency", "label": "Compr. Frequency", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Status_OutdoorTemp", "state": "10.9 °C", "stateDescription": { "pattern": "%.0f", "readOnly": true, "options": [  ] }, "editable": true, "type": "Number:Temperature", "name": "NibeRESTAPI_Status_OutdoorTemp", "label": "Outdoor Temp.", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Status_AlarmInfo", "state": "NULL", "stateDescription": { "pattern": "%s", "readOnly": true, "options": [  ] }, "editable": true, "type": "String", "name": "NibeRESTAPI_Status_AlarmInfo", "label": "Alarm Info", "tags": [ "nibe2vera" ], "groupNames": [  ] }, { "link": "https://192.168.XXXX:8443/rest/items/NibeRESTAPI_Status_HotWaterTop", "state": "46.3 °C", "stateDescription": { "pattern": "%.0f", "readOnly": true, "options": [  ] }, "editable": true, "type": "Number:Temperature", "name": "NibeRESTAPI_Status_HotWaterTop", "label": "Hot Water Top", "tags": [ "nibe2vera" ], "groupNames": [  ] } ]
2021-10-01T14:04:02.173Z <Engine:5:Engine.js:1367> _process_reaction_queue() wake-up!
2021-10-01T14:04:02.200Z <Engine:5:Engine.js:1328> _process_reaction_queue() running task 1179 { "tid": 1179, "id": "rule-kt9u8aur:S", "rule": "rule-kt9u8aur", "__reaction": [RuleReaction#rule-kt9u8aur:S], "next_step": 2, "status": 0, "ts": 1633097040135, "parent": --null--, "__resolve": --function--, "__reject": --function--, "__promise": [object Promise], "attempts": 1 }
2021-10-01T14:04:02.201Z <Engine:NOTICE> Resuming reaction Nibe uplink SiteSensor<SET> (rule-kt9u8aur:S) from step 2
2021-10-01T14:04:02.204Z <Engine:5:Engine.js:1537> Engine#1 request header accept: application/json
2021-10-01T14:04:02.205Z <Engine:5:Engine.js:1539> Engine#1 reaction rule-kt9u8aur:S step 2 request url https://192.168.XXXX:8443/rest/items?tags=nibe2vera2&recursive=false data { "method": "GET", "timeout": 15000, "size": 8192, "agent": [object Object], "headers": [object Headers] }
2021-10-01T14:04:02.208Z <Engine:INFO> Nibe uplink SiteSensor<SET> all actions completed.
2021-10-01T14:04:02.209Z <Engine:5:Engine.js:1332> _process_reaction_queue() task returned, new status -1; task 1179
2021-10-01T14:04:02.212Z <Engine:5:Engine.js:1367> _process_reaction_queue ending with 0 in queue; none delayed/ready; waiting

(and then it continues with another http request in the same reaction, 2 s delay in between)

[toggledbits]

@tunnus said in SiteSensor migration:

What seems to happen is that if my global expression/variable is empty, fresh data is initially stored to that, but subsequent queries are "ignored", data is not replaced.
In the log I can see new/fresh values, but for some reason this new data is not used

You're going to have to describe this in more detail.

[tunnus]

@toggledbits I'll PM you relevant logs