[Solved]Global Expression with null value and Auto-Evaluation

[vezinpi]

Hi, I am not a software developper so I apologise in advance...
MSR 20002 Docker on Raspberry PI 4

I am using MSR and global expressions to control my house humidifier. To calculate the target humidity level, the outside temperature is used within a simple formula.

From time to time and for short period, the outside temperature, coming from a Hass sensor is showing a value "null" which is screwing up my target level. Ideally, I would like to retain the current value when the OutsideTemp is not available (i.e. null)

Here are my global expressions to perform this:

OutsideTemp: Simply get the outside temperature (that is the one showing "null" from time to time.
CurrentHumid: Current house humidity
CalculatedHumid: This is where the target humidity is calculated. (Used to be 45 + round(OutsideTemp)
LowHumidity: This is the flag that tell the humidifier to start working.

CalculatedHumid and LowHumid are use in a rule to update sensor in HASS.
In addiditon LowHumidity is used in 2 rules to turn the Humidifier on and off.

At first, I was crashing MSR because CalculatedHumid was auto evaluating constantly. After a couple of attempt I disabled "Auto-Evaluation". From what I can tell, all is working fine which is a bit surprising considering what the documentation says about disabling auto-evaluation.

2 questions:

1- Since auto-evaluation is disable, how come the CalculatedHumid is still evaluating when the temperature changes?

2- Is there a better way to filter out a null value while retaining the previous valid value.

Thanks

[toggledbits]

If you just want it to keep the last value when the device is handing back null, you can use the order of operations to your advantage: the assignment to the variable is the last thing that happens in the evaluation, so if you change OutsideTemp like this:

    getEntity( 'hass>sensor_etc' )?.attributes?.temperature_sensor?.value ?? OutsideTemp

This will do the entity fetch and member accesses (with extra guards), and if the result of that is null, it just returns the current value of OutsideTemp. So as long as the entity fetch clause returns null, the previous temperature value will be saved as the current temperature value.

Edit: Added an additional guard to the above expression (immediately after the getEntity()'s closing paren).

[vezinpi]

This is a good day. I have learned something again.

Many thanks for your support.

[vezinpi]

@toggledbits I might have celebrated a bit too soon on that one. After doing the change all seemed fine. Tested the null value and a temperature change and all was working as advertised. After a certain time I am getting an error and the container crashes and cannot be re-started. By looking at the container log, I can see some issues with the file sys_alerts.json. Indeed the file is still in the storage folder but has a size of 0 byte and is complety empty. I replaced the empty file by the original one (I keept a copy of all files of the reactor folder after the original install) and was able to restart the container but it eventually crashed again. I changed the OutsideTemp expression back to what is was and I am back to normal. I am in the process of collecting log info and whatever might be relevant to investigate. It does seemed similar to my original problem where the expression is self trigering and get caught in a loop. I will be in and out this week so it may take some time before I get back to you. In the interim here is a screen capture of the error:

[therealdb]

it's a classic situation of stack overflow. you'd probably just set a value (like "unavailable") on startup and a constraint on the rule. That's what I'm doing anyway for a couple of things that send spurious values from time to time.

[wmarcolin]

@vezinpi

In the engine group config (reacotr.yaml), add this parameter:

update_rate_limit: 300

This solved my problem and did not harm the system at all.

Thanks.

[toggledbits]

Can you post all the expressions as you have them now? That's a pretty unusual error message. There's a distinction between evaluation and compilation, and I'm very meticulous about making sure the message reports the right one. The alert shows that the compilation is the problem, which suggests the expression isn't written properly somehow. I need to see what you've got.

[toggledbits]

@wmarcolin update_rate_limit has no effect on global variable evaluation, FYI. That's a control/config for rule evaluations. Global variables have no equivalent config value, it's really not needed. I think there's (a) a problem with the expression that's causing a compilation error, and (b) a problem handling the compilation error that's causing the loop/stack overflow. Once (b) is fixed we'll be back to "no rate limiting needed here" status.

[wmarcolin]

@toggledbits ok, understood!
Thanks.

[vezinpi]

Here it is, as requested. Getting the logs, standby...

[vezinpi]

MSR Logs. It is quite long with the middle repeating OutsideTemp so I copied the start and the end. After that it repeats. I did preserved the logs so if you want more let me know.

This is how (I think) it started

GlobalExpression#CurrentHumid
[latest-22002]2022-01-04T17:33:01.558Z Rule:5:813:Rule.js Rule#rule-grpqosf66u Humidifier On requesting eval; expression-state-changed from GlobalExpression#CurrentHumid
[latest-22002]2022-01-04T17:33:01.559Z Rule:5:813:Rule.js Rule#rule-grpqoyiumi Humidifier Off requesting eval; expression-state-changed from GlobalExpression#CurrentHumid
[latest-22002]2022-01-04T17:33:01.560Z Rule:5:813:Rule.js Rule#rule-kwz0myt1 Low Humidity To Hass requesting eval; data-changed from Data#states:expr-global-CurrentHumid
[latest-22002]2022-01-04T17:33:01.561Z Rule:5:1009:Rule.js Rule#rule-kwz0myt1 evaluation in progress; waiting for completion
[latest-22002]2022-01-04T17:33:01.562Z Rule:5:813:Rule.js Rule#rule-grpqosf66u Humidifier On requesting eval; data-changed from Data#states:expr-global-CurrentHumid
[latest-22002]2022-01-04T17:33:01.563Z Rule:5:1009:Rule.js Rule#rule-grpqosf66u evaluation in progress; waiting for completion
[latest-22002]2022-01-04T17:33:01.564Z Rule:5:813:Rule.js Rule#rule-grpqoyiumi Humidifier Off requesting eval; data-changed from Data#states:expr-global-CurrentHumid
[latest-22002]2022-01-04T17:33:01.565Z Rule:5:1009:Rule.js Rule#rule-grpqoyiumi evaluation in progress; waiting for completion
[latest-22002]2022-01-04T17:33:01.566Z Engine:5:984:Engine.js [Engine]Engine#1 var CurrentHumid changed, evaluating dependent child LowHumidity
[latest-22002]2022-01-04T17:33:02.407Z Engine:ERR [Engine]Engine#1: error evaluating expression OutsideTemp: [RangeError]RangeError: Maximum call stack size exceeded
[latest-22002]2022-01-04T17:33:02.411Z Engine:ERR [Engine]Engine#1: expression: getEntity( "hass>sensor_ottawa_kanata_orleans_temperature" ).attributes?.temperature_sensor?.value ?? OutsideTemp
[latest-22002]2022-01-04T17:33:02.411Z Engine:ERR [Engine]Engine#1: path LowHumidity->CalculatedHumid->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp

And this is how it ended

OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp->OutsideTemp
[latest-22002]2022-01-04T17:33:02.415Z Engine:CRIT RangeError: Maximum call stack size exceeded
RangeError: Maximum call stack size exceeded
at /opt/reactor/server/lib/Logger.js:434:21
at Array.forEach (<anonymous>)
at Logger._emit_streams (/opt/reactor/server/lib/Logger.js:428:57)
at Logger._emit_streams (/opt/reactor/server/lib/Logger.js:436:71)
at Logger._emit_streams (/opt/reactor/server/lib/Logger.js:436:71)
at Logger._emit (/opt/reactor/server/lib/Logger.js:423:35)
at Logger.err (/opt/reactor/server/lib/Logger.js:452:31)
at Engine._eval_expr (/opt/reactor/server/lib/Engine.js:954:57)
at Engine._expr_resolve (/opt/reactor/server/lib/Engine.js:1217:38)
at _resolve (/opt/reactor/common/lexp.js:1217:35)
[latest-22002]2022-01-04T17:33:02.416Z Engine:ERR [Engine]Engine#1: error evaluating expression OutsideTemp: [RangeError]RangeError: Maximum call stack size exceeded
[latest-22002]2022-01-04T17:33:02.416Z Engine:ERR [Engine]Engine#1: expression: getEntity( "hass>sensor_ottawa_kanata_orleans_temperature" ).attributes?.temperature_sensor?.value ?? OutsideTemp

[toggledbits]

OK. I'm busy this afternoon, but will look at it later today.

[toggledbits]

OK. I think I have this nailed down. Sorry it took me so long, I was working on an edge case to make sure the system stays in control under the worst of conditions. Is essence, as I suspected, the first issue is that the dependency loop check had an error and wasn't always working. In addition, I've added additional checks for more cases where a loop may evade the first check. Also, the system should remain in control if a loop does sneak by the checks, rather than hanging.

Build will be published shortly...

[toggledbits]

@vezinpi I've added one additional guard to the expression in the post linked below. You should make that edit as well during your upgrade.

https://smarthome.community/post/11053

[vezinpi]

@toggledbits said in Global Expression with null value and Auto-Evaluation:

OK. I think I have this nailed down. Sorry it took me so long

You have nothing to be sorry about. Being aware of a potential issue, evaluating the problem, developping a solution and posting a new version in the same day, is really remarquable and almost unheard of now a day.

Thanks you for your dedication.

[vezinpi]

Installed the new version yesterday along with the new guard and all is still fine this morning. Considering that it was failing very quickly before, I consider that issue fixed.