Errors after updating to MQTTController build 25139

[toggledbits]

Build 21063

• 0000081: Implement RunLua action (on system entity for each VeraController instance)
• 0000080: Fix a problem running Vera scenes
• 0000075: Limit time delays to 24 hours (a multi-day delay on a schedule, specifically >24 days, could overflow JavaScript's 32-bit signed integer millisecond timers)

Other work:

• Fixed an issue with "changes" operator restoring configured values too literally (was inserting word "null" rather than blank/empty string)
• Progress on getting "set variable" action working. It's pretty much working for global expressions/variables. Still working on rule-based variables. PR 5 remains open.
• A lot of "linting". The old-school software-types will know what I mean.

[cw-kid]

Can confirm #80 and #81 and "issue with "changes" operator" are all working OK.

I've very nearly moved all my PLEG conditions / actions over to MSR.

I am waiting to be able to use an Expression value in a rule, for my "Master" dimmer virtual devices.

And for the Scene Activation stuff being exposed, for Fibaro Dimmer 2 modules for double / triple click events.

Also need to figure out how to setup a motion activated light rule.

[toggledbits]

I'm getting close on expressions. 21063 has working Set Variable for global expressions. Rule-based expressions is going to require some fairly large changes to the way evaluation of rules is done (not in the evaluation itself, but how they are scheduled). I could have pulled that off yesterday but it was getting late and the changes are not without risk, so I decided to defer. Once the configuration works, and Set Variable works, then I'll need to integrate the substitution capability into reaction arguments, etc. So still a couple of days, I think. That's PR#5 if you want to track it.

Scene activation is going to be tricky because Vera doesn't expose enough information over user_data or status requests. Doable, for sure, but it will need to be done slightly differently from the way you do it on Vera. It's the same thing openLuup runs into with scene controllers and sl_ prefixed state variables. I don't see a PR for that capability, though. Go ahead and open one.

[toggledbits]

Build 21064

• 0000082 - Support for ZWave scene controllers (system capability and Vera)

Other work:

Much time spent today getting Set Variable action update and running. Docs still need updates. The new substition syntax is ${{variable-name-or-expression}}

[toggledbits]

Just realized that one of the changes for variable substitution was causing an issue on some actions. I've respun the release and put it up. If you downloaded before 22:35 US Eastern, please re-download and install.

[toggledbits]

Build 21065

No PRs fixed today.

Work continued on expressions, and although I think I could mark PR #5 resolved, I'm going to leave it pending any reports related to them. I suspect there will be some discussion needed, at least, because they operate somewhat differently from Reactor for Vera/openLuup.

[toggledbits]

Build 21067

No PRs fixed today.

Ongoing work on expressions in the UI, and a number of other cleanups in the UI, for Rule editing in particular. Expression substitution using ${{...}} should be work in Reactions. Still working on conditions, so don't expect any success there yet.

[toggledbits]

Build 21068

• 0000089 - Entity action substitution values don't update after first run
• 0000090 - Expression not saving when modified
• 0000091 - Expression UI issues
• 0000092 - Expression UI issues

Expressions in conditions are still not available. I've chased UI issues most of the day today.

[toggledbits]

Build 21069

• 0000095: Fix annoying change of attribute when changing device selection
• 0000096: value_sensor.value now allows any data type
• 0000098: global expressions recreated with same name remember old values of prior expression

Lots of organizational and administrative work today as well.

[toggledbits]

Build 21072

• 0000108: Editing an extended attribute after having a system attribute selected applies validation rules of system attribute
• 0000107: Improve button consistency between rules and reactions
• 0000104: Error when attempting to recreate variable used in rule
• 0000095: Also fix in Reactions (reselect of action if device changes)
• 0000100: Make sl_CentralSceneAttr available for Vera

The x_vera_state capability is gone, and has been replaced with x_vera_svc_XXXX capabilities, wherein XXXX is the service ID on the Vera device. This slows startup, as additional queries must be made of every Vera device at initialization; the user_data query does not provide sufficient information. The Hubitat and Hass controllers also model this behavior. It's very tidy on Hubitat and Hass, I'm happy to say (and there is no performance penalty on these platforms).

The bad news (Vera users): If you are using x_vera_state attributes in your rules, you will need to edit them and choose the newer capability. Good news: this model will probably never change.

[toggledbits]

Build 21073:

• 0000110: Problem with formatting and reselection of Interval conditions in "relative to condition TRUE". Good time to mention that this mode is now DEPRECATED. This functionality was long ago replaced in Reactor for Vera by pulse output mode, but I didn't deprecate the feature in RFV at the time; should have. It is deprecated now in MSR.

Ongoing UI work, including exporting of group to new Rule, and importing of Rule to group.

[toggledbits]

Build 21074

• 0000110 - Interval condition not restoring related condition (option) correctly on edit
• 0000111 - Hass identified service not available (androidtv)
• 0000112 - Expression error
• 0000113 - Bridged Vera systems shows no devices (corrupt geofencing data)
• No PR - Fix precedence issue with lexpjs "each" and "first" statements

This version is also ready for testing under systemd on RPi (I've been running it that way all day). I will publish more on that later; I'm being called to the dinner table.

[toggledbits]

Build 21075:

• 0000114: Fix loss of condition options on device change
• 0000115: Add restart button to Tools page
• No PR: Update lexpjs to fix && and || return values.

Continuing work on UI cleanups are in the fore these days. Also work getting the HTTP API squared away sufficiently that I can port my automated test tool and start writing new test scripts.

[toggledbits]

Build 21076:

• 0000122 - Pushover notifications fail (dup/same cause as 119)
• 0000121 - "in" operator malfunction (Conditions)
• 0000120 - getEntity() fails to find entity when using name
• 0000119 - SMTP Notifications fail
• 0000106 - Include ID in entities list of expression builder
• 0000063 - Reorganize entity list in triggers and constraints

PRs 63 and 106 are addressed by a long-planned change finally implemented which I call the "Entity Picker". It operates similar to the Entities tab at the top leve of the UI, providing filters for source controller, group and capability. It also provides a name search that filters as you type. And, it remembers the last set of filters, so when you bring it up, your last search is intact. This has been deployed in all locations where entity selection was previously done with a drop-down menu.

[toggledbits]

Build 21077:

• 0000123: Rename of rule sometimes doesn't stick
• 0000124: Request action URL check doesn't allow variable substitution

Notification now supports (hopefully) CallMeBot. The dist-config contains an updated notifications.yaml with a section for CMB. It looks like this:

notification:
  # other method configurations not shown

  # CallMeBot
  CallMeBot:
    profiles:
      default:
        # api_url - Access URL for CallMeBot API being used
        api_url: http://api.callmebot.com/start.php
        #
        # api_key - (if needed) API key (use for Facebook API, WhatsApp API, etc.).
        #           If the API you are using does not require it, leave it
        #           commented out or blank.
        #api_key: "place api key here"
        #
        # source - (if needed) Source field for some APIs. If yours does not require
        #          it, leave it commented out or blank.
        #source: "source-value"
        #
        # phone - (if needed) Phone field for some APIs. If yours does not require
        #         it, leave it commented out or blank.
        #phone: "phone-number"
        #
        # user - (if needed) User field for some APIs. If yours does not require
        #        it, leave it commented out or blank.
        #user: "username"
        #
        # lang - (optional) Language/voice; default "en-US-Standard-C".
        #        See CallMeBot docs for possible values.
        #lang: "en-US-Standard-C"
        #
        # rpt - (optional) Number of times to repeat message; default: 1.
        #rpt: 1

If you run into an issue, logs will help here.

[toggledbits]

Build 21078:

• 0000122: Prowl notifications not working
• 0000125: Add name search to Entities list (like Entity Picker)
• 0000129: Exit after creating rule leaves empty rule
• 0000130: Improve "last value" display for expressions

[toggledbits]

Build 21080:

• 0000135: Fix animation when rule SETs
• 0000127: Overly verbose error messages on expression syntax errors

Additional work:

• Rule status detail now shows expression values.
• Sun info now available in a "Sun Information" entity managed by the system controller.
• "Solar noon" option added to "Sunrise/Sunset" conditions.

[toggledbits]

Build 21081:

• 0000139: Unconfigured notification methods causing incomplete UI display/missing actions

Additional:

• In the system location, one can now set name to the name of the desired location for default weather (sometimes OpenWeatherMap comes up with some odd place names). The city name will be used by default.
• Work on expression substitutions in conditions is feature-complete but not thoroughly tested yet. Note that substitutions are only permitted in Entity Attribute and Expression Value condition operands, and in about half of the condition options. The latter is very much by design, as the proper operation of the condition options requires a deterministic (stable, unchanging) value between evaluations for proper operation of the option in many cases (e.g. pulse timing and counts cannot be set by expression values).
• The evaluation context for rules is now more persistent, which allows a global expression that is referred to twice or more in a rule or other global expressions to be only evaluated once (previously, the global variable was evaluated on every reference, which could have side-effects for expressions used as counters, array-builders, etc.).

I've also closed/resolved a number of "wish list" PRs in Mantis. Rather than leaving these open until they are eventually dealt with, they are memorialized by the "suspended" resolution state. This makes it easy to search out the wish list (just search on resolution == suspended). These PRs can be re-opened when the time comes for work. Also, I'd like to see things bantered about in the community forums before they turn into wish-list PRs, if possible.

[toggledbits]

Build 21082:

• 0000140: Round-trip move of expression between rule and global and back doesn't manage state correctly

Work proceeding apace on the new alerts/trouble notification system.

[toggledbits]

Build 21084:

• 0000147 Issues (multiple) handling headers for request actions
• 0000146 Additional native capabilities for Hubitat (plus fixes to hs_color)

IMPORTANT: Please re-edit your Request action headers (if any) to get them into the new form, and correct at least one of the bugs related to the UI's contribution to the PR.

Additional:

• Timing animations are now removed from disabled rules.
• Expression value changes in the Rule status detail card are now properly animated.
• A data corruption when dragging conditions between groups has been found and fixed.
• Improved error handling in the Request action, including the storage function (could throw an unhandled promise rejection when the target variable was in a disabled rule).
• The display of "Last Updated" on the Entities list was stalling; this was reported by somebody some time ago but I couldn't find the PR for it, forgot about it, and then noticed it again earlier today.
• Improve error handling around the loading of Notifier subclasses, and modify the API slightly to move some additional repetitious implementation up to the base class.

A couple more notes on Request action headers: the RFC (2616, Section 2.4 specifically) says that header (field) names are not case-sensitive. In my experience, this is true for 99% of servers you run into. And Murphy is on board (i.e. the server you want to use is guaranteed to be in the 1% remaining). MSR makes no assumption about the case of the header/field name. However, the underlying node-fetch implementation does, effectively converting all headers to lower case. This may cause problems on some servers, but it's the server that is not RFC-compliant, not node-fetch or MSR, so you are likely not to see a fix for this if you run into a server where it's a problem (I'm guessing it will be the hackish HTTP stack on some cheap WiFi-based microcontroller that rears its ugly head here).

The other thing is that the RFC allows a header to be specified multiple times, the net effect being a comma-separated list of header/field values in order of appearance. That is, specifying an "Accepts" header twice, once with "application/json" and another with "text/xml" would be handled (in compliance with the RFC) as "Accepts: application/json, text/xml". Unfortunately, node-fetch uses a JavaScript Map object for its underlying implementation of headers, and that only permits one value per key -- you can't have multiple "Accepts" headers, each subsequent will simply replace any prior. Probably a rare circumstance, but nonetheless, I have the MSR Request action storing the headers you provide in its own format, not the implementation form, and MSR can handle repeated header/field names; if you use this feature, MSR will simply do the work that the server would normally do to concatenate the same-named fields on its end, before handing the headers to node-fetch.

So, while I can't work around node-fetch's corruption/overrule of the header case, I can provide a workaround for the unfortunate choice of data structure in node-fetch's header implementation. Onward.