Need help reducing false positive notifications

[wmarcolin]

Hi

Is it possible to access the information that the MSR has about the last date and time that a device suffered a change? The information that appears in the Entities area?

Use Case: I want to monitor all devices, and those that are not communicating/acting for more than 48 hours, take notice and action

Of course there are some that will not communicate by their nature, such as a water valve, so a list of exceptions would be created.

The principle is almost the same as Hubitat's WatchDog APP, but I want to have my own, and not be dependent on Hubitat's communication or action.

Thanks.

[wmarcolin]

@toggledbits hi!

Is it possible to get the information?

Thanks.

[toggledbits]

Everyone is used to looking at the attributes of entities, but the structure of an entity itself has properties. Actually, attributes is a property of entities (it's a map containing key/value pairs). The entity's name, id, and canonical_id are also properties of the entity. You could refer to these in an expression the same way you refer to attributes (as a property), and many of you are already doing this to get the name of an entity for some purpose (alert messages, commonly): getEntity( 'someid' ).name.

There is a lastupdate property on an entity, but it may not work like you think/want it to. The lastupdate property is modified any time an entity is modified, and this would include modifications to other properties, like its name. It also tends to be modified when controllers start up or reconnect, because they re-inventory the hub and reconcile all of the entities and their capabilities at that time. In any case, the "Last Modified" time shown on the Entities list is that time, but again, it is not necessarily the time that an attribute was modified, and more particularly, it is not the time that an attribute you care about was last modified (there are lots of attributes on most entities that people never look at). It is not usable for the purpose you describe, and there is no alternative other than looking for changes in attribute values.

[wmarcolin]

@toggledbits thanks for your comment.

Based on it, I will try to progress with the idea of actually knowing that the timestamp is from an action of the device and not from updates or restarts as you comment.

An additional help, you correctly comment that there are a number of attributes as you mention canonical_id. How is it possible to know these attributes? Where would they be exposed?

Thanks.

[toggledbits]

OK, let me make sure you understand:

1. lastupdate is not a reliable indicator of device updates; there are many other things that will cause it to change, thereby invalidating its usefulness for the purpose you propose.
2. canonical_id is not an attribute, it is a property. The attributes structure is itself a property, but the values contained in it are not properties, they are attributes. Properly terminology is important here, because if you start talking about canonical_id (incorrectly) as an attribute, you and/or others will start thinking you can get to it via entity.attributes.canonical_id, which is incorrect. Properties are directly accessible on the entity (entity.canonical_id is correct). @therealdb has already opened a PR for more description of entities and capabilities, and I will add to that some discussion of attributes and properties at a later date.

[wmarcolin]

@toggledbits ok, thanks!

Well after a few hours of thinking about your comments, lastupdate really doesn't serve the purpose of monitoring. There should be something that informs when the entity.attributes.door_sensor.state (motion_sensor...) changes state, and then it will be monitored.

Now I understand why APP monitors that have Hubitat used battery_power.since as a reference, because it is the closest to a communication of the device effectively passing its state. But in this situation it excludes everything that does not work with battery power.

Well back to another old discussion, that in Vera we had the zwave_device.failed that helped in this problem identification.

Well anyway it was good to understand a little of the theme attribute and property, and I'm waiting for the information you commented that @therealdb requested.

Thanks!

[gwp1]

I could be missing the boat here completely but it sounds like Vera was passing different/additional attributes than Hubitat is/does, yes? If Hubs won't pass it then MSR wouldn't know of it and, therefore, be unable to action on it.

Or am I missing the aforementioned watercraft?

[toggledbits]

@gwp1 said in Last Modifed Information:

I could be missing the boat here completely but it sounds like Vera was passing different/additional attributes than Hubitat is/does, yes?

That's right. Vera has its urn:micasaverde-com:serviceId:HaDevice1/CommFailure state variable, which at least applies to Z-Wave parent devices. There does not seem to be an analogue for this in Hubitat or Hass, but, of the two, I think Hubitat is most likely to fix it itself, and it may also be possible for a third-party app to address it (I have not ruled out abandoning Maker API as the interface for MSR and writing my own, but I have other things I want to spend my time on in this project right now).

[wmarcolin]

@toggledbits

Sorry for the insistence with the subject

I exchanged a few messages with the developer of the Hubitat Z-Wave Mesh Details APP, Tony Fleisher also asked if there would be information about the last activity of the device with the z-wave radio.

He just answered me, that in the last version was incorporated the information extension, see his comment:

As to your question, the closest we have on Hubitat is the "Last Activity At" date, which you can see in the "Last Activity" column on the Devices page or at the end of the device details pages in the data section.

The latest version of the Z-Wave Mesh Tool has this available if you follow these steps:

1. Authorize Extended device data for your z-wave devices.
2. Select "Last Active Time" in the Additional Columns drop-down.

Looking at the information presented, I see that the date and time presented, effectively is not of a reboot or update of the device name, for example, it would actually be closer to what would be the last communication of the device with the hub.

I searched for this information in the MSR and don't see it, surely as he comments it is an extension of the device information.

Would it be possible for you @toggledbits, your kindness to perhaps look at this information, if we would be able to access it, or would it depend on the Make API?

Thanks, and again sorry for the insistence.

[toggledbits]

@wmarcolin said in Last Modifed Information:

surely as he comments it is an extension of the device information.

Then we should be able to see it in Reactor.

At this point, Reactor can only work with information that is passed through Maker API and Hubitat's events websocket, and it needs to be directly presented as attributes on the devices as those interfaces report them. If they do add attributes to the existing native Z-Wave devices, then those would be automatically immediately visible in the listed attributes of the Entities list.

Anything presented in the Hubitat UI is of no use if it does not make its way out of either Maker API, the events websocket, or another API that I might be able to access (the last option being the least desirable/most work).

[wmarcolin]

@toggledbits

Ok, I get it, and that is what I thought, the Make API has to publish this additional information, otherwise of course the MSR won't be able to see it.

So this goes along with your comments that you would be working on a way to not use the Make API for MSR and open up a wider universe of integration, is that correct?

Anyway, I did the search below, and I see that all devices have capabilities.x_hubitat_extra_attributes which is an object. How do I read this object, I lack the knowledge here to work out this additional step to the query. Could you help me?

Thanks.

[toggledbits]

Why don't you just look at the Entities list?

[wmarcolin]

@toggledbits

Yes, I look everything up in Entities and then try to elaborate queries to use in the actions. My question was more in the sense of knowledge/learning, how to read what appears as [object Object], if it is possible.

[toggledbits]

OK. Well you can't get to attributes by going through the capabilities property. You've never done it that way before, when you reference other attributes on an entity. The attributes under ...extra... work the same as any other... power_switch, dimming, binary_sensor...

I think you're over-thinking it.

[wmarcolin]

@toggledbits

[toggledbits]

LOL!

OK. Let's work from something you probably already do... checking the battery level. You are probably using an expression, either in a global variable or a group filter, that does this: entity.attributes.battery_power.level < 0.5.

This is an access through several objects to get to the final member, level. The top object is the entity itself, supplied either by the context (in the case of a group filter as shown), or some variant of getEntity(). Either way, the result is an object that contains the definition of the entity.

Adding .attributes to that, to make entity.attributes gets to the attributes property of the entity, which is itself an object that contains objects, the inner objects being identified by a capability name. So...

Adding .battery_power then, to make entity.attributes.battery_power gets us to the object that contains the attributes for the battery_power capability. That capability publishes one attribute...

Adding .level, to make entity.attributes.battery_power.level, which is the member that contains the (numeric) level, 0 to 1, representing the charge state of the battery.

So accessing any other capabilities attributes is done the same way... entity.attributes.x_hubitat_extra.somename

You can also enumerate the contents of an object using the each...in statement. For objects, in particular, the two-identifier syntax is useful because objects contain key/value pairs, and you want access to both the key and the value. So each v,k in obj: something will make the key and value available in the variables k and v to the something expression, for every key/value pair on obj.

[wmarcolin]

@toggledbits again a class of knowledge, thank you!

The part of checking the battery level I'm already using since you launched the Dynamic Group Controller, and of course already expanded to several other features, such as tracking the tripped devices.

Anyway, I also opened a ticket so that the Make API can contemplate this extension of information, which is not really published as the date of the last device x hub communication.

[wmarcolin]

Hi @toggledbits !!

After your class in January, I started using Dynamic Group Controller to manage devices that have no changes after a period to alert me of an eventual problem (48 hours for one group and 7 days for another).

        "dgc_lastupdate_48h":
          name: DGC Last Update Up 48 hours
          select:
            - include_controller:
              - hubitat
            - exclude_entity:
              - "hubitat>54"
              - "hubitat>257"
              - "hubitat>56"
              filter_expression: >
                entity.lastupdate < (time()-172800000)
            "dgc_lastupdate_7d":
              name: DGC Last Update Up 7 days
              select:
                - include_entity:
                  - "hubitat>107"
                  - "hubitat>1569"
                  - "hubitat>483"
          filter_expression: >
                      entity.lastupdate < (time()-604800000)         

It works and helped a lot, but as you already said, it can fail because a simple name update or reset of the HE/MSR can update everything and not indicate failures. Even updating the battery does not mean that the device is working. It can send a signal, but having no action does not help to identify the failure.

So I don't know if I could include in the Thanksgiving wishlist maybe something like Last Action, where it would store the date and time of when the device performs an action, i.e., the primary attribute that triggers a Trigger, for example.

Sorry to bother you with this subject again, but with a very large number of devices that I already have, it is becoming increasingly difficult to identify failures, a device that stops working, and then impacts the entire automation.

Thanks.