Solved Is it possible for MSR to act on Nest hello (doorbell pressed)?

[Fanan]

Hi!
I have a Nest hello, and as a integration in Home Assistent. I would want to be able to create a automation/reaction when someone press the doorbell button. The problem is that this event/service will only appear as triggers (in HA):
I can't access the doorbell as an entity. I can only access it if I open the 'doorbell' device in HA.
My question is therefor; is it possible to access the doorbell and it's triggers within MSR?
I have MSR build 21307-1746e27 and HA core-2021.11.2.
Have a great day!
/Fanan

[Matohl]

@fanan OT: I can not answer your question but I see that you are using HA 11.2. Do you use the Vera integration in HA as well? If so, how does it work since I have seen some breaking changes in 2021.11 with respect to entities?

[gwp1]

@fanan Does MSR show the Nest under Entities? If so, clicking the > (chevron) will show you what's exposed to MSR.

[toggledbits]

If HomeAssistant is showing a device Doorbell, it should have a matching entity (you may need to enable it in the HA integration; it could be disabled/hidden). MSR will make an entity for it if HA publishes it, and it will be visible as @gwp1 describes.

It could also be that it's in a domain that MSR doesn't know how to handle.

Post the hass_states.json file from your logs directory here (DM to me), and I can see if it's published, and if so, what its domain is and mapping should be.

[Fanan]

@toggledbits I sent you an email with my log file.
@gwp1 No, MSR does not show the Nest doorbell under Entities. It do show the Camera though.
@Matohl So far so good, using the Vera integration!

[toggledbits]

OK, as I suspected, there's no data there for anything other than the camera feed. You need to go into the integration, go to the device, and enable the sensor entities (probably marked hidden) for that device. You should then see binary sensors related to the camera in Lovelace. Once you have that, restart MSR and it should pick up those sensors as entities.

[Fanan]

@toggledbits the problem is that's it's not a entities, only a trigger. So it's not hidden. I have other cameras with hidden entities, so it's not that i don't know where our how to find it. I can only access it as a trigger when i do a automation within HA, as in the pic in my first post.

[toggledbits]

The camera itself is a device. That's where you go. I assume you configured this integration through the UI, as it seems like a "first class" integration, not one of the components you configure directly in configuration.yaml or other. I don't have these devices, so I can't help you beyond that. But the typical way is to go to the parent device, and enable its children, in effect. At least, it works this way with every other integration I have, including other non-Nest cameras.

[kfxo]

It appears from the Nest- Home Assistant docs that the doorbell generates a nest_event (doorbell_chime). Unfortunately I do not believe there is any way for MSR to see these events. I have brought this up previously with respect to Emulated Roku. My work around is to use a native HA automation to detect the event and trigger a helper object that MSR can see, in my case I place the text of the event in a input_text helper which MSR can see.

[Fanan]

@kfxo Thanks - I believe this is the way to go. I'll give it a try in a few days.
/Fanan

[toggledbits]

@Fanan After a bit more research, it may be there's something I can do on the Reactor side. I can't trial or test anything, of course, because I don't have those devices or use that integration, so I'm hoping you'll be my hands and eyes, and it may take a couple of passes to get something working that way...

[toggledbits]

OK. There's a build 21314 now posted. It allows you to set log_events: true in the config section of your HassController (in reactor.yaml). This will produce a log of events to logs/hass_events.log. Add that config, restart MSR, trigger some events with your camera, and then set log_events: false and restart MSR. Then DM or email me the hass_events.log file.

[Fanan]

@toggledbits Thanks! You just got an email!

[toggledbits]

OK. I see the events. I think this can be handled. If I get time to look at this over the weekend, I certainly will, but expect Monday at the latest.

[Fanan]

@toggledbits No rush. Take care of your son and enjoy the weekend!

[toggledbits]

OK. I think this is doable on the version you have, which already has some enhancements I carried in from the MQTTController side.

First, turn off log_events in the config for now; I don't think we'll be needing it. Restart MSR and remove the hass_events.log file from the logs directory.

Add the following in the config section of your HassController configuration (in reactor.yaml). The event_targets line should be indented to the same level as source and log_events; I've put a config line in to show that, but don't copy it, it's just for reference. I'll explain what all this does after:

    config:
      # ... your other config stuff is above this comment; append these lines:
      event_targets:
        "nest_hello_motion":
          name: "Nest Hello Motion"
          capabilities: 
            - motion_sensor
          events:
          - event:
              event_type: nest_event
              data:
                device_id: "148552cdf7eb1caeb5dad5f6a3874821"
                type: "camera_motion"
            response:
              "motion_sensor.state": true
              "motion_sensor.since":
                expr: "time()"
        "nest_hello_person":
          capabilities: 
            - binary_sensor
          name: "Nest Hello Person"
          events:
          - event:
              event_type: nest_event
              data:
                device_id: "148552cdf7eb1caeb5dad5f6a3874821"
                type: "camera_person"
            response:
              "binary_sensor.state": true
              "binary_sensor.since":
                expr: "time()"
       "nest_doorbell":
          capabilities: 
            - binary_sensor
          name: "Nest Doorbell"
          events:
          - event:
              event_type: nest_event
              data:
                device_id: "148552cdf7eb1caeb5dad5f6a3874821"
                type: "doorbell_chime"
            response:
              "binary_sensor.state": true
              "binary_sensor.since":
                expr: "time()"

This config is going to create three entities in Reactor, the first with ID nest_hello_motion the second with ID nest_hello_person, and third with ID "nest_doorbell". Each of them works about the same, but they look at different Nest events.

The nest_hello_motion config sets the entity up with the motion_sensor capability (only). It watches for a Home Assistant event of type nest_event that contains in its data the device ID shown and the more specific type camera_motion. When Reactor gets a matching event, it will set the motion_sensor.state attribute on the Reactor entity to true, and update the timestamp in the since attribute for that capability.

The nest_hello_person config works the same, but looks for Nest's camera_person event. And of course, the nest_doorbell config is looking at the same Home Assistant event type and device ID, but matching only the specific doorbell_chime event.

Now for the tricky part (in case this hasn't been tricky enough). These events from Nest have no reset. That is, the Nest integration tells you when it detects motion, but not when it stops detecting motion. That means that the state attribute values can never be anything but true in the Reactor entity (there's no way to reset it to false). Therefore, to test if the motion (or person, or doorbell) event has occurred, then, you instead will have to test if since (an extension attribute for the capability) changes, which it will every time the event is received. This is a bit of a shame, because it requires special handling for Nest different from most other devices, but then, that's nothing new. My Amcrest and other cameras do not work this way — their events send true and false detection events so you can tell when the camera no longer detects motion. Alas, not with Nest. So, if at some point in future you need to know when motion is no longer detected, because Nest doesn't tell us you'll have to use condition options to infer it (e.g. test that since has not changed for a certain period). Should work fine, it's just not as easy as when the device/driver gives you an affirmative reset event.

To generalize, this structure allows you to receive and respond to any Hass event:

      event_targets:    # this section starts the event-receiving entities
        "my_virtual_entity_id":     # Assign an ID to your entity; each entity must have a unique ID
          name: "My Entity Name"   # This is optional but recommended, so you have a friendly name
          capabilities:  # define an array of capabilities to be modified by the event
          - capability_name    # first capability
          - capability_name    # second, etc., as many as you need, but always at least ONE
          events:  # define an array of events that modify capability attributes on the entity
          - event:  # start of an event; each element of the events array begins this way
              event_type: "type of HomeAssistant event"
              data:  # optional section, if further matching to the event data is required
                data_field: "data_value"    # data field to be checked/matched to data_value
                data_field: "data_value"    # as many as you need, but each data_field must be unique
            response:  # begin the (required) response section for handling the event
              "capability.attribute":
                expr: "expression"   # expression to get attribute value from event data
              # repeat "capability.attribute" section for all attributes modified by the event

This isn't complete for documentation, I'm writing it now. Definitely will be an advanced user feature.

Also, just FYI, Hass seems to be modifying its person entities when detected by the camera as well, so that's a sneaky thing that's changing when the Nest integration is handling the event on its end. See Hass entity binary_sensor.person_vid_ytterdorren for example (and there should be a Reactor entity with a similar ID, just the dot replaced with underscore).

[kfxo]

I was messing around with this and I think there is an indentation issue starting at line 8. See below for what indentation worked for me.

      event_targets:    # this section starts the event-receiving entities
        "my_virtual_entity_id":     # Assign an ID to your entity; each entity must have a unique ID
          name: "My Entity Name"   # This is optional but recommended, so you have a friendly name
          capabilities:  # define an array of capabilities to be modified by the event
            - capability_name    # first capability
            - capability_name    # second, etc., as many as you need, but always at least ONE
          events:  # define an array of events that modify capability attributes on the entity
            - event:  # start of an event; each element of the events array begins this way
                event_type: "type of HomeAssistant event"
                data:  # optional section, if further matching to the event data is required
                  data_field: "data_value"    # data field to be checked/matched to data_value
                  data_field: "data_value"    # as many as you need, but each data_field must be unique
              response:  # begin the (required) response section for handling the event
                "capability.attribute":
                  expr: "expression"   # expression to get attribute value from event data
                  # repeat "capability.attribute" section for all attributes modified by the event

[toggledbits]

Yeah, there's some ambiguity/disagreement with respect to indenting of array elements, and yamllint and the library I'm using to parse YAML disagree. Bottom line, do what works, and what you've shown is typical for the way I indent such arrays. I'll make sure the docs reflect that.

[kfxo]

This is off topic to the original post so maybe it should be moved but it is directed towards the Home Assistant event_targets discussed above.

I defined an event_target in my system for emulated roku that looks like this:

    event_targets:
      "LR_Emulated_Roku_Home_Keypress":
        name: "LR Emulated Roku Home Keypress"
        capabilities:
          - button
        events:
          - event:
              event_type: "roku_command"
              data:
                source_name: "LR_Roku"
                type: "keypress"
                key: "Home"
            response:
              "button.since":
                expr: "time()"
              "button.state": true

This works great, detects the event and stores the time and True in the respective button entity attributes. However upon restarting MSR the values of the button entity attributes clear to null (e.g. button.since=null).

Is this expected behavior or should these values be maintained on restart?

[toggledbits]

This is expected behavior for the moment, but will be changing. MSR has never had a need to persist entities over restarts because the controllers have the current data. It's not even an issue for MQTT, because retained messages, when used properly, update state correctly on restart/reconnect.

But, I have been using a version of MSR in my own home that persists entities across restarts, for other reasons, and that will also serve to smooth out the bump for event handling here. I still have a few more wrinkles to smooth out... like how to detect when the user has changed their custom configuration for an entity, modified their own custom declaration of a capability, etc. These are rarely-used features, but I nonetheless want them to require less user intervention (e.g. manual forced purge of retention data, etc).