[Reactor] Bug when sending MQTT boolean payloads

[MikeReadington]

@CatmanV2 Well, at least I know it was not just me. I knew this day was coming because he had said something in August when it was broken that he didn't know if it was fixable because of Apple's API. I am thinking they kicked everything that did not use 2FA authentication.

@therealdb I set up Owntracks on Hubitat because it was easy, and I could play with it. Looks like it is very flexible. How hard is it on the battery? I realize mileage will vary a lot on this, but that was a feature I liked about the iPhone locator app. It ran using Apple's "Find My" that was already running on the phone. It's slow to update, but if I needed to know where the phone was, I would force a poll of location from the app. Since I started refining my logic in MSR I have been using manual polling almost exclusively. Thanks again for the point in this direction.

[MikeReadington]

Something of note from OwnTracks with iOS.

"Why do location publishes sometimes seem to cease to work?
Here's what happens:
If the app goes into Background, the connection is disconnected because the app cannot maintain the TCP connection in background.
If you bring the app back into foreground, the app will re-connect.
When a location change is recorded (no matter if manual, significant, move or region mode), a message is prepared and an attempt to connect to the broker is started. If there is an error, connect will be retried after 2, 4, 8, ..., 64, 64, .... seconds until it succeeds.
BUT...
If the application is in background or is sent to the background, retries are restarted eventually - depending on iOS' background execution model about every 10 minutes.
With automatic modes switched off, even when bringing the app to foreground, no new location updates are generated."

[CatmanV2]

@mikereadington said in iPhone location or alternative.:

I am thinking they kicked everything that did not use 2FA authentication.

Quite quite possible

C

[CatmanV2]

@mikereadington and this precisely explains the issues I saw. And makes perfect sense (I'm engaged with a company that is developing an a for iOS and Apple's restrictions and what happens when an App backgrounds (read 'is stopped') are quite severe.

Taking this, and my own recent experience into account, I don't think Owntracks is going to be viable. In fact, given the limitations from Apple I'm not sure anything is going to be viable for a geofence

C

[therealdb]

iOS has a background system to preserve battery life. It’s designed to be reliable and to not impact battery life. If you define your zones (I have home and office) at the very same moment you’ll cross the boundaries, a message is sent. I’m on my 2nd year with the system and never missed a bit. I also have a Bayesian sensor to detect if I’m connected to WiFi, and this is used as a second factor to confirm I’m home. But generally speaking, the entry/exit is totally demanded to OwnTracks.

[CatmanV2]

@therealdb It is indeed so designed, but that's not what I am seeing in terms of behaviour. Of course I may have misconfigured it on my phone so I am happy to be corrected!

(but I'll probably bug you for help)

But not right now

C

[gwp1]

@CatmanV2 I concur with @therealdb here, I've been using the OwnTracks on iOS via the Hubitat app integrations and it works well. I also use, as a secondary, a Home Assistant integration called iCloud3 which mitigates the 2FA issues you found with iPhone Locator. I've seen no noticeable battery issues (once I had iC3 configured correctly.)

@toggledbits got me into OwnTracks when he had his Logo trial up and running.

[CatmanV2]

Well I fully expect to be wrong I know that Geolocation is one of the use cases that Apple permit (or at least tolerate) in their process (and it's documented in the Owntracks literature as well as being the workaround for our devs.

I strongly suspect I fubarred the config somewhere (possibly as basic as not allowing full location services, for example). I had a crack this morning and I have the phone claiming it's connected, and reactor claiming it's connected (yay)

I need to get the entity installed but that's another day

C

[CatmanV2]

Trying to follow the posting guidelines.....

  - id: mqqt
    enabled: true
    implementation: MQTTController
    name: Mosquito
    config:
      source: "mqtt://192.168.70.249:8883/"
      username: xxxxx 
      password: xxxxx
      entities:
        catman_iphone:
          name: "Catman Phone"
          topic: FFCEB043-B975-490E-B051-198473F6A504
          uses_template: owntracks_in_region

Claims to be valid YAML in https://yamlchecker.com/

But when I add the last 4 lines to my reactor.yaml msr will not start:

Reactor log stops:

[latest-22274]2022-10-08T17:32:39.316Z <Engine:NOTICE> [Engine]Engine#1 has shut down.
[latest-22274]2022-10-08T17:32:39.316Z <app:NOTICE> Closing Structure...
[latest-22274]2022-10-08T17:32:39.317Z <Structure:INFO> Structure#1 Stopping controllers...
[latest-22274]2022-10-08T17:32:39.317Z <VeraController:NOTICE> VeraController#vera stopping
[latest-22274]2022-10-08T17:32:39.321Z <VeraController:ERR> Controller VeraController#vera is off-line!
[latest-22274]2022-10-08T17:32:39.366Z <MQTTController:NOTICE> MQTTController#mqqt stopping, sending LWT
[latest-22274]2022-10-08T17:32:39.366Z <DynamicGroupController:NOTICE> DynamicGroupController#groups stopping
[latest-22274]2022-10-08T17:32:39.369Z <DynamicGroupController:ERR> Controller DynamicGroupController#groups is off-line!
[latest-22274]2022-10-08T17:32:39.369Z <SystemController:NOTICE> SystemController#reactor_system stopping
[latest-22274]2022-10-08T17:32:39.371Z <SystemController:ERR> Controller SystemController#reactor_system is off-line!
[latest-22274]2022-10-08T17:32:39.372Z <Structure:INFO> Structure#1 Final data sync...
[latest-22274]2022-10-08T17:32:39.379Z <Structure:NOTICE> Structure Structure#1 stopped
[latest-22274]2022-10-08T17:32:39.379Z <app:NOTICE> Stopping timers...
[latest-22274]2022-10-08T17:32:39.380Z <app:NOTICE> Shutdown complete, process ID 27723

As soon as I replace the original version of reactor.yaml

  - id: mqqt
    enabled: true
    implementation: MQTTController
    name: Mosquito
    config:
      source: "mqtt://192.168.70.249:8883/"
      username: xxxx 
      password: xxxx

Everything springs to life.

I can't see an error in my syntax. Do I need to have the region in there (although I've tried both with and without)

It's going to be something obvious / stupid but hopefully I'm learning at least a little

TIA

C

[therealdb]

@catmanv2 the logs are not complaining, so maybe the error was written before.

values are escaped with " in the docs, give it a try (ie: "catman_iphone" and topic: "FFCEB043-B975-490E-B051-198473F6A504")

[CatmanV2]

Thanks

Tried

  entities:
        catman_iphone:
          name: "Catman Phone"
          topic: "FFCEB043-B975-490E-B051-198473F6A504"
          uses_template: owntracks_in_region
          regionName: Home

and

  entities:
        "catman_iphone":
          name: "Catman Phone"
          topic: "FFCEB043-B975-490E-B051-198473F6A504"
          uses_template: owntracks_in_region
          regionName: Home

With no discernible difference

C

[toggledbits]

@catmanv2 Topics for me are not UUIDs, but rather the content of the Device ID field in the Identification section of the configuration. This is a value you set, not a value determined by OwnTracks. That's Android. I no longer have an Apple device I can use for this.

        owntracks_patrick_home_template:
          name: "Patrick Home Template"
          uses_template: owntracks_in_region
          topic: phrs21
          regionName: Home

[MikeReadington]

Thanks for all of the input.

For whatever reason I am having the same spotty relationship with OwnTracks that @CatmanV2 has. One minute I think it's solved, the next minute it's missed a trigger.

With the success others have had, including @gwp1 with the same method of deployment, I must have something wrong. I'll keep working on it as time permits.

Thanks again

[CatmanV2]

@mikereadington iPhone or Android?

C

[MikeReadington]

@catmanv2 iPhone. I just returned home and it got a location trigger, but it was far from an instant trigger. The phone was in sleep for at least 30 minutes before I crossed the perimeter, and I would say the delay was in the area of 45 seconds.

[CatmanV2]

45 seconds I can live with, personally but might not be right for your use case. (My fence is about 5km out)

The thing I'm struggling with right now is testing it. I think I need to cross the fence. Logically I make the region much smaller for testing purposes but other issues at work today.

C

[gwp1]

@catmanv2 testing OwnTracks can be accomplished by changing from substantial to move in the app and then, well, moving.

[CatmanV2]

@gwp1 said in iPhone location or alternative.:

@catmanv2 testing OwnTracks can be accomplished by changing from substantial to move in the app and then, well, moving.

Thanks, I can see the data arriving from Owntracks in Mosquitto (if you hit publish, for example) but what I can't do is get the MQQT entity to have anything but 'null' as its primary value. I can see MQQT is publishing to reactor as well, so confidence is reasonably high....

My "logic" is that MSR is looking for the 'Enter' or 'Leave' payload which you only get when you transition into / out of a region.

I was expecting to go somewhere today but the optician was ill and I've been tied up on calls and meetings all day so not had a chance to set a small region and wander up the road.

Of course my logic could be utterly wrong, but wanted to test it before asking for more help.

Because then I can save what little credibility I have and use it asking for help as to why the second phone (Mrs C's) doesn't have primary value. I found an error in the logs though which I'll dig at later

Cheers

C

[MikeReadington]

@catmanv2 Yeah, my use case is a little more particular. I don't have an issue with never getting notification changes, they are just more delayed than I would like. I think once you get MQTT and MSR talking a 5km is going be more than relievable enough.

I had iPhone locator poll/force location refresh when I came in my driveway and triggered the magnetic sensor. I needed the location to be accurate and updated by the time I reached either my house or my building on property. The driveway is about 300m long as the crow flies for region radius purposes, and traversal to either location is much less than 45 seconds.

For what it is worth, I do see instant (and I mean instant instant) location changes like @therealdb and @gwp1 reported when the app is open or recently accessed. The delay seems to come from the app being backgrounded and the phone idle for a long period.

@gwp1 with HASS iCloud3, can you remote request a location refresh of the phone? Meaning, could I have polling suspended until I ask for an update? I think I could get exactly what I am looking for with a combination of the two and some MSR logic.

[CatmanV2]

@mikereadington said in iPhone location or alternative.:

I had iPhone locator poll/force location refresh when I came in my driveway and triggered the magnetic sensor. I needed the location to be accurate and updated by the time I reached either my house or my building on property. The driveway is about 300m long as the crow flies for region radius purposes, and traversal to either location is much less than 45 seconds.

Given my thermostat has a 5 minute update. 45 seconds is just dandy.

The lights and so on are never going to be an issue.

C