Advice reqeusted to migrate MSR from Bare Metal to Container

[therealdb]

I'm slowly migrating all my stuff to MQTT under MSR, so I have a central place to integrate everything (and, in a not-so-distant future, to remove virtual devices from my Vera and leave it running zwave only).

Anyway, here's my reactor-mqtt-contrib package:

GitHub - dbochicchio/reactor-mqtt-contrib: Contrib MQTT templates for Reactor

Contrib MQTT templates for Reactor. Contribute to dbochicchio/reactor-mqtt-contrib development by creating an account on GitHub.

Simply download yaml files (everything or just the ones you need) and you're good to go.

I have mapped my most useful devices, but I'll add others soon. Feel free to ask for specific templates, since I've worked a lot in the last weeks to understand and operate them.

The templates are supporting both init and query, so you have always up-to-date devices at startup, and the ability to poll them. Online status is supported as well, so you can get disconnected devices with a simple expression.

Many-many thanks to @toggledbits for its dedication, support, and patience with me and my requests

[gwp1]

@therealdb And kudos to you for sharing all of this with the Greater Good.

[therealdb]

I've pushed a couple of fixes, plus my new template for owntracks, that's supporting all the info as attributes.

GitHub - dbochicchio/reactor-mqtt-contrib: Contrib MQTT templates for Reactor

Contrib MQTT templates for Reactor. Contribute to dbochicchio/reactor-mqtt-contrib development by creating an account on GitHub.

This template will create a device that has:

• a string sensor with the current region
• a binary sensor that's true when the user is at home
• locations info via location capability
• battery info for the device (including percentage and charging status)

@gwp1 @CATMANV2

[gwp1]

@therealdb Very timely, this. I'm poking at using MQTT/OwnTracks to replace the Home Assistant integration.

[therealdb]

@gwp1 I was using my own virtual devices on Vera + some code in my own mqtt consumer, and I'm slowly migrating all of them to native MSR devices. I'm keeping virtual devices on Vera just for dashboard and some other minor automations.
next on my list: my EV charger and maybe ESPPresence.

[CatmanV2]

Oh wow. That looks very interesting indeed! Thanks!

C

[gwp1]

@therealdb I just this week started into ESPresense with my MQTT > Home Assistant > msr.

Three rooms so far. Also looking to use it at a buddy’s house to trigger garage door when I’m by.

[CatmanV2]

@therealdb
Thanks so much for this. I seem to have fallen at the first hurdle:
I grabbed zip file from Git hub and unzipped it. SCPed the owntracks_sensor.yaml file over to my bare metal Debian Bullseye install.
Created reactor/config/mqtt_templates

catman@openluup:~/reactor/config$ ls -l
total 124
-rw-r--r-- 1 catman catman   232 May 26  2022 0-README.txt
-rw-r--r-- 1 catman catman   302 May 26  2022 cameras.yaml
-rw-r--r-- 1 catman catman   150 May 26  2022 dashboard.json
-rw-r--r-- 1 catman catman   399 May 26  2022 local_hass_devices.yaml
-rw-r--r-- 1 catman catman   434 May 26  2022 local_hubitat_devices.yaml
-rw-r--r-- 1 catman catman  1945 May 26  2022 logging.yaml
drwxr-xr-x 2 catman catman  4096 Dec 11 18:00 mqtt_templates
-rw-r--r-- 1 catman catman  8960 Jul 19 16:10 notification.yaml
-rw-r--r-- 1 catman catman  8959 Jul 19 15:55 notification.yaml.orig
-rw-r--r-- 1 catman catman  8963 Jul 19 16:03 notification.yaml.telegram
-rw-r--r-- 1 catman catman  8960 Jul 19 17:15 notification.yaml.working
-rw-r--r-- 1 catman catman 12221 Dec 11 18:12 reactor.yaml
-rw-r--r-- 1 catman catman 11827 Oct 10 18:07 reactor.yaml.change
-rw-r--r-- 1 catman catman 11896 May 26  2022 reactor.yaml.orig
-rw-r--r-- 1 catman catman 12039 Dec 11 18:01 reactor.yaml.working

The file is there

catman@openluup:~/reactor/config/mqtt_templates$ ls -l
total 4
-rw-r--r-- 1 catman catman 3301 Dec 11 18:00 owntracks_sensor.yaml

And there is an entry in reactor.yaml:

        rachel_owntracks:
          name: "Rachel Location"
          prefix: rachel
          topic: iphone
          uses_template: owntracks_sensor
          homeRegionName: Home

But

Obviously something really silly but damned if I can spot it

Could the template yaml file have been corrupted?

TIA!

C

[toggledbits]

Are you on the latest version of MQTTController?

[CatmanV2]

I am now, thanks!
All looking good

C

[gwp1]

@catmanv2 I'm watching your trials with this and trying to learn from your successes and losses.

What do you have configured in the actual iOS OwnTracks app as far as:

TrackerID
DeviceID
UserID

I think this is where I'm getting lost. I'm testing with HiveMQ and have the iPhone connecting (phone shows connected) and MSRs MQTTController is connected just fine to HiveMQ - it's just MSR and the iphone that are missing each other.

  - id: mqtt2
    name: MQTT HiveMQ
    enabled: true
    implementation: MQTTController
    config:
      # Replace IP with that of your MQTT broker below
      source: "mqtts://7exxxxxxxxxxxxxxx0d0.s2.eu.hivemq.cloud:8883"
      username: "<username>"
      password: "<password>"
      entities:
        owntracks_iphone14p: #entity ID - must be unique
          name: "iPhone14p location" # friendly name
          prefix: "iphone14p"
          topic: "john_iphone" #the device name of my phone is John iPhone
          uses_template: owntracks_sensor
          homeRegionName: "Home"

In MQTTController I do see:
...but that's nowhere the detail I'd expect from @therealdb's template (which is properly placed in the correct location:

I am brand new to MQTT so I'm likely missing something horribly obvious.

Any guidance is appreciated.

[CatmanV2]

@gwp1
TrackerID is CM
DeviceID is catmaniphone
UserID is my configured username for my Mosquito server (so should be your Hive username)

Because all of my stuff is local I can see the messages being posted to the logs. Not sure if you can do that with your hive set up?

At a guess you are not connecting to the Hive server. Does the iPhone app say 'connected'. Can you see Reactor getting messages from the Hive server?

I'm not 100% sure that's the right location for the config file. I guess if you're running reactor from /home/reactor/Documents it's right though. Just an odd choice (although Windows is not my thing) !

C

[gwp1]

@catmanv2 yep, configured the Hive user and password and is shows as “connected” in the iOS app. I’ve not found anywhere in their dashboard to verify messages being received into the broker. I’ve not seen anything come in from the broker to msr.

I’m running a bare metal install on a RPi. Not a Linux expert so it ended up in an interesting folder structure but it works just fine so re-orging isn’t a priority.

[CatmanV2]

@gwp1 That's a Linux setup?!

OK, can you read the log from reactor?

Also, are you really sure that Reactor is able to connect to HiveMQ?

C

[gwp1]

@catmanv2 Back home now.

This leads me to believe it's connected to Hive:

I see nothing in the logs because a.) not entirely should what it would look like, and b.) there's nothing happening atm.

[CatmanV2]

Hmm, yes tough. I share your suspicion that it's Hive. No interest in firing up Mosquito on your Linux box? Takes about 10 minutes. OK there's an increase in your vulnerability but.....

C

[toggledbits]

The docs for MQTTController describe how to enable topic capture, so you can see if you are getting anything at all from Hive (this is probably preferable to turning up the log level in the log file for MQTTController). I'd suggest that as a start. See what, if anything, it's sending at all.

Ref: https://reactor.toggledbits.com/docs/MQTTController/#troubleshooting-and-debugging

[therealdb]

@catmanv2 I hope you’ll enjoy it as much as I am

[gwp1]

@toggledbits @CatmanV2 Unless I'm just not far enough thru coffee this morning, this is what I see in MSR's mqtt logs and it looks like it's data from my iPhone8 (which I fired up via wifi to use as a test phone.)

2022-12-12T14:02:02.896Z "owntracks/msrmqtt/iphone8" {"_type":"location","acc":35,"alt":10,"batt":100,"bs":1,"conn":"o","inregions":["Home"],"inrids":["adf34c"],"lat":32.94714,"lon":-80.013836,"m":1,"p":101.826,"t":"u","tid":"i8","tst":1670853720,"vac":15}
2022-12-12T14:02:02.998Z "owntracks/msrmqtt/iphone8" {"_type":"location","acc":10,"alt":10,"batt":100,"bs":1,"conn":"o","inregions":["Home"],"inrids":["adf34c"],"lat":32.946934,"lon":-80.013887,"m":1,"p":101.827,"tid":"i8","tst":1670853722,"vac":8,"vel":0}

So why isn't this showing here:

My entity config is:

        owntracks_iphone8: #entity ID - must be unique
          name: "iPhone8 location" # friendly name
          prefix: "iphone8"
          topic: "test_iphone"
          uses_template: owntracks_sensor
          homeRegionName: "Home"

I feel like I'm just "this close" and I just don't have something aligned properly.

[toggledbits]

Yes, your configuration of prefix and topic don't make a string that matches what is being received. Based on what it is coming in and how those fields template configuration are used, prefix should be msrmqtt and topic should be just iphone8.

Conspicuous give-away: your test_iphone value doesn't appear in the topic or payload of what you're receiving.

@therealdb this may need a bit more documentation, since the relevance of prefix and topic aren't immediately clear (at least, not to me) without looking at the template itself and seeing how they are used, and likely the phone configuration as well (i.e. describe how configuration fields on the phone may appear in the final event topic).