Migrating Zwave & ZigBee stuff to MQTT - guidance needed.

a-lurker

OK it works with them all set to skip although the documentation says blank, skip or http:// is OK . Might be useful to initialize them to skip.

akbooer

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

Hope this is of some help.

Certainly is! I'll pull something together and post it in the testing branch. Might need a bit of to and fro to get going properly.

akbooer

@a-lurker

There is a new version (v22.11.27) in the testing branch.

If you connect to the openLuup MQTT server, then it should detect the "bridge/devices" message and create some generic devices with some partial info about them in some variables. Just a basic test to show that the fundamentals work.

When the message is first detected, openLuup will create a Zigbee2MQTT bridge, a room called Zigbee and the child devices in that room. Here's what I get with the sample code you provided above (excellent starting info, by the way!) ...

therealdb

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

@therealdb

OK it works with them all set to skip although the documentation says blank, skip or http:// is OK . Might be useful to initialize them to skip.

Yep, time to update docs and the lib. Thanks for letting me know.

akbooer

I've slightly refined the individual device variables.

Should be easy to select at least some device types from this...

a-lurker

@akbooer

All works as advertised!! Excellent work. I still don't see the last image of the variables - still see the table pointers; as per your early image. Probably the latest code hadn't been committed to GitHub, judging by your commit times & post times.

I installed the openLuup testing branch and restarted. All worked as normal. Then I restarted the zigbee2mqtt server forcing it to output the "bridge/devices" message and everything got going in openLuup - great.

This is going to provide a lot possibilities. Currently 2507 possibilities! Downside is I may have to go and buy a few more devices.

Here's the start up log with a few edits and minus the date time. Might be useful to have say "none" or "****" for UserName & Password in the log. The "My Lua ver 0.50 debug:" is some of my test code:

Thank you very much for your efforts.

openLuup.io.server:: MQTT:1883 connection from XXX.XX.XX.XX tcp{client}: 0x47e20b0
openLuup.mqtt:: CONNECT tcp{client}: 0x47e20b0
openLuup.mqtt:: ClientId: mqttjs_19f169d2
openLuup.mqtt:: WillTopic: zigbee2mqtt/bridge/state
openLuup.mqtt:: WillMessage: {"state":"offline"}
openLuup.mqtt:: UserName: ****
openLuup.mqtt:: Password: ****
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: Topic ignored : zigbee2mqtt/bridge/state
openLuup.mqtt:: SUBSCRIBE tcp{client}: 0x47e20b0
openLuup.mqtt:: Packet Id: 0x2972
openLuup.mqtt:: Topic: zigbee2mqtt/#
openLuup.mqtt:: mqttjs_19f169d2 SUBSCRIBE to zigbee2mqtt/# tcp{client}: 0x47e20b0
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: Topic ignored : zigbee2mqtt/bridge/info
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: New Zigbee detected: 0x84b4dbfffecc0c6f
openLuup.luup:: creating room [36] Zigbee
luup.create_device:: [20001] D_GenericZigbeeDevice.xml /  / D_GenericZigbeeDevice.json   (GenericZigbeeDevice)
luup.zigbee2mqtt:0: New Zigbee detected: 0x01234567890abcde
luup.create_device:: [20002] D_GenericZigbeeDevice.xml /  / D_GenericZigbeeDevice.json   (GenericZigbeeDevice)
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: Topic ignored : zigbee2mqtt/bridge/groups
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: Topic ignored : zigbee2mqtt/bridge/extensions
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: Topic ignored : zigbee2mqtt/bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup.zigbee2mqtt:0: Topic ignored : zigbee2mqtt/bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x47e20b0
luup_log:0: My Lua ver 0.50 debug: topic = zigbee2mqtt/0x01234567890abcde
luup_log:0: My Lua ver 0.50 debug: device_temperature = 18
luup_log:0: My Lua ver 0.50 debug: illuminance = 316
luup_log:0: My Lua ver 0.50 debug: occupancy = false

akbooer

Great! Yes, haven’t committed the latest.

So that’s enough to get you going, then?

What else is needed? Obviously device type Identification and variable updates…

a-lurker

Device detection would be good. Hoping to see but asking a lot:

Occupancy sensor as per the device already shown in zigbee2mqtt/bridge payload.
RGB & white adjustable light bulb in this link. It's pretty typical. RGB would be optional. ie some bulbs being white only tunable.
Something like this Chinese dual outlet.. This would produce two virtual switches.
Hue switch/buttons. and similarly something like this single button. or this one.

The above should allow a Hue hub replacement in most cases? Noting the Hue switch is pretty tricky to replicate. Seems like it's effectively a scene controller.

The Occupancy sensor also has illuminance and temperature but certainly at this stage; just need the device's primary function.

I note in the documents some of the properties are in round brackets eg:

light (state, brightness, color_temp, color_xy), effect, power_on_behavior, linkquality

I don't know but I suspect that those properties in round braces are sub properties of the property of 'light" and that would be reflected in the MQTT device definition json as such. I need to add in a few more devices to my zigbee network to be definitive.

I had a look at the ZWave JS UI . Probably not overly different form Zigbee.

akbooer

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

0x01234567890abcde is the "ieee_address" of the device and it can be associated with a "friendly_name" of your own choice, such as "bedroom1/left_corner/motion"

This would seem to be a somewhat ambiguous convention, since MQTT topics are also subdivided with the '/' symbol. Especially since these can then be suffixed...

zigbee2mqtt/FRIENDLY_NAME/availability
zigbee2mqtt/FRIENDLY_NAME/set
zigbee2mqtt/FRIENDLY_NAME/get

...and so on.

I suppose we have to go with it.

akbooer

@a-lurker

New testing version v22.11.28 with the updated variable listing and also (perhaps) device identification for:

lights (displayed as switch)
switches (displayed as scene controller)
occupancy (displayed as motion sensor)
generic

The identified devices should have some of their standard serviceIds and variables.

Can I get a more complete bridge/devices JSON to try?

a-lurker

@akbooer
This is all coming along nicely - thanks.

I've set up a Hue color/tuneable white light bulb as it's probably the most complicated of the lights. This is what it replies with when you do a "zigbee2mqtt/0xaa55aa55aa55aa55/set" with payload {"state": "TOGGLE"}

{
  "color_mode": "xy",
  "linkquality": 255,
  "state": "ON"
}

Not sure if the "color_mode" is actually changeable. ie whether it's always XY color (CIE 1931 color space) or whether it can be RGB style. I note the Ikea bulbs just seem to be XY space.

If I change a color in the zigbee2MQTT web page it sends (not sure about all the back slashes):

'zigbee2mqtt/0xaa55aa55aa55aa55',
payload '{\"color\":{\"x\":0.4339,\"y\":0.5008},\"color_mode\":\"xy\",\"linkquality\":255,\"state\":\"ON\"}'"}

this is the reply from the bulb.

{
  "color": {
    "x": 0.6942,
    "y": 0.2963
  },
  "color_mode": "xy",
  "linkquality": 255,
  "state": "ON"
}

Here's "zigbee2mqtt/bridge/devices" for the light bulb only. You see there is a further lower level "features" below "exposes" ie "description,exposes,features". I've just cut & pasted the light bulb json description part below. It's just concatenated with the coordinator and the occupancy sensor, that you have already been using. I hope this is not getting unduly complex.

The coordinator and the occupancy sensor are listed here in the json and this is the light bulb:

... ,{
    "date_code": "20210331",
    "definition": {
      "description": "Hue white and color ambiance E26/E27/E14",
      "exposes": [
        {
          "features": [
            {
              "access": 7,
              "description": "On/off state of this light",
              "name": "state",
              "property": "state",
              "type": "binary",
              "value_off": "OFF",
              "value_on": "ON",
              "value_toggle": "TOGGLE"
            },
            {
              "access": 7,
              "description": "Brightness of this light",
              "name": "brightness",
              "property": "brightness",
              "type": "numeric",
              "value_max": 254,
              "value_min": 0
            },
            {
              "access": 7,
              "description": "Color temperature of this light",
              "name": "color_temp",
              "presets": [
                {
                  "description": "Coolest temperature supported",
                  "name": "coolest",
                  "value": 153
                },
                {
                  "description": "Cool temperature (250 mireds / 4000 Kelvin)",
                  "name": "cool",
                  "value": 250
                },
                {
                  "description": "Neutral temperature (370 mireds / 2700 Kelvin)",
                  "name": "neutral",
                  "value": 370
                },
                {
                  "description": "Warm temperature (454 mireds / 2200 Kelvin)",
                  "name": "warm",
                  "value": 454
                },
                {
                  "description": "Warmest temperature supported",
                  "name": "warmest",
                  "value": 500
                }
              ],
              "property": "color_temp",
              "type": "numeric",
              "unit": "mired",
              "value_max": 500,
              "value_min": 153
            },
            {
              "access": 7,
              "description": "Color temperature after cold power on of this light",
              "name": "color_temp_startup",
              "presets": [
                {
                  "description": "Coolest temperature supported",
                  "name": "coolest",
                  "value": 153
                },
                {
                  "description": "Cool temperature (250 mireds / 4000 Kelvin)",
                  "name": "cool",
                  "value": 250
                },
                {
                  "description": "Neutral temperature (370 mireds / 2700 Kelvin)",
                  "name": "neutral",
                  "value": 370
                },
                {
                  "description": "Warm temperature (454 mireds / 2200 Kelvin)",
                  "name": "warm",
                  "value": 454
                },
                {
                  "description": "Warmest temperature supported",
                  "name": "warmest",
                  "value": 500
                },
                {
                  "description": "Restore previous color_temp on cold power on",
                  "name": "previous",
                  "value": 65535
                }
              ],
              "property": "color_temp_startup",
              "type": "numeric",
              "unit": "mired",
              "value_max": 500,
              "value_min": 153
            },
            {
              "description": "Color of this light in the CIE 1931 color space (x/y)",
              "features": [
                {
                  "access": 7,
                  "name": "x",
                  "property": "x",
                  "type": "numeric"
                },
                {
                  "access": 7,
                  "name": "y",
                  "property": "y",
                  "type": "numeric"
                }
              ],
              "name": "color_xy",
              "property": "color",
              "type": "composite"
            },
            {
              "description": "Color of this light expressed as hue/saturation",
              "features": [
                {
                  "access": 7,
                  "name": "hue",
                  "property": "hue",
                  "type": "numeric"
                },
                {
                  "access": 7,
                  "name": "saturation",
                  "property": "saturation",
                  "type": "numeric"
                }
              ],
              "name": "color_hs",
              "property": "color",
              "type": "composite"
            }
          ],
          "type": "light"
        },
        {
          "access": 2,
          "description": "Triggers an effect on the light (e.g. make light blink for a few seconds)",
          "name": "effect",
          "property": "effect",
          "type": "enum",
          "values": [
            "blink",
            "breathe",
            "okay",
            "channel_change",
            "finish_effect",
            "stop_effect"
          ]
        },
        {
          "access": 1,
          "description": "Link quality (signal strength)",
          "name": "linkquality",
          "property": "linkquality",
          "type": "numeric",
          "unit": "lqi",
          "value_max": 255,
          "value_min": 0
        }
      ],
      "model": "9290012573A",
      "options": [
        {
          "access": 2,
          "description": "Controls the transition time (in seconds) of on/off, brightness, color temperature (if applicable) and color (if applicable) changes. Defaults to `0` (no transition).",
          "name": "transition",
          "property": "transition",
          "type": "numeric",
          "value_min": 0
        },
        {
          "access": 2,
          "description": "When enabled colors will be synced, e.g. if the light supports both color x/y and color temperature a conversion from color x/y to color temperature will be done when setting the x/y color (default true).",
          "name": "color_sync",
          "property": "color_sync",
          "type": "binary",
          "value_off": false,
          "value_on": true
        }
      ],
      "supports_ota": true,
      "vendor": "Philips"
    },
    "endpoints": {
      "11": {
        "bindings": [],
        "clusters": {
          "input": [
            "genBasic",
            "genIdentify",
            "genGroups",
            "genScenes",
            "genOnOff",
            "genLevelCtrl",
            "touchlink",
            "manuSpecificPhilips2",
            "lightingColorCtrl",
            "manuSpecificUbisysDimmerSetup"
          ],
          "output": [
            "genOta"
          ]
        },
        "configured_reportings": [],
        "scenes": []
      },
      "242": {
        "bindings": [],
        "clusters": {
          "input": [
            "greenPower"
          ],
          "output": [
            "greenPower"
          ]
        },
        "configured_reportings": [],
        "scenes": []
      }
    },
    "friendly_name": "0xaa55aa55aa55aa55",
    "ieee_address": "0xaa55aa55aa55aa55",
    "interview_completed": true,
    "interviewing": false,
    "manufacturer": "Signify Netherlands B.V.",
    "model_id": "LCT010",
    "network_address": 34172,
    "power_source": "Mains (single phase)",
    "software_build_id": "1.88.1",
    "supported": true,
    "type": "Router"
  }

zigbee2mqtt has individual device set up/review pages. You can see here that all the colors can be set up for example. The tooltip shows underlying stuff. In this case for color_hs.

akbooer

@a-lurker

New testing version v22.11.29 should identify (some) lights, and it's plausible that the on/off control will work (the dimmer doesn't, and the internal state doesn't yet get updated.) Would be interested in the log when trying this.

a-lurker

@akbooer

Making great progress. The light is recognized and can be turned on & off. Also if I change the color with the zigbee2mqtt web page and then turn the light off and on in AltUI the color is retained.

Here is one observed issue: I have in my own code with a subscription to the occupancy sensor and using that to set a switch in VirtualHTTP. Has been working and has now stopped working. You see the code here - line 415.. Looks like something in openLuup has stopped that functioning? See error in log below:

luup_log:209: VirtualDevices[3.0-beta7@209](initializeMqtt@421):initializeMqtt(209,{ PowerStatusOff={ Value="0", Service="urn:upnp-org:serviceId:SwitchPower1", Variable="Status" }, BrightnessValue={ Variable="LoadLevelStatus", Service="urn:upnp-org:serviceId:Dimming1" }, PowerStatusOn={ Value="1", Service="urn:upnp-org:serviceId:SwitchPower1", Variable="Status" } }) - openLuup: true
luup_log:209: VirtualDevices[3.0-beta7@209](subscribeToMqtt@414):subscribeToMqtt(209,nil,{ deviceID=209, opts={ Value="0", Service="urn:upnp-org:serviceId:SwitchPower1", Variable="Status" } })

openLuup.context_switch::  ERROR: [dev #209] ./L_VirtualLibrary.lua:415: attempt to concatenate field 'topic' (a nil value)

openLuup.scheduler:: job aborted : ./L_VirtualLibrary.lua:415: attempt to concatenate field 'topic' (a nil value)
openLuup.scheduler:: [211] Zigbee2MQTT device startup
luup.set_failure:: status = 0
luup.variable_set:: 211.urn:micasaverde-com:serviceId:HaDevice1.CommFailure was: 0 now: 0 #hooks:0
luup.variable_set:: 211.urn:micasaverde-com:serviceId:HaDevice1.CommFailureTime was: 0 now: 0 #hooks:0
openLuup.scheduler:: [211] Zigbee2MQTT device startup completed: status=true, msg=OK, name=Zigbee2MQTTBridge

Start up of openLuup and then connecting to zigbee2mqtt server follows:

openLuup.mqtt:: CONNECT tcp{client}: 0x2d4a968
openLuup.mqtt:: ClientId: mqttjs_2080d9ee
openLuup.mqtt:: WillTopic: zigbee2mqtt/bridge/state
openLuup.mqtt:: WillMessage: {"state":"offline"}
openLuup.mqtt:: UserName: ****
openLuup.mqtt:: Password: ****
openLuup.mqtt:: CONNECT tcp{client}: 0x25141a0
openLuup.mqtt:: ClientId: mqtt-explorer-27e89756
openLuup.mqtt:: WillTopic: 
openLuup.mqtt:: WillMessage: 
openLuup.mqtt:: UserName: ****
openLuup.mqtt:: Password: ****
openLuup.mqtt:: SUBSCRIBE tcp{client}: 0x25141a0
openLuup.mqtt:: Packet Id: 0xa5b3
openLuup.mqtt:: Topic: #
openLuup.mqtt:: mqtt-explorer-27e89756 SUBSCRIBE to # tcp{client}: 0x25141a0
openLuup.mqtt:: SUBSCRIBE tcp{client}: 0x25141a0
openLuup.mqtt:: Packet Id: 0xa5b4
openLuup.mqtt:: Topic: $SYS/#
openLuup.mqtt:: mqtt-explorer-27e89756 SUBSCRIBE to $SYS/# tcp{client}: 0x25141a0
openLuup.mqtt:: SUBSCRIBE tcp{client}: 0x2d4a968
openLuup.mqtt:: Packet Id: 0x6437
openLuup.mqtt:: Topic: zigbee2mqtt/#
openLuup.mqtt:: mqttjs_2080d9ee SUBSCRIBE to zigbee2mqtt/# tcp{client}: 0x2d4a968
openLuup.mqtt:: PUBLISH tcp{client}: 0x2d4a968
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2d4a968
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2d4a968
luup.zigbee2mqtt:0: Topic ignored : bridge/state
openLuup.server:: request completed (871430 bytes, 55 chunks, 630 ms) tcp{client}: 0x29113e0
openLuup.server:: GET /data_request?id=user_data&output_format=json&DataVersion=768995292&_=1669768933474 HTTP/1.1 tcp{client}: 0x29113e0
openLuup.server:: request completed (1679360 bytes, 105 chunks, 1182 ms) tcp{client}: 0x29113e0

Restart of zigbee2mqtt server to force devices to be announced by the zigbee2mqtt server. You see the zigbee devices being detected and set up by openLuup.

openLuup.mqtt:: CONNECT tcp{client}: 0x2ac56b0
openLuup.mqtt:: ClientId: mqttjs_689eab85
openLuup.mqtt:: WillTopic: zigbee2mqtt/bridge/state
openLuup.mqtt:: WillMessage: {"state":"offline"}
openLuup.mqtt:: UserName: ****
openLuup.mqtt:: Password: ****
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: Topic ignored : bridge/state
openLuup.mqtt:: SUBSCRIBE tcp{client}: 0x2ac56b0
openLuup.mqtt:: Packet Id: 0x8099
openLuup.mqtt:: Topic: zigbee2mqtt/#
openLuup.mqtt:: mqttjs_689eab85 SUBSCRIBE to zigbee2mqtt/# tcp{client}: 0x2ac56b0
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: Topic ignored : bridge/info
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: New Zigbee detected: 0xRedacted1
luup.create_device:: [20001] D_GenericZigbeeDevice.xml /  / D_GenericZigbeeDevice.json   (GenericZigbeeDevice)
luup.zigbee2mqtt:0: New Zigbee detected: 0xRedacted2
luup.create_device:: [20002] D_MotionSensor1.xml /  / D_MotionSensor1.json   (urn:schemas-micasaverde-com:device:MotionSensor:1)
luup.zigbee2mqtt:0: New Zigbee detected: 0xRedacted3
luup.create_device:: [20003] D_DimmableLight1.xml /  / D_DimmableLight1.json   (urn:schemas-upnp-org:device:DimmableLight:1)
openLuup.server:: request completed (16218 bytes, 2 chunks, 4645 ms) tcp{client}: 0x2456b18
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: Topic ignored : bridge/groups
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: Topic ignored : bridge/extensions
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2ac56b0

Here is the light being turned on & off noting that it actually does go on & off but I'm unclear about all the "Topic ignored". eg "0x_redacted_light_bulb/set/state"

openLuup.server:: GET /data_request?id=action&output_format=json&DeviceNum=20003&serviceId=urn:upnp-org:serviceId:SwitchPower1&action=SetTarget&newTargetValue=1 HTTP/1.1 tcp{client}: 0x28ba940
luup.call_action:: 20003.urn:upnp-org:serviceId:SwitchPower1.SetTarget 
luup.call_action:: action will be handled by parent: 211
luup.zigbee2mqtt:0: Topic ignored : 0x_redacted_light_bulb/set/state
openLuup.server:: request completed (35 bytes, 1 chunks, 16 ms) tcp{client}: 0x28ba940
luup.watch_callback:: 12.urn:a-lurker-com:serviceId:Paradox_IP150_wps1.Zone_018 called [3]variableWatchCallback() function: 0x156a978
openLuup.mqtt:: PUBLISH tcp{client}: 0x2b88a08
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2b88a08
luup.zigbee2mqtt:0: Topic ignored : 0x_redacted_light_bulb
luup.variable_set:: 10021.urn:micasaverde-com:serviceId:ZWaveNetwork1.LastPollSuccess was: 1669791951 now: 1669792831 #hooks:0
openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=769116357&Timeout=60&MinimumDelay=1500&_=1669769862259 HTTP/1.1 tcp{client}: 0x1f620a0
openLuup.server:: request completed (7375 bytes, 1 chunks, 120 ms) tcp{client}: 0x1f620a0
openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=769116361&Timeout=60&MinimumDelay=1500&_=1669769862260 HTTP/1.1 tcp{client}: 0x1f620a0
openLuup.server:: GET /data_request?id=action&output_format=json&DeviceNum=20003&serviceId=urn:upnp-org:serviceId:SwitchPower1&action=SetTarget&newTargetValue=1 HTTP/1.1 tcp{client}: 0x28ba940
luup.call_action:: 20003.urn:upnp-org:serviceId:SwitchPower1.SetTarget 
luup.call_action:: action will be handled by parent: 211
luup.zigbee2mqtt:0: Topic ignored : 0x_redacted_light_bulb/set/state
openLuup.server:: request completed (35 bytes, 1 chunks, 4 ms) tcp{client}: 0x28ba940
openLuup.mqtt:: PUBLISH tcp{client}: 0x2b88a08
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2b88a08
luup.zigbee2mqtt:0: Topic ignored : 0x_redacted_light_bulb
openLuup.server:: request completed (6642 bytes, 1 chunks, 3272 ms) tcp{client}: 0x1f620a0
openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=769116362&Timeout=60&MinimumDelay=1500&_=1669769862261 HTTP/1.1 tcp{client}: 0x1f620a0
openLuup.server:: GET /data_request?id=action&output_format=json&DeviceNum=20003&serviceId=urn:upnp-org:serviceId:SwitchPower1&action=SetTarget&newTargetValue=0 HTTP/1.1 tcp{client}: 0x28ba940
luup.call_action:: 20003.urn:upnp-org:serviceId:SwitchPower1.SetTarget 
luup.call_action:: action will be handled by parent: 211
luup.zigbee2mqtt:0: Topic ignored : 0x_redacted_light_bulb/set/state
openLuup.server:: request completed (35 bytes, 1 chunks, 4 ms) tcp{client}: 0x28ba940
openLuup.mqtt:: PUBLISH tcp{client}: 0x2b88a08
luup.zigbee2mqtt:0: Topic ignored : bridge/logging
openLuup.mqtt:: PUBLISH tcp{client}: 0x2b88a08
luup.zigbee2mqtt:0: Topic ignored : 0x_redacted_light_bulb
openLuup.server:: request completed (6870 bytes, 1 chunks, 4973 ms) tcp{client}: 0x1f620a0

Here is the light being turned on & off being reported by zigbee2mqtt in the linux terminal.

Zigbee2MQTT:info  2022-11-30 18:20:30: MQTT publish: topic 'zigbee2mqtt/0x_redacted_light_bulb', payload '{"brightness":42,"color":{"x":0.5267,"y":0.4133},"color_mode":"color_temp","color_temp":500,"linkquality":224,"state":"ON"}'
Zigbee2MQTT:info  2022-11-30 18:20:35: MQTT publish: topic 'zigbee2mqtt/0x_redacted_light_bulb', payload '{"brightness":42,"color":{"x":0.5267,"y":0.4133},"color_mode":"color_temp","color_temp":500,"linkquality":232,"state":"ON"}'
Zigbee2MQTT:info  2022-11-30 18:20:40: MQTT publish: topic 'zigbee2mqtt/0x_redacted_light_bulb', payload '{"brightness":42,"color":{"x":0.5267,"y":0.4133},"color_mode":"color_temp","color_temp":500,"linkquality":224,"state":"OFF"}'

Zigbee2MQTT:info  2022-11-30 18:26:51: MQTT publish: topic 'zigbee2mqtt/0x_redacted_occupancy_sensor', payload '{"battery":100,"detection_interval":30,"device_temperature":19,"illuminance":77,"linkquality":132,"motion_sensitivity":"medium","occupancy":false,"power_outage_count":0,"trigger_indicator":false,"voltage":3189}'

akbooer

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

Here is one observed issue: I have in my own code with a subscription to the occupancy sensor and using that to set a switch in VirtualHTTP. Has been working and has now stopped working. You see the code here - line 415.. Looks like something in openLuup has stopped that functioning? See error in log below:

This is clearly a nil topic value in the call. Where that is supposed to come from, I don't know.

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

Here is the light being turned on & off noting that it actually does go on & off but I'm unclear about all the "Topic ignored". eg "0x_redacted_light_bulb/set/state"

The ignored topics are the reason that the openLuup device state is not changing – I simply haven't coded the monitoring side of things yet. The next step.

I'm unclear because of your log edits whether the devices are using their ieee_address or their friendly_name as MQTT topics. AFAIK, it's legal to use either? I need to make some extra arrangements if so.

Which devices are currently not being correctly identified?

a-lurker

"The ignored topics are the reason that the openLuup device state is not changing – I simply haven't coded the monitoring side of things yet" - makes sense. Yes next step.
"your log edits": - sorry about this. Being a bit paranoid. All the ieee_address(s) and their associated friendly_name(s) are equal to each other. I haven't changed them. They are still the hex "mac" addresses of the devices.
"AFAIK, it's legal to use either? " It would certainly appear so. The use of the friendly_name(s) would certainly be needed to do topic filtering. ie plus & hash. But I've made no attempt so far to change the friendly_name(s). Trying to keep the change in all the variables to a minimum.
"This is clearly a nil topic value in the call" - this is my separate code. I subscribe to the occupancy sensor and the openLuup handler routine is coded to set a virtual switch. Seems the topic is getting screwed up in the handler but may be red herring, so please don't take too much notice. However as far as I can see @thereadldb has not made any recent code changes.
"Which devices are currently not being correctly identified?" - errhh none! Working very well so far.

If I need to provide more info please let me know.

akbooer

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

"Which devices are currently not being correctly identified?" - errhh none! Working very well so far.

Oh! ... wasn't expecting that!

Currently working on the device variable status updates.

akbooer

@a-lurker said in Migrating Zwave & ZigBee stuff to MQTT - guidance needed.:

Being a bit paranoid.

I did, at least, replace username and password in the MQTT diagnostics with "****"!

These wouldn't anyway appear in the log at all if you had not enabled DEBUG in that module.

akbooer

New testing version v22.11.30 should now mirror the device status (on/off and brightness) and also enable the openLuup device on/off control to work. Brightness level not yet implemented.

Battery levels should be shown too (once the device has woken up.)

Does the occupancy sensor work too?

a-lurker

@akbooer
Looking good - using AltUI for the following:

Light bulb:

Works:

on/off does control the bulb
variable LoadLevelStatus reflects any brightness changes
virtually all variables are loaded with a few exceptions - see below

Doesn't work:

brightness slider (as you noted)
light bulb icon does not change state in UI
variable LoadLevelTarget is always zero
variable linkquality is not loaded from the mqtt messages
variable effect is not loaded from the mqtt messages

Motion detector:

Works:

red dagger shows in UI when triggered
last trigger time shows in the UI (30 secs later as per its set up parms)
battery icon shows with percentage
virtually all variables are loaded with a few exceptions - see below

Not loaded from the mqtt messages:

detection_interval
device_temperature
illuminance
illuminance_lux
linkquality
motion_sensitivity
occupancy
trigger_indicator
voltage

Other:

variable "BatteryLevel" (and UI icon) shows "100" and variable "battery" shows "numeric". mqtt exposes "battery", so maybe some mix up or duplication here. They're probably the same thing - one mios & one zigbee.

Start up problem:
Conditions:

devices have all been previously detected and built and are available in AltUI.
the zigbee2mqtt server is running.

OK now restart openLupp. Status of all devices are not updated by any incoming mqtt messages from then on. Log says: "Topic Ignored : 0x_ieee_address".

Referring to file L_Zigbee2MQTTBridge.lua: function handle_friendly_names line 407 . Table "devices" is empty, so "dno" and "D" are nil.

Reason: it appears the "devices" table is loaded by "function init_device" and this is only called when the devices are first created in "function create_devices".

The devices table needs to be loaded if the devices have already been created. The work around is to restart the zigbee2mqtt server, so it publishes the topic "zigbee2mqtt/devices".

Logging:
Another slight issue with logging: The log showed for example on occasion : "bridge/logging" when it's actually "zigbee2mqtt/bridge/logging". The prefix zigbee2mqtt is missing and it makes a little unclear what topic is actually being ignored. Just makes it a bit harder to debug.

Console -> openLuup -> About:
There is item labelled "Debug" - how do you set this to true?

Everything is coming together!

a-lurker

Next device I'm hoping to try out: a Xiaomi button.

The above has similarities with the Hue dimmer switch

And then there is the magic cube - that's a very interesting one. eg you can just rotate it to dim/undim lights. Flip it and control something else.

OpenLuup : Loading Global Modules

openLuup_install.lua - URL changed?

openLuup charting and forward slashes in variable names

openLuup log files - LuaUPnP.log and LuaUPnP_startup.log

openLuup: Version Log

ZWaveJS and MQTT

Awesome MQTT - GitHub list

GetSolarCoords - correct results?

Truncated web pages using wireguard over long distances

Console view: - No Live refreshes via :3480/console?page=devices

openLuup and room names

Need to get rid of RulesEngine

Migrating Zwave & ZigBee stuff to MQTT - guidance needed.

DisplayLine1 & 2 - Ability to align text?

AltUI access over VPN

Problem with luup.call_delay nexted in luup.call_timer

OpenLuup installation

Unexpected stop of openLuup

Migrating Zwave & ZigBee stuff to MQTT - guidance needed.

Recent Topics