openLuup: MQTT server
-
@akbooer Below is the relevant output of a typical packet between mosquitto and a mosquitto bridged instance. In this case, mosquitto is sending update data to the bridge regarding a tasmota device/switch I use to remotely reboot my Vera. The format is definitely MQTT 3.1 and not 5.0, as 5.0 would not parse correctly in the wireshark viewer. The data payload is at the top of the window as Wireshark will truncate long messages. I can PM you the entire capture as it's not much, but may contain technical info that's best kept private. Let me know.
I'll try to capture some traffic between openLuup and mosquitto later.
{"Version":"9.1.0(tasmota)","BuildDateTime":"2020-11-07T11:57:45","Module or Template":"Gosund-WP5","RestartReason":"Software/System restart","Uptime":"6T05:50:22","Hostname":"power_MainVera-0278","IPAddress":"10.17.2.33","RSSI":"100","Signal (dBm)":"-17","WiFi LinkCount":5,"WiFi Downtime":"0T00:00:10","MqttCount":14,"LoadAvg":19} Frame 9: 433 bytes on wire (3464 bits), 433 bytes captured (3464 bits) on interface eth0, id 0 Ethernet II, Src: Advansus_0a:8c:3a (00:19:0f:0a:8c:3a), Dst: 96:62:08:fb:22:8a (96:62:08:fb:22:8a) Internet Protocol Version 4, Src: 10.17.2.41, Dst: 10.17.2.110 Transmission Control Protocol, Src Port: 40664, Dst Port: 1882, Seq: 3, Ack: 3, Len: 367 MQ Telemetry Transport Protocol, Publish Message Header Flags: 0x30, Message Type: Publish Message, QoS Level: At most once delivery (Fire and Forget) 0011 .... = Message Type: Publish Message (3) .... 0... = DUP Flag: Not set .... .00. = QoS Level: At most once delivery (Fire and Forget) (0) .... ...0 = Retain: Not set Msg Len: 364 Topic Length: 30 Topic: tele/power_MainVera/HASS_STATE Message [truncated--see above]: {"Version":"9.1.0(tasmota)","BuildDateTime":"2020-11-07T11:57:45","Module or Template":"Gosund-WP5","RestartReason":"Software/System restart","Uptime":"6T05:50:22","Hostname":"power_MainVera-0278","IPAddress":"10.17.2
-
@akbooer
And a ping response from bridge to main instance:Frame 33: 68 bytes on wire (544 bits), 68 bytes captured (544 bits) on interface eth0, id 0 Ethernet II, Src: 96:62:08:fb:22:8a (96:62:08:fb:22:8a), Dst: Advansus_0a:8c:3a (00:19:0f:0a:8c:3a) Internet Protocol Version 4, Src: 10.17.2.110, Dst: 10.17.2.41 Transmission Control Protocol, Src Port: 1882, Dst Port: 40664, Seq: 5, Ack: 2851, Len: 2 MQ Telemetry Transport Protocol, Ping Response Header Flags: 0xd0, Message Type: Ping Response 1101 .... = Message Type: Ping Response (13) .... 0000 = Reserved: 0 Msg Len: 0
-
The Connect packet:
MQ Telemetry Transport Protocol, Connect Command Header Flags: 0x10, Message Type: Connect Command Msg Len: 94 Protocol Name Length: 4 Protocol Name: MQTT Version: Unknown (132) Connect Flags: 0xec, User Name Flag, Password Flag, Will Retain, QoS Level: At least once delivery (Acknowledged deliver), Will Flag 1... .... = User Name Flag: Set .1.. .... = Password Flag: Set ..1. .... = Will Retain: Set ...0 1... = QoS Level: At least once delivery (Acknowledged deliver) (1) .... .1.. = Will Flag: Set .... ..0. = Clean Session Flag: Not set .... ...0 = (Reserved): Not set Keep Alive: 60 Client ID Length: 14 Client ID: Thing.MosquittoBridge Will Topic Length: 43 Will Topic: $SYS/broker/connection/Thing.MosquittoBridge/state Will Message Length: 1 Will Message: 0 User Name Length: 6 User Name: YYYYYY Password Length: 10 Password: XXXXXXXXXX
-
The connect ACK:
MQ Telemetry Transport Protocol, Connect Ack Header Flags: 0x20, Message Type: Connect Ack 0010 .... = Message Type: Connect Ack (2) .... 0000 = Reserved: 0 Msg Len: 2 Acknowledge Flags: 0x00 0000 000. = Reserved: Not set .... ...0 = Session Present: Not set Reason Code: Success (0)
-
@akbooer
The subscribe packet:Frame 156: 81 bytes on wire (648 bits), 81 bytes captured (648 bits) on interface eth0, id 0 Ethernet II, Src: Advansus_0a:8c:3a (00:19:0f:0a:8c:3a), Dst: 96:62:08:fb:22:8a (96:62:08:fb:22:8a) Internet Protocol Version 4, Src: 10.17.2.41, Dst: 10.17.2.110 Transmission Control Protocol, Src Port: 36446, Dst Port: 1882, Seq: 147, Ack: 9, Len: 15 MQ Telemetry Transport Protocol, Unsubscribe Request Header Flags: 0xa2, Message Type: Unsubscribe Request 1010 .... = Message Type: Unsubscribe Request (10) .... 0010 = Reserved: 2 Msg Len: 5 Message Identifier: 2 Topic Length: 1 Topic: # MQ Telemetry Transport Protocol, Subscribe Request Header Flags: 0x82, Message Type: Subscribe Request 1000 .... = Message Type: Subscribe Request (8) .... 0010 = Reserved: 2 Msg Len: 6 Message Identifier: 3 Topic Length: 1 Topic: # Requested QoS: At most once delivery (Fire and Forget) (0)
-
Finally, the subscribe ACK:
MQ Telemetry Transport Protocol, Subscribe Ack Header Flags: 0x90, Message Type: Subscribe Ack 1001 .... = Message Type: Subscribe Ack (9) .... 0000 = Reserved: 0 Msg Len: 3 Message Identifier: 3 Reason Code: Granted QoS 0 (0)
-
@Buxton
Thanks so much for taking the time to capture those snapshots.What I see is:
- PUBLISH – requests QoS 0
- PINGRESP – standard fixed format
- CONNECT – v3.1.1 protocol, asks for WillQoS 1 and CleanSession 0
- CONNACK – responds with SessionPresent 0
- SUBSCRIBE – ask for QoS 0
- SUBACK - grants QoS 0
This all seems fine (as, indeed, it is) but, more importantly, appears to be quite consistent with what the openLuup MQTT server can offer.
Given, that it would be revealing to see how the Mosquitto / openLuup bridge fails...
Much appreciate the help (hence the upvotes to each of the posted images!)
AK
-
The update to include Mqtt and Shelly support is a great addition to OpenLuup!
I updated to the latest version (2021 Release 3.11b) to start testing the MQTT functionality. The first aim for me is to get data into OpenLuup from my Tasmota based sensors, and also in the next step to get my Shelly devices into OpenLuup with MQTT.
Today I use Tasmota rules, SiteSensor and http commands from the Shellies, migrating to MQTT will make this much more simple and robust.I have got it to work with setting up the server with credentials. I can see in my test Tasmota device that it connects to the server, and I can also see in MqttExplorer that it connects. So far so good!
However what I have not quite understood is how to get the subscription of device data into e.g. a virtual temp or humidity sensor in OpenLuup.
@akbooer In the instruction it says that one should use a standard Luup request handler to subscribe, where do I put that? Also how would the handler code typically look like?
As a comparison, below is what I used in the settings.json that I used fo the Mqtt Bridge from @therealdb.
"Devices": [ { "ClientID": "tasmota_test", "TopicName": "tele/tasmota_test/SENSOR", "TopicPath": "AM2301.Temperature", "DeviceID": 216, "Service": "urn:upnp-org:serviceId:TemperatureSensor1", "Variable": "CurrentTemperature" } ],
Sorry for the stupid questions!
-
There are no stupid questions... just poor documentation.
You're referring to the instructions here?
https://smarthome.community/topic/20/openluup-version-log/12?_=1615649462664
Not entirely sure where you are putting your code? Lua Startup??
-
ArcherSreplied to akbooer on Mar 13, 2021, 3:47 PM last edited by ArcherS Mar 13, 2021, 10:49 AM
@akbooer Yes it the instructions in you post February 11th.
At the moment I have not put any code anywhere since I am not sure where it should go, and what code to write!
(For the example from the bridge from @therealdb it is in a separate .json file. I think it may have been confusing of me to add it to the post. I added it just to show the topic, OL deviceId, etc for my test device.)
-
Ah, OK. If I understand it, you want to subscribe to that topic (not quite sure about the TopicPath variable) and update the device variable referenced.
So all you need, in Startup Lua is:
function MyMQTThandler (topic, message) luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", message, 216) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR/AM2301.Temperature")
I may have the topic wrong, I'm just guessing that I should concatenate the two parts in your JSON, but maybe not.
You can easily generalise this to work for more sensors and use wildcard topic subscriptions.
-
ArcherSreplied to akbooer on Mar 13, 2021, 4:18 PM last edited by ArcherS Mar 13, 2021, 11:20 AM
@akbooer Without adding the function and handler into the Startup Lua I noticed I get this error message when doing "Reload Luup engine" that I have not seen with the old OpenLuup version:
3/13/2021, 5:12:40 PM #1: openLuup:./openLuup/timers.lua:349: bad argument #1 to 'max' (number expected, got nil)
Do you know what this can be?
This is what I have added into Startup Lua:
luup.attr_set ("openLuup.MQTT.Port", 1883) luup.attr_set ("openLuup.MQTT.Username", "user") luup.attr_set ("openLuup.MQTT.Password", "passwd")
-
New to me, I haven't seen that on my three systems currently running the latest. But the error message is a start for me to look.
Speaking of errors, what I have seen is an AltUI startup error (in the delayed startup code, so not quite the first bits mentioning AltUI in the log.) I know what this is, and how to fix it, and have suggested that to amg0. But it doesn't happen on all my systems, so have yet to track down the real root cause. Would be interested if you see anything similar.
-
@akbooer this is what is in the logs surrounding the error, not sure if it could give som more clues:
2021-03-13 17:26:15.180 luup.register_handler:: global_function_name=Shelly_MQTT_Handler, request=mqtt:shellies/# 2021-03-13 17:26:15.180 openLuup.io.server:: starting MQTT:1883 server on port: 1883 tcp{server}: 0x55dba190f988 2021-03-13 17:26:15.180 openLuup.scheduler:: starting 2021-03-13 17:26:15.180 openLuup.scheduler:: [2] openLuup device startup 2021-03-13 17:26:15.180 luup_log:2: v21.3.11 2021-03-13 17:26:15.180 luup_log:2: sync in 104.8 s 2021-03-13 17:26:15.181 luup.variable_watch:: callback=housemode_watcher, watching=2.openLuup.HouseMode 2021-03-13 17:26:15.181 luup.register_handler:: global_function_name=openLuup_email, request=openLuup@openLuup.local 2021-03-13 17:26:15.181 luup.register_handler:: global_function_name=openLuup_images, request=images@openLuup.local 2021-03-13 17:26:15.181 luup.register_handler:: global_function_name=openLuup_events, request=events@openLuup.local 2021-03-13 17:26:15.181 luup.register_handler:: global_function_name=openLuup_mailbox, request=mail@openLuup.local 2021-03-13 17:26:15.181 luup.chdev.append:: [AltAppStore] Alternate App Store 2021-03-13 17:26:15.181 luup.chdev.sync:: [2] openLuup, syncing children 2021-03-13 17:26:15.181 openLuup.context_switch:: ERROR: [dev #2] ./openLuup/timers.lua:349: bad argument #1 to 'max' (number expected, got nil) 2021-03-13 17:26:15.182 openLuup.scheduler:: job aborted : ./openLuup/timers.lua:349: bad argument #1 to 'max' (number expected, got nil) 2021-03-13 17:26:15.182 openLuup.scheduler:: [3] Alternate UI device startup 2021-03-13 17:26:15.182 luup_log:3: ALTUI: initstatus(3) starting version: v2.53b 2021-03-13 17:26:15.182 openLuup.scheduler:: [3] Alternate UI device startup completed: status=, msg=, name=
-
Ah, excellent snippet from the log, giving context.
I've found it, and can fix it, but for the moment can you add to your MQTT configuration in Startup:
luup.attr_set ("openLuup.MQTT.PublishDeviceStatus", "0")
and see if that fixes it.
-
@akbooer There is still something that does not work, the device does not get updated.
This is what it says on the Tasmota device:
19:14:10 MQT: Connected 19:14:10 MQT: tele/tasmota_test/LWT = Online (retained) 19:14:10 MQT: cmnd/tasmota_test/POWER = 19:17:31 MQT: tele/tasmota_test/STATE = {"Time":"2021-03-13T19:17:31","Uptime":"0T00:15:09","UptimeSec":909,"Heap":27,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":5,"Wifi":{"AP":1,"SSId":"BeachAC","BSSId":"FC:EC:DA:D1:7A:64","Channel":11,"RSSI":88,"Signal":-56,"LinkCount":1,"Downtime":"0T00:00:03"}} 19:17:31 MQT: tele/tasmota_test/SENSOR = {"Time":"2021-03-13T19:17:31","AM2301":{"Temperature":26.2,"Humidity":43.8,"DewPoint":12.9},"TempUnit":"C"}
This is what it says in MqttExplorer:
LuaStartup:
function MyMQTThandler (topic, message) luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", message, 216) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR/AM2301.Temperature")
It does connect and it seems as if everything in the Lua Startup matches the topic etc. Still no update of the device, strange.
I need to doubble check everything and see so that there are no typos... -
No, it’s fine. I’ve just not understood the previous syntax. The Explorer output gives exactly the required info. I’ll send some new code later (busy just now.)
-
In fact, just change the topic to tele/tasmota_test/SENSOR. It should update the temperature variable with the whole JSON string, which will actually need decoding.
58/179