openLuup: MQTT server
-
Ah, excellent snippet from the log, giving context.
I've found it, and can fix it, but for the moment can you add to your MQTT configuration in Startup:
luup.attr_set ("openLuup.MQTT.PublishDeviceStatus", "0")
and see if that fixes it.
-
Ah, OK. If I understand it, you want to subscribe to that topic (not quite sure about the TopicPath variable) and update the device variable referenced.
So all you need, in Startup Lua is:
function MyMQTThandler (topic, message) luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", message, 216) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR/AM2301.Temperature")
I may have the topic wrong, I'm just guessing that I should concatenate the two parts in your JSON, but maybe not.
You can easily generalise this to work for more sensors and use wildcard topic subscriptions.
@akbooer There is still something that does not work, the device does not get updated.
This is what it says on the Tasmota device:
19:14:10 MQT: Connected 19:14:10 MQT: tele/tasmota_test/LWT = Online (retained) 19:14:10 MQT: cmnd/tasmota_test/POWER = 19:17:31 MQT: tele/tasmota_test/STATE = {"Time":"2021-03-13T19:17:31","Uptime":"0T00:15:09","UptimeSec":909,"Heap":27,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":5,"Wifi":{"AP":1,"SSId":"BeachAC","BSSId":"FC:EC:DA:D1:7A:64","Channel":11,"RSSI":88,"Signal":-56,"LinkCount":1,"Downtime":"0T00:00:03"}} 19:17:31 MQT: tele/tasmota_test/SENSOR = {"Time":"2021-03-13T19:17:31","AM2301":{"Temperature":26.2,"Humidity":43.8,"DewPoint":12.9},"TempUnit":"C"}
This is what it says in MqttExplorer:
LuaStartup:
function MyMQTThandler (topic, message) luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", message, 216) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR/AM2301.Temperature")
It does connect and it seems as if everything in the Lua Startup matches the topic etc. Still no update of the device, strange.
I need to doubble check everything and see so that there are no typos... -
In fact, just change the topic to tele/tasmota_test/SENSOR. It should update the temperature variable with the whole JSON string, which will actually need decoding.
@akbooer I changed the topic to tele/tasmota_test/SENSOR and now the temp variable is updated with the complete string:
{"Time":"2021-03-13T19:59:43","AM2301":{"Temperature":26.6,"Humidity":41.6,"DewPoint":12.5},"TempUnit":"C"}
which means that it actually gets updated.
One step in the right direction. -
The basic code, without error checking, would be:
local json = require "openLuup.json" function MyMQTThandler (topic, message) local info = json.decode (message) local t = info.AM2301.Temperature luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", t, 216) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR")
-
The basic code, without error checking, would be:
local json = require "openLuup.json" function MyMQTThandler (topic, message) local info = json.decode (message) local t = info.AM2301.Temperature luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", t, 216) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR")
@akbooer thanks for the assistance and patience, now it works!
Edit: Added the humidity to the code:
local json = require "openLuup.json" function MyMQTThandler (topic, message) local info = json.decode (message) local t = info.AM2301.Temperature local h = info.AM2301.Humidity luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", t, 216) luup.variable_set ("urn:micasaverde-com:serviceId:HumiditySensor1", "CurrentLevel", h, 228) end luup.register_handler ("MyMQTThandler", "mqtt:tele/tasmota_test/SENSOR")
-
No need to have these as parameters... just make sure they’re in scope. The best way to do this is to put them in the same closure. You could even write a factory method to do this. There’s a trade-off between having multiple handlers for specific things and one which does many things. A lot of the choice will be governed by the topic structure. If, as is likely, a single handler manages a whole topic sub-tree, then you’ll probably want to subscribe to a wildcard topic of the form aaa/bbb/#.
-
@Buxton
Thanks so much for taking the time to capture those snapshots.What I see is:
- PUBLISH – requests QoS 0
- PINGRESP – standard fixed format
- CONNECT – v3.1.1 protocol, asks for WillQoS 1 and CleanSession 0
- CONNACK – responds with SessionPresent 0
- SUBSCRIBE – ask for QoS 0
- SUBACK - grants QoS 0
This all seems fine (as, indeed, it is) but, more importantly, appears to be quite consistent with what the openLuup MQTT server can offer.
Given, that it would be revealing to see how the Mosquitto / openLuup bridge fails...
Much appreciate the help (hence the upvotes to each of the posted images!)
AK
@akbooer Yes but as you can see on the connect, the version is "Version: Unknown (132)"
This is what is causing the problem. After much searching and trying different configs, I stumbled on the following which solved the problem. From mosquitto.org
try_private [ true | false ] If try_private is set to true, the bridge will attempt to indicate to the remote broker that it is a bridge not an ordinary client. If successful, this means that loop detection will be more effective and that retained messages will be propagated correctly. Not all brokers support this feature so it may be necessary to set try_private to false if your bridge does not connect properly. Defaults to true.
So I set the attribute to false in my bridge config and immediately connected openLuup to the mosquitto broker. The connect packet shows the right version, and with a luup reload, all of my mosquitto broker topics populated in mqtt explorer that was pointed at openLuup. However, I don't see the topics in the mqtt console on openLuup?? Which is odd because I not only see the openLuup topics in explorer, but I see the topics actively changing.
I'm not a good one to suggest code changes, but since this mosquitto setting defaults to true, can you try to incorporate the try_private flag in openLuup's MQTT server.... It took a long time to track this down and I imagine anyone else that tries to connect the two servers will be in for a similar bug fix adventure.
-
@akbooer Yes but as you can see on the connect, the version is "Version: Unknown (132)"
This is what is causing the problem. After much searching and trying different configs, I stumbled on the following which solved the problem. From mosquitto.org
try_private [ true | false ] If try_private is set to true, the bridge will attempt to indicate to the remote broker that it is a bridge not an ordinary client. If successful, this means that loop detection will be more effective and that retained messages will be propagated correctly. Not all brokers support this feature so it may be necessary to set try_private to false if your bridge does not connect properly. Defaults to true.
So I set the attribute to false in my bridge config and immediately connected openLuup to the mosquitto broker. The connect packet shows the right version, and with a luup reload, all of my mosquitto broker topics populated in mqtt explorer that was pointed at openLuup. However, I don't see the topics in the mqtt console on openLuup?? Which is odd because I not only see the openLuup topics in explorer, but I see the topics actively changing.
I'm not a good one to suggest code changes, but since this mosquitto setting defaults to true, can you try to incorporate the try_private flag in openLuup's MQTT server.... It took a long time to track this down and I imagine anyone else that tries to connect the two servers will be in for a similar bug fix adventure.
-
No need to have these as parameters... just make sure they’re in scope. The best way to do this is to put them in the same closure. You could even write a factory method to do this. There’s a trade-off between having multiple handlers for specific things and one which does many things. A lot of the choice will be governed by the topic structure. If, as is likely, a single handler manages a whole topic sub-tree, then you’ll probably want to subscribe to a wildcard topic of the form aaa/bbb/#.
@akbooer when you have a lot of devices, it could become a lot of code to do the same things. I found the design of callbacks for variables watches very powerful for the very same reason. Are you saying that instead of a string for a separate function, I could just pass a closure as an argument?
-
@akbooer when you have a lot of devices, it could become a lot of code to do the same things. I found the design of callbacks for variables watches very powerful for the very same reason. Are you saying that instead of a string for a separate function, I could just pass a closure as an argument?
@therealdb said in openLuup: MQTT server:
Are you saying that instead of a string for a separate function, I could just pass a closure as an argument?
No, not that, although at the openLuup object-level API you can just pass a local function, rather than a string with the global function name.
I guess I’m not seeing your difficulty, can you supply a (pseudo-)code example of what you’d like to be able to do?
-
@akbooer Yes but as you can see on the connect, the version is "Version: Unknown (132)"
This is what is causing the problem. After much searching and trying different configs, I stumbled on the following which solved the problem. From mosquitto.org
try_private [ true | false ] If try_private is set to true, the bridge will attempt to indicate to the remote broker that it is a bridge not an ordinary client. If successful, this means that loop detection will be more effective and that retained messages will be propagated correctly. Not all brokers support this feature so it may be necessary to set try_private to false if your bridge does not connect properly. Defaults to true.
So I set the attribute to false in my bridge config and immediately connected openLuup to the mosquitto broker. The connect packet shows the right version, and with a luup reload, all of my mosquitto broker topics populated in mqtt explorer that was pointed at openLuup. However, I don't see the topics in the mqtt console on openLuup?? Which is odd because I not only see the openLuup topics in explorer, but I see the topics actively changing.
I'm not a good one to suggest code changes, but since this mosquitto setting defaults to true, can you try to incorporate the try_private flag in openLuup's MQTT server.... It took a long time to track this down and I imagine anyone else that tries to connect the two servers will be in for a similar bug fix adventure.
@buxton said in openLuup: MQTT server:
This is what is causing the problem. After much searching and trying different configs, I stumbled on the following which solved the problem.
Thanks SO MUCH for all the hard work, this is gold dust! I’ll see what I can do.
However, I don't see the topics in the mqtt console on openLuup?? Which is odd because I not only see the openLuup topics in explorer, but I see the topics actively changing.
Not an issue, I think. This is a list of SUBSCRIBED topics, not PUBLISHED ones. So I think it is working. A little test code with a wildcard subscription could confirm.
I'm not a good one to suggest code changes, but since this mosquitto setting defaults to true, can you try to incorporate the try_private flag in openLuup's MQTT server.... It took a long time to track this down and I imagine anyone else that tries to connect the two servers will be in for a similar bug fix adventure.
Absolutely... but I’m still not understanding what the difference is... I will dig further.
Many thanks again.
-
@akbooer something like this
function MyMQTThandler (topic, msg, state) local info = json.decode (message) local t = state.path luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", t, state.deviceID) end luup.register_handler ("MyMQTThandler", "mqtt:tele/#/SENSOR", {deviceID=255, description ="outdoor", path = "info.AM2301.Temperature"}) luup.register_handler ("MyMQTThandler", "mqtt:tele/#/SENSOR", {deviceID=266, description ="indoor", path = "info.DS18B20.Temperature"})
Passing a generic state will make handlers more compact and reusable.
-
@buxton said in openLuup: MQTT server:
This is what is causing the problem. After much searching and trying different configs, I stumbled on the following which solved the problem.
Thanks SO MUCH for all the hard work, this is gold dust! I’ll see what I can do.
However, I don't see the topics in the mqtt console on openLuup?? Which is odd because I not only see the openLuup topics in explorer, but I see the topics actively changing.
Not an issue, I think. This is a list of SUBSCRIBED topics, not PUBLISHED ones. So I think it is working. A little test code with a wildcard subscription could confirm.
I'm not a good one to suggest code changes, but since this mosquitto setting defaults to true, can you try to incorporate the try_private flag in openLuup's MQTT server.... It took a long time to track this down and I imagine anyone else that tries to connect the two servers will be in for a similar bug fix adventure.
Absolutely... but I’m still not understanding what the difference is... I will dig further.
Many thanks again.
@akbooer said in openLuup: MQTT server:
... I will dig further.
...and have now done so.
@Buxton, would it be at all possible to try this (latest development v21.3.14b) with the original bridging configuration? My implementation now, effectively, ignores the try_private flag, so will still behave as an ordinary client. Any magic of topic filtering, and the like, will still have to be configured on Mosquitto (if required.) It might be wise to limit the topics sent to those which are actually needed.
-
@akbooer something like this
function MyMQTThandler (topic, msg, state) local info = json.decode (message) local t = state.path luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", t, state.deviceID) end luup.register_handler ("MyMQTThandler", "mqtt:tele/#/SENSOR", {deviceID=255, description ="outdoor", path = "info.AM2301.Temperature"}) luup.register_handler ("MyMQTThandler", "mqtt:tele/#/SENSOR", {deviceID=266, description ="indoor", path = "info.DS18B20.Temperature"})
Passing a generic state will make handlers more compact and reusable.
@therealdb said in openLuup: MQTT server:
something like this
I see.
Well, as I said, you can do it right now like this:
local mqtt = require "openLuup.mqtt" local function MyMQTThandler (info) return function (topic, message) local state = json.decode (message) local t = state.path luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature", t, state.deviceID) end end mqtt.register_handler (MyMQTThandler {deviceID=255, description ="outdoor", path = "info.AM2301.Temperature"}, "tele/+/SENSOR") mqtt.register_handler (MyMQTThandler {deviceID=266, description ="indoor", path = "info.DS18B20.Temperature"}, "tele/+/SENSOR")
So, I obviously haven't been able to test exactly this, but I tried instead:
function MyHandler (state) return function (topic, message) luup.log ((json.encode {MQTTtest = { topic = topic, message = message, state = state}})) end end local mqtt = require "openLuup.mqtt" mqtt.register_handler (MyHandler ("the rain Spain"), "shellies/#") mqtt.register_handler (MyHandler {"the", "rain","in","Italy"}, "shellies/#")
So there are two calls to
MyHandler
, which is actually a factory methodsto produce callbacks with thestate
parameter in the closure of that function. The handler itself simply logs its parameters as a JSON structure. An extract from the log shows that each has been called, in response to a message published by a Shelly, but with their ownstate
(one is a string, the other an array of strings.)2021-03-14 17:25:26.217 luup_log:0: {"MQTTtest":{ "message":"0", "state":"the rain Spain", "topic":"shellies/shellyix3-68C63AFA97F5/input/0" }} 2021-03-14 17:25:26.217 luup_log:0: {"MQTTtest":{ "message":"0", "state":["the","rain","in","Italy"], "topic":"shellies/shellyix3-68C63AFA97F5/input/0" }}
The only small issue I see is that you used this wildcard subscription topic:
tele/#/SENSOR
which isn't valid for MQTT, I think you meant:tele/+/SENSOR
, and the other small issue with that is that I haven't (yet) implemented the '+' wildcard, although you can certainly use '/#' at the end of a topic, as I did in the above example.Note, also, that when calling the object-level method in openLuup, you don't use the
mqtt:
prefix, but just the topic itself (since you are directly calling the MQTT request_handler function.)This might seem a bit cumbersome, and I will take a look at adding the extra parameter in the way that you suggest.