openLuup email server

[therealdb]

@akbooer something like this

function MyMQTThandler (topic, msg, state)
   local info = json.decode (message)
   local t = state.path
   luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature",  t, state.deviceID)
  
 end
  
 luup.register_handler ("MyMQTThandler", "mqtt:tele/#/SENSOR", {deviceID=255, description ="outdoor", path = "info.AM2301.Temperature"})

luup.register_handler ("MyMQTThandler", "mqtt:tele/#/SENSOR", {deviceID=266, description ="indoor", path = "info.DS18B20.Temperature"})

Passing a generic state will make handlers more compact and reusable.

[akbooer]

@akbooer said in openLuup: MQTT server:

... I will dig further.

...and have now done so.

@Buxton, would it be at all possible to try this (latest development v21.3.14b) with the original bridging configuration? My implementation now, effectively, ignores the try_private flag, so will still behave as an ordinary client. Any magic of topic filtering, and the like, will still have to be configured on Mosquitto (if required.) It might be wise to limit the topics sent to those which are actually needed.

[akbooer]

@therealdb said in openLuup: MQTT server:

something like this

I see.

Well, as I said, you can do it right now like this:

local mqtt = require "openLuup.mqtt"

local function MyMQTThandler (info)
  return function (topic, message)
    local state = json.decode (message)
    local t = state.path
    luup.variable_set ("urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature",  t, state.deviceID)
  end
end
 
mqtt.register_handler (MyMQTThandler {deviceID=255, description ="outdoor", path = "info.AM2301.Temperature"}, "tele/+/SENSOR")
mqtt.register_handler (MyMQTThandler {deviceID=266, description ="indoor", path = "info.DS18B20.Temperature"}, "tele/+/SENSOR")

So, I obviously haven't been able to test exactly this, but I tried instead:

function MyHandler (state)
  return function (topic, message)
    luup.log ((json.encode {MQTTtest = {
      topic = topic, 
      message = message,
      state = state}}))
  end
end

local mqtt = require "openLuup.mqtt"
mqtt.register_handler (MyHandler ("the rain Spain"), "shellies/#")
mqtt.register_handler (MyHandler {"the", "rain","in","Italy"}, "shellies/#")

So there are two calls to MyHandler, which is actually a factory methodsto produce callbacks with the state parameter in the closure of that function. The handler itself simply logs its parameters as a JSON structure. An extract from the log shows that each has been called, in response to a message published by a Shelly, but with their own state (one is a string, the other an array of strings.)

2021-03-14 17:25:26.217   luup_log:0: {"MQTTtest":{
    "message":"0",
    "state":"the rain Spain",
    "topic":"shellies/shellyix3-68C63AFA97F5/input/0"
  }}
2021-03-14 17:25:26.217   luup_log:0: {"MQTTtest":{
    "message":"0",
    "state":["the","rain","in","Italy"],
    "topic":"shellies/shellyix3-68C63AFA97F5/input/0"
  }}

The only small issue I see is that you used this wildcard subscription topic: tele/#/SENSOR which isn't valid for MQTT, I think you meant: tele/+/SENSOR, and the other small issue with that is that I haven't (yet) implemented the '+' wildcard, although you can certainly use '/#' at the end of a topic, as I did in the above example.

Note, also, that when calling the object-level method in openLuup, you don't use the mqtt: prefix, but just the topic itself (since you are directly calling the MQTT request_handler function.)

This might seem a bit cumbersome, and I will take a look at adding the extra parameter in the way that you suggest.

[therealdb]

Thanks for the clarification! I was obviously referring to + instead of #.

The idea is to write very generic routines to push sensors data.

[Buxton]

@akbooer Yes, I was thinking along those lines as the bridge config allows filters. The latest openLuup version now works fine with try_private flag set to default (true). Thanks for nailing this down and your work is definitely appreciated. Below is the connection to openLuup.

Here's my Mosquitto config for anyone who wants to bridge the two brokers:

allow_anonymous true
password_file /mosquitto/data/PW.txt
listener 1883
connection openLuup
address 127.0.0.1:1882
topic # out 0
topic # in 0
cleansession false
notifications true
username XXXXX
password YYYYYYYYYY
bridge_protocol_version mqttv311

Most of these settings can/should be modified to suit one's particular needs, but the settings should be employed. The password file for mosquitto needs to be encrypted with mosquitto's built-in encryption tool. The directions are straightforward and are described in on-line documents.

[akbooer]

Excellent news – thanks again for all the detective work.

[akbooer]

@therealdb said in openLuup: MQTT server:

The idea is to write very generic routines to push sensors data.

The latest development version 21.3.19 supports an additional parameter to luup.register_handler() as per your suggestion. For which, thanks.

Lua Startup:

function test_MQTT_callback (...)
  luup.log ((json.encode {...}))
end

luup.register_handler ("test_MQTT_callback", "mqtt:test", {a = "A", b = "B"})
luup.register_handler ("test_MQTT_callback", "mqtt:test", {c = "C", d = "D"})

Lua Log:

2021-03-19 16:28:21.850   luup_log:0: ["test","hello world",{
  "a":"A",
  "b":"B"
}]
2021-03-19 16:28:21.850   luup_log:0: ["test","hello world",{
  "c":"C",
  "d":"D"
}]

[therealdb]

Thanks! That's great!

[PerH]

Seems like the openLuup MQTT server works well now, i'm just wondering if theres a plan to make a UI for configuration of this? Perhaps as a plugin visually, with the possibility of adding sensors that is linked to a MQTT subscription? mabye even autogeneration of sensors/buttons?

I haven't dug in to this myself yet, but the shelly like sure looks interesting, and there's offcourse a lot of other use cases too..

[akbooer]

Glad to hear that things are working well for you. It is for me too. Here's my server stats for the last 5 days that the system has been up (no related errors in the log, either)

I'm not great at UIs, designing or building, but what did you have in mind? I've tried to make the "device as sensor" direction as easy as possible, since ALL variable changes can be published/subscribed by simply setting the openLuup.MQTT.PublishVariableUpdates attribute to true.

For the other direction (control/display) you can already turn devices on/off with the relay/deviceNo topic with arguments of 0 or 1. But you're wanting to create arbitrary devices which can display and/or be controlled by MQTT topic message contents?

One thing I have learned about this sort of configuration UI functionality is that it can be very time consuming to implement and it gets used very, very rarely. So sometimes a little bit of manual configuration is worth tolerating.

Edit: I should add that the Shelly MQTT bridge does generate and configure newly discovered devices completely automatically – but this is mainly due to the rich and carefully designed topic/message structure which they employ. Tasmotas could be made to do the same and, indeed, the basic infrastructure for doing so is already there too.

[PerH]

I'm just thinking that generally, the functionality will probably reach out to more people without editing code to make it work.

Say you have a Plugin device called MQTT server, with a control panel with an "Enable server" button. (This could offcourse just happen when you add the plugin in "plugins" tab, like with Vera Bridge?)
Below could have a table list of registered topics and subscriptions (and attached child device, if existing), with the possibility to add/remove/ignore?
A button for "Create Device" with entrys for "Device type", "topic", ... <-whatever you need to assign the recieved/sent value to a sensor/button.

Again, i'm not fully versed in how this works yet, "just spitballing".

[akbooer]

Sounds like we need just a small extension to the Virtual HTTP device plugin? Let's ask @therealdb

[Elcid]

Hi @akbooer , I installed the latest developement version of openLuup and have enabled mqtt on an I3 shelly device, I have not set any user or password. The issue I have is that openluup has not discovered the device. In the mqqt server page I have 1 internal mqtt connection and the publised/messages/ received is currently at 24000. I have set port in startup lua to 1884 and shelly is configured to 1884. Any Idea of my error may be?

[akbooer]

What settings have you used in your Lua Startup to enable the MQTT server? Specifically, have you set openLuup.MQTT.Port?

[therealdb]

@akbooer said in openLuup: MQTT server:

Sounds like we need just a small extension to the Virtual HTTP device plugin? Let's ask @therealdb

I was thinking of adding special support for mqtt under openluup, with protocol handler mqtt:// under virtual device for actions. This will work for you?

[akbooer]

Sounds about right!

[Elcid]

@akbooer
yes I set port to 1884
luup.attr_set ("openLuup.MQTT.Port", 1884)

[akbooer]

...and when you say "enabled mqtt on an I3 shelly device", that means that you have also set the Server IP parameter of the Shelly device to point to the openLuup IP and the 1884 port:

(for me it's 1883)

and you are just keeping all the other defaults on Shelly?

I'd like to see the main log after a system reload, preferably for a few minutes to give the Shelly time to update its status over MQTT.

[Elcid]

@akbooer
the device is configured as

and log shows

openLuup.historian:: using memory cache size (per-variable): 1024
2021-04-13 21:09:55.722   luup.register_handler:: global_function_name=Shelly_MQTT_Handler, request=mqtt:shellies/#
2021-04-13 21:09:55.737   luup.register_handler:: global_function_name=Tasmota_MQTT_Handler, request=mqtt:tele/#
2021-04-13 21:09:55.738   luup.register_handler:: global_function_name=Tasmota_MQTT_Handler, request=mqtt:cmnd/#
2021-04-13 21:09:55.738   luup.register_handler:: global_function_name=Tasmota_MQTT_Handler, request=mqtt:stat/#
2021-04-13 21:09:55.739   openLuup.io.server:: starting MQTT:1884 server on port: 1884 tcp{server}: 0xf3f88c8
2021-04-13 21:09:55.744   openLuup.scheduler:: starting
2021-04-13 21:09:55.747   openLuup.scheduler:: [2]     openLuup device startup
2021-04-13 21:09:55.748   luup_log:2: v21.4.8
2021-04-13 21:09:55.749   luup_log:2: sync in 4.3 s
2021-04-13 21:09:55.751   luup.variable_watch:: callback=housemode_watcher, watching=2.openLuup.HouseMode
2021-04-13 21:09:55.751   luup.register_handler:: global_function_name=openLuup_email, request=openLuup@openLuup.local
2021-04-13 21:09:55.752   luup.register_handler:: global_function_name=openLuup_images, request=images@openLuup.local
2021-04-13 21:09:55.753   luup.register_handler:: global_function_name=openLuup_events, request=events@openLuup.local
2021-04-13 21:09:55.754   luup.register_handler:: global_function_name=openLuup_mailbox, request=mail@openLuup.local
2021-04-13 21:09:55.754   luup.chdev.append:: [AltAppStore] Alternate App Store
2021-04-13 21:09:55.755   luup.chdev.sync:: [2]     openLuup, syncing children
2021-04-13 21:09:55.756   luup_log:2: starting MQTT $SYS/broker statistics
2021-04-13 21:09:55.760   luup_log:2: 7Mb, 1.2%cpu, 0.0days

no other mention of mqtt.

this is from reload of luup and waiting 5 mins.

[akbooer]

Do you have MQTT Explorer, and can you see messages sent by Shelly there?