openweather plugin ?

[sender]

Maybe you know, maybe you don't. I use this explore to view real time events and topic updates on my broker:

MQTT Explorer

Very handy to discover topics, see updates etc.

[rafale77]

Thank you,

It is actually a forked version of the plugin @akbooer shared as well and...

"This plugin provides the ability to publish out any user defined variable to an MQTT Broker."

Which again is not what I am trying to do. I am trying to get a client to subscribe and read messages. Not to publish which involves for the plugin to be constantly connected to the broker and listen.

[kfxo]

@rafale77 It does allow you to subscribe.

You can subscribe to a topic by creating a child device :

Add a new device and choose a type (e.g. "D_BinaryLight1.xml" or "D_TemperatureSensor1.xml").
Reload LUUP engine.
Change the attribut "id_parent" with the id of the MQTT plugin device.
Reload LUUP engine and refresh your browser.
You should see variables "mqttTarget" and "mqttTopic" in your newly created device.
Set the topic you want to subcribe to and the target (format: service,variable=(formula in LUA)).
Reload LUUP engine.
If the payload of the received message is in JSON, the plugin will try to decode it and put it in the variable "payload" in the context of the LUA formula.

[rafale77]

I will look into it. Thanks... I am so close with my own plugin though...

[kfxo]

I thought it might help with writing your plugin. Looking forward to see what you come up with.

[rafale77]

Well I learned something from it. My problem with opening the listening client was not due to cpu threading but because it was blocking the socket. Unless I want to proceed with a polling strategy, this particular library is not going to work... Ohh well, back to square one with a different library.

[therealdb]

Slightly OT. Overall I like lua, but I miss the simple approach to non-blocking code offered by c# and even JavaScript. That’s particularly bad when doing network/http call, because it’s very difficult to write async code.

[akbooer]

There’s no fundamental difficulty in writing non-blocking I/O. The openLuup server module and scheduler allow callbacks on incoming. The HTTP async code has a separate module simply because of its entanglement within the LuaSocket HTTP module.

[therealdb]

@akbooer said in MQTT Plugin:

There’s no fundamental difficulty in writing non-blocking I/O.

well, I was speaking about an async/await first approach, with lambdas/promises. co-routines are just OK.
maybe I'm too biased because I'm used to the other approach and I still spend most of my time writing C#/Javascript instead of LUA. I wish it was simpler - let's say - to execute 3 HTTP calls under Vera in parallel, each having its own response handler inline.

[akbooer]

@therealdb said in MQTT Plugin:

I wish it was simpler - let's say - to execute 3 HTTP calls under Vera in parallel, each having its own response handler inline.

So this isn't good enough...?

local async = require "http_async"

async.request ("http://request1",
  function  (response, code, headers, statusline)
    -- response code 1 here
  end)

async.request ("http://request2",
  function  (response, code, headers, statusline)
    -- response code 2 here
  end)

async.request ("http://request3",
  function  (response, code, headers, statusline)
    -- response code 3 here
  end)

[therealdb]

Oh, I missed this.
It could greatly benefit to my own HTTP plug-in. Any plan to release as a plug-in on Vera itself?

[akbooer]

Yes, it already works on Vera.

[akbooer]

...if you excuse a link to the other place:

May 11, 2019

openLuup: Asynchronous I/O

The latest openLuup development release has a new file in the openLuup/ folder: http_async.lua. This now implements asynchronous HTTP and HTTPS requests (depending on which scheme is provided in the requested URL.) If you want to use it, you simply require it in your code like this: local async...

[rafale77]

Well my frustration is not so much with http, though I did have my run ins with it and ended up using libcurl due to... digest not working somehow on luajit with luasocket but at this moment, I am finding that:

1. the luarocks version of the MQTT library uses the luasocket in a blocking fashion. It does connect fine to the MQTT broker though.
2. the vera-mqtt plugin is full of little bugs causing it to not work as a subscription client. For example the library fails to declare the socket variable, It crashes with an empty socket call to SSL when SSL is not needed etc... I have not been able to make it connect to the mosquitto broker even after fixing all the problems I found...

Sadly, I may have to move back to python and have home-assistant being the MQTT bridge.

[therealdb]

As I already said, I wrote my own bridge in c#. I might make it more general purpose and release it. It’s a process running inside .net core and as ha-bridge could be installed separately, but pointing to openluup.
Let’s see if I find some motivation. It’s very lightweight, since it’s just one library running in a console app, and .NET Core 3.1 is very performant.

[kfxo]

@rafale77 said in MQTT Plugin:

2. the vera-mqtt plugin is full of little bugs causing it to not work as a subscription client. For example the library fails to declare the socket variable, It crashes with an empty socket call to SSL when SSL is not needed etc... I have not been able to make it connect to the mosquitto broker even after fixing all the problems I found...

Sadly, I may have to move back to python and have home-assistant being the MQTT bridge.

I have had good success with the plugin I linked to above. Currently I have 9 different child devices subscribing to at least 15 different topics. I did modify the plugin slightly, not to get it to work, but to allow for child devices to have any designated number of subscriptions, include availability, state, and command topic subscriptions, and I added actions/functions for reconnecting to the mqtt server and reconnecting subscriptions.

[rafale77]

Well I made some progress and got it to connect to the broker but it is extremely fragile. Digging into it deeper, it actually relies on a periodic call of a "handler" function in the library to check for messages from the broker. The call fails regularly and is not very well written in the sense that it defeats the purpose of MQTT by doing a poll.
There are a number of forks of this library which attempted various fixes and they have had various degree of success.
Not judging the plugin itself but rather the MQTT library. I actually took quite a bit from the plugin for these tests. Still work in progress but I am pretty pessimistic.

[rafale77]

Update: I now have a kind of a working solution and it is interesting...
The fundamental problem is that setting up an MQTT client to listen to the broker blocks the luasocket and therefore openLuup.
The two libraries use different approaches to the problem.
The older library used by the vera MQTT plugin uses a handler function which polls the broker and gets and closes immediately regardless of getting a message or not. It therefore won't block the luasocket but it comes back empty 99% of the time and really relies on luck to catch a message as it can miss them. Basically unusable.
The newer lua library does the same but waits for a message back and blocks the luasocket while waiting. If one sets a high polling frequency without getting a message back, openLuup hangs. My solution to this has been to significantly increase the frequency of messages from the publishing client to unblock the socket. It really is no longer event based and this generates a lot of traffic.

If anyone has better suggestions (@therealdb I look forward to your bridge). I am also thinking that openLuup could use moving all its http traffic from luasocket to libcurl...

Edit:

I ended up giving up on trying to make it work with luasocket and spent 2min setting up 4 MQTT subscriptions on Home Assistant and another 10 to build the bridges back into openLuup. I really hoped to be able to do have a direct pipe into openLuup...

[therealdb]

@rafale77 right now I'm using it to process incoming messages and update variables on Vera/Openluup. Do you want to call service endpoints as well? I can make it really generic and call HTTP endpoints, if needed. So you can
I don't use it so much to send messages, but I think we could add this later, if needed.
I can also try to put a relay feature, to receive incoming messages and then publish them back to another broker.

It's based on a JSON config file and you have to map it manually, but then it's transparent. Give me a couple of days to polish and I'll publish it on GitHub, with instructions on how to build, and install it.

[akbooer]

@rafale77 said in MQTT Plugin:

The fundamental problem is that setting up an MQTT client to listen to the broker blocks the luasocket and therefore openLuup.
The two libraries use different approaches to the problem.

This just sounds like a lack of understanding on behalf of the module authors, ...or possibly me. I can’t believe that it could possibly be so difficult.

These are the two libraries you referenced st the start?