openLuup email server

[PerH]

Ok, think i found an issue?

All I did in openluup was to start MQTT in the lua_startup.

Then tried to connect something, and suddenly it started running wild with alot of these messages (and CPU at 95%):

2021-02-12 17:50:51.847   luup.incoming_callback:: function: 0x55694ad24e40 ERROR: table: 0x55694b5b18a0
2021-02-12 17:50:51.847   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b2257c0
2021-02-12 17:50:51.847   luup.incoming_callback:: function: 0x55694ac43980 ERROR: table: 0x55694b2257c0
2021-02-12 17:50:51.847   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b225ea0
2021-02-12 17:50:51.847   luup.incoming_callback:: function: 0x55694ad24e40 ERROR: table: 0x55694b225ea0
2021-02-12 17:50:51.848   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b13bae0
2021-02-12 17:50:51.848   luup.incoming_callback:: function: 0x55694ac43980 ERROR: table: 0x55694b13bae0
2021-02-12 17:50:51.848   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b13c200
2021-02-12 17:50:51.848   luup.incoming_callback:: function: 0x55694ad24e40 ERROR: table: 0x55694b13c200
2021-02-12 17:50:51.848   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b139d60
2021-02-12 17:50:51.848   luup.incoming_callback:: function: 0x55694ac43980 ERROR: table: 0x55694b139d60
2021-02-12 17:50:51.848   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b13a440
2021-02-12 17:50:51.848   luup.incoming_callback:: function: 0x55694ad24e40 ERROR: table: 0x55694b13a440
2021-02-12 17:50:51.848   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b9519a0
2021-02-12 17:50:51.848   luup.incoming_callback:: function: 0x55694ac43980 ERROR: table: 0x55694b9519a0
2021-02-12 17:50:51.848   openLuup.context_switch::  ERROR: [dev #0] table: 0x55694b952080
2021-02-12 17:50:51.848   luup.incoming_callback:: function: 0x55694ad24e40 ERROR: table: 0x55694b952080

When I stopped the sender (stopped the container it was running in), the errors kept coming until I reloaded luup.

The sender was this, which is a program that parses data from Tibber Pulse into JSON. Tibber Pulse sends instant power data from my house power input.
It has two tobpics, do i have to establish these in the broker first?

[rafale77]

Already a feature request: could the broker force users to use credentials like mosquitto does?

[akbooer]

@rafale77 said in openLuup: MQTT server:

could the broker force users to use credentials like mosquitto does?

Absolutely not a problem. How would you like to be able to specify the credentials.

On your other problem... I’m working on it.

[rafale77]

Credentials: Could be attributes of openLuup? somehow encoded?

[PerH]

Question: does the broker have to have the topics ready, or will it create them when some unit connects with topics?

[rafale77]

I believe topics are created by subscribers/publishers. They are a "header" to the "message".

[akbooer]

The server/broker needs no configuration aside from the port number (except, soon, for defining login credentials.)

The joy of MQTT is that you don’t need to know who is listening, where they are (IP address), who is sending, or what they’re sending. It’s just that if you’re subscribed to a topic, and someone publishes an application message with that topic, then you receive it.

[akbooer]

@rafale77

The latest development release of openLuup, v21.2.16 should have fixed the error condition that you were seeing. I was using the socket.try() function incorrectly in an attempt to catch errors in a better way!

I’ve also fixed the multiple subscriber issue that I was seeing. Still some more to do, including login credentials, but would appreciate any feedback at this stage.

The Shelly bridge also seems to be working well with it... very responsive.

[rafale77]

Sounds good, Will get on it in the next few hours. I just finished refactoring a bunch of my scene lua code moving a couple of global state variables into virtual switch devices. I am trying to not convolute too many changes.

[rafale77]

@akbooer

Looking good so far! Will run more tests today.

Now I am thinking how much you could do by making the console an mqtt client and channel the console as a UI to openluup....
It is a minor thing but ALTUI, relying on regular http API polling, is actually pretty taxing on the server. If only it could use websocket or MQTT...
And z-way bridge could use this too!

[akbooer]

Yes, I know, I know... MQTT is so simple, but it opens up so many possibilities (and a lot more rewriting work to go along with that!)

I should update my ZWay installation and see what the MQTT server offers. The Shelly bridge is by far the simplest bridge implementation I've ever done, just because it uses MQTT.

BTW, I've implemented login credentials for the server and that'll be in the next release.

[Buxton]

Hey AK,

Good to see you back in the saddle! I use MQTT quite a bit in Home Assistant and though I wanted to do the same in openLuup, the MQTT plugin was not cutting it for me as the polling was slow to catch updates. I too need credentials built in to the MQTT openLuup client as the latest docker mosquito build now more or less mandates a user name and PW. I have all my devices working with the updated broker so I don't want to roll back to test in openLuup. Let me know when you finish credentials and I'll pull down your changes.

[akbooer]

Hi there,

Oh, they’re there already...

@akbooer said in openLuup: Version Log:

Development Branch: 2021 Release 2.18

MQTT server now supports login credentials.

To set Username and Password for MQTT connections use the following commands in Lua Startup:

luup.attr_set ("openLuup.MQTT.Username", "---username---")
luup.attr_set ("openLuup.MQTT.Password", "---password---")

Note, though, that this implementation in openLuup is an MQTT server (broker) and not a client.

[Buxton]

@akbooer ok thx. Mosquito allows bridging so I need to play around with things to see what works best. The goal will be virtual switches in openLuup that will map completely to my tasmota devices. That way I can integrate things like reactor with MQTT traffic.

[akbooer]

That should work, although I’ve not tried it. The handling of Shelly virtual devices with MQTT is very slick, so hopefully you’ll have a similar experience.

[Buxton]

I get a partial connection:

2021-03-01 23:01:15.829 openLuup.io.server:: MQTT:1882 connection closed tcp{client}: 0x559e58efeff8
2021-03-01 23:01:15.829 openLuup.mqtt:: RECEIVE ERROR: closed tcp{client}: 0x559e58efeff8
2021-03-01 23:01:21.819 openLuup.io.server:: MQTT:1882 connection from 10.17.2.40 tcp{client}: 0x559e583ea0b8

It took a while to get that far as networking with docker can be difficult, and mosquitto was passing its container network address to openLuup--an address that is on a completely different subnet to the host machine running docker. Anyway, this is about as far as I can go with out some sort of packet analysis.

[akbooer]

OK, thanks for that. You can turn on extra debug log messages with this Lua Startup line

luup.openLuup.mqtt.ABOUT.DEBUG = true

[akbooer]

@Buxton

Also been looking at the Mosquitto configuration documentation here (the section on Bridging Controllers):

mosquitto.conf man page

I note that there are username/password defaults which may not be appropriate.

It’s also possible that it relies on some $SYS topics, but debug logging should show this. I’d like to see the log of the original connection from your broker.

[Buxton]

The mosquitto log just shows the connection attempts. In the below, my mosquitto bridge is named openLuup and I'm running openLuup on the same machine as my docker host so the IP address is the same--10.17.2.40. Port 1882 is the openLuup MQTT port

1614713171: Connecting bridge openLuup (10.17.2.40:1882)
1614713237: Connecting bridge openLuup (10.17.2.40:1882)
1614713303: Connecting bridge openLuup (10.17.2.40:1882)
1614713369: Connecting bridge openLuup (10.17.2.40:1882)
1614713435: Connecting bridge openLuup (10.17.2.40:1882)

The docker compose file is below. I had to change to "network_mode: host" as that was the only way to forward the machine host IP address to openLuup. My other MQTT clients specify the broker IP in a device field, so there was no need to expose the entire container before.

mosquitto:
container_name: mosquitto
image: eclipse-mosquitto:latest
restart: always
depends_on:

• hass

expose:

- "1883"

- "9001"

ports:

- "1883:1883"

- "9001:9001"

network_mode: host
volumes:

• /home/kevinb/mosquitto/data:/mosquitto/data
• /home/kevinb/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf
• /home/kevinb/mosquitto/log:/mosquitto/log
  environment:
• TZ=US/Pacific

The mosquitto config is below:

allow_anonymous true
password_file /mosquitto/config/PW.txt
listener 1883
connection openLuup
address 10.17.2.40:1882
topic # out 0
topic # in 0
cleansession false
notifications true
username yyyyyyy
password xxxxxxx

I will check packets later with wireshark when I have some free time.
Thx

[Buxton]

LOL the large font above is from using a pound sign to comment out the line.