openLuup email server

[ArcherS]

@akbooer installed v21.4.26 on my test Pi and it seems to work.
The AltUi tags are back. The Shellies seem to work, also the Tasmota seem to update.
No errors in the log so far.

[CatmanV2]

@akbooer said in openLuup: MQTT server:

FWIW, I've now updated, and hopefully corrected, the code – v21.4.26 is the latest development version. So, I'd be very grateful if anyone is brave enough to test this.

I'm really glad of this error being highlighted, beause it's caused me to re-think somewhat the openLuup object API, and I'm rather happier with the outcome now. No guarantees, of course, that this fixes all the problems, but I hope that it is a step in the right direction.

PS: One of my systems exhibited the AltUI missing buttons, but, curiously, one did not. However, they both appear to work OK after this update.

Is this ready for prime time? No biggy if not, yet

Cheers

C

[akbooer]

@catmanv2

I’m using it in my 'production' system. I’m not planning any major enhancements (indeed, there isn’t much more to add) and all the Shelly and Tasmota bridges need is support for more devices.

So it works for me (I don’t bridge to another MQTT server myself.)

[CatmanV2]

Thanks. Updated OpenLuup to 21.4.26 and the tags are back

C

[powisquare]

Hi guys - just received some Shelly H & Ts. Is there any documentation on how to connect these to mqtt (in novice speak )? Have read the thread but am getting slightly confused - do I need just the openluup mqtt server or another plugin also?

[akbooer]

No other plug-in required… Shelly and Tasmota plugins are built-in. For Shellies, you just need to plug in the IP address of openLuup as the MQTT server.

[powisquare]

Will I need to set ports or do anything in Lua Startup still?

[akbooer]

@powisquare

Yes, you need simply to set up a port for the MQTT server

luup.attr_set ("openLuup.MQTT.Port", 1883)

In Lua Startup, and set the same for the Shellies.

[powisquare]

OMG. Genius. Many thanks indeed! Note to self - for Shelly .. Internet & Security/Advanced - Developer Settings/Enable MQTT

[powisquare]

The first Shelly I tried worked with the bridge created and device created in the Shellies room. The next ones also appear in openLuup/Servers/Received Connections list but don't appear as devices. Have I missed a step?

[akbooer]

No, I don't think so. Where are you looking at the devices (AltUI / openLuup console)?

Anything in the log?

[powisquare]

Should have looked first ! Red highlights a win here; am seeing this ... (is there a proper way of copying and pasting?)

2021-06-12 11:10:01.904 openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=492370363&Timeout=60&MinimumDelay=1500&_=1623492371898 HTTP/1.1 tcp{client}: 0x1e96db0
2021-06-12 11:10:19.108 openLuup.io.server:: MQTT:1883 connection from 192.168.1.156 tcp{client}: 0x1d97790
2021-06-12 11:10:19.152 openLuup.mqtt:: shellyht-E01A8B SUBSCRIBE to shellies/command tcp{client}: 0x1d97790
2021-06-12 11:10:19.153 openLuup.mqtt:: shellyht-E01A8B SUBSCRIBE to shellies/shellyht-E01A8B/command tcp{client}: 0x1d97790
2021-06-12 11:10:19.156 luup.shelly:83: New Shelly announced: shellyht-E01A8B
2021-06-12 11:10:19.279 openLuup.context_switch:: ERROR: [dev #83] ./openLuup/client.lua:96: missing realm or nonce in received WWW-Authenticate header
2021-06-12 11:10:19.280 openLuup.mqtt:: ERROR publishing application message for mqtt:shellies/announce : ./openLuup/client.lua:96: missing realm or nonce in received WWW-Authenticate header
2021-06-12 11:10:19.393 openLuup.server:: request completed (1891 bytes, 1 chunks, 17488 ms) tcp{client}: 0x1e96db0

[akbooer]

@powisquare said in openLuup: MQTT server:

Should have looked first ! Red highlights a win here; am seeing this ... (is there a proper way of copying and pasting?)

2021-06-12 11:10:01.904 openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=492370363&Timeout=60&MinimumDelay=1500&_=1623492371898 HTTP/1.1 tcp{client}: 0x1e96db0
2021-06-12 11:10:19.108 openLuup.io.server:: MQTT:1883 connection from 192.168.1.156 tcp{client}: 0x1d97790
2021-06-12 11:10:19.152 openLuup.mqtt:: shellyht-E01A8B SUBSCRIBE to shellies/command tcp{client}: 0x1d97790
2021-06-12 11:10:19.153 openLuup.mqtt:: shellyht-E01A8B SUBSCRIBE to shellies/shellyht-E01A8B/command tcp{client}: 0x1d97790
2021-06-12 11:10:19.156 luup.shelly:83: New Shelly announced: shellyht-E01A8B
2021-06-12 11:10:19.279 openLuup.context_switch:: ERROR: [dev #83] ./openLuup/client.lua:96: missing realm or nonce in received WWW-Authenticate header
2021-06-12 11:10:19.280 openLuup.mqtt:: ERROR publishing application message for mqtt:shellies/announce : ./openLuup/client.lua:96: missing realm or nonce in received WWW-Authenticate header
2021-06-12 11:10:19.393 openLuup.server:: request completed (1891 bytes, 1 chunks, 17488 ms) tcp{client}: 0x1e96db0

Yes, there is, there's a little code icon </> in the full Reply window (not the Quick one):

2021-06-12 11:10:01.904   openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=492370363&Timeout=60&MinimumDelay=1500&_=1623492371898 HTTP/1.1 tcp{client}: 0x1e96db0
2021-06-12 11:10:19.108   openLuup.io.server:: MQTT:1883 connection from 192.168.1.156 tcp{client}: 0x1d97790
2021-06-12 11:10:19.152   openLuup.mqtt:: shellyht-E01A8B SUBSCRIBE to shellies/command tcp{client}: 0x1d97790
2021-06-12 11:10:19.153   openLuup.mqtt:: shellyht-E01A8B SUBSCRIBE to shellies/shellyht-E01A8B/command tcp{client}: 0x1d97790
2021-06-12 11:10:19.156   luup.shelly:83: New Shelly announced: shellyht-E01A8B
2021-06-12 11:10:19.279   openLuup.context_switch::  ERROR: [dev #83] ./openLuup/client.lua:96: missing realm or nonce in received WWW-Authenticate header
2021-06-12 11:10:19.280   openLuup.mqtt:: ERROR publishing application message for mqtt:shellies/announce : ./openLuup/client.lua:96: missing realm or nonce in received WWW-Authenticate header
2021-06-12 11:10:19.393   openLuup.server:: request completed (1891 bytes, 1 chunks, 17488 ms) tcp{client}: 0x1e96db0

Oooh, that's a nasty error. I don't have any HT devices, I see that they are battery operated and wake up from time to time. I do have a button 1, which does a similar thing, but I haven't seen this problem.

It may be that the device's HTTP server isn't active during the MQTT Announce – the Shelly plugin normally gets a full configuration over HTTP, not MQTT, so I may have to change this approach for certain devices. I see there is a 120ms delay between the Announce message and the HTTP error – a long time for things to have happened in between...

...I will read up more on the device. Any thoughts anyone??

= = = = = =

Edit: have you set a Username and Password? ...perhaps that's an issue.

[powisquare]

Yes I have - on all of them. I seem to recall, when setting up the first one, that openLuup recognised a button press. Perhaps that's how it worked. These sensors wake up on an hourly basis, but when you wake them up manually they are awake for a minute or so (it seems).

[powisquare]

Sorry - my bad. Device that's working does not have a password. I'll have another go.

Boom - all working

[akbooer]

So, I need to fix HTTP access for device with a password, it seems. That makes sense given the 'nonce' error message.

[ArcherS]

I have a Mqtt device that reads the electrical consumption in the house and posts it via Mqtt to OpenLuup. The device posts this every minute:

What I have in Lua Startup for this sensor is (for 8 of the reported values):

function ElmatareMatarstallning (topic, message)
    local matare=message/1000
    -- local matare_clean=tointeger(matare)
  luup.variable_set ("urn:micasaverde-com:serviceId:GenericSensor1", "CurrentLevel",  matare, 95) --set value to device
end
...
luup.register_handler ("ElmatareMatarstallning", "mqtt:sensors/power/p1meter/consumption")

In general this works really well!

I have on some occations seen errors in the log related to this reporting and also a few crashes. At least once the last entry in the log was the following and then it crashed:

2021-08-15 11:03:52.703   openLuup.context_switch::  ERROR: [dev #0] ./openLuup/mqtt.lua:477: attempt to get length of local 'TopicName' (a nil value)
2021-08-15 11:03:52.704   luup.incoming_callback:: function: 0x23d2a88 ERROR: ./openLuup/mqtt.lua:477: attempt to get length of local 'TopicName' (a nil value)

I have seen in MqttExplorer that the device very seldom but sometimes also has sent rouge messages for some reason, e.g:

I am not sure why the messages sometimes get wrong or even nil, I will see what could be done on the device, it could be that I need to send less often, that the device deteriorates over time, intermitent bad wifi-connection, or something else. I have not seen this behaviour on any other device.

Another thought is if it would be possible to make to mqtt handler code more robust so that OpenLuup does not crash when this happens? By looking at the log above the topic name is reported being "nil" just before the crash. On the other hand; on at least one other occation I have seen this and it did not crash, so maybe at least it does not lead to a crash every time.

2021-08-15 21:26:29.714   luup.variable_set:: 89.urn:micasaverde-com:serviceId:GenericSensor1.CurrentLevel was: 507 now: 510 #hooks:0
2021-08-15 21:26:29.718   luup.variable_set:: 90.urn:micasaverde-com:serviceId:GenericSensor1.CurrentLevel was: 756 now: 791 #hooks:0
2021-08-15 21:26:29.722   luup.variable_set:: 91.urn:micasaverde-com:serviceId:GenericSensor1.CurrentLevel was: 1491 now: 1486 #hooks:0

2021-08-15 21:26:29.739   openLuup.context_switch::  ERROR: [dev #0] ./openLuup/mqtt.lua:477: attempt to get length of local 'TopicName' (a nil value)
2021-08-15 21:26:29.739   luup.incoming_callback:: function: 0x1e8b230 ERROR: ./openLuup/mqtt.lua:477: attempt to get length of local 'TopicName' (a nil value)

2021-08-15 21:26:29.740   openLuup.io.server:: MQTT:1883 connection closed  tcp{client}: 0x213f010
2021-08-15 21:26:29.740   openLuup.mqtt:: PUBREL: UNIMPLEMENTED packet type: 6 tcp{client}: 0x213f010
2021-08-15 21:26:29.836   openLuup.io.server:: MQTT:1883 connection from 192.168.1.191 tcp{client}: 0x25faaf0

I have had problems in the past with OpenLuup crashing when I setup Mqtt initially (before I had the electrical reporting device). Since then I now run a new OpenLuup on a Raspberry Pi 4 (my Zway server is on a Pi 3B on the same network). I still on rather rare occations get OpenLuup crashes unfortunately. I never quite got down to the reason behind the crashes that time. However reducing the mqtt load by disabling some devices has usually stopped OpenLuup from crashing when it occurs.

During summer I had a series of crashes when I was out of the house (naturally ). Checking over my VPN I could see that the power device had gone down, so the above could be the reason that time also. Since I was not at home I could not restart the power Mqtt device (it is not Tasmota and lacks that possibility unfortunately).

The Pi is currently on OpenLuup v21.6.12 and AltUi v2.53c.

Sorry for a slightly disorganised post, any advice or clues would be welcome on what this could be and hopefully how to mitigate it would be much appreciated.

[therealdb]

@archers I don't want to substitute @akbooer, but you could try this quick fix at line 477:

if TopicName == nil or #TopicName == 0 then

to just ignore the message.

[akbooer]

Thanks for reporting this in detail.

I'll take a look at the MQTT server and have a go at making it more robust.

[akbooer]

OK, development version 21.8.16 has a fix (I hope).