SmartHome Community

Buxton

@archers Very Cool gadget. I soldered to the flash pins, which given the size of the contacts was challenging, but managed to make it work. Now waiting for my BLE sensors to arrive.

Energy Monitoring built into the dual relay!

Buxton

@akbooer I think I found the culprit causing the random connection disconnects to Mosquitto. The LWT payload for a given device is a simple string, which doesn't seem to decode in the json decode call. So I added the below code to send a json string to the decoder. The errors have disappeared and I now see the LWT variable in the service variables. The variable reads "LWT : Online"

--line 165
local valid = {SENSOR = true, STATE = true, RESULT = true, LWT = true}

--line 172
  -- begin code
  local info, err = json.decode (message)
   
  if message then
	if not info then  -- json did not decode because of single string parameter
		message = '{' .. '"' .. mtype .. '"' .. ':' .. '"' .. message .. '"' .. '}'
	end
  end
  -- end code

  
  local info, err = json.decode (message)

This is probably not the way you would handle the error, but it does work.

Buxton

No, you can still configure UDP. You just need login credentials now.

If I have a few moments, I will try to set up one of my Pi machines this weekend with an instance of Mosquitto. My HA server that runs my docker Mosquitto is headless, and I run Ubuntu server, so command line captures of packets are just a drag. My Pi has an hdmi port so I should be able to load the gui version of wireshark, and then test/capture the traffic between the two instances. I will let you know.

Buxton

I've been waiting for something like this. Very cool. Thx for the post

Buxton

@rafale77 hey Rafale,

I went down the very same road a while back and threw in towel because the polling by the MQTT plugin created CPU drags that stopped openLuup from functioning "reliably". The instability was also in part because I use two other must-have plugins that rely on polling, and I imagine that the combination of the three was creating a scenario that caused intermittent failures. And I too ended up implementing MQTT in Home Assistant and then using RealDB's virtual HTTP plugin to send commands to my WiFi devices--albeit not knowing the status of the devices in openLuup after the send.

I'm looking at RigPapa's socket proxy and WebSocket plugins to see if I can transform my polling plugins to Async. The MQTT plugin is too complex for me to convert though, so if you take a crack at it, and are successful, I would very much appreciate you publishing your results, as MQTT is becoming a must for me.

toggledbits/sockproxyd

A socket proxy for Vera/openLuup systems. Contribute to toggledbits/sockproxyd development by creating an account on GitHub.

toggledbits/LuWS

A WebSocket client implementation for Luup (Vera and openLuup) systems, with optional async receive (responsive) - toggledbits/LuWS

Buxton

@akbooer Tasmota energy sensor

{
  "Time": "2021-03-28T21:51:01",
  "ENERGY": {
    "TotalStartTime": "2020-06-07T00:10:43",
    "Total": 2356.063,
    "Yesterday": 7.056,
    "Today": 6.459,
    "Period": 24,
    "Power": 285,
    "ApparentPower": 302,
    "ReactivePower": 99,
    "Factor": 0.95,
    "Voltage": 123,
    "Current": 2.453
  }
}

Buxton

@akbooer Below is the relevant output of a typical packet between mosquitto and a mosquitto bridged instance. In this case, mosquitto is sending update data to the bridge regarding a tasmota device/switch I use to remotely reboot my Vera. The format is definitely MQTT 3.1 and not 5.0, as 5.0 would not parse correctly in the wireshark viewer. The data payload is at the top of the window as Wireshark will truncate long messages. I can PM you the entire capture as it's not much, but may contain technical info that's best kept private. Let me know.

I'll try to capture some traffic between openLuup and mosquitto later.

{"Version":"9.1.0(tasmota)","BuildDateTime":"2020-11-07T11:57:45","Module or Template":"Gosund-WP5","RestartReason":"Software/System restart","Uptime":"6T05:50:22","Hostname":"power_MainVera-0278","IPAddress":"10.17.2.33","RSSI":"100","Signal (dBm)":"-17","WiFi LinkCount":5,"WiFi Downtime":"0T00:00:10","MqttCount":14,"LoadAvg":19}



Frame 9: 433 bytes on wire (3464 bits), 433 bytes captured (3464 bits) on interface eth0, id 0
Ethernet II, Src: Advansus_0a:8c:3a (00:19:0f:0a:8c:3a), Dst: 96:62:08:fb:22:8a (96:62:08:fb:22:8a)
Internet Protocol Version 4, Src: 10.17.2.41, Dst: 10.17.2.110
Transmission Control Protocol, Src Port: 40664, Dst Port: 1882, Seq: 3, Ack: 3, Len: 367
MQ Telemetry Transport Protocol, Publish Message
    Header Flags: 0x30, Message Type: Publish Message, QoS Level: At most once delivery (Fire and Forget)
        0011 .... = Message Type: Publish Message (3)
        .... 0... = DUP Flag: Not set
        .... .00. = QoS Level: At most once delivery (Fire and Forget) (0)
        .... ...0 = Retain: Not set
    Msg Len: 364
    Topic Length: 30
    Topic: tele/power_MainVera/HASS_STATE
    Message [truncated--see above]: {"Version":"9.1.0(tasmota)","BuildDateTime":"2020-11-07T11:57:45","Module or Template":"Gosund-WP5","RestartReason":"Software/System restart","Uptime":"6T05:50:22","Hostname":"power_MainVera-0278","IPAddress":"10.17.2

Buxton

@buxton In the above, I'm seeing an extra closing right hand bracket in the JSON string.

Buxton

@akbooer I hate to add more to the pile... but I'm still seeing a receive error for the connection to mosquitto. openLuup 2021.04.29b

Here's the log error:

2021-05-01 14:22:38.815   openLuup.io.server:: MQTT:1882 connection closed  tcp{client}: 0x5579919e9c58
2021-05-01 14:22:38.816   openLuup.mqtt:: RECEIVE ERROR: closed tcp{client}: 0x5579919e9c58
2021-05-01 14:22:43.935   luup.io.incoming:: bytes received: 51, status: OK tcp{client}: 0x557990e7f528
2021-05-01 14:22:48.435   luup.io.incoming:: bytes received: 51, status: OK tcp{client}: 0x557990e7f528
2021-05-01 14:22:49.763   luup.variable_set:: 10181.urn:micasaverde-com:serviceId:EnergyMetering1.KWHReading was: 1619904116 now: 1619904168 #hooks:0
2021-05-01 14:22:50.158   openLuup.io.server:: HTTP:3480 connection closed openLuup.server.receive closed tcp{client}: 0x557991971ff8
2021-05-01 14:22:53.834   openLuup.io.server:: MQTT:1882 connection from 127.0.0.1 tcp{client}: 0x5579920dc0b8
2021-05-01 14:22:53.834   openLuup.mqtt:: client is in ERROR empty
2021-05-01 14:22:53.834   openLuup.mqtt:: credentials is in ERROR empty
2021-05-01 14:22:53.834   openLuup.mqtt:: subscriptions is in ERROR empty

using the below error trapping in function "MQTTservlet"

  local function MQTTservlet (client)
  
	if client == nil then
	  _log ("client is in ERROR nil")
	else
	  	if table.concat(client) == "" then
		_log ("client is in ERROR empty")
		else
	   _log (table.concat {"MQTT ERROR: ", table.concat(client)})
	    end
	end 
	
	if credentials == nil then
	  _log ("credentials is in ERROR nil")
	else
	  	if table.concat(credentials) == "" then
		_log ("credentials is in ERROR empty")
		else
	   _log (table.concat {"MQTT ERROR: ", table.concat(credentials)})
	    end
	end 
	
	if subscriptions == nil then
	  _log ("subscriptions is in ERROR nil")
	else
	  	if table.concat(subscriptions) == "" then
		_log ("subscriptions is in ERROR empty")
		else
	    _log (table.concat {"MQTT ERROR: ", table.concat(subscriptions)})
	    end
	end
	
    return function () incoming (client, credentials, subscriptions) end
  end

I can't find a deeper layer in the stack where I can trap for the incoming message to see what's in the message that is throwing the error. As near as I can tell, if openLuup tries to connect to a running mosquitto instance, then it fails to see the topics and messages, and passes empty--but not nil--strings when the servlet interface sees incoming bytes.

If I restart mosquitto, openLuup then sees the topics and messages and the error messages stop--and the connection to mosquitto remains stable.

This behavior does not occur when I aim an IOT device directly at openLuup--in that the connection to the device always resumes when openLuup reloads--in other words, I don't need to restart the IOT device to enable the connection.

Buxton

@akbooer
And a ping response from bridge to main instance:

Frame 33: 68 bytes on wire (544 bits), 68 bytes captured (544 bits) on interface eth0, id 0
Ethernet II, Src: 96:62:08:fb:22:8a (96:62:08:fb:22:8a), Dst: Advansus_0a:8c:3a (00:19:0f:0a:8c:3a)
Internet Protocol Version 4, Src: 10.17.2.110, Dst: 10.17.2.41
Transmission Control Protocol, Src Port: 1882, Dst Port: 40664, Seq: 5, Ack: 2851, Len: 2
MQ Telemetry Transport Protocol, Ping Response
    Header Flags: 0xd0, Message Type: Ping Response
        1101 .... = Message Type: Ping Response (13)
        .... 0000 = Reserved: 0
    Msg Len: 0

Buxton

Buy yourself one of these to get accurate energy usage : https://www.energyswitching.co.uk/best-energy-monitors-of-2021/ The US equivalent is "Kill a Watt" and that brand of device works well, though I don't believe they make a UK version.
Once you dial in the actual watts to your calcs, you'll probably see some of your unexplained energy use disappear. When I went through this process, and as mentioned above by several, the vampire devices in my home were the main contributors to my unknown energy usage. I was able to claw some of that back by using myriad smart switches for always on (sleeping) devices, albeit with a loss of instant on capability.

Buxton

@akbooer I'm good as I don't have additional topics that must be added at the moment. So the code change is probably the way to go.

Buxton

@akbooer After updating to the lastest development version (21.6.12b), I did a direct copy and paste of the above attr code into startup lua, and it appears that openLuup hangs on the ipairs iteration, because the ipairs statement is seeing a string instead of a table. I was then able to boot openLuup by removing the "or" clause :

--local prefixes = config.Prefix or {"tele", "tasmota/tele"}       -- subscribed Tasmota prefixes
  local prefixes = {"tele", "tasmota/tele"}       -- subscribed Tasmota prefixes
  for _, prefix in ipairs (prefixes) do
    luup.register_handler ("Tasmota_MQTT_Handler", "mqtt:" .. prefix .. "/#", prefix)   -- * * * MQTT wildcard subscription * * *
  end

The topic table has the same problem.

I was able to see the error message when using "./openLuup_reload" at the linux command prompt.

Buxton

@akbooer Can I ask you for one more addition. I'd like to be able to set tasmota prefixes and suffixes via lua startup. Something like:

attr ("openLuup.Tasmota.Prefix", "tele", "tasmota/tele","stat")
attr ("openLuup.Tasmota.Topic", "SENSOR", "STATE","RESULT", "LWT")

You'd also need to change the upper case error check on the topic as some of the full topic suffixes are not all in upper case:

local tasmota, mtype = tasmotas: match "^(.-)/(.+)"

Buxton

@akbooer Awesome. Cool lua magic using the exception return with "or". That should cover all text messages that are topic filtered.

Buxton

@akbooer Yes, that should work. However, the LWT MQTT specification calls for a standard message, so a tasmota upgrade could change the text message to a json message. Is there a way to test the message for a non-json string, and then filter accordingly. I fear the hardcoded flags will inevitably break....

Buxton

@akbooer I used openluup json. I tried dkjson, but there was no change in effect.

Buxton

@akbooer I think I found the culprit causing the random connection disconnects to Mosquitto. The LWT payload for a given device is a simple string, which doesn't seem to decode in the json decode call. So I added the below code to send a json string to the decoder. The errors have disappeared and I now see the LWT variable in the service variables. The variable reads "LWT : Online"

--line 165
local valid = {SENSOR = true, STATE = true, RESULT = true, LWT = true}

--line 172
  -- begin code
  local info, err = json.decode (message)
   
  if message then
	if not info then  -- json did not decode because of single string parameter
		message = '{' .. '"' .. mtype .. '"' .. ':' .. '"' .. message .. '"' .. '}'
	end
  end
  -- end code

  
  local info, err = json.decode (message)

This is probably not the way you would handle the error, but it does work.

Buxton

Maintaining status messages to BlueIris for things like occupancy (established through a unifi plugin on openLuup). If the MQTT mosquitto connection disconnects or if an Mqtt client reboots, the retained openLuup message would make sure that BlueIris is in sync. Not a big deal as there are other ways to deal with this, but I asked as I didn't want to reinvent anything already functional.

@therealdb has some great library MQTT routines in his virtual switch, so I should be able to create a device without too much trouble.

Buxton

Ok thx. Do you have a retain flag in the MQTT object?

Ways to identify and calculate energy usage (watts) ?

(Tip) Pushover and Original Reactor using openLuup / ALTUI

Is Telegram working?

Device information in string or array

Multi-System Reactor Developer Preview AVAILABLE

TTS in MSR?

Change instruction in Dome DMS01 device

OpenSprinkler Plugin Timer Keeps Resetting

P1 electrical smart meter reader

Help with MSR Rule

restricting lights from a certain hour

IKEA Sonos Symfonisk Speaker TTS

MSR - long-leap updating

Luup : Hue Energy (watts) Calculator

MSR on 2 QNAP devices

Version 1.0 Pre-release Discussion

openLuup: Tasmota MQTT Bridge

openLuup: MQTT server

Z-way token Expiration

Historian

Buxton

@Buxton

Best posts made by Buxton

toggledbits/sockproxyd

toggledbits/LuWS

Latest posts made by Buxton