Sorry if this has been covered before, just curious why triggers in openluup are not consistent..
I looked at a scene i’d created a while back via ALTUI using the Console view and noticed it didn’t show any Triggers, which was strange as it was my main front door event 🙂 . So I added the door tripped trigger again, but I’ve just noticed I now how two tiggers using this view.
25bfe00a-d63e-4dc1-a501-23e779c64379-image.png
In ALTUI it shows this.
AK. Was doing an openLuup install and the installer errored with:
openLuup_install 2019.02.15 @akbooer getting openLuup version tar file from GitHub branch master... un-zipping download files... getting dkjson.lua... lua5.1: openLuup_install.lua:45: GitHub download failed with code 500 stack traceback: [C]: in function 'assert' openLuup_install.lua:45: in main chunk [C]: ?The installer code was executing this URL:
http://dkolf.de/src/dkjson-lua.fsl/raw/dkjson.lua?name=16cbc26080996d9da827df42cb0844a25518eeb3Running it manually gives:
dkolf.de The script could not be run error-free. Please check your error log file for the exact error message. You can find this in the KIS under "Product Management > *YOUR PRODUCT* > *CONFYGUAR* > Logfiles". Further information can be found in our FAQ. The script could not be executed correctly. Please refer to your error log for details about this error. You find it in your KIS under item "Product Admin > *YOUR PRODUCT* > *CONFIGURE* > Logfiles". Further information can also be found in our FAQ.I'm thinking the dkjson code URL has been changed. On dkolf.de there is a download link:
http://dkolf.de/dkjson-lua/dkjson-2.8.luaand dkjson code also seems to be in GitHub (I presume this is the same code?):
https://github.com/LuaDist/dkjson/blob/master/dkjson.luaI'm don't know what dkolf.de looked like previously but I do see the dkjson code has been updated as of 2024-06-17. Hope this helps.
Oh - and by the way the dkjson.lua file seems to have been downloaded OK by the installer - error or no error, so go figure.
It’s been a while since I looked at openLuup as it had been running nicely and quietly in the background doing some basic tasks. With my VeraPlus looking like it’s finally succumbing to old age, I want to shift a number of the global module I have over to openLuup.
To do this, I have added the files (example would be xxpushover.lua to the cmh-ludl folder and the following to the startup
require “xxpushover”
The xxpushover.lua file itself starts with the following..
module("xxpushover", package.seeall)
And I always have a line in these files to allow me to check it’s been read in the start up related logs, which in this case it is..
The challenge I’m having is that when I try to call any of the functions within the module, it returns the following error..
"[string "ALTUI - LuaRunHandler"]:1: attempt to index global 'xxpushover' (a nil value)”
I’m no doubt missing something obvious, can anyone help me find out what it is ? Many thanks
Currently I have some Whisper files used by DataYours that been working well for ages and do what I want.
One of the files is called Watts_L1.d.wsp and uses this retention from "storage_schemas_conf" in openLuup file virtualfilesystem.lua:
[day] pattern = \.d$ retentions = 1m:1dInside the actual "Watts_L1.d.wsp" file is a header like so:
1, 86400, 0, 1 84, 60, 1440The 1, 86400 is one minute & one day (in minutes) as per the retention listed above. As a side issue I would like to know what the other header values mean ie what's the syntax here?
New challenge: I now have three Shelly variables named:
em1/0/act_power
em1/1/act_power
em1/2/act_power
with a device ID of "10006" and a SID of "shellypro3em"
And I would like to plot them using the Historian, just like I do with Watts_L1.d.wsp in DataYours. So I need a file in the history directory for the data. So I looked at doing this:
local whisper = require "openLuup.whisper" -- Syntax: history/0.deviceNumber.shortServiceId.variableName local filename = "history/0.10006.shellypro3em.em1/0/act_power.wsp" local archives = "1m:1d" whisper.create (filename,archives,0)Problem is that the variable names contains forward slashes, which are invalid filename characters. What to do?
Also should the retentions now be (to suit the latest openLuup software)?:
local archives = "1m:1d,10m:7d,1h:30d,3h:1y,1d:10y"Also "shellypro3em" is not a "shortServiceID" as per those listed in "servertables.lua". So can "shellypro3em" be used instead? ie can both short and long service IDs be used in the above call to whisper.create?
To try and minimized the frequency of writing to the SD card I want to move these log files to a RAM drive, like I already do with /var/log. Is there an 'official' way of doing this?
_John.
A list of openLuup releases including the latest developments…
master – stable, and infrequently updated, development – latest updates and bug fixes, testing – use only when advised!A long while ago (May, 2015) I wrote my 2000-th post on another forum: openLuup - running unmodified plugins on any machine.
Now rehosted at https://community.ezlo.com/t/openluup-running-unmodified-plugins-on-any-machine/187412
Here’s the gist of it:
...I want to work in a more open and stable [Vera] environment...
...All would be solved if Luup was open source and could be run on the plethora of cheap and reliable hardware available today. But it’s not. But we could get something like that effect if we engineered a sufficient subset of Luup to run on such a platform. Could it be done? What would we need?
1. UI
2. scheduler
3. web server
4. Luup compatible API
5. Device and Implementation xml file reader
6. Zwave bridge to Vera
7. runs most plugins without modification
What we wouldn’t need is UPnP.
What have we (nearly) got already?
We have, courtesy of @amg0, the most excellent AltUI: Alternate UI to UI7, and that, I think, is probably the hardest one to do in the above list. Items 2 - 5, and 7, I’ve prototyped, in pure Lua, and posted elsewhere: DataYours on Raspberry Pi, running selected plugins unmodified, including: DataYours, EventWatcher, Netatmo, RBLuaTest, altUI. See screenshot attached.Is it worth the effort? Probably not. Will I pursue this quest? Yes.
openLuup was the result.
Hoping you could tell us a bit about your experiences with ZWaveJS and MQTT.
Akbooer: it would be good if openLuup was added to the awesome mqtt resources list.
How to contribute is described here.
Looks like the GetSolarCoords() doesn't return the correct results. Right Ascension (RA) and
Declination (DEC) look OK. They presumably must be, as I have a light that goes on at sunset at the correct time for years.
Altitude and Azimuth look incorrect. They both have the hour angle in common, so I'm wondering if it's incorrect and hence the sidereal time. Should be able to convert the angle to hours and check it against this clock:
The formula used looks like Compute sidereal time on this page. Might be some mix up between JD2000 that has a 12 hour offset. Could also be some issue with the hour angle.
I'm assuming all Right Ascension (RA) and
Declination (DEC) are degrees plus & minus from north.
Likewise Altitude (ALT) and Azimuth (AZ) are in degrees?
Bit of caution: I haven't looked at this too closely, so may be barking up the wrong tree. It probably doesn't help living near Greenwich.
This site may also be helpful.
PS did you have a look at the link in my last PM?
Set up:
a) Many many many many kms from home: laptop connected to modem router. Router running wireguard client to create a virtual network.
b) Home: modem router running wireguard server. openLuup pi4 connect to router and also a PC and other stuff, etc.
The problem: When accessing charts, AltUI or the openLupp console the web pages are returned OK up to the point where they are truncated and therefore fail to display anything useful.
Note this all works fine over short distances eg around a major city (I tested it) but not seemingly at world wide distances. ie network delays seem to be the issue here? Windows TeamViewer works fine overy the exact same network/wireguard set up. That's how I was able to get the openLuup logs shown below.
Here is any example of openLuup trying to return a chart:
2023-09-04 21:31:20.463 openLuup.io.server:: HTTP:3480 connection from 10.0.0.2 tcp{client}: 0x55aed35038 2023-09-04 21:31:20.464 openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=316885191&Timeout=60&MinimumDelay=1500&_=1692128389970 HTTP/1.1 tcp{client}: 0x55ae538348 2023-09-04 21:31:20.465 openLuup.server:: GET /data_request?id=lu_status2&output_format=json&DataVersion=316885191&Timeout=60&MinimumDelay=1500&_=1692129024374 HTTP/1.1 tcp{client}: 0x55addbe1e8 2023-09-04 21:31:20.477 openLuup.server:: GET /data_request?id=lr_render&target={temp_first_floor.w,temp_ground_floor.w,temp_back_wall_of_office.w,temp_inside_roof.w,temp_jps_bedrm_north.w,temp_outside.w}&title=Temperatures&height=750&from=-y&yMin=0&yMax=40 HTTP/1.1 tcp{client}: 0x55aed35038 2023-09-04 21:31:20.478 luup_log:6: DataGraph: drawing mode: connected, draw nulls as: null 2023-09-04 21:31:20.502 luup_log:6: DataGraph: Whisper query: CPU = 23.122 mS for 2016 points 2023-09-04 21:31:20.532 luup_log:6: DataGraph: Whisper query: CPU = 22.952 mS for 2016 points 2023-09-04 21:31:20.561 luup_log:6: DataGraph: Whisper query: CPU = 22.738 mS for 2016 points 2023-09-04 21:31:20.575 luup_log:6: DataGraph: Whisper query: CPU = 9.547 mS for 2016 points 2023-09-04 21:31:20.587 luup_log:6: DataGraph: Whisper query: CPU = 9.569 mS for 2016 points 2023-09-04 21:31:20.598 luup_log:6: DataGraph: Whisper query: CPU = 9.299 mS for 2016 points 2023-09-04 21:31:20.654 luup_log:6: visualization: LineChart(2016x7) 196kB in 51mS 2023-09-04 21:31:20.655 luup_log:6: DataGraph: render: CPU = 51.219 mS for 6x2016=12096 points 2023-09-04 21:31:20.755 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x55aed35038' sending 2 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:20.855 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x55aed35038' sending 6 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.037 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x55aed35038' sending 2 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.138 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x55aed35038' sending 6 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.332 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x55aed35038' sending 2 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.432 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x55aed35038' sending 6 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.507 openLuup.server:: error 'closed' sending 196367 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.507 openLuup.server:: ...only 144000 bytes sent 2023-09-04 21:31:21.507 openLuup.server:: error 'closed' sending 2 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.507 openLuup.server:: ...only 0 bytes sent 2023-09-04 21:31:21.507 openLuup.server:: error 'closed' sending 5 bytes to tcp{client}: 0x55aed35038 2023-09-04 21:31:21.507 openLuup.server:: ...only 0 bytes sent 2023-09-04 21:31:21.507 openLuup.server:: request completed (196367 bytes, 10 chunks, 1030 ms) tcp{client}: 0x55aed35038 2023-09-04 21:31:21.517 openLuup.io.server:: HTTP:3480 connection closed openLuup.server.receive closed tcp{client}: 0x55aed35038 2023-09-04 21:31:22.824 openLuup.io.server:: HTTP:3480 connection from 10.0.0.2 tcp{client}: 0x55aea22c88Re: socket.select() not ready to send
Is there some sort of timeout I change; to see if this can make this work?
Note that openLuup is still running everything flawlessly for ages now, including the more recent addtions of ZigBee stuff. Much appreciated.
Hi @akbooer
Just bringing this over as suggested..
I’ve started to use the console view a lot more, mainly for it’s look and simplicity , but I noticed it does not do any live updates compared to ALTUI, you have to do a full browser reload. Is that by design, or is mine not working?
Also if I want to go strait to the console view, rather than into ALTUI, I recall seeing something abut altering that in the guide by for the life of me I can’t find it. Is it possible to do, if so how would I do that..
You suggested this was something you were looking at ? Also you said You don't need a "full browser reload", just click on the display menu item to refresh the screen. - what do you mean by `display menu?
Very minor issue: was messing about renaming a few rooms and ended up with a room being listed twice. One with the room's contents and the other with no room contents.
It simply turns out one room name had a trailing space. It is possible in both AltUI and the openLuup console to create a room name with a trailing space. Once having done so chaos then ensues, as the rooms are not necessarily treated as different and become difficult to manipulate.
Just need to trim white space off room names. Haven't tested if it's possible to add in leading spaces. That may also be possible.
Hey guys...
Long time... 😉
Since my first day with Vera, I'm using RulesEngine from @vosmont to handle complex rules that will do something based on multiple condition base on "true/false" and also based on time.
Do you think I will be able to do that directly with LUA in openLuup ?
For example..
IF bedroom-motion1 is not detecting motion for 15 minutes
AND
IF bedroom-motion2 is not detecting motion for 15 minutes
AND
IF current-time is between 6am and 11pm
AND
IF binary-light1 is OFF
AND
IF binary-light2 is OFF
THEN
execute LUA code
WAIT 2 minute
execute LUA code
BUT IF any "conditions" failed while in the "THEN" , It need to stop...
I currently have around 60 rules like that 😞
Currently I have a Vera and Hue hub all reliably controlled by openLuup with AltUI, plus any number of plugins. Been working really well for a few years now. However would like to head for a more MQTT based set up. Eliminate the Hue hub and hopefully eliminate Vera by using ZWAVE JS UI. Noting that Zwavejs2mqtt has been renamed to Z-Wave JS UI. Probably also run the stuff using Docker. Just because. Everything would end up on the one computer for easier management. Erhhh that's the hope.
Some of the new Zigbee Aqara stuff is very good and inexpensive plus it fits in with HomeKit. Also the Aqara battery powered stuff looks to have a good battery lifetime: ie suggested up to five years. The battery operated Hue buttons I have; have lasted for ages. Would like to use zigbee2mqtt with a SonOff dongle, which would allow access to the over two and half thousand devices zigbee2mqtt now supports:
Zigbee2MQTT
AK has the MQTT stuff working in openLuup. Have played around with it and it works well, as one would expect. Love the UDP to MQTT code.
Shellys are great and also very inexpensive and they spit out & accept MQTT but I would prefer to stay away from WiFi. Not meshed and higher power consumption. Horses for courses.
Now here's the query:
Got about forty or more ZWave twin light switches, plus a few other ZWave bits & pieces such as blind controllers. Then there are the Hue devices on top of that. That's a lot of virtual devices to set up in openLuup. What's an appropriate way to do this?
It seems there is no "auto magic bridge set up". Do I need to use say @therealdb's Virtual Devices plugin that supports MQTT or is there some other approach?
I have to confess I still don't understand the master child approach in that plugin. Seems one light switch would have all the other light switches hanging off it? Helps Vera but not a problem with openLuup - why is that? Suspect AK's good coding beats Vera's?
GitHub - dbochicchio/vera-VirtualDevices: Virtual HTTP Devices plug-in for Vera and openLuup
GitHub - dbochicchio/vera-VirtualDevices: Virtual HTTP Devices plug-in for Vera and openLuup
Virtual HTTP Devices plug-in for Vera and openLuup - dbochicchio/vera-VirtualDevices
Setting up manually say 100 virtual devices is a bit much to ask. I had a look at hacking the user_data.json file. Good approach till you see all the UIDs and the individually numbered ControlURL and EventURLs that need to be set up.
I need some way of say of creating about 80 light switches in "No room" or in say the "ZWave upgrade" room. Or say some sort of code that could go through all my existing bridged ZWAVE devices in openLuup and create virtual devices for each one. I caould then use the openLuup console to name them and place them in their rooms:
openLuup_IP_address:3480/console?page=devices_table
At that point I could hack the user_data.json file to insert the MQTT topics fairly easily for each? Plus any other fine tuning needed.
Then the old ZWave stuff could be swapped over to ZWAVE JS UI and all the virtual MQTT devices would be ready to go or am I dreaming? Then delete all the old Vera bridged stuff. I'm not too fussed about scene code and the like, as a I have all my code in one block, that is set up in the openLuup start up.
It seems that with ZWay you can create all the ZWave device by doing some sort of interrogation of ZWay's API? Seems also to be the case with the Shelleys?
So any ideas, suggestions or code snippets are welcome on how to move towards MQTT and in particular ZWAVE JS UI and zigbee2mqtt.
I'm in no hurry as openLuup is performing nicely, with the old Vera handling all my ZWave devices.
Hi
Just wondering if it’s possible when writing plugins to set if the text shown via DisplayLine1/2 can be left, right or centre aligned ?
Bit of an odd one this:
Bare metal install on Debian Bullseye (Intel NUC)
I've noticed when travelling, I connect to my L2TP VPN and I cannot get AltUI to update. I just get 'Waiting Initial Data'
Specifically this is in Chrome:
Version 108.0.5359.94 (Official Build) (x86_64)
In Chrome I can access and control everything via the Openluup console.
In Chrome I can also access and control everything via the Z-Wave expert UI and Z-Wave UI
In Safari I get a more complete view of AltUI but loads of errors along the lines of:
the module or function ALTUI_PluginDisplays.drawBinaryLight does not exist, check your configuration
Homewave on my iOS devices is fine across the same VPN config,
I can ssh into all my servers
Not a huge issue, just curious if anyone has any thoughts of what I might tweak to resolve it?
(FWIW I also access my IMAP and SMTP servers across the same VPN with no issues, as well as remote desktop. Also MS Reactor on the same host as Openluup)
TIA for any thoughts
C
Hi Ak,
Not sure when it started as it took me a while to notice.
I have a function on a luup.call_timer to turn on a switch and then use a luup.call_delay to turn it off a minute later. This is done by the same global function, but on the luup.call_delay i get a message in the log : "luup.call_delay:: unknown global function name: HouseDevice1_PumpCommand"
This is in the init function:
luup.call_timer("HouseDevice1_PumpCommand", 2, "2:15:00", hm_Heating.PumpHealthRunDay, hm_Heating.PumpCMD.HEALTH.."1", true)This is in the function to schedule to off command giving the global function name not found:
luup.call_delay("HouseDevice1_PumpCommand", hm_Heating.PumpHealthOnDuration, hm_Heating.PumpCMD.HEALTH.."0")Is it because I use the "TRUE" parameter that is openLuup specific so the timer does not fire just once?
Running v21.7.25, may be time to update?
Cheers Rene
Hi, I have been trying to install OpenLuup on MacOS but I am failing, so far.
Is there a step-by-step instruction (for MacOS) to follow?
After installing LuaRocks, luasec, luafilesystem and luasocket I then try to run lua5.1 openLuup_install.lua and then get the messages below.
Any ideas and proposals are appreciated.
Regards
Jan
openLuup_install 2019.02.15 @akbooer
lua5.1: openLuup_install.lua:18: module ‘socket.http’ not found:
no field package.preload[‘socket.http’]
no file ‘./socket/http.lua’
no file ‘/usr/local/share/lua/5.1/socket/http.lua’
no file ‘/usr/local/share/lua/5.1/socket/http/init.lua’
no file ‘/usr/local/lib/lua/5.1/socket/http.lua’
no file ‘/usr/local/lib/lua/5.1/socket/http/init.lua’
no file ‘./socket/http.so’
no file ‘/usr/local/lib/lua/5.1/socket/http.so’
no file ‘/usr/local/lib/lua/5.1/loadall.so’
no file ‘./socket.so’
no file ‘/usr/local/lib/lua/5.1/socket.so’
no file ‘/usr/local/lib/lua/5.1/loadall.so’
stack traceback:
[C]: in function ‘require’
openLuup_install.lua:18: in main chunk
OpenLuup unavailable
-
I've pushed a new v21.3.24b release which includes the new checks, but, hopefully, without the deleterious effect on AltUI (or anything else.)
To make it easy to revert, I've also tagged the v21.3.23 version, so you can always revert to that by typing it into the update box.
-
Have not upgraded to 24b yet but this is what I am seeing:
2021-03-24 12:36:01.348 openLuup.io.server:: HTTP:3480 connection from 192.168.0.55 tcp{client}: 0x7fb153b396c8 2021-03-24 12:36:01.350 openLuup.server:: GET /data_request?id=variableset&serviceId=urn:upnp-org:serviceId:VSwitch1&Variable=Text2&DeviceNum=22&Value=10 HTTP/1.1 tcp{client}: 0x7fb153b396c8 2021-03-24 12:36:01.350 luup.variable_set:: 22.urn:upnp-org:serviceId:VSwitch1.Text2 was: 9 now: 10 #hooks:0 2021-03-24 12:36:01.351 openLuup.server:: error 'closed' sending 2 bytes to tcp{client}: 0x7fb153b396c8 2021-03-24 12:36:01.351 openLuup.server:: ...only 0 bytes sent 2021-03-24 12:36:01.351 openLuup.server:: error 'closed' sending 5 bytes to tcp{client}: 0x7fb153b396c8 2021-03-24 12:36:01.351 openLuup.server:: ...only 0 bytes sent 2021-03-24 12:36:01.351 openLuup.server:: request completed (2 bytes, 1 chunks, 1 ms) tcp{client}: 0x7fb153b396c8 2021-03-24 12:36:01.351 openLuup.io.server:: HTTP:3480 connection closed openLuup.server.receive closed tcp{client}: 0x7fb153b396c8I am nearly 100% convinced that the hangs are due to luasocket error or timeout handling. You can see here that likely the client which here is home-assistant sending an http call to update a device on openLuup, apparently closed the tcp connection to which openLuup is trying to send something. As a result, I am getting an error. The humor is that both home assistant and openLuup are on the same machine. I am seeing this type of errors a few times a day but have not had a hang up since my previous post (I have updated luasocket and luasec since by the way) so I don't know if they are related to this error. However my previous experience from last year seem to indicate a bug in luasocket causing it to hold up the entire lua thread.
-
@akbooer Upgraded to v21.2.24b, it works as supposed so far.
Also I modified my strategy a bit. I have now turned off the http reporting from the sensors. They generate quite a lot of http calls, the screenshot below is from approx 13:00 to 22:00.
Worth noting is that they have been doing this http reporting for a few months without any stability problems, the traffic they generate could however be part of seeing the problems on my particular server perhaps.
The three test devices reporting over Mqtt are left on, we will see what happens.
-
Today at 12:08 it crashed again.
Not many Mqtt calls since the restart yesterday evening.
Also very few http server sockets:
Log file errors before the crash:
.. 2021-03-25 12:05:33.879 openLuup.io.server:: MQTT:1883 connection closed tcp{client}: 0x563ca4502428 2021-03-25 12:05:33.879 openLuup.mqtt:: RECEIVE ERROR: closed tcp{client}: 0x563ca4502428 2021-03-25 12:05:33.879 openLuup.mqtt:: TasmotaCO2An UNSUBSCRIBE from cmnd/TasmotaCO2An_fb/# tcp{client}: 0x563ca4502428 2021-03-25 12:05:33.879 openLuup.mqtt:: TasmotaCO2An UNSUBSCRIBE from cmnd/tasmotas/# tcp{client}: 0x563ca4502428 2021-03-25 12:05:33.879 openLuup.mqtt:: TasmotaCO2An UNSUBSCRIBE from cmnd/TasmotaCO2An/# tcp{client}: 0x563ca4502428 2021-03-25 12:05:33.891 openLuup.mqtt:: TasmotaCO2An SUBSCRIBE to cmnd/TasmotaCO2An/# tcp{client}: 0x563ca41ea188 2021-03-25 12:05:33.894 openLuup.mqtt:: TasmotaCO2An SUBSCRIBE to cmnd/tasmotas/# tcp{client}: 0x563ca41ea188 .. 2021-03-25 12:08:44.921 openLuup.io.server:: MQTT:1883 connection closed tcp{client}: 0x563ca41ea188 2021-03-25 12:08:44.921 openLuup.mqtt:: RECEIVE ERROR: closed tcp{client}: 0x563ca41ea188 2021-03-25 12:08:44.921 openLuup.mqtt:: TasmotaCO2An UNSUBSCRIBE from cmnd/TasmotaCO2An_fb/# tcp{client}: 0x563ca41ea188 2021-03-25 12:08:44.921 openLuup.mqtt:: TasmotaCO2An UNSUBSCRIBE from cmnd/tasmotas/# tcp{client}: 0x563ca41ea188 2021-03-25 12:08:44.921 openLuup.mqtt:: TasmotaCO2An UNSUBSCRIBE from cmnd/TasmotaCO2An/# tcp{client}: 0x563ca41ea188 2021-03-25 12:08:44.925 openLuup.server:: request completed (11424 bytes, 1 chunks, 3813 ms) tcp{client}: 0x563ca3ebc488 .. 2021-03-25 12:08:44.935 openLuup.io.server:: MQTT:1883 connection closed tcp{client}: 0x563ca3b9d058 2021-03-25 12:08:44.935 openLuup.mqtt:: RECEIVE ERROR: closed tcp{client}: 0x563ca3b9d058 2021-03-25 12:08:44.935 openLuup.io.server:: MQTT:1883 connection closed tcp{client}: 0x563ca3f8a488 2021-03-25 12:08:44.935 openLuup.mqtt:: RECEIVE ERROR: closed tcp{client}: 0x563ca3f8a488 ... 2021-03-25 12:08:44.938 openLuup.io.server:: MQTT:1883 connection closed tcp{client}: 0x563ca47549f8 2021-03-25 12:08:44.938 openLuup.mqtt:: closed tcp{client}: 0x563ca47549f8 2021-03-25 12:08:44.938 openLuup.mqtt:: tasmota_test UNSUBSCRIBE from cmnd/tasmota_test_fb/# tcp{client}: 0x563ca47549f8 2021-03-25 12:08:44.938 openLuup.mqtt:: tasmota_test UNSUBSCRIBE from cmnd/tasmota_test/# tcp{client}: 0x563ca47549f8 2021-03-25 12:08:44.938 openLuup.mqtt:: tasmota_test UNSUBSCRIBE from cmnd/tasmotas/# tcp{client}: 0x563ca47549f8 2021-03-25 12:08:44.938 openLuup.mqtt:: ERROR publishing application message for mqtt:cmnd/tasmota_test/POWER : closed 2021-03-25 12:08:44.939 openLuup.io.server:: MQTT:1883 connection closed tcp{client}: 0x563ca428dc98 2021-03-25 12:08:44.939 openLuup.mqtt:: closed tcp{client}: 0x563ca428dc98 2021-03-25 12:08:44.939 openLuup.mqtt:: TasmotaCO2Ax UNSUBSCRIBE from cmnd/TasmotaCO2Ax/# tcp{client}: 0x563ca428dc98 2021-03-25 12:08:44.939 openLuup.mqtt:: TasmotaCO2Ax UNSUBSCRIBE from cmnd/tasmotas/# tcp{client}: 0x563ca428dc98 2021-03-25 12:08:44.939 openLuup.mqtt:: TasmotaCO2Ax UNSUBSCRIBE from cmnd/TasmotaCO2Ax_fb/# tcp{client}: 0x563ca428dc98 2021-03-25 12:08:44.939 openLuup.mqtt:: ERROR publishing application message for mqtt:cmnd/TasmotaCO2Ax/POWER : closed -
The log sequences for MQTT seem normal, simply denoting that a connection has been dropped by the client, so then it's unsubscribed. A while later, presumably in the bits you've edited out, it reconnects. Might be interesting to know if there were any successful transactions going on in between. What's the KeepAlive configured as on your clients?
But I'm thinking this is not a very fruitful way forward. I am really, really struggling to think what may have changed, other than the fact that there is simply more I/O going on in the system. I tend to agree with @rafale77 above.
I have not fully run out of options with the LuaSocket library, so there will be a bit more experimentation. I'm hoping that a eureka moment will strike and I realize that this is something stupid that I've done (wouldn't be the first time.)
-
Just loaded 3.25b and will continue to test. I have not had any other crash myself but again, I have not been using the MQTT server on my "production" system. I continue to suspect as I had evidence of last year that the lua thread hangs when luasocket tries to send too many packets into a closed connections and certainly the MQTT broker running in the same lua instance as openLuup increases that probability. For me it was homewave making http polling calls which suddenly went unanswered. Maybe @ArcherS, could try updating the luasocket version to the development version? It helped for me. Another idea might be to run the MQTT server on a separate lua instance? I am getting the idea from the UPnP proxy plugin which technically is really a separate server on another lua instance launched by the plugin device.
-
3.25b has yet more checking, so it's worth a go.
The thing about the openLuup MQTT implementation is that it is a server, not a client. What this means is that instead of messages going from:
client A --> PUBLISH over TCP --> server --> PUBLISH over TCP --> client BYou get:
If client A is running on openLuup
client A --> server --> PUBLISH over TCP --> client Bor if client B is runing on openLuup
client A --> PUBLISH over TCP --> server --> client Bor both under openLuup (less likely)
client A --> server --> client BThe conclusion is that splitting the server to a separate machine actually generates more TCP traffic, not less.
-
True, it generates more TCP traffic to the machine and overall but... it may reduce traffic within the openLuup/socket instance since there could be traffic not destined/subscribed to openLuup but maybe I misunderstood something...
edit: I am still seeing this packets sending into a closed connection error appearing occasionally on 25b so the keepalive may not help in my case given how short the time is.
-
Indeed, the keepalive shouldn't make much difference since MQTT has a ping/response transaction to ostensibly keep the channel open anyway. It was worh a try. On my Synology Docker system, I'm not seeing any dropouts like that at all. Once again, closed connections should be handled correctly and aren't, in themselves, necessarily sinister. It's not even totally clear that this is the reason to hang the system. Perplexing.
-
@akbooer I think the KeepAlive is 15 seconds when I read up on Tasmota KeepAlive.
The connection from the devices to the server seem stable, I have seldom seen any dropouts in the device console anyway.Updated to v21.2.25b and restarted, we will se what happens!
Also enabledluup.attr_set ("openLuup.MQTT.PublishVariableUpdates", true), I had turned it off to bring down the load on messages. -
I looked through the log files for errors, the red highlighting is very good to highlight the errors.
This one is Homewave (192.168.1.127) on my phone. I noticed it was slow to connect. No crash though. I had a few more similar errors in the log a bit later.
2021-03-25 20:39:31.205 openLuup.io.server:: HTTP:3480 connection from 192.168.1.127 tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.207 openLuup.server:: GET /data_request?output_format=json&id=lu_status HTTP/1.1 tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.531 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x557d7a41acf8' sending 2 bytes to tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.732 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x557d7a41acf8' sending 6 bytes to tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.925 openLuup.server:: error 'closed' sending 2 bytes to tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.925 openLuup.server:: ...only 0 bytes sent 2021-03-25 20:39:31.925 openLuup.server:: error 'closed' sending 6 bytes to tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.925 openLuup.server:: ...only 0 bytes sent 2021-03-25 20:39:31.925 openLuup.server:: error 'closed' sending 857193 bytes to tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.925 openLuup.server:: ...only 128000 bytes sent 2021-03-25 20:39:31.925 openLuup.server:: error 'closed' sending 2 bytes to tcp{client}: 0x557d7a41acf8 2021-03-25 20:39:31.925 openLuup.server:: ...only 0 bytes sent 2021-03-25 20:39:31.925 openLuup.server:: error 'closed' sending 5 bytes to tcp{client}: 0x557d7a41acf8A few Whisper errors, could be related to a Luup reload at that I made at that time:
2021-03-25 19:28:41.905 openLuup.context_switch:: ERROR: [dev #0] ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) 2021-03-25 19:28:41.905 openLuup.scheduler:: 55.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature ERROR ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) function: 0x557d78862090 2021-03-25 19:28:42.325 openLuup.context_switch:: ERROR: [dev #0] ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) 2021-03-25 19:28:42.325 openLuup.scheduler:: 246.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature ERROR ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) function: 0x557d78862090 2021-03-25 19:28:42.326 openLuup.context_switch:: ERROR: [dev #0] ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) 2021-03-25 19:28:42.326 openLuup.scheduler:: 247.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature ERROR ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) function: 0x557d78862090 2021-03-25 19:28:42.363 openLuup.context_switch:: ERROR: [dev #0] ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) 2021-03-25 19:28:42.363 openLuup.scheduler:: 11669.urn:upnp-org:serviceId:TemperatureSensor1.CurrentTemperature ERROR ./openLuup/whisper.lua:101: bad argument #1 to 'format' (number expected, got string) function: 0x557d78862090 2021-03-25 19:28:42.758 luup_log:3: ALTUI: startupDeferred, called on behalf of device:3 -
The Homewave one is interesting. Might just be a slow connection. I'll make the timeout for socket.select() larger (probably user-configurable.) So HomeWave works OK anyway?
Ah, very interesting. I know what the second one is. Brand new variables are (in Vera, hence openLuup too) given the default value of EMPTY. These messages come from the HIstorian (expecting a numeric value) which you have enabled?? I suspect that they will go away after the first update (so after the system has been running a while, so long as the actual device is working.)
-
The homewave errors are the same as my occasional home assistant errors. My homewave is nowhere slow and was the initial source of my openLuup hangs. I just retested it with a transition between cellular+VPN to wifi and while I used to be able to cause openLuup to crash nearly 100% of the time doing that, I am no longer able to do it. Last year, the big factor was my upgrade of luasocket to the current github master branch which although it reports as 3.0rc1 is actually scm-2. This made a night and day difference and made even homewave feel significantly more responsive. I have since also moved to running openLuup to luajit with rapidjson which also is another performance improvement. It is possible that I accidentally downgraded my luasocket while troubleshooting something else at some point in the past few months which lead to the 2 crashes a couple of days ago. I have since been unable to crash openLuup again. Just a data point... I am still seeing these errors in the log but again, no crash.
This is the error I am seeing when I am switching network connection on homewave while it is polling openLuup:
2021-03-25 13:59:40.533 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x7f3a6d48d9d8' sending 2 bytes to tcp{client}: 0x7f3a6d48d9d8 2021-03-25 13:59:46.651 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x7f3a6d6c5520' sending 2 bytes to tcp{client}: 0x7f3a6d6c5520 2021-03-25 13:59:47.192 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x7f3a6d6c5520' sending 2 bytes to tcp{client}: 0x7f3a6d6c5520 2021-03-25 13:59:47.392 openLuup.server:: error 'socket.select() not ready to send tcp{client}: 0x7f3a6d6c5520' sending 6 bytes to tcp{client}: 0x7f3a6d6c5520 -
@akbooer yes Homewave works as it should. At times it is a bit slow to connect, usually quite ok.
I have enabled Historian and have connected it to Grafana in which I have some graphs.
@rafale77 I tried to switch between cellular&vpn and wifi with Homewave and it seems to work without crashing OpenLuup. I am using Qvpn from Qnap.
-
Any update?
I just inadvertently stress tested openLuup's socket after all the updates this morning:
I went working in the yard cutting wood this morning which trips my ipcam motion sensor which then trips HomeAssistant object detection which then scans my camera stream using a deep learning model and when it detects a person sends an http call to openLuup to trip a security sensor on which I run a scene to do other things ranging from notifications to announcement depending on housemode etc... This happens at a rate of 5Hz (one call every 200ms)... You can see that the high frequency of http calls on top of the regular polling (I poll z-way at rate of 2Hz for example) made openLuup's CPU utilization jump! yet no crash...
Edit: And I checked the log to discover a lot more errors of sending bytes to a closed connection but as I said, it handled like a champ.
-
5% peak load? Shameful!
I’ve been stress testing too, and haven’t been able to make anything crash recently. I’m crafting a new Docker to take over from my main, and ageing, BeagleBone Black system, and the I/O there just whistles along with cpu at under 0.2%. I also see that RapidJson is available as a standard install of Alpine, so I may include that.
My conclusion, following yours, is that this is low-level LuaSocket issues on out of date libraries, which only show a problem when moderately stressed. I don’t think there’s anything more I can do at the openLuup coding level.
-
@akbooer said in OpenLuup unavailable:
5% peak load? Shameful!
I know! right? lol! As you know I optimize things to the edge... running on luajit, having turned JIT back on and rapidjson which I highly recommend over cjson because it enables not having to failover to dkjson if you so desire. I also regularly look at library updates which admittedly for lua, are sadly rare. I even tested a number of forks of luajit and found nothing worth the trouble at this point.
-
@rafale77 @akbooer two days ago the system was quite slow and non-responsive. I then decided to disable the Mqtt server in OpenLuup to see what would happen. Now after two days no crash, so it seems as if this is somehow casing the crashes. Small progress perhaps.
Edit:
What I still have noticed is that the system has been quite slow on v21.3.25b and v2.3.24b with at times very long response times. Much worse that what it was 3-4 days ago for some reason, before updating from v2.3.21 to v2.3.24b.
On 2.3.21 the system was quite snappy, but after some time crashed without warning.
Now I updated to 21.3.26 and so far it seems much more responsive that on v2.3.24b / v21.3.25b.I thought I would updated the Lua sockets libraries, LuaFileSystem etc to the latest versions, but now I cannot find the openLuup > System > Required list list.
This is what I had alst time I checked a week ago:
Hints on what to update and how to do it much appreciated.
Edit: The update to 21.3.26 initially seems to have made the system more repsonsive for whatevere reason. Good news, let's see if it stays like that.
I see that I am on LuaFileSystem 1.6.3 that seems to be from 2015 and @rafale77 you are on 1.8.0 that is from 2020. This seems like something that could need updating. -
