luaJIT
-
I just compiled and installed the OpenResty branch of luaJIT along with updating libc.
After reboot, the cpu load is now... ~1.5%
You can see the steps below:
On the left it is using the lua5.1 interpreter hovering around 2.8%, the middle plateau is using a 2017 beta version of luaJIT: ~2.2%, the rightmost plateau at ~1.5% is after the latest update...
As a reminder I previously reduced the cpu load by swapping the json module from cjson to rapidjson which only lowered the "idle" cpu load baseline from 3.2% to 2.8% but was most impactful when the dkjson module was in use (i.e ALTUI page loading) which reduced the spike from >10% down to <5%.... to now not even showing a peak. It remains <2% (they are the little peaks on the right side going up to ~1.8%). Overall these steps reduced idle CPU load by ~50% and ALTUI browsing load by ~95%. -
Really interesting (and quite new) paper from the same stable as Lua, discussing, amongst other things, JIT in the context of high performance computing and Lua.
http://www.inf.puc-rio.br/~hgualandi/papers/Gualandi-2020-SCP.pdf
-
guys....
so we can just install luajit package and instead of calling
lua5.1 openLuup/init.lua
we can use luajit openLuup/init.lua ?
guys....
so we can just install luajit package and instead of calling
lua5.1 openLuup/init.lua
we can use luajit openLuup/init.lua ?
Yup Drop-in replacement. It even reuses all the lua5.1 library folders.
Pallene seems real enough, in active development:
Go on, @rafale77, give it a go!
Only thing which makes me hesitant is lua5.3... Are there deprecation issues?
-
guys....
so we can just install luajit package and instead of calling
lua5.1 openLuup/init.lua
we can use luajit openLuup/init.lua ?
Yup Drop-in replacement. It even reuses all the lua5.1 library folders.
Pallene seems real enough, in active development:
Go on, @rafale77, give it a go!
Only thing which makes me hesitant is lua5.3... Are there deprecation issues?
Only thing which makes me hesitant is lua5.3... Are there deprecation issues?
Oh, absolutely. The whole arrangement for function environments changed in v5.2.
I was thinking you might just want to fire it up on a stand-alone program, but I'll ask again when I have a v5.3 version of openLuup. It's actually 'only' a matter of changing a few places in the loader and scheduler modules. I've never done it because I felt it necessary to retain as much compatibility with Vera as possible. But now, it really doesn't matter – aside from installing Lua in the first place, there should anyway be no apparent change for any plugin or from the user POV.
I actually have more need of this in my other current project, so perhaps I'll try it there first.
Anyway, as with your investigations for LuaJIT, it's nice to have an option for future speed-ups as our HA needs get more sophisticated.
-
-
Well I finally found something which does not work on luajit: http-digest.
I will be experimenting with some other http libraries...edit: I am thinking about switching my one plugin requiring digest authentication to cURL:
Wondering if it could be a good alternative to luasocket for openluup's http construct...
-
I have fixed my digest problem using the luajit-request library which indeed relies on the libcurl integration in luaJIT's ffi.
I am not quite sure why it broke the http-digest library though but overall cURL may be a better supported/more robust solution than luasocket for http...I also found this:
-
I have fixed my digest problem using the luajit-request library which indeed relies on the libcurl integration in luaJIT's ffi.
I am not quite sure why it broke the http-digest library though but overall cURL may be a better supported/more robust solution than luasocket for http...I also found this:
-
You are right... this library won't work on non POSIX OS so it may not be the best choice.
My setup works now but I am a bit off a tangent having to use an ffi library to fix a digest call.
I was just trying to bring it back to something which can be used more broadly in case others run into this problem:
on lua5.1... works as is with the old http-digest
on luajit... need to switch to luajit-request which depends on ffi-libcurl.cqueues is what that library uses for... async calls. I am wondering how it compares to your async implementation.
-
So... I modified the client.lua file on openLuup to switch to the luajit-request libcurl wrapper I posted above just to see what it does... it dropped my cpu utilization further down to 1.1% from 1.5% at idle while not really impacting the memory load. interesting... My openLuup instance does rely on a lot of http calls, local ones are being handled by the servlet which has not been changed so only changing the ones it is making to the cloud and other local machines seems to be having an impact in spite of their relatively low frequency...
Edit: Well I kidded myself... The drop was actually due to updating the z-way bridge this morning and missing the fact that my openLuup json file was no longer there as I had manually switched everything to rapidjson, essentially killing part of the zway bridge. After correcting back to rapidjson the drop is really within noise (1.5% vs. 1.3%)
-
Reviving this thread... Now I am tempted to give Pallene a go just to see if it would have any problem with openLuup.
An interesting read here:
Given the amount of array handling openLuup does, I wonder if there is any gain Vs. LuaJIT.
@akbooer, thoughts about a lua5.3/5.4 version of openLuup?
-
Reviving this thread... Now I am tempted to give Pallene a go just to see if it would have any problem with openLuup.
An interesting read here:
Given the amount of array handling openLuup does, I wonder if there is any gain Vs. LuaJIT.
@akbooer, thoughts about a lua5.3/5.4 version of openLuup?
I considered this in depth a while ago (Aug 2018) here:
https://community.getvera.com/t/will-vera-ever-move-beyond-lua-5-1-5/199571/3
...but to bring the discussion onto this forum, here it is:
=======
I think you have to ask “why”?
Lua is a tiny language which, from the outset was meticulously designed to be concise, compact, transportable, and easy to use. It’s a huge credit to the implementers that the things which make it so attractive have changed so little since its inception.
If you look at the development history, outlined here: https://www.lua.org/versions.html, then you’ll see:
5.2
Its main new features are yieldable pcall and metamethods, new lexical scheme for globals, ephemeron tables, new library for bitwise operations, light C functions, emergency garbage collector, goto statement, and finalizers for tables.
5.3
Its main new features are integers, bitwise operators, a basic utf-8 library, and support for both 64-bit and 32-bit platforms.
5.4
Work on Lua 5.4 has begun but there is no release date yet nor a roadmap.
- new generational mode for garbage collection
- userdata can have multiple user values
- debug information about function arguments and returns
- new implementation for math.random
I don’t believe that most casual users would care much about the 5.2 additions, or even know what they meant. 5.1 already has metamethods, coroutines, ephemeral tables, etc. It seems to me that it’s really rather few problems which actually need more than this.
In 5.3, integers are more of an internal thing, there are already bit libraries, but I would grant that utf-8 support might be desirable for systems development. However, just for home automation logic…?
I must admit that I am a huge fan of Lua. I have to thank Vera for introducing me to the language. I chose 5.1 as the implementation language for openLuup, simply to be compatible with Vera. Whilst I’m confident that I could upgrade to 5.2 and beyond (significant changes in function environments due to the lexical scheme for globals, would be the main task) the concept of Vera/MiOS doing so fills me with dread… ever had a problem with one of their firmware upgrades…?
=======
...now, I may have moved on since I wrote this over two years ago, so I'm up for discussion.
Pallene is a somewhat different matter, and I'd be very keen to try this in a couple of areas (specifically, my Prolog-based HA logic engine.) There are benchmarks comparing it to JIT and it shines in a few places, but I don't see a pressing need.