Disaster recovery and virtualisation

[akbooer]

@parkerc said in Discover IR codes for lost remote:

Please could someone help me to translate this into Lua

for file in $IRDB_BASE/codes/*/*.csv:
    for line in $file:
        (cmd_name,protocol,D,S,F) = split $line ','
         if match('power', $cmd_name):
               echo "Trying signal $protocol $D $S $F" 
               pronto = $(irptransmogrifier render -n D=$D,S=$D,F=$F $protocol)
               sendIr $pronto
               sleep $a_while
       endif
   endfor
endfor

becomes, in Lua:

local lfs = require "lfs"
local dir = "codes/"   -- make this your path to the files
local split = "[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"  -- (cmd_name,protocol,D,S,F)

for file in lfs.dir(dir) do
    if file: match "%.csv$" then
      for line in io.lines(dir .. filename)do
        local cmd_name,protocol,D,S,F = list: match (split)
        if 'power' == cmd_name then
          print ("$protocol $D $S $F",  protocol,D,S,F)
        end
      end
   end
end

I've just replaced the IR send with a print statement for now.

[parkerc]

Thanks @akbooer

Some quick questions .

1. Where would be the best place to store/sync the .csv files ?
2. FYI to help test, here is an example/extract of the Samsung TV .csv

[functionname,protocol,device,subdevice,function
INPUT SOURCE,NECx2,7,7,1
POWER,NECx2,7,7,2
1,NECx2,7,7,4
2,NECx2,7,7,5
3,NECx2,7,7,6
VOLUME +,NECx2,7,7,7
4,NECx2,7,7,8
5,NECx2,7,7,9
6,NECx2,7,7,10
VOLUME -,NECx2,7,7,11
7,NECx2,7,7,12
8,NECx2,7,7,13
9,NECx2,7,7,14
MUTE,NECx2,7,7,15
CHANNEL -,NECx2,7,7,16
0,NECx2,7,7,17
CHANNEL +,NECx2,7,7,18

By the looks of it to get the pronto version of the code, you would have to used the IrpTransmogrifier tool - https://github.com/bengtmartensson/IrpTransmogrifier

Where would be the best place to install that - is Vera a safe place or should I look elsewhere e.g a Pi ?

[akbooer]

What were you thinking of using to send the codes?

I note that luup has an ir table with a single function in it. Also, thanks to @a-lurker it's implemented in openLuup. IT mentions GC100 in the comments:

-- IR module
-- thanks to @a-lurker for this.
-- see: http://forum.micasaverde.com/index.php/topic,37268.0.html

local ir = {
  pronto_to_gc100 = function (pronto)
    -- replace the pronto code preamble with the GC100 preamble

...but I have absolutely no idea about its functionality, not ever having used an IR devices at this level. I would have thought that these codes ARE the pronto codes?? (...or are they IRP notation protocols?)

[parkerc]

@akbooer said in Discover IR codes for lost remote:

What were you thinking of using to send the codes?

I was ideally looking to use my Broadlink RM-Pro device and @a-lurker ‘s Vera plugin

@akbooer said in Discover IR codes for lost remote:

I have absolutely no idea about its functionality, not ever having used an IR devices at this level. I would have thought that these codes ARE the pronto codes?? (...or are they IRP notation protocols?)

I’m getting a bit out of my depth here now too,..

which is so often the case when I embark on these kind of HA adventures .

My interpretation is that when using irdb, you have to convert the code for them them to be Pronto codes. - http://irdb.tk/convert/

If I use my Samsung TV as an example, that seems to use the necx2:7.7 protocol - which I’ve also seen written as NECx2, device 7.7. Either way that seems to be a very common TV code set for Samsung, Akai and some other brands.

So, looking at their api, I assumed I could at least try to call a specific code.

http://irdb.tk/api/code/?brand=Samsung&devicetype=TV

And then..

http://irdb.tk/api/code/?brand=Samsung&devicetype=TV&protocol=NECx2&device=7&subdevice=7&functionname=POWER

but alas no ir code was returned only this json, which perhaps is not surprise...

{
  "meta": {
    "model": "code", 
    "next": "", 
    "page": 1, 
    "previous": ""
  }, 
  "objects": [
    {
      "function": "2", 
      "protocol": "NECx2", 
      "subdevice": "7", 
      "devicetype": "TV", 
      "device": "7", 
      "functionname": "POWER"
    }
  ]
}

Any ideas on their api - http://irdb.tk/api ?

[a-lurker]

parkec

You haven't told us what hardware or software (MiOS/openLuup) you are using. If you read AK's code above you would just have to point to where the files are. Presumably the file(s) could be on a USB stick. You want to get AK's coe running first, on just one file first - one step at a time.

Later you need this bit:

bengtmartensson commented 16 hours ago •

Just install IrpTransmogrifier as shell command and call it (as shell program) using (I think) os.execute or io.popen in Lua. No need to call java from lua
pronto = $(irptransmogrifier render -n D=$D,S=$D,F=$F $protocol)

The database looks like it could be huge and would take a while (who knows - a day?) to cycle around all the IR codes. Are you going to sit there and watch your device all day to catch when it turns on? AK's code needs to get the pronto code and then send it. Something like this (but totally untested):

-- Send an IR code every two seconds
-- Function needs to be global as it's a delay timeout target
function sendProntoCodeToBroadLinkDevice(prontoCode)
   local BROADLINK_DEVICE_ID = 49
   luup.call_action('urn:a-lurker-com:serviceId:IrTransmitter1', 'SendProntoCode', {ProntoCode = prontoCode}, BROADLINK_DEVICE_ID)
end

function scanIndexFiles ()
    local lfs = require "lfs"
    local dir = "codes/"   -- make this your path to the files
    local split = "[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"  -- (cmd_name,protocol,D,S,F)
    
    local prontoCode = ''

    for file in lfs.dir(dir) do
        if file: match "%.csv$" then
          for line in io.lines(dir .. filename) do
            local cmd_name,protocol,D,S,F = list: match (split)
            if 'power' == cmd_name then
              print ("$protocol $D $S $F",  protocol,D,S,F)

              local irptransmogrifierPath = "insert path here"

              -- this may all need work depending on whatever "irptransmogrifier" does
              local command = string.format("%sirptransmogrifier render -n D=%s,S=%s,F=%s protocol",irptransmogrifierPath,D,S,F,protocol)
              local handle = assert(io.popen(command))
              local prontoCode = handle:read("*a")
              handle:close()

              local INTERVAL_SECS = 2
              luup.call_delay('sendProntoCodeToBroadLinkDevice', INTERVAL_SECS, prontoCode)

            end
          end
       end
    end
end

scanIndexFiles()

Note: luup.call_delay only works in the openLuup window test window - I'm still not clear why. But the Vera and AltUI test windows won't run it correctly. You need the delay so the BroadLink device has time to send the code. (the function sleep() is not suitable)

[parkerc]

@a-lurker said in Discover IR codes for lost remote:

You haven't told us what hardware or software (MiOS/openLuup) you are using.

Sorry,, I’m looking first to Vera/MIOS , with the Broadlink RM Pro4, and your plug-in.

@a-lurker said in Discover IR codes for lost remote:

The database looks like it could be huge and would take a while (who knows - a day?) to cycle around all the IR codes. Are you going to sit there and watch your device all day to catch when it turns on? AK's code needs to get the pronto code and then send it. Something like this (but totally untested):

To avoid sitting around as the devices in question register for an IP, which I can fix, I was thinking I can watch for that variable to be updated .e.g via ping test or another option could be to place a light sensor right in front of the power LED to react when the light comes on..

@a-lurker said in Discover IR codes for lost remote:

You want to get AK's coe running first, on just one file first - one step at a time.

Agreed, and here’s an example .csv of the Samsung TV code/set they store. /codes/

functionname,protocol,device,subdevice,function
INPUT SOURCE,NECx2,7,7,1
POWER,NECx2,7,7,2
1,NECx2,7,7,4
2,NECx2,7,7,5
3,NECx2,7,7,6
VOLUME +,NECx2,7,7,7
4,NECx2,7,7,8
5,NECx2,7,7,9
6,NECx2,7,7,10
VOLUME -,NECx2,7,7,11
7,NECx2,7,7,12
8,NECx2,7,7,13
9,NECx2,7,7,14
MUTE,NECx2,7,7,15
CHANNEL -,NECx2,7,7,16
0,NECx2,7,7,17
CHANNEL +,NECx2,7,7,18
LAST,NECx2,7,7,19
MENU,NECx2,7,7,26
INFO,NECx2,7,7,31
AD/SUBT,NECx2,7,7,37
EXIT,NECx2,7,7,45
E-MANUAL,NECx2,7,7,63
TOOLS,NECx2,7,7,75
GUIDE,NECx2,7,7,79
RETURN,NECx2,7,7,88
CUSOR UP,NECx2,7,7,96
CURSOR DOWN,NECx2,7,7,97
CURSOR RIGHT,NECx2,7,7,98
CURSOR LEFT,NECx2,7,7,101
ENTER,NECx2,7,7,104
CH LIST,NECx2,7,7,107
SMART HUB,NECx2,7,7,121
3D,NECx2,7,7,159

And overall size wise, I assume - as they are not storing every individual code and Ir format/type, instead it’s just the protocol and some other identifier - the whole irdb/code folder seems reasonably small.

[therealdb]

Are you sure you can’t open those devices and look at the electronics? It seems to be simpler than this, to me

Maybe, also, some repair shop could help you. They usually deal with these kind of things and maybe they have something smarter to identify them. May be worth a try.

[parkerc]

@therealdb said in Discover IR codes for lost remote:

Are you sure you can’t open those devices and look at the electronics? It seems to be simpler than this, to me

These items are racked up so it;s not quite as easy as I’d like that option to be - but I am weighing up the hassle of doing that vs doing this

The benefit of making this code/process work is that a) i will have learnt something new - b) This could then be something g that helps others and c) it would give the Broadlink plug-in access to a huge ir bank of codes for people to use.

[parkerc]

Hi,

I’ve been working with the code that was shared (thanks so much), and have trimming it down a bit to work it through in sections, but - here’s my current issue - any ideas ?

(I) if I put the .csv file in the main search folder it does not return any errors, but it also does not print anything out.

/mnt/nas/vera/ircodes/7,7.csv

LuaTest 1.7

Lua file: /etc/cmh-ludl/luatest.lua

Results
No errors
Runtime: 51.4 ms
Code returned: nil

Print output
(none)

Code
   1    
   2       local lfs = require "lfs" 
   3       local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
   4       local split = "[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"  -- (cmd_name,protocol,D,S,F) 
   5        
   6       local prontoCode = '' 
   7    
   8       for file in lfs.dir(dir) do 
   9           if file: match "%.csv$" then 
  10             for line in io.lines(dir .. filename) do 
  11               local cmd_name,protocol,D,S,F = list: match (split) 
  12               if 'power' == cmd_name then 
  13                 print ("$protocol $D $S $F",  protocol,D,S,F) 
  14               end 
  15             end 
  16          end 
  17      end

• When I run it again, I get this, should filename be local or specified ?

LuaTest 1.7

Lua file: /etc/cmh-ludl/luatest.lua

Results
Runtime error: Line 9: attempt to concatenate global 'filename' (a nil value)

Locals
[main]
lfs={ _VERSION="LuaFileSystem 1.6.2", symlinkattributes=function, lock_dir=function, dir=function, _DESCRIPTION="LuaFileSystem is a Lua library developed to complement the set of functions related to file systems offered by the standard Lua distribution", setmode=function, unlock=function, _COPYRIGHT="Copyright (C) 2003-2012 Kepler Project", currentdir=function, touch=function, mkdir=function, lock=function, link=function, attributes=function, chdir=function, rmdir=function }
dir="mnt/nas/vera/ircodes/"
split="[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"
prontoCode=""

Print output
(none)

Code
   1       local lfs = require "lfs" 
   2       local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
   3       local split = "[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"  -- (cmd_name,protocol,D,S,F) 
   4        
   5       local prontoCode = '' 
   6    
   7       for file in lfs.dir(dir) do 
   8           if file: match "%.csv$" then 
   9             for line in io.lines(dir .. filename) do 
  10               local cmd_name,protocol,D,S,F = list: match (split) 
  11               if 'POWER' == cmd_name then 
  12                 print ("$protocol $D $S $F",  protocol,D,S,F) 
  13               end 
  14             end 
  15          end 
  16      end

II) I know we are doing this in sections, but as there are many layers of manufacturer folders and device type sub folders, will the eventual end code systematically search for all *.csv files within all the levels below? E.g.

/mnt/nas/vera/ircodes/Samsung/TV/7,7.csv
/mnt/nas/vera/ircodes/Panasonic/TV/128,0.csv
/mnt/nas/vera/ircodes/Panasonic/TV/128,1.csv
/mnt/nas/vera/ircodes/Panasonic/TV/128,4.csv
/mnt/nas/vera/ircodes/Panasonic/TV/128,9.csv
/mnt/nas/vera/ircodes/Panasonic/DVD Player/128,0.csv /mnt/nas/vera/ircodes/Panasonic/DVD Player/160,0.csv /mnt/nas/vera/ircodes/Panasonic/DVD Player/160,16.csv
/mnt/nas/vera/ircodes/Panasonic/DVD Player/160,18.csv
/mnt/nas/vera/ircodes/Panasonic/DVD Player/160,28.csv
/mnt/nas/vera/ircodes/Panasonic/DVD Player/160,34.csv
/mnt/nas/vera/ircodes/Panasonic/DVD Player/160,4.csv
/mnt/nas/vera/ircodes/Panasonic/Unknown_EUR7617010/170,3csv

I found this online, is that the sort of thing we will need to factor i later on..

Alex  /  Jan 12, 2012

Lua function to traverse all files in a folder recursively

Here is a little function that uses Lua File System (lfs module) in order to traverse all the files in a folder and its sub folders (and their sub folders) recursively. DIR_SEP=”\” &#82…

If I remove the ‘filename’ variable call, and specify a file, it then does this.

LuaTest 1.7

Lua file: /etc/cmh-ludl/luatest.lua

Results
Runtime error: Line 10: attempt to index global 'list' (a nil value)

Locals
[main]
lfs={ _VERSION="LuaFileSystem 1.6.2", symlinkattributes=function, lock_dir=function, dir=function, _DESCRIPTION="LuaFileSystem is a Lua library developed to complement the set of functions related to file systems offered by the standard Lua distribution", setmode=function, unlock=function, _COPYRIGHT="Copyright (C) 2003-2012 Kepler Project", currentdir=function, touch=function, mkdir=function, lock=function, link=function, attributes=function, chdir=function, rmdir=function }
dir="mnt/nas/vera/ircodes/"
split="[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"
prontoCode=""

Print output
(none)

Code
   1   local lfs = require "lfs" 
   2       local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
   3       local split = "[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"  -- (cmd_name,protocol,D,S,F) 
   4        
   5       local prontoCode = '' 
   6    
   7       for file in lfs.dir(dir) do 
   8           if file: match "%.csv$" then 
   9             for line in io.lines(dir .. '7,7.csv') do 
  10               local cmd_name,protocol,D,S,F = list: match (split) 
  11               if 'POWER' == cmd_name then 
  12                 print ("$protocol $D $S $F",  protocol,D,S,F) 
  13               end 
  14             end 
  15          end 
  16      end

[a-lurker]

"list" should be "line"

And /mnt/nas/vera/ircodes/Panasonic/Unknown_EUR7617010/170,3csv files are expected to end in ".csv"

And "file" should probably be changed to "filename"

Refer: https://keplerproject.github.io/luafilesystem/manual.html

Also I see you changed "power" to "POWER", which is correct but it's safer to do this: 'POWER' == cmd_name:upper() to guarantee you get uppercase. You are also using commas in your filenames. While this is allowed in Linux, I'm not so sure it should be encouraged. If that's how they come, so be it.

[parkerc]

Thanks, I’ve made a few adjustments based on feedback, and using just one .csv file for now, which it seems to find ok, based on the print output, but it unfortunately doesn’t extract the POWER information?

LuaTest 1.7

Lua file: /etc/cmh-ludl/luatest.lua

Results
No errors
Runtime: 9.9 ms
Code returned: nil

Print output
7,7.csv     

Code
   1    
   2       local lfs = require "lfs" 
   3       local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
   4       local split = "[^,]+,[^,]+,[^,]+,[^,]+,[^,]+"  -- (func_name,protocol,D,S,F) 
   5        
   6       local prontoCode = '' 
   7    
   8       for filename in lfs.dir(dir) do 
   9           if filename: match "%.csv$" then 
  10                  print(filename) 
  11             for line in io.lines(dir .. filename) do 
  12               local func_name,protocol,D,S,F = line: match (split) 
  13                  -- print(line) 
  14                  -- print (line: match (split)) 
  15               if 'POWER' == func_name:upper() then 
  16                 print ("$protocol $D $S $F",  protocol,D,S,F) 
  17                    print (func_name) 
  18               end 
  19             end 
  20          end 
  21      end

[akbooer]

My fault. A rookie error in the code (which, obviously, I hadn't tested.)

For your purposes, it's probably good enough to use the search string:

local split = "([^,]+),(%.*)" -- (func_name,protocol,D,S,F)

So this I have tested, and it works.

local lfs = require "lfs" 
local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
local split = "^([^,]+),(.*)"  -- (func_name,protocol,D,S,F) 
 
local prontoCode = '' 

for filename in lfs.dir(dir) do 
    if filename: match "%.csv$" then 
      print(filename) 
      for line in io.lines(dir .. filename) do 
        local func_name,etc = line: match (split) 
        if 'POWER' == func_name:upper() then 
          print ("$protocol $D $S $F",  etc) 
        end 
      end 
   end 
end

giving this output, for example:

7,7.csv
$protocol $D $S $F 	NECx2,7,7,2

[parkerc]

Thanks @akbooer

If I’m to use the irptransmogrifier command line interface, I’m going to need to get the format into something like this.

irptransmogrifier render -p -n D=7,S=7,F=2 NECx2

Here is the latest code,

local lfs = require "lfs" 
local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
local split = "^([^,]+),(.*)"  -- (func_name,protocol,D,S,F) 
 
local prontoCode = '' 

for filename in lfs.dir(dir) do 
    if filename: match "%.csv$" then 
      print(filename) 
      for line in io.lines(dir .. filename) do 
        local func_name,etc = line: match (split) 
        if 'POWER' == func_name:upper() then 
          print ("$protocol $D $S $F",  etc) 
	   for word in string.gmatch(etc, '([^,]+)') do
    	     print(word)
	      end
        end 
      end 
   end 
end

Which I’ve tried to update so that the returned values can be seen individually

Print output
7,7.csv
$protocol $D $S $F NECx2,7,7,2
NECx2
7
7
2

However I theoretically need them created into individual variables, however I’m not able to get name them ?

This was my attempt, which didn’t work..

local protocol, device, subdevice, function2 = string.match(etc, '(%a+) (%d+) (%d+) (%d+)')
print('Protocol = ' .. protocol)
print('Device = ' .. device)
print('SubDevice = ' .. subdevice)
print('Function = ' .. function2)

[a-lurker]

Once you get the variables you need, use this line instead of the above - I left out the "-p"

local command = string.format("%sirptransmogrifier render -p -n D=%s,S=%s,F=%s protocol",irptransmogrifierPath,D,S,F,protocol)

[parkerc]

Thanks, but that’s ultimate where I’m struggling.

I can’t work out how to take those 4 comma separated values, and make each one identifiable as a separate variable - D,S,F, protocol

This line returns what I need,

print ("$protocol $D $S $F",  etc) ,

which tells me the values i need are in etc so any idea why this didn’t work ?

local protocol, D, S, F = string.match(etc, '(%a+) (%d+) (%d+) (%d+)')
print('Protocol = ' .. protocol)
print('D = ' .. D)
print('S = ' .. S)
print('F = ' .. F)

[akbooer]

@parkerc said in Discover IR codes for lost remote:

any idea why this didn’t work ?

...because the syntax is wrong.

I've lost count of the number of times that I may have suggested to you that it might just be worthwhile to learn a little bit more about Lua. The effort that you are putting into this shows that a bit of learning will be a very good investment.

This will split the parameters:

local params = "([^,]+),([^,]+),([^,]+),([^,]+)"
local protocol, D,S,F = etc: match(params)

[parkerc]

@akbooer said in Discover IR codes for lost remote:

I've lost count of the number of times that I may have suggested to you that it might just be worthwhile to learn a little bit more about Lua.

I know, and believe me when I say. I honestly try to lean aspects of Lua when I can. but - I don’t know if you recall my responses in the past - my job and other commitments leave me so little time to learn anything for any length of time. And when I do find a pocket of time to play, I’m eager to create something new and interesting , and obviously use that as time to learn in the process.

To make it worse, not being a programmer, and not being able to work with LUA on a daily basis, so much of what I learn is sadly lost - through lack of use..

Not meant to be a sob story, just something I have to work with - hence I’m always so grateful for your help and the help of others..

[parkerc]

It looks like I finally have the main function in good shape Next on the list is to understand how the IrpTransmogrifier element fits in.

FYI - I’ve added a logging facility to the code, so that I can keep track of the .csv files accessed and the codes sent.

function scanIndexFiles ()
	local lfs = require "lfs" 
	local dir = "mnt/nas/vera/ircodes/"   -- make this your path to the files 
	local split = "^([^,]+),(.*)"  -- (func_name,protocol,D,S,F) 

    local prontoCode = ''

    for filename in lfs.dir(dir) do 
    	if filename: match "%.csv$" then 
  	    print('Filename =' ..filename) 
 	     for line in io.lines(dir .. filename) do 
			local func_name,etc = line: match (split) 
			local params = "([^,]+),([^,]+),([^,]+),([^,]+)"
			local protocol, D,S,F = etc: match(params)
				if 'POWER' == func_name:upper() then 
				-- print ("$protocol $D $S $F",  etc) 
				print('Protocol = ' .. protocol)
				print('D = ' .. D)
				print('S = ' .. S)
				print('F = ' .. F)

			  local irptransmogrifierPath = "insert path here"

              -- this may all need work depending on whatever "irptransmogrifier" does
			  
              local command = string.format("%sirptransmogrifier render -n D=%s,S=%s,F=%s protocol",irptransmogrifierPath,D,S,F,protocol)
			  print(command)
              local handle = assert(io.popen(command))
              local prontoCode = handle:read("*a")
              handle:close()
			  print(prontoCode)

				local time = os.date("%Y-%m-%d %H:%M:%S") 
				local file = io.open("/www/test_write.txt", 'a') 
				file:write(time .. "; " .. filename .. "; " .. protocol .. "; " .. D .. "; " .. S .. "; " .. F .. "\n")
				file:close()
			  
              local INTERVAL_SECS = 2
              --luup.call_delay('sendProntoCodeToBroadLinkDevice', INTERVAL_SECS, prontoCode)

            end
          end
       end
    end
end

[parkerc]

Current sticking point is - what do I do about the ‘Java’ requirement for getting IrpTransmogrifier to run on Vera.. Any recommendations?

To help narrow down my options; I’ve also posted this question on the Vera forum to see if anyone installed/used it.. -

Oct 4, 2020

Need a Java virtual machine (JVM, "Java runtime environment") to run something

Looking online, there are a few out there… http://jamvm.sourceforge.net/ Here’s an OpenWrt and JamVM here → Multithreaded Randomness: Java on OpenWRT … be good to know if anyone has installed/used one of them on Vera.

Hoping this is the final piece to the puzzle and then we’re hopefully good to go....

JamVM seems the most likely...

[akbooer]

I’d be really surprised if that fitted. What type of Vera is it?