(Last Updated: July 3, 2020)

My migration from Vera, or what I did on my holidays



  • Well here we go. A slow, considered migration from Vera to something (I hope) more reliable.
    Hopefully we'll end up with a decent documented process for people as right now I'm still not quite clear 🙂
    Starting point:
    I have a 'mature' Mios set up running on an extrooted Vera Plus with roughly 100 devices, both physical and virtual
    Integration with Alexa both voice control and TTS
    iPhones for Geolocation
    Volumio devices for music playing around the house.
    I have no user defined scenes, all automation is run from Reactor.

    My impetus is final dissatisfaction with the quality of UI7 software delivery and operational rigour (for full disclosure I am a senior IT Operations leader with circa 20 years experience delivering properly stable systems, so I probably have un-realistic expectations 😄 )

    So initial steps:

    1. Raspberry Pi 3 B+ (I think this is my 5, and 6th Pi in total. I may have a problem)
    2. Raspbian
    3. OpenLuup
    4. Verabridge

    That I think is stage 1

    C



  • Well I'm going to stop now before I get too far ahead of myself. I'm quite au fait with Linux and so far simples.
    (I'm doing this on an Intel NUC I have lying around just for simplicity before the Pi arrives next week)
    Assuming you have a nice, up to date Debian install, and using the documentation here:

    First install Lua:

    # apt-get update
    # apt-get install lua5.1
    # apt-get install lua-socket
    # apt-get install lua-filesystem
    # apt-get install lua-sec
    

    Create your cmh-ludl directory. I used /etc/cmh-ludl for backwards compatability with Vera

    #mkdir /etc/cmh-ludl
    

    cd into that directory and then

    #wget https://github.com/akbooer/openLuup/raw/master/Utilities/openLuup_install.lua
    

    from the same directory:

    # lua5.1 openLuup_install.lua
    

    I had a momentary gotcha as lua5.1 is not in my $PATH. I might add it later but I had to use:
    #/usr/bin/lua openLuup_install.lua

    Assuming that all looks good check

    http://you_ip_address:3480 with a browser and be amazed that AltUI is already up and running.

    Next steps are to make OpenLuup start at boot. There are several documented ways. I went with systemctl for consistency.
    This is documented here

    However:

    sudo nano /etc/cmh-ludl/run_openLuup.sh
    

    Then

    #!/bin/bash
    echo "Starting openLuup server"
    echo "To see tail of logfile: tail -f ./out.log"
    cd /etc/cmh-ludl
    sudo rm ./out.log
    nohup ./openLuup_reload >> out.log 2>&1 &
    

    Make it executable

    sudo chmod +x run_openLuup.sh
    

    Create the service

    sudo nano /etc/systemd/system/openluup.service
    

    Add this code

    [Unit]
    Description=openLuup and AltUI Server for Vera 3
    Wants=network.target
    After=network.target
    
    [Service]
    Type=forking
    WorkingDirectory=/etc/cmh-ludl
    ExecStart=/bin/bash /etc/cmh-ludl/run_openLuup.sh
    
    [Install]
    WantedBy=multi-user.target
    

    Then enable it and make it so it starts at boot:

    sudo systemctl enable openluup
    sudo systemctl start openluu
    

    Reboot to check and it's all good

    So far so simple!

    C



  • Next we need to bridge to the Vera by installing Vera bridge from the Altui plugins

    In AltUI go to More>Plugins and hit the update button which is the chasing arrows next to the 'x' for delete. This will automatically install.

    When that's done head to the devices tab and hit the fresh and shiny Vera Bridge icon.
    Press 'Attributes' and look for the 'IP' box.
    Enter in there the IP of your physical Vera box (I typoed it the first time)
    When prompted, save the configuration and go to Misc>Reload Luup

    Now when you go to Devices you should see all your existing devices cloned in there.

    Not should as this is my first issue. Not all of mine have come over. And some of the ones that have are incomplete (e.g. switches that don't have a switch....)

    Errors in the LuaLogs example:

    2020-06-19 13:33:19.815   luup.create_device:: [10009] D_TemperatureSensor1.xml /  /    ()
    2020-06-19 13:33:19.815   openLuup.chdev:: ERROR: unable to read XML file D_TemperatureSensor1.xml
    

    Any thoughts as to where to go next?

    Also I have no 'GetVeraFiles' as the VeraBridge says clearly 'No Vera'

    I can see clearly communication to it in the log....

    2020-06-19 13:30:08.905   luup_log:5: http://192.168.70.6/port_3480/data_request?id=action&DeviceNum=41&newTargetValue=0&action=SetTarget&serviceId=urn:upnp-org:serviceId:SwitchPower1
    2020-06-19 13:30:09.237   openLuup.server:: request completed (2479 bytes, 1 chunks, 8189 ms) tcp{client}: 0x146b964
    

    Browsing some other posts I also have

    4	2020-06-19 13:33:20.332	5	5	Abort	0.199	4	plugin: VeraBridge	./openLuup/devices.lua:538: table index is nil
    

    Which might help

    TIA!

    C



  • It looks like it is missing the xml file. Have you imported the files from the vera? There is a command in the verabridge device to do just that. It will copy the content of the "cmh-lu" and "cmh-ludl" folders into the openluup "files" folder. If not, try importing them manually? Maybe you don't have the latest openLuup and verabridge version?



  • I can't as the button doesn't exist.

    VeraBridge device still says 'No Vera' even thought the IP is most definitely correct, oh and I pulled the files direct
    so OpenLuup is 20.5.22
    VeraBridge is from Master 20.4.30

    C



  • Hmm will wait for @akbooer on this one. I have never seen this and since I no longer have a verabridge or a vera, I can't even check.
    Try copying the temperature xml file from the vera and put it in openLuup?

    Edit: I just checked, and mine was definitely imported. It maybe a chicken and egg problem caused by a file import failure. Maybe try importing the entire /etc/cmh-lu and /etc/cmh-ludl content and dump them in /etc/cmh-ludl/files on openLuup. I suspect something has changed at some point on the vera firmware making this import function fail.



  • Yeah, that didn't help

    Still getting this as well:
    19/06/2020, 14:06:24 #4:VeraBridge:./openLuup/devices.lua:538: table index is nil

    I shall google while I wait for the esteemed @akbooer to enlighten me 🙂

    C



  • Hi Catman,

    Things you may need to install as well: cjson, dkjson, bit,.... jeez, I wish i made notes the last time i installed openLuup 😮

    Cheers Rene



  • @mrFarmer said in My migration from Vera, or what I did on my holidays:

    Hi Catman,

    Things you may need to install as well: cjson, dkjson, bit,.... jeez, I wish i made notes the last time i installed openLuup 😮

    Cheers Rene

    Rather why I'm doing this 😄

    So I have cjson and dkjson

    Did you mean something like 'bit' or was it a typo of 'but'?

    Nothing is different. I have deleted and re-installed the VeraBridge. Still says 'No Vera' even with the correct IP address, and only about 15 devices.

    Cheers

    C



  • Rather strange as your last error seem to indicate that the vera API returned an empty device table... Why would that be? Failing to decode json should have prompted another error but you already have that covered. All the files are in your openLuup installation folder right? (Mine is in /etc/cmh-ludl)



  • I have files in /etc/cmh/ludl however only the files that were installed but the Openluup script. Do you mean I should copy the .xml.lzo files to /etc/cmh-ludl (the same as my install?) or into /etc/cmh-ludl/files (where I copied them earlier)

    Cheers

    C



  • OpenLuup won’t decompress the lzo files. Therefore you would need to download them through UI7, one by one and then upload them in the openLuup installation. This is what the getverafiles does...
    Otherwise you could also SCP into the vera and run an lzo decompress of all the files before downloading them.
    The primary concern here though is why openLuup gets an empty device table from the vera now. It seems like you initially did get devices right? That’s why it was failing to create 10009 due to the missing xml file. This meant that it actually connected and that your vera device id 9 is a temperature sensor which openLuup was trying to mirror but was missing a file to do so.
    Now your error seems to be altogether different: It seems to not see the vera’s returned json file and that puzzles me.

    To decompress the files on the vera the command is:

    pluto-lzo -d file.lzo newfilename
    


  • It also has the rooms correctly cloned, though none of the devices that do exist are in the correct rooms.

    Assuming that the devices are created in numerical order makes no sense either as I have 10061 in OpenLuup for example

    I also have 10041 which works, but is in the wrong room (a simple switch)
    1009 is the temperature reading of a Fibaro 3-in-1

    Migh highest ID on Openluup is 79, the control for my boiler, everything from that is missing. Everything down is there but faulty.

    I think, given that part of the purpose here is to create a migration guide, I'll possess my sole in patience for @akbooer
    It's probably user error 😉

    C



  • I think they are missing because the xml files for these devices are missing from the openLuup installation which is the first problem you reported. This appears to indicate that the verabridge did connect. Why it failed to import the xml verafiles and why it subsequently shows that it can't connect is what is so strange.



  • @CatmanV2 said in My migration from Vera, or what I did on my holidays:

    http://you_ip_address:3480 with a browser and be amazed that AltUI is already up and running.

    🙂

    @CatmanV2 said in My migration from Vera, or what I did on my holidays:

    I shall google while I wait for the esteemed @akbooer to enlighten me

    Not seen this before, either. I’ll check the code, but, meantime, it’s not an issue with security settings on Vera, is it? Need to allow some ‘unsafe’ actions, IIRC.



  • @mrFarmer said in My migration from Vera, or what I did on my holidays:

    Things you may need to install as well: cjson, dkjson, bit,....

    There's nothing you need to install.

    dkjson is actually loaded by the openLuup_install script.



  • @CatmanV2

    This is the moment to look at the logs:

    • Startup log
    • Standard log, at the the start of it, after a system startup.

    These most easily available from the openLuup console Logs menu, or directly from:

    http://IP:3480/openLuup?page=log

    and

    http://IP:3480/openLuup?page=startup_log

    ...I'm confused by the 'Vera is running but No Vera shown'.



  • Thanks @akbooer

    Startup log and Lua log don't tell me anything that I find useful

    2020-06-19 15:57:41.027   :: openLuup STARTUP :: /etc/cmh-ludl
    2020-06-19 15:57:41.027   openLuup.init::        version 2020.05.01  @akbooer
    2020-06-19 15:57:41.041   openLuup.scheduler::   version 2020.01.25  @akbooer
    2020-06-19 15:57:41.051   openLuup.wsapi::       version 2019.08.12  @akbooer
    2020-06-19 15:57:41.051   openLuup.servlet::     version 2020.01.29  @akbooer
    2020-06-19 15:57:41.051   openLuup.client::      version 2019.10.14  @akbooer
    2020-06-19 15:57:41.052   openLuup.io::          version 2019.11.29  @akbooer
    2020-06-19 15:57:41.052   openLuup.server::      version 2020.03.20  @akbooer
    2020-06-19 15:57:41.055   openLuup.scenes::      version 2020.03.16  @akbooer
    2020-06-19 15:57:41.056   openLuup.chdev::       version 2020.03.07  @akbooer
    2020-06-19 15:57:41.056   openLuup.userdata::    version 2020.03.31  @akbooer
    2020-06-19 15:57:41.057   openLuup.requests::    version 2020.04.15  @akbooer
    2020-06-19 15:57:41.057   openLuup.gateway::     version 2020.03.08  @akbooer
    2020-06-19 15:57:41.058   openLuup.smtp::        version 2018.04.12  @akbooer
    2020-06-19 15:57:41.060   openLuup.historian::   version 2020.02.12  @akbooer
    2020-06-19 15:57:41.061   openLuup.luup::        version 2020.03.21  @akbooer
    2020-06-19 15:57:41.061   openLuup.pop3::        version 2018.04.23  @akbooer
    2020-06-19 15:57:41.062   openLuup.compression:: version 2016.06.30  @akbooer
    2020-06-19 15:57:41.062   openLuup.timers::      version 2019.05.03  @akbooer
    2020-06-19 15:57:41.062   openLuup.logs::        version 2018.03.25  @akbooer
    2020-06-19 15:57:41.062   openLuup.json::        version 2020.05.20  @akbooer
    2020-06-19 15:57:41.062   openLuup.init:: using Cjson 2.1.0 for fast decoding
    2020-06-19 15:57:41.062   luup.create_device:: [1] D_ZWaveNetwork.xml /  /    ()
    2020-06-19 15:57:41.062   openLuup.chdev:: ERROR: unable to read XML file I_ZWave.xml
    2020-06-19 15:57:41.068   luup.create_device:: [2] D_openLuup.xml / I_openLuup.xml / D_openLuup.json   (openLuup)
    2020-06-19 15:57:41.068   openLuup.init:: loading configuration user_data.json
    2020-06-19 15:57:41.068   openLuup.userdata:: loading user_data json...
    2020-06-19 15:57:41.072   luup.variable_set:: 2.openLuup.HouseMode was: EMPTY now: 1 #hooks:0
    2020-06-19 15:57:41.072   openLuup.userdata:: loading rooms...
    2020-06-19 15:57:41.072   openLuup.userdata:: room#1 'MiOS-50103066'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#2 'Garden'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#3 'Driveway'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#4 'Hall'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#5 'Living Room'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#6 'Server Room'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#7 'Porch'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#8 'ReactorCore'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#9 'Garage'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#10 'Master Bedroom'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#11 'Media_Centre'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#12 'Office'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#13 'X-Lab'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#14 'Dining Room'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#15 'Greenhouse'
    2020-06-19 15:57:41.072   openLuup.userdata:: room#16 'Kitchen'
    2020-06-19 15:57:41.072   openLuup.userdata:: ...room loading completed
    2020-06-19 15:57:41.072   openLuup.userdata:: loading devices...
    2020-06-19 15:57:41.073   luup.create_device:: [1] D_ZWaveNetwork.xml / X /    (urn:schemas-micasaverde-com:device:ZWaveNetwork:1)
    2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.starts' (a function value)
    2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.altui_split' (a function value)
    2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.template' (a function value)
    2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.trim' (a function value)
    2020-06-19 15:57:41.113   luup.create_device:: [3] D_ALTUI.xml / I_ALTUI.xml / D_ALTUI_UI7.json   (urn:schemas-upnp-org:device:altui:1)
    2020-06-19 15:57:41.117   luup.create_device:: [4] D_AltAppStore.xml / I_AltAppStore.xml / D_AltAppStore.json   (urn:schemas-upnp-org:device:AltAppStore:1)
    2020-06-19 15:57:41.123   luup.create_device:: [7] D_VeraBridge.xml / I_VeraBridge.xml / D_VeraBridge.json   (VeraBridge)
    2020-06-19 15:57:41.123   luup.create_device:: [10002] D_ZigbeeNetwork.xml /  /    ()
    2020-06-19 15:57:41.123   openLuup.chdev:: ERROR: unable to read XML file D_ZigbeeNetwork.xml
    2020-06-19 15:57:41.123   luup.create_device:: [10003] D_BluetoothNetwork.xml /  /    ()
    2020-06-19 15:57:41.123   openLuup.chdev:: ERROR: unable to read XML file D_BluetoothNetwork.xml
    2020-06-19 15:57:41.123   luup.create_device:: [10004] D_SceneController1.xml /  /    ()
    2020-06-19 15:57:41.124   openLuup.chdev:: ERROR: unable to read XML file D_SceneController1.xml
    2020-06-19 15:57:41.124   luup.create_device:: [10006] D_SceneController1.xml /  /    ()
    2020-06-19 15:57:41.124   openLuup.chdev:: ERROR: unable to read XML file D_SceneController1.xml
    2020-06-19 15:57:41.125   luup.create_device:: [10007] D_SceneController1.xml /  /    ()
    2020-06-19 15:57:41.125   openLuup.chdev:: ERROR: unable to read XML file D_SceneController1.xml
    2020-06-19 15:57:41.126   luup.create_device:: [10008] D_MotionSensor1.xml / X / D_MotionSensorWithTamper1.json   (urn:schemas-micasaverde-com:device:MotionSensor:1)
    2020-06-19 15:57:41.127   luup.create_device:: [10009] D_TemperatureSensor1.xml /  /    ()
    2020-06-19 15:57:41.127   openLuup.chdev:: ERROR: unable to read XML file D_TemperatureSensor1.xml
    2020-06-19 15:57:41.127   luup.create_device:: [10010] D_LightSensor1.xml /  /    ()
    2020-06-19 15:57:41.127   openLuup.chdev:: ERROR: unable to read XML file D_LightSensor1.xml
    2020-06-19 15:57:41.130   luup.create_device:: [10011] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.131   luup.create_device:: [10015] D_DimmableLight1.xml /  /    ()
    2020-06-19 15:57:41.131   openLuup.chdev:: ERROR: unable to read XML file D_DimmableLight1.xml
    2020-06-19 15:57:41.132   luup.create_device:: [10018] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.134   luup.create_device:: [10019] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.136   luup.create_device:: [10020] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.137   luup.create_device:: [10023] D_ALTUI.xml / X / D_ALTUI_UI7.json   (urn:schemas-upnp-org:device:altui:1)
    2020-06-19 15:57:41.138   luup.create_device:: [10026] D_DayTime.xml /  /    ()
    2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_DayTime.xml
    2020-06-19 15:57:41.138   luup.create_device:: [10027] D_HouseModes.xml /  /    ()
    2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_HouseModes.xml
    2020-06-19 15:57:41.138   luup.create_device:: [10031] D_DigitalSecurityCamera2.xml /  /    ()
    2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_DigitalSecurityCamera2.xml
    2020-06-19 15:57:41.138   luup.create_device:: [10033] D_DigitalSecurityCamera2.xml /  /    ()
    2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_DigitalSecurityCamera2.xml
    2020-06-19 15:57:41.140   luup.create_device:: [10037] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.141   luup.create_device:: [10041] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.143   luup.create_device:: [10057] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
    2020-06-19 15:57:41.144   luup.create_device:: [10061] D_DimmableLight1.xml /  /    ()
    2020-06-19 15:57:41.144   openLuup.chdev:: ERROR: unable to read XML file D_DimmableLight1.xml
    2020-06-19 15:57:41.144   luup.create_device:: [10062] D_DimmableLight1.xml /  /    ()
    2020-06-19 15:57:41.144   openLuup.chdev:: ERROR: unable to read XML file D_DimmableLight1.xml
    2020-06-19 15:57:41.144   luup.create_device:: [10079] D_Heater1.xml /  /    ()
    2020-06-19 15:57:41.144   openLuup.chdev:: ERROR: unable to read XML file D_Heater1.xml
    2020-06-19 15:57:41.145   openLuup.userdata:: loading scenes...
    2020-06-19 15:57:41.145   openLuup.userdata:: number of scenes = 0
    2020-06-19 15:57:41.145   openLuup.userdata:: ...scene loading completed
    2020-06-19 15:57:41.145   openLuup.userdata:: loading installed plugin info...
    2020-06-19 15:57:41.145   openLuup.userdata:: [openLuup] openLuup (20.5.22)
    2020-06-19 15:57:41.145   openLuup.userdata:: [AltAppStore] Alternate App Store (20.3.30)
    2020-06-19 15:57:41.145   openLuup.userdata:: [VeraBridge] VeraBridge (20.4.30)
    2020-06-19 15:57:41.145   openLuup.userdata:: [8246] Alternate UI (2.49b.)
    2020-06-19 15:57:41.145   openLuup.userdata:: [Z-Way] Z-Way (not.installed)
    2020-06-19 15:57:41.145   openLuup.userdata:: [Arduino] MySensors (not.installed)
    2020-06-19 15:57:41.145   openLuup.userdata:: ...user_data loading completed
    2020-06-19 15:57:41.145   openLuup.init:: running _openLuup_STARTUP_
    2020-06-19 15:57:41.145   luup_log:0: startup code completed
    2020-06-19 15:57:41.145   openLuup.init:: init phase completed
    2020-06-19 15:57:41.145   :: openLuup LOG ROTATION :: (runtime 0.0 days) 
    

    Is the start up log. There are no immediate errrors in the Lua log, but happy to poke it.

    What I mean by 'No Vera shown' is that the VeraBridge insists that there no Vera on the ip address:

    I'd attach a screenshot but I can't see how!

    C



  • @CatmanV2 said in My migration from Vera, or what I did on my holidays:

    There are no immediate errrors in the Lua log, but happy to poke it.

    It’s not errors that I’m specifically looking for just now, but startup sequences and versions. The first few hundred lines should suffice.



  • OK, after getting the log requested above, can you just try updating openLuup from the Plugins page, by pressing the update button on the openLuup Console Plugins page.


Log in to reply