My migration from Vera, or what I did on my holidays

CatmanV2

Well here we go. A slow, considered migration from Vera to something (I hope) more reliable.
Hopefully we'll end up with a decent documented process for people as right now I'm still not quite clear
Starting point:
I have a 'mature' Mios set up running on an extrooted Vera Plus with roughly 100 devices, both physical and virtual
Integration with Alexa both voice control and TTS
iPhones for Geolocation
Volumio devices for music playing around the house.
I have no user defined scenes, all automation is run from Reactor.

My impetus is final dissatisfaction with the quality of UI7 software delivery and operational rigour (for full disclosure I am a senior IT Operations leader with circa 20 years experience delivering properly stable systems, so I probably have un-realistic expectations )

So initial steps:

Raspberry Pi 3 B+ (I think this is my 5, and 6th Pi in total. I may have a problem)
Raspbian
OpenLuup
Verabridge

That I think is stage 1

C

CatmanV2

Well I'm going to stop now before I get too far ahead of myself. I'm quite au fait with Linux and so far simples.
(I'm doing this on an Intel NUC I have lying around just for simplicity before the Pi arrives next week)
Assuming you have a nice, up to date Debian install, and using the documentation here:

openLuup/Documentation/openLuup User Guide.pdf at master · akbooer/openLuup

a pure-Lua open-source emulation of the Vera Luup environment - akbooer/openLuup

First install Lua:

# apt-get update
# apt-get install lua5.1
# apt-get install lua-socket
# apt-get install lua-filesystem
# apt-get install lua-sec

Create your cmh-ludl directory. I used /etc/cmh-ludl for backwards compatability with Vera

#mkdir /etc/cmh-ludl

cd into that directory and then

#wget https://github.com/akbooer/openLuup/raw/master/Utilities/openLuup_install.lua

from the same directory:

# lua5.1 openLuup_install.lua

I had a momentary gotcha as lua5.1 is not in my $PATH. I might add it later but I had to use:
#/usr/bin/lua openLuup_install.lua

Assuming that all looks good check

http://you_ip_address:3480 with a browser and be amazed that AltUI is already up and running.

Next steps are to make OpenLuup start at boot. There are several documented ways. I went with systemctl for consistency.
This is documented here
http://forum.micasaverde.com/index.php/topic,48431.0.html

However:

sudo nano /etc/cmh-ludl/run_openLuup.sh

Then

#!/bin/bash
echo "Starting openLuup server"
echo "To see tail of logfile: tail -f ./out.log"
cd /etc/cmh-ludl
sudo rm ./out.log
nohup ./openLuup_reload >> out.log 2>&1 &

Make it executable

sudo chmod +x run_openLuup.sh

Create the service

sudo nano /etc/systemd/system/openluup.service

Add this code

[Unit]
Description=openLuup and AltUI Server for Vera 3
Wants=network.target
After=network.target

[Service]
Type=forking
WorkingDirectory=/etc/cmh-ludl
ExecStart=/bin/bash /etc/cmh-ludl/run_openLuup.sh

[Install]
WantedBy=multi-user.target

Then enable it and make it so it starts at boot:

sudo systemctl enable openluup
sudo systemctl start openluu

Reboot to check and it's all good

So far so simple!

C

CatmanV2

Next we need to bridge to the Vera by installing Vera bridge from the Altui plugins

In AltUI go to More>Plugins and hit the update button which is the chasing arrows next to the 'x' for delete. This will automatically install.

When that's done head to the devices tab and hit the fresh and shiny Vera Bridge icon.
Press 'Attributes' and look for the 'IP' box.
Enter in there the IP of your physical Vera box (I typoed it the first time)
When prompted, save the configuration and go to Misc>Reload Luup

Now when you go to Devices you should see all your existing devices cloned in there.

Not should as this is my first issue. Not all of mine have come over. And some of the ones that have are incomplete (e.g. switches that don't have a switch....)

Errors in the LuaLogs example:

2020-06-19 13:33:19.815   luup.create_device:: [10009] D_TemperatureSensor1.xml /  /    ()
2020-06-19 13:33:19.815   openLuup.chdev:: ERROR: unable to read XML file D_TemperatureSensor1.xml

Any thoughts as to where to go next?

Also I have no 'GetVeraFiles' as the VeraBridge says clearly 'No Vera'

I can see clearly communication to it in the log....

2020-06-19 13:30:08.905   luup_log:5: http://192.168.70.6/port_3480/data_request?id=action&DeviceNum=41&newTargetValue=0&action=SetTarget&serviceId=urn:upnp-org:serviceId:SwitchPower1
2020-06-19 13:30:09.237   openLuup.server:: request completed (2479 bytes, 1 chunks, 8189 ms) tcp{client}: 0x146b964

Browsing some other posts I also have

4	2020-06-19 13:33:20.332	5	5	Abort	0.199	4	plugin: VeraBridge	./openLuup/devices.lua:538: table index is nil

Which might help

TIA!

C

rafale77

It looks like it is missing the xml file. Have you imported the files from the vera? There is a command in the verabridge device to do just that. It will copy the content of the "cmh-lu" and "cmh-ludl" folders into the openluup "files" folder. If not, try importing them manually? Maybe you don't have the latest openLuup and verabridge version?

CatmanV2

I can't as the button doesn't exist.

VeraBridge device still says 'No Vera' even thought the IP is most definitely correct, oh and I pulled the files direct
so OpenLuup is 20.5.22
VeraBridge is from Master 20.4.30

C

rafale77

Hmm will wait for @akbooer on this one. I have never seen this and since I no longer have a verabridge or a vera, I can't even check.
Try copying the temperature xml file from the vera and put it in openLuup?

Edit: I just checked, and mine was definitely imported. It maybe a chicken and egg problem caused by a file import failure. Maybe try importing the entire /etc/cmh-lu and /etc/cmh-ludl content and dump them in /etc/cmh-ludl/files on openLuup. I suspect something has changed at some point on the vera firmware making this import function fail.

CatmanV2

Yeah, that didn't help

Still getting this as well:
19/06/2020, 14:06:24 #4:VeraBridge:./openLuup/devices.lua:538: table index is nil

I shall google while I wait for the esteemed @akbooer to enlighten me

C

mrFarmer

Hi Catman,

Things you may need to install as well: cjson, dkjson, bit,.... jeez, I wish i made notes the last time i installed openLuup

Cheers Rene

CatmanV2

@mrFarmer said in My migration from Vera, or what I did on my holidays:

Hi Catman,

Things you may need to install as well: cjson, dkjson, bit,.... jeez, I wish i made notes the last time i installed openLuup

Cheers Rene

Rather why I'm doing this

So I have cjson and dkjson

Did you mean something like 'bit' or was it a typo of 'but'?

Nothing is different. I have deleted and re-installed the VeraBridge. Still says 'No Vera' even with the correct IP address, and only about 15 devices.

Cheers

C

rafale77

Rather strange as your last error seem to indicate that the vera API returned an empty device table... Why would that be? Failing to decode json should have prompted another error but you already have that covered. All the files are in your openLuup installation folder right? (Mine is in /etc/cmh-ludl)

CatmanV2

I have files in /etc/cmh/ludl however only the files that were installed but the Openluup script. Do you mean I should copy the .xml.lzo files to /etc/cmh-ludl (the same as my install?) or into /etc/cmh-ludl/files (where I copied them earlier)

Cheers

C

rafale77

OpenLuup won’t decompress the lzo files. Therefore you would need to download them through UI7, one by one and then upload them in the openLuup installation. This is what the getverafiles does...
Otherwise you could also SCP into the vera and run an lzo decompress of all the files before downloading them.
The primary concern here though is why openLuup gets an empty device table from the vera now. It seems like you initially did get devices right? That’s why it was failing to create 10009 due to the missing xml file. This meant that it actually connected and that your vera device id 9 is a temperature sensor which openLuup was trying to mirror but was missing a file to do so.
Now your error seems to be altogether different: It seems to not see the vera’s returned json file and that puzzles me.

To decompress the files on the vera the command is:

pluto-lzo -d file.lzo newfilename

CatmanV2

It also has the rooms correctly cloned, though none of the devices that do exist are in the correct rooms.

Assuming that the devices are created in numerical order makes no sense either as I have 10061 in OpenLuup for example

I also have 10041 which works, but is in the wrong room (a simple switch)
1009 is the temperature reading of a Fibaro 3-in-1

Migh highest ID on Openluup is 79, the control for my boiler, everything from that is missing. Everything down is there but faulty.

I think, given that part of the purpose here is to create a migration guide, I'll possess my sole in patience for @akbooer
It's probably user error

C

rafale77

I think they are missing because the xml files for these devices are missing from the openLuup installation which is the first problem you reported. This appears to indicate that the verabridge did connect. Why it failed to import the xml verafiles and why it subsequently shows that it can't connect is what is so strange.

akbooer

@CatmanV2 said in My migration from Vera, or what I did on my holidays:

http://you_ip_address:3480 with a browser and be amazed that AltUI is already up and running.

@CatmanV2 said in My migration from Vera, or what I did on my holidays:

I shall google while I wait for the esteemed @akbooer to enlighten me

Not seen this before, either. I’ll check the code, but, meantime, it’s not an issue with security settings on Vera, is it? Need to allow some ‘unsafe’ actions, IIRC.

akbooer

@mrFarmer said in My migration from Vera, or what I did on my holidays:

Things you may need to install as well: cjson, dkjson, bit,....

There's nothing you need to install.

dkjson is actually loaded by the openLuup_install script.

akbooer

@CatmanV2

This is the moment to look at the logs:

Startup log
Standard log, at the the start of it, after a system startup.

These most easily available from the openLuup console Logs menu, or directly from:

http://IP:3480/openLuup?page=log

and

http://IP:3480/openLuup?page=startup_log

...I'm confused by the 'Vera is running but No Vera shown'.

CatmanV2

Thanks @akbooer

Startup log and Lua log don't tell me anything that I find useful

2020-06-19 15:57:41.027   :: openLuup STARTUP :: /etc/cmh-ludl
2020-06-19 15:57:41.027   openLuup.init::        version 2020.05.01  @akbooer
2020-06-19 15:57:41.041   openLuup.scheduler::   version 2020.01.25  @akbooer
2020-06-19 15:57:41.051   openLuup.wsapi::       version 2019.08.12  @akbooer
2020-06-19 15:57:41.051   openLuup.servlet::     version 2020.01.29  @akbooer
2020-06-19 15:57:41.051   openLuup.client::      version 2019.10.14  @akbooer
2020-06-19 15:57:41.052   openLuup.io::          version 2019.11.29  @akbooer
2020-06-19 15:57:41.052   openLuup.server::      version 2020.03.20  @akbooer
2020-06-19 15:57:41.055   openLuup.scenes::      version 2020.03.16  @akbooer
2020-06-19 15:57:41.056   openLuup.chdev::       version 2020.03.07  @akbooer
2020-06-19 15:57:41.056   openLuup.userdata::    version 2020.03.31  @akbooer
2020-06-19 15:57:41.057   openLuup.requests::    version 2020.04.15  @akbooer
2020-06-19 15:57:41.057   openLuup.gateway::     version 2020.03.08  @akbooer
2020-06-19 15:57:41.058   openLuup.smtp::        version 2018.04.12  @akbooer
2020-06-19 15:57:41.060   openLuup.historian::   version 2020.02.12  @akbooer
2020-06-19 15:57:41.061   openLuup.luup::        version 2020.03.21  @akbooer
2020-06-19 15:57:41.061   openLuup.pop3::        version 2018.04.23  @akbooer
2020-06-19 15:57:41.062   openLuup.compression:: version 2016.06.30  @akbooer
2020-06-19 15:57:41.062   openLuup.timers::      version 2019.05.03  @akbooer
2020-06-19 15:57:41.062   openLuup.logs::        version 2018.03.25  @akbooer
2020-06-19 15:57:41.062   openLuup.json::        version 2020.05.20  @akbooer
2020-06-19 15:57:41.062   openLuup.init:: using Cjson 2.1.0 for fast decoding
2020-06-19 15:57:41.062   luup.create_device:: [1] D_ZWaveNetwork.xml /  /    ()
2020-06-19 15:57:41.062   openLuup.chdev:: ERROR: unable to read XML file I_ZWave.xml
2020-06-19 15:57:41.068   luup.create_device:: [2] D_openLuup.xml / I_openLuup.xml / D_openLuup.json   (openLuup)
2020-06-19 15:57:41.068   openLuup.init:: loading configuration user_data.json
2020-06-19 15:57:41.068   openLuup.userdata:: loading user_data json...
2020-06-19 15:57:41.072   luup.variable_set:: 2.openLuup.HouseMode was: EMPTY now: 1 #hooks:0
2020-06-19 15:57:41.072   openLuup.userdata:: loading rooms...
2020-06-19 15:57:41.072   openLuup.userdata:: room#1 'MiOS-50103066'
2020-06-19 15:57:41.072   openLuup.userdata:: room#2 'Garden'
2020-06-19 15:57:41.072   openLuup.userdata:: room#3 'Driveway'
2020-06-19 15:57:41.072   openLuup.userdata:: room#4 'Hall'
2020-06-19 15:57:41.072   openLuup.userdata:: room#5 'Living Room'
2020-06-19 15:57:41.072   openLuup.userdata:: room#6 'Server Room'
2020-06-19 15:57:41.072   openLuup.userdata:: room#7 'Porch'
2020-06-19 15:57:41.072   openLuup.userdata:: room#8 'ReactorCore'
2020-06-19 15:57:41.072   openLuup.userdata:: room#9 'Garage'
2020-06-19 15:57:41.072   openLuup.userdata:: room#10 'Master Bedroom'
2020-06-19 15:57:41.072   openLuup.userdata:: room#11 'Media_Centre'
2020-06-19 15:57:41.072   openLuup.userdata:: room#12 'Office'
2020-06-19 15:57:41.072   openLuup.userdata:: room#13 'X-Lab'
2020-06-19 15:57:41.072   openLuup.userdata:: room#14 'Dining Room'
2020-06-19 15:57:41.072   openLuup.userdata:: room#15 'Greenhouse'
2020-06-19 15:57:41.072   openLuup.userdata:: room#16 'Kitchen'
2020-06-19 15:57:41.072   openLuup.userdata:: ...room loading completed
2020-06-19 15:57:41.072   openLuup.userdata:: loading devices...
2020-06-19 15:57:41.073   luup.create_device:: [1] D_ZWaveNetwork.xml / X /    (urn:schemas-micasaverde-com:device:ZWaveNetwork:1)
2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.starts' (a function value)
2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.altui_split' (a function value)
2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.template' (a function value)
2020-06-19 15:57:41.113   openLuup.scheduler.sandbox:: device 3 defined 'string.trim' (a function value)
2020-06-19 15:57:41.113   luup.create_device:: [3] D_ALTUI.xml / I_ALTUI.xml / D_ALTUI_UI7.json   (urn:schemas-upnp-org:device:altui:1)
2020-06-19 15:57:41.117   luup.create_device:: [4] D_AltAppStore.xml / I_AltAppStore.xml / D_AltAppStore.json   (urn:schemas-upnp-org:device:AltAppStore:1)
2020-06-19 15:57:41.123   luup.create_device:: [7] D_VeraBridge.xml / I_VeraBridge.xml / D_VeraBridge.json   (VeraBridge)
2020-06-19 15:57:41.123   luup.create_device:: [10002] D_ZigbeeNetwork.xml /  /    ()
2020-06-19 15:57:41.123   openLuup.chdev:: ERROR: unable to read XML file D_ZigbeeNetwork.xml
2020-06-19 15:57:41.123   luup.create_device:: [10003] D_BluetoothNetwork.xml /  /    ()
2020-06-19 15:57:41.123   openLuup.chdev:: ERROR: unable to read XML file D_BluetoothNetwork.xml
2020-06-19 15:57:41.123   luup.create_device:: [10004] D_SceneController1.xml /  /    ()
2020-06-19 15:57:41.124   openLuup.chdev:: ERROR: unable to read XML file D_SceneController1.xml
2020-06-19 15:57:41.124   luup.create_device:: [10006] D_SceneController1.xml /  /    ()
2020-06-19 15:57:41.124   openLuup.chdev:: ERROR: unable to read XML file D_SceneController1.xml
2020-06-19 15:57:41.125   luup.create_device:: [10007] D_SceneController1.xml /  /    ()
2020-06-19 15:57:41.125   openLuup.chdev:: ERROR: unable to read XML file D_SceneController1.xml
2020-06-19 15:57:41.126   luup.create_device:: [10008] D_MotionSensor1.xml / X / D_MotionSensorWithTamper1.json   (urn:schemas-micasaverde-com:device:MotionSensor:1)
2020-06-19 15:57:41.127   luup.create_device:: [10009] D_TemperatureSensor1.xml /  /    ()
2020-06-19 15:57:41.127   openLuup.chdev:: ERROR: unable to read XML file D_TemperatureSensor1.xml
2020-06-19 15:57:41.127   luup.create_device:: [10010] D_LightSensor1.xml /  /    ()
2020-06-19 15:57:41.127   openLuup.chdev:: ERROR: unable to read XML file D_LightSensor1.xml
2020-06-19 15:57:41.130   luup.create_device:: [10011] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.131   luup.create_device:: [10015] D_DimmableLight1.xml /  /    ()
2020-06-19 15:57:41.131   openLuup.chdev:: ERROR: unable to read XML file D_DimmableLight1.xml
2020-06-19 15:57:41.132   luup.create_device:: [10018] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.134   luup.create_device:: [10019] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.136   luup.create_device:: [10020] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.137   luup.create_device:: [10023] D_ALTUI.xml / X / D_ALTUI_UI7.json   (urn:schemas-upnp-org:device:altui:1)
2020-06-19 15:57:41.138   luup.create_device:: [10026] D_DayTime.xml /  /    ()
2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_DayTime.xml
2020-06-19 15:57:41.138   luup.create_device:: [10027] D_HouseModes.xml /  /    ()
2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_HouseModes.xml
2020-06-19 15:57:41.138   luup.create_device:: [10031] D_DigitalSecurityCamera2.xml /  /    ()
2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_DigitalSecurityCamera2.xml
2020-06-19 15:57:41.138   luup.create_device:: [10033] D_DigitalSecurityCamera2.xml /  /    ()
2020-06-19 15:57:41.138   openLuup.chdev:: ERROR: unable to read XML file D_DigitalSecurityCamera2.xml
2020-06-19 15:57:41.140   luup.create_device:: [10037] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.141   luup.create_device:: [10041] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.143   luup.create_device:: [10057] D_BinaryLight1.xml / X / D_BinaryLight1.json   (urn:schemas-upnp-org:device:BinaryLight:1)
2020-06-19 15:57:41.144   luup.create_device:: [10061] D_DimmableLight1.xml /  /    ()
2020-06-19 15:57:41.144   openLuup.chdev:: ERROR: unable to read XML file D_DimmableLight1.xml
2020-06-19 15:57:41.144   luup.create_device:: [10062] D_DimmableLight1.xml /  /    ()
2020-06-19 15:57:41.144   openLuup.chdev:: ERROR: unable to read XML file D_DimmableLight1.xml
2020-06-19 15:57:41.144   luup.create_device:: [10079] D_Heater1.xml /  /    ()
2020-06-19 15:57:41.144   openLuup.chdev:: ERROR: unable to read XML file D_Heater1.xml
2020-06-19 15:57:41.145   openLuup.userdata:: loading scenes...
2020-06-19 15:57:41.145   openLuup.userdata:: number of scenes = 0
2020-06-19 15:57:41.145   openLuup.userdata:: ...scene loading completed
2020-06-19 15:57:41.145   openLuup.userdata:: loading installed plugin info...
2020-06-19 15:57:41.145   openLuup.userdata:: [openLuup] openLuup (20.5.22)
2020-06-19 15:57:41.145   openLuup.userdata:: [AltAppStore] Alternate App Store (20.3.30)
2020-06-19 15:57:41.145   openLuup.userdata:: [VeraBridge] VeraBridge (20.4.30)
2020-06-19 15:57:41.145   openLuup.userdata:: [8246] Alternate UI (2.49b.)
2020-06-19 15:57:41.145   openLuup.userdata:: [Z-Way] Z-Way (not.installed)
2020-06-19 15:57:41.145   openLuup.userdata:: [Arduino] MySensors (not.installed)
2020-06-19 15:57:41.145   openLuup.userdata:: ...user_data loading completed
2020-06-19 15:57:41.145   openLuup.init:: running _openLuup_STARTUP_
2020-06-19 15:57:41.145   luup_log:0: startup code completed
2020-06-19 15:57:41.145   openLuup.init:: init phase completed
2020-06-19 15:57:41.145   :: openLuup LOG ROTATION :: (runtime 0.0 days)

Is the start up log. There are no immediate errrors in the Lua log, but happy to poke it.

What I mean by 'No Vera shown' is that the VeraBridge insists that there no Vera on the ip address:

I'd attach a screenshot but I can't see how!

C

akbooer

@CatmanV2 said in My migration from Vera, or what I did on my holidays:

There are no immediate errrors in the Lua log, but happy to poke it.

It’s not errors that I’m specifically looking for just now, but startup sequences and versions. The first few hundred lines should suffice.

akbooer

OK, after getting the log requested above, can you just try updating openLuup from the Plugins page, by pressing the update button on the openLuup Console Plugins page.

15+ Smart Home Automation Ideas from Pros

Finally gave up on my Ezlo Plus

Another transition from Vera to OpenLuup And now to HASS - PerH

Half-way migration from Vera

My Home Automation Story

My migration from Vera, or what I did on my holidays

Not so much a blog

My Wandering

This is me

My Home Automation Setup...

My migration from Vera, or what I did on my holidays

openLuup/Documentation/openLuup User Guide.pdf at master · akbooer/openLuup

Recent Topics