Disaster recovery and virtualisation

[parkerc]

A while back the z-wave module on my veraplus stopped working and as it couldnt be fixed, I bought an external usb z-wave dongle UZB1 (which I plugged in and setup on ‘port’ /dev/ttyACM0 ) which made it operational again, and I’ve been using it again since.

Recently I picked up a new (well 2nd hand) VeraPlus unit and tried to do a full system/zwave network restore from the original one, but it didn’t work.

The restore looked like it worked fine, as it brought everything over, but when nothing worked, I checked the z-wave ‘port’ settings on the new unit, and it too had been set to use an external usb (/dev/ttyACM0) which it obviously doesn’t have , but when I changed it back to the onboard zwave module (/dev/ttyS0) all the z-wave devices disappeared.

To fix this, it sounds like I need to do a controller shift to get the details off the UZB1 key and onto the zwave chip on the veraplus board ?

However not having done one before I’m not sure if the process is - any ideas ?

[rafale77]

I am afraid a controller shift will not help in this case. The problem stems from rigidity of the restore tool of the vera. What you are observing is completely expected.
The vera controller backup contains two things:

1. The user-data file along with all your plugin data.
2. The zwave dongle network backup.

When you did the restore from backup step on your new vera, your user-data had your port as /dev/ttyACM0 so during the restoration the vera attempted to restore the zwave network to that port which obviously did not exist. What you need to do now that you changed the zwave serial port back to the vera onboard chip, is to manually grab the latest dongle backup file from the old vera (it's in /etc/cmh-ludl) and move it to the new vera in the same location. Use SFTP to do this.
Then SSH into the new vera and run

touch /etc/cmh-ludl/dongle.restore

then reload luup. It should normally restore your previous zwave network configuration into the new vera zwave chip.
Another alternative is of course to just plug your uzb zwave dongle into the new vera.... I actually prefer using USB these days for exactly this kind of moves.

To be clear on what a controller shift does:
It is a combination of inclusion of a new controller and moving the zwave master flag from the master controller to the new controller. The problem with this is the node numbering and will leave you with the vera not being node 1 on your zwave network. It could be ok but some devices may not like it. Sensors for example may have issues with their lifeline associations. The lifeline association is essentially a direct relation to the target controller a device reports its data to. The controller shift was very relevant for the time when zwave was just controlling lights from a controller which was a remote control. After that it has become less and less useful.

[parkerc]

Thanks @rafale77

Coincidentally Vera support got back to me with the following instructions. (FYI)

To do the Z-Wave network shift on the controller, you need to follow these steps:

• Make sure both controllers are connected to the same LAN.
• Go to "User & Account info" -> "Security" -> make sure the option "Enable UPnP" is checked.
• Go to "Settings" -> "Net & Wi-Fi" -> make sure the option "Auto-detect devices in my home network" is checked.
• On the controller that will send the network, go to "Settings" -> "Z-Wave Settings" -> "Advanced" -> click "Go" on "Controller shift."
• On the controller that will receive the network, go to "Settings" -> "Z-Wave Settings" -> "Advanced" -> click "Go" on "Copy Z-Wave network from a primary controller."
• The process should take place in the next few minutes.

[rafale77]

@parkerc

Nope, not the best answer. That will get you a new controller with a new zwave node ID. It will work for switches and lightbulbs but anything which needs instant status or sensor reporting will fail.

You will have to do a lot of reconfiguration of devices and some devices may not work. When I migrated my vera zwave network to z-way, I ended doing a lot of cleanup because of the number of times I had migrated from vera to vera. The screenshot below is one of the reason why I favor z-way. I have full visibility of what the associations are, what they are and I can update them. They are configuration parameters inside the devices. Not in the controller just like the neighbor node information.

In your case "cloning" the zwave dongle is the best answer but it is not documented or supported officially by vera.

[parkerc]

@rafale77

How is the controller shift instructions provided by Vera above , differ from your suggestion to manually grab the latest dongle backup file from the old vera (in /etc/cmh-ludl) and move it to the new vera, into the same location and then run ‘touch /etc/cmh-ludl/dongle.restore’

The both seem to be copying the Z-Wave network from a original (primary) controller to the new one ?

[rafale77]

No it is actually very different.

The controller shift is an actual zwave protocol function, not a vera function. What it does is:

1. include new controller (meaning add new controller to network and give it a new node ID, send network data to new controller over zwave (RF))
2. assign new controller as master, unassigned current controller master flag.

The old controller will remain node 1 until you use the new controller to exclude it. Actually I can't remember with if it actually excludes itself. I don't believe it does. You end up with 2 controllers on your network. One master, one slave and you have shifted the master from one to the other.

The process I gave you clones all the data from one controller chip to the other using the backup file over the serial API of the zwave dongle. It means that they will both have the same node ID "1" and you don't need to do anything to your devices. You however have to take down the old controller or your network will be confused with 2 controllers with the same ID. Or if you tinker like me, keep the old USB dongle as a backup in case the internal one fails so you can stick it in just in case but in any case, you should unplug it and not use both at the same time.

The whole upnp think they ask you to do is to make sure the two vera see one another so they can transfer your user-data... including your security key but you can do that using backup/restore.

[parkerc]

Does the version of z-wave firmware play any part in this ?

My UZB1 usb stick is on 4.05 L:1 - Role: Suc SIS: YES PRI: YES
The new Vera is on 6.01 L:1 - Role: Master SIS: NO PRI: YES

[rafale77]

The vera upgrades from SDK to SDK very well(unless the new chip has an even newer firmware which the vera did not know of). It is part of their firmware update process. It is downgrading which is more of a challenge. I don't think you'll have any problems.

Fun fact, I had even upgraded my vera's zwave chip firmware far beyond what vera officially supported and had no problems.

[parkerc]

Hi @rafale77

With your help, could we make this ‘shift’ run almost like a batch like process on the same device, by that I mean.

After doing a backup of the old Vera and a restore onto the new one.
Could I move the UZB1 USB dongle onto the new VeraPlus, ensuring it’s operational and working first under /dev/ttyACM0
Then is there a set of commands I could run that does the cloning straight off from /dev/ttyACM0 (UZB1) onto to /dev/ttyS0 (onboard) ?
And then ideally at the end of that, could it update the new Vera z-wave config to now look at /dev/ttyS0 (onboard) allowing me to move the UZB1 dongle (and store it away) ?

[rafale77]

It's not a very hard thing to do but I unfortunately gave away most of my veras and only have a couple of bricked ones left so I can't test this.
There was another way which was to manipulate the zwave serial port inside your backup user-data.json which is what I had to do but it takes decompressing your backup and the user-data.json.lzo and the compress it back and then restore that backup.
The silliness of the backup restore process is that it is very rigid and doesn't allow you to restore to a different serial port than what your original backup file. What I gave you should work though... manually grab the last dongle file from one unit, move it to the other one and with your UI already showing the new target serial port, start a dongle restore.

As I previously said, I also highly recommend you stay with the usb stick. It likely has greater range and upgradeability.

[parkerc]

@rafale77 said in Controller shift - How do you do it ?:

I also highly recommend you stay with the usb stick. It likely has greater range and upgradeability.

I’d actually seriously consider that, but the upgrading route for the z-way UZB1 dongle, seems so unnecessarily complicated .

One of the reasons for getting the new Vera was because a Fibaro RGBW Module was appearing as many multiple nodes, and Vera support had a look and seemed to suggest it was due to the z-wave version I was running.

parkerc said in Controller shift - How do you do it ?:

My UZB1 usb stick is on 4.05 L:1 - Role: Suc SIS: YES PRI: YES
The new Vera is on 6.01 L:1 - Role: Master SIS: NO PRI: YES

[rafale77]

upgrading firmware on a UZB is actually quite easy...
what is the brand of your stick?

Sep 28, 2018

Updating firmware on Silicon Labs Z-wave UZB USB sticks

There may be occasions when it becomes necessary to update the firmware of Silicon Labs Z-wave UZB controller USB sticks. Firmware updates may be needed when new SDKs are released or a change in UZB functionality is required. UZB firmware can be updated using Silicon Labs Windows based Z-Wave...

[parkerc]

@rafale77 said in Controller shift - How do you do it ?:

what is the brand of your stick?

Z-wave.me - ZMEEUZB

UZB

[rafale77]

Ahhhhh then this is what you need:

https://z-wave.me/zmeserialupdater-manual/

and it is indeed a little more complicated due to the need of following this crazy map:

https://service.z-wave.me/expertui/uzb-stats/versions-graph.html

[parkerc]

Yep, that’s it and I think you even helped me a while back, as that ‘crazy map’ was doing me head in.

Trying to follow it drove me crazy !

[rafale77]

I can help you read through it and break down the commands to upgrade if you want me to. What is your current firmware version?

[parkerc]

@rafale77 said in Controller shift - How do you do it ?:

What you need to do now that you changed the zwave serial port back to the vera onboard chip, is to manually grab the latest dongle backup file from the old vera (it's in /etc/cmh-ludl) and move it to the new vera in the same location. Use SFTP to do this.
Then SSH into the new vera and run
touch /etc/cmh-ludl/dongle.restore

then reload luup. It should normally restore your previous zwave network configuration into the new vera zwave chip

Hi @rafale77

Picking this up again as i have managed to find a bit of time - and looking on my Vera I can’t see that a dongle.restore.sh file, plus the dongle files seem to be in /etc/cmh rather than /etc/cmh-ludl see below..

root@MiOS_55100454:/# find -name dongle*.*
./etc/cmh/dongle.4.5.dump.0
./etc/cmh/dongle.4.5.dump.1
./etc/cmh/dongle.4.5.dump.2
./etc/cmh/dongle.4.5.dump.3
./etc/cmh/dongle.4.5.dump.4
./etc/cmh/dongle.4.5.dump.5
./ezmi/cmh/dongle.4.5.dump.0
./ezmi/cmh/dongle.4.5.dump.1
./ezmi/cmh/dongle.4.5.dump.2
./ezmi/cmh/dongle.4.5.dump.3
./ezmi/cmh/dongle.4.5.dump.4
./ezmi/cmh/dongle.4.5.dump.5
root@MiOS_55100454:/#

Any ideas where the dongle.restore file might be or if it’s called something different ?

[parkerc]

Have looked here too, which has helped a bit.. https://smarthome.community/topic/199/problem-migration-z-wave-nw-from-vera-to-uzb1-dongle?_=1632067117298

The issue seems to be the dongle.restore file requirements

If it helps, i found this within the ./mios/www/cgi-bin/cmh/restore.sh file

if [ "$FORM_dongle" == "1" ]; then
                # LuaUPnP will restore the Z-Wave backup file if it finds the 'dongle.restore.go' file.
                # To prevent LuaUPnP from restoring a dongle backup until the next boot,
                # we'll save the file as 'dongle.restore', and the 'Start_cmh.sh' script will rename it to 'dongle.restore.go'.
                log "Create $MIOS_CONF_PATH/dongle.restore"
                touch "$MIOS_CONF_PATH/dongle.restore" 2>>$log_file
        fi

And then looking in ./usr/bin/Start_cmh.sh the reference there is..

# to prevent LuaUPnP from restoring a dongle backup until the next boot, 
# the web ui saves .restore and LU looks for .go
mv /etc/cmh/dongle.restore /etc/cmh/dongle.restore.go || /bin/true

Sadly not much further forward…

[CatmanV2]

I don't want to be stupid, but have you actually made a backup? Unless it's automatic and storing locally.....

C

[parkerc]

Yep, I have made a local back up. (Inc z-wave network) and have it stored locally

Both Veras are on the network, I first went into the veraplus with the usb zwave dongle and did the full backup, saved it locally and then logged into the VeraSecure and did the restore.

Everything but the zwave devices was restored and based on what @rafale77 had said before my issue is that it’s trying to restore back to an /dev/ttyACM0 dongle, rather than the onboard one /dev/ttyS0

I somehow need to get the zwave records onto the right tty..