Disaster recovery and virtualisation

[parkerc]

Considering how long I’ve been using HA systems, where I was a relatively early adopter of z-wave, I’ve never really explored the Z-Wave Classes and Commands.

Don’t get me wrong I understand (well at least I think I do) the overall objective and the design of the classes and the command, but my exposure to it has always been hidden behind a software/application layer.. No bad thing your might say, but the more I got to know the Lua language , the more I wanted to know more about the Z-wave Classes and Command - and how I could interact with them a little more directly.

FYI - I’m still very much in the learning phase, and welcome any insight/advice from those more experienced than me.

For this particular ‘How To’, I’m going to start off with a link to all the recorded z-wave classes and commands, this is a great place to see the length and breadth of what z-wave supports.

https://www.silabs.com/documents/login/miscellaneous/SDS13548-List-of-defined-Z-Wave-Command-Classes.xlsx

My first objective is to see if I can request a battery report from a device.. (Make sure your device support this class)

To do that I need to look up the Commands_Class for Battery, and note down it’s associated value, which in this case is 0x80

COMMAND_CLASS_BATTERY	0x80	Management	Active	

After that, I then need to get the value associated with the specific command I want to send, which in this case is the Battery Get one from the following list..

BATTERY_GET				0x02	controlling	supporting
BATTERY_REPORT			0x03	supporting	controlling
BATTERY_HEALTH_GET		0x04	controlling	supporting	 
BATTERY_HEALTH_REPORT	0x05	supporting	controlling

Those two looks up have now provided me with the following two item values …

Battery Command Class = 0x80 
BATTERY_HEALTH_GET Command = 0x04

Next I need to send them to my Vera/Z-wave controller , specifying the z-wave ‘node’ I want to target (make sure it is the node number (alt.I’d, NOT the Device No.) , which in this case is node 3 (device no. 64) - using a command like the following....

luup.call_action('urn:micasaverde-com:serviceId:ZWaveNetwork1','SendData',{Node='3',Data='0x80 0x04'},1) 

Or externally..

http://192.168.1.10/port_3480/data_request?id=action&DeviceNum=1&serviceId=urn:micasaverde-com:serviceId:ZWaveNetwork1&action=SendData&Node=3&Data=0x80-0x04

Any/all feedback welcome..

[toggledbits]

Very well constructed. A couple of details I'd point out:

1. A subtle mistake users often make in this procedure is sending the SendData command to the device. It cannot be stressed enough that the SendData action is always and only sent to device #1, the ZWave Network device (i.e. the last parameter of call_action() for SendData is always 1). A lot of users will get the node ID parameter set correctly, construct the data string correctly, and then send the action to the device they want to control rather than the ZWave Network device #1, and then puzzle over why it doesn't work.
2. The hex may be abbreviated, for example, x80 (the leading zero is not required), and any value may also be given in decimal form, and you can mix decimal and hex (e.g. x70 4 10 1 15 sets configuration parameter decimal 10 to the one-byte decimal value 15 on the target device).
3. Many ZWave commands require multi-byte values. For example, to send a 16-bit integer (range 0-65535 unsigned or -32768 to 32767 signed), you must send two bytes. To send 4096, for example, you cannot just put 4096 or 0x1000 in your data string, as these are not valid byte values (which are 0-255/0x00-0xff range). If you look at the hex representation of 4096 -- 0x1000 -- you see four hexadecimal digits; split them into pairs and send them like this: 0x00 0x10. Notice the order of the two bytes has been reversed. This is referred to as little endian in network communications: the least significant byte is sent first, followed by progressively more significant bytes. A four-byte (32 bit) value, for example 0x12345678, would be sent as 0x78 0x65 0x34 0x12. Notice that the order of bytes is reversed, but within the bytes, there is no reversal of bits (that is 0x78 does not become 0x87). If the command you are using requires a byte count, remember to account for this splitting in your number of bytes (the 0x1000 example adds 2 to the length of the command because it takes two bytes to represent the one 16-bit number). Always send leading zeroes to get the expected number of bytes: if a 16-bit integer is expected for the command and you need to send 0, you must send 0 0 or 0x00 0x00.
4. There is no easy way on Vera to intercept any data that may be a reply to the request/command. If you issue a command that replies with a packet that Luup does not understand, it will be logged but not otherwise handled, or available. This is a big short-coming of Vera in terms of allowing third-party support of ZWave devices. Hubitat, for example, has solved this problem, and third party ZWave device drivers are quite common, and not difficult to use, write or support.
5. There is no way that I am aware of to do this on eZLO hubs with current firmware (there is no documented equivalent to SendData). I have pointed this out; the reply thus far has been deafening silence, not even so much as an acknowledgement that it may be added to a list for future consideration..

[a-lurker]

All a bit tricky. The OpenZWave people spent ages reverse engineering the protocol. The logViewer plugin I wrote - it's for Vera/OpenWRT only - was subsequently augmented by @gengen. Putting the logging into verbose mode invokes his "ZShark". It allows a good view of the ZWave interchanges. Definitely don't leave your Vera in the logging verbose mode fo any length of time.

[therealdb]

I have a Qubino ZMNHBD1 that I'm trying to reconfigure without messing with device numbers, but it's not really working.

I'm sending param 13 2-byte device to 0, but it doesn't seem to work (this will disable auto-off after 1 sec, since in 7.32 beta there's a bug preventing auto-off from being updated, so I've repurposed all my devices doing this, attaching external relays, or using Shellies).

luup.call_action("urn:micasaverde-com:serviceId:ZWaveNetwork1", "SendData",
    { Node='128', Data= "x70 4 13 2 0"}, 1 )

Any hints? Device is set to auto configure to off. Next step is to remove, reset the device and pair again, but I'd like to try this route first.

[toggledbits]

The first thing that stands out is latter part of my point #3 above: you are saying in the codes that parameter 13 is coming with a 2-byte value, but you are only sending one byte after the count. Try sending x70 4 13 2 0 0 so that you are sending the upper and lower bytes of the two-byte value. Remember that SendData sends bytes, not values. It's up to you to "decompose" your values into the correct number and sequence of bytes that represents the value.

One thing I also forgot to mention in my missive above for Vera in particular, is that the refresh and data validity around the "Device Options" in the UI is almost non-existent, and the values shown are often wrong. You have to do the following in order to see correct values:

1. Set to monitor only any parameter in Device Options that you intend to control via SendData. If you don't do this, every Luup restart will reconfigure the device and clobber your values, and all you will ever see is what Luup wants to set or has set.
2. Some users turn off auto-reconfigure, and it is often recommended quite casually in the other forums. IMO, this is probably the worst advice I see being given in that community today, at least when given casually. It should only be used as a last resort for devices that errantly destroy and create children and wreak havoc with your device numbering, scenes and automations. But in any case, if you turn it off, the UI will never show you correct Device Option parameter values. Because...
3. When inspecting values, the device must be reconfigured in order for the value to be picked up by Device Options, as during ZWave startup is the only time Luup sets or fetches these values. So if you are looking at the Device Options UI after a SendData, it will be wrong, and even after a hard refresh, it will be wrong (relative to your expectations), unless and until you reload Luup, or, reconfigure the device. Sometimes, you can get away with hitting "Configure Now", but I find that sometimes, for reasons I fear I will ever understand, attempting to just say "Configure Now" fails when configuration during a Luup reload works, an inexplicable dilemma (they should be one and the same function and work identically). And after either reload or "Configure Now", you must hard-refresh your browser, because the UI does not see changes in the state variables they use for these values reliably.

Also keep in mind that some devices will not keep the value you send, they will only act on it. For example, some of my Aeotec devices will reset their parameters to factory default if you set the value of a parameter to 255. When looking at the value, it is always reported as 0, never as 255, even though it gets the value when sent and does in fact reset itself as directed. Likewise my siren will accept the number of a melody to play in a parameter, but that parameter always reports as 0, even after playing melody 1, 10, 77, ....

[therealdb]

@toggledbits said in Z-Wave Classes & Commands:

Try sending x70 4 13 2 0 0 s

that did the trick. I really didn't paid attention to this detail. it works this way, in fact. I'm not really monitoring the params, but I know the relay will not do a second click after 1 sec after sending on, so I'm know it's working well. I'm not messing so much with parameters these days, since the system is almost stable and I've added many new devices (I still have 3-4 added but not already deployed, but that's another story). Thanks, as always!

[Black Cat]

@toggledbits Would it be possible to add a function to MSR that returns the parameters of a Z-Wave device?

Vera comes close but you need to know the parameter to be able to find the setting of it, punching in 254 values one at a time is very laborious and time consuming, especially if you find that there are no further parameters......FWIW .Homeseer doesn't have this ability at all, perhaps someone can comment of HASS or other HA platforms in use?
Some will question why do you need to do this?
Apart from discovering any unknown parameters just because they are there, Z-Wave device manufacturers do not publish all parameters that are used in a device - generally because they have been added after the device has been certified and publishing it renders Certification void.
Would it be useful? Yes, as an example a Device we had in a Z-Wave net didn't behave as expected. An email exchange between the manufacturer divulged a "hidden parameter" which when switched on caused the device to behave correctly. If we had known that there was an extra Parameter then we would have tested it to find out what it did. It may or may not have worked but we would have had the opportunity to try it beforehand.
Anyway, it's useful to know when engaging with manufacturers, it gives you the upper hand......

[toggledbits]

At this stage, probably not, just for the lack of consistency. The issues:

1. Vera, we know, keeps the data in state variables, and we get what we get. You can get that today -- just look at the attribute on the entity in MSR and see as much information as it could publish. On a Vera ZWave device, look for x_vera_svc_micasaverde_com_ZWaveDevice1.VariablesGet.
2. On Hass, not only does the API not tell you what integration drives an entity, but even if you forced that association by declaring the ZWave service, the service itself doesn't publish any attributes/data from the integration through the API. So the configuration variables aren't visible at all. Nor is the device's manufacturer info, model number, or even its ZWave node ID. Mind you, Hass has all of this information, it must in order to function, it just doesn't publish any of it through its APIs.
3. On Hubitat, pretty much the same story as Hass: the API doesn't tell you what driver supplies the device, so you don't even know it's ZWave, and there's no data available attached to the device that's published through the API related to ZWave, anyway.
4. On eZLO, they do identify the device by its parent "protocol" (zwave), and give you the ZWave node ID and a couple of flags, but no other useful information that I've seen. There aren't even API commands available for single-node things like poll a node or set a config value, only network-wide commands for things like start/stop inclusion or reset the Zwave chip and delete the entire network (i.e. not things you usually want to do in an automation). Since I'm probably the only person who cares if those other things exist and I'm persona non grata over there, they're probably not getting done any time soon, whether I report them or not. I haven't used any devices on my eZLO system that have configuration parameters, so I can't tell if or how it publishes those. If someone has included on their eZLO system a device that normally uses configuration parameters, like an Aeotec Multisensor, etc., please send me via PM your logs/ezlo_data_list.json file. I'd love to see if it has anything.

Very low on my priorities (particularly with the big changes to the core/Engine of late) has been direct integration with ZWave-JS. That would expose all the right information, I'm sure. But that's really no small task, and in essence, transforms MSR from an automation tool into the realm of being a hub itself. Maybe that's not bad.