Disaster recovery and virtualisation

[rafale77]

The controllers on the drawing are the best ones in that box in my opinion. As I said, almost every controller actually tries to fulfill the pyramid. Home Assistant has a pretty good GUI now though its controls of automation from the GUI is poor and the automation logic itself is pretty incomplete causing some people to run node-red as the front end for Home-Assistant. I run openLuup...
For the Integration layer, I run both Home-Assitant with all its integration component and openLuup and its many vera plugins, manually bridged together.
For the backend, I run z-way into openLuup and Bellows into Home-Assistant.

All boxes are important but each box depends on the box below it. As an illustration, when I start my system, I have to start it from bottom to top. When a controller is built and tested, it should be done from bottom to top as well in order to make sense and be effiscient.

The z-way-server also does all three and each API: the zwave API, JS API, Automation API, pretty much corresponds to each of the APIs I drew in order from bottom to top. It is just very good at the bottom layer but not as good as others as you go up the boxes. The best part of vera was the middle box with the community plugins and pretty complete virtual device set and is what openLuup has improved on.

[rafale77]

From my reading of different forums it doesn't look like Homeseer is doing all that great with their HS4 release. So between them and vera going down dead ends, it looks like Z-way and the in beta openzwave-QT will soon be the only contenders for zwave and I would slide in hubitat as third in spite of their lack of S2 support... Pretty sad to see good choices get smaller and smaller.

[CatmanV2]

I think part of the problem is that there are too many standards to chose from. None of them have a complete offering in terms of the hardware, the big boys have tried to create closed ecosystems (also not complete, or lacking in some other way) and very few (none) of the current offerings are succeeding at supporting the whole shooting match.

Feels like there are no decent devs / architects out there with a focus on a clear end model and some decent funding that's not making bad technical decisions.

C

[rafale77]

I think also that there is a business problem issue. The controller is absolutely critical and getting the right hardware and software is a lot of work. However they are being sold as a very low cost to attract new customers into the ecosystem while the real business is actually in the devices. The controller is only an enabler. Homeseer was the only one selling theirs at a high price... And then there is open source... Vera tried very hard to differentiate from the rest, attempting self maintenance and trying to get a lot of smart in the zwave management, taking control away from their customers. It tanked because not only was it mostly unnecessary but above all, it was very poorly designed. Let's see how the others do...

[CatmanV2]

@rafale77 that's a good point.

However, I fear that seeing how the others 'do' is pretty much how they've 'done' I had a X10 system running off a USB 'thing' and Indigo automation software. That was about 15 years ago. If anything it was easier and more reliable than stuff today (let's talk about how my magnetic door sensors don't un trip if you don't have the door open for more than about 3 seconds).

There is a significant difference that I have more money to throw at it than I did then, but progress seems, well, erratic. What do we have? Switches, dimmers and thermostats really.....

C

[sender]

I run QT-openzwave... beta but very steady and very easy! After the holiday it will by bye bye vera... almost all logics are already transferred to home assistant and node-red just some pleg and the "garaga door plugin" running on vera... I moved all problematic devices to QT and since then it's rock solid. Even Vera is fast. And since zwave devices are included (secure) in qt zwave I did not even know they were also so good and fast

[therealdb]

Vera’s zwave is a mess and I’m giving them a compliment.
I still think the idea behind Vera was good, the execution made it what it is. With more attention to details and the ability to really fix bugs, it could have been a good solution.

I still want a plug and play solution with easy customization, but maybe we’re just at the beginning and this market needs to reach its peak to become more mainstream.

[rafale77]

The closest to a plug and play local all in one is probably Hubitat at this point but they have their limitations in each area. For a large system with a lot of plugins, it seems like the RAM and CPU on their unit is a limiter.
Home Assistant is also going through some strange strategic changes and is really not plug and play... though that's what they aim towards but the rate of breaking changes and tedious "templating" for automation has me much prefer openLuup at this point. It is a bit of pitching lua vs. python+yaml...

[akbooer]

@rafale77 said in The Home Automation Controller Pyramid:

much prefer openLuup at this point

AT THIS POINT ...?

[rafale77]

haha... yeah and for the foreseeable future. I just started learning a little python because all the machine learning libraries seem to have a python API so it makes things much easier... But for home automation, I for one am way too invested in lua and two, don't see python as being suitable... which is, I am guessing, why Home Assistant has supplemented it with yaml. Gosh and I am not even a coder/developer by trade... I just am enjoying the learning.

[therealdb]

to be fair, ezlo seems to have implemented Zwave (and Zigbee) stack in a modern way. but I'm not convinced at all by their new programming environment. they're adopting an approach very similar to HA's, where you can launch scripts to intercept events, instead of having services always executing. So, openluup it's probably the safest bet in terms of features and stability for luup refugees.

[therealdb]

and yes, I don't like python neither. I learned LUA quite easily (being a dev for the latest 25 years helped) and I was really tempted to build my own engine running C# (on Linux, it's xplat now thanks to .NET Core) at some point, and move all my logic there, and use some hub just to send ZWave commands. I have already tons of code doing integrations with Vera, MiLight, OpenSprinkler, MQTT running in my own C# service, but it was just for fun.
The real problem is more about building a compelling UI, today, and offer side services (Alexa/GoogleHome, remote access, etc) than anything else.
A couple of friends approached me (because you know, home automation addiction has grown in me) because they wanted to build something and I always told them to not do anything like I did, because it needs maintenance. I dream of a day when we'll get a real plug&play system. We're not there, clearly.

[CatmanV2]

@therealdb said in The Home Automation Controller Pyramid:

to be fair, ezlo seems to have implemented Zwave (and Zigbee) stack in a modern way.

But what they (appear) to be shipping a beta seems way too immature. When would you think there's a production product? Feels to me like 2021 some time?

C

[therealdb]

@CatmanV2 said in The Home Automation Controller Pyramid:

But what they (appear) to be shipping a beta seems way too immature. When would you think there's a production product? Feels to me like 2021 some time?

yes. current system is barely usable. They are targeting next September, and they can improve fast, because they want to release a new build every 2 weeks. The real problem is their priorities: in order to migrate, you have to unpair and pair again all your devices. It's impossible for me (70+ devices, sometimes buried into boxes I really don't know where, since they were first installed by my electricians when building the house) and for many more people.
The concerning problem is that it's quite impossible for them to have plug-ins ready, because their new runtime is completely different and porting is not easy. But from a Zwave standpoint, they seem to have learned the lesson, by decoupling devices from implementation, and by allowing device definition to be updated separately. We'll see, it's probably too early and a lot of customers will migrate to anything better when the product will be really ready.

[CatmanV2]

@therealdb said in The Home Automation Controller Pyramid:

next September

2020 or 2021? 2021 seems more likely. Short sprints only means lots of progress if you're actually capable of writing code. Given the number of recurring issues, this seems something that may be beyond them.
And who's going to hang around for 2021?

C

[akbooer]

@rafale77 said in The Home Automation Controller Pyramid:

I just started learning a little python because all the machine learning libraries seem to have a python API so it makes things much easier... But for home automation, I for one am way too invested in lua and two, don't see python as being suitable...

My view entirely! Python is a huge language compared to Lua. Its real plus is the vast number of libraries, but the huge minus is the size of the system. Parts of openLuup (the Historian's graphite_cgi module, and the Whisper database were translated from Graphite's Python code, so I had to learn it a bit.

@CatmanV2 said in The Home Automation Controller Pyramid:

I think part of the problem is that there are too many standards to chose from.

Indeed... always the problem with 'Standards'.

[CatmanV2]

@akbooer

Indeed... always the problem with 'Standards'.

Of course, we only need one. TCP/IP would do just dandy!

C

[akbooer]

@rafale77 said in The Home Automation Controller Pyramid:

all the machine learning libraries seem to have a python API so it makes things much easier...

Yes, that seems to be the case. One AI application I use applies neural nets to image processing for astro-photography.

However...

... another big project at the moment (which has been on the “to do” list for a very long time), is a Prolog interpreter (written in Lua, of course.) I’m currently catching up on the last 30 year’s worth of scientific papers in the field of logic and constraint programming. Recently came across a 2017 paper outlining, what I consider to be, a real breakthrough in implementing the Prolog language, so that’s where my time is going.

If it all turns out well, it may even be included as an openLuup plugin! It also touches on this project...

https://smarthome.community/topic/146/ai-deep-learning-implementation-shifting-gear-in-home-automation

[rafale77]

@therealdb said in The Home Automation Controller Pyramid:

to be fair, ezlo seems to have implemented Zwave (and Zigbee) stack in a modern way. but I'm not convinced at all by their new programming environment. they're adopting an approach very similar to HA's, where you can launch scripts to intercept events, instead of having services always executing. So, openluup it's probably the safest bet in terms of features and stability for luup refugees.

More modern way than vera, no question. Anything else would have been pretty difficult to do but from my observation has been that they are still basing a lot of their focus on addressing the vera problems... which are very vera centric and no other platform has: They still want to go after devices one by one rather than look at zwave command class overall. I heard from other device vendors that they want to charge them to test their devices. They appear to be looking at a very closed and controlled device support environment and the whole idea of having a collocated mesh of controllers connected over IP as I discussed back at the old place displays their ignorance of how Zwave works. Again it is focused on fixing vera problems no other zwave system has, in a complicated way, "vera style".
But what made me walk away is the significant downgrade in the hardware vs. vera. A huge disappointment showing clearly that they tried to use the cheapest possible, ignoring reliability and future scalability (the i/o capability ranging from the bus to the ethernet chipset and the grade of the cpu and memory silicon are all downgraded vs the vera plus) and second the huge changes in the API which I felt was unnecessary and requiring a lot of work to adjust to the discontinuity to adopt something which is unreliable, not as mature in terms of feature and absolutely less scalable than what we have while many other solutions are available on the market. What is the point? They also seem to insisted on building their firmware backwards starting from the bling bling, eye candies at the top of the pyramid from 2 years ago and only now working on the core base which of course will delay everything because of the dependencies. They also definitely have gone cloud first which will makes the vision of having a cloud independent system much more challenging to implement. So indeed lots of pain (exclude include), lots of changes (plugins and API), to downgrade a system (hardware) to get a less reliable and scalable system which for now remains cloud dependent... no thank you. In comparison my migration to zway on openluup, though not plug and play involved none of the exclusion/inclusion, very few plugin and no API changes to get to much more scalable system on my own hardware (which I can later upgrade and move around) and has been cloud independent from day 1. It is much faster, more reliable, more integrated and scalable than ezlo can ever be.

What they should have done was to put design a core firmware from the beginning (the base of the pyramid), release it on a much more modern and scalable hardware (something like an atomic pi or an nvidia jetson, or even stay on the vera plus hardware rather than a $7 board used for middle school lego projects), put all these UI and mobile app work on the back-burner and put their efforts on making the core radio control stack work on the same API as vera (put the second layer on top of the base of the pyramid). Then work on migrating all the existing plugin and integrations. These should have been done last year... and now they could have had a product upon which they could build further integrations, UI and mobile app at their leisure, having migrated over the existing vera customer base. They are currently a competitor for no one... though they see themselves as being competitors to even this forum which sells nothing... I have not followed any of their progress but really have lost interest since I felt they were heading full speed towards a dead end.

[rafale77]

@akbooer said in The Home Automation Controller Pyramid:

@rafale77 said in The Home Automation Controller Pyramid:

all the machine learning libraries seem to have a python API so it makes things much easier...

Yes, that seems to be the case. One AI application I use applies neural nets to image processing for astro-photography.

However...

... another big project at the moment (which has been on the “to do” list for a very long time), is a Prolog interpreter (written in Lua, of course.) I’m currently catching up on the last 30 year’s worth of scientific papers in the field of logic and constraint programming. Recently came across a 2017 paper outlining, what I consider to be, a real breakthrough in implementing the Prolog language, so that’s where my time is going.

If it all turns out well, it may even be included as an openLuup plugin! It also touches on this project...

https://smarthome.community/topic/146/ai-deep-learning-implementation-shifting-gear-in-home-automation

That would be awesome, I am myself thinking about eventually getting some of the video processing working directly on openLuup rather than going through home assistant...