Advice reqeusted to migrate MSR from Bare Metal to Container

[Cadwizzard]

Info:
Home Assistant Yellow - 2023.12.3
Reactor Community Add-on
SSL/SFTP enabled on the HA Yellow

I'm trying to migrate from a Pi and MSR installed with docker compose to the community add-on inside HA.
I located all my rules etc in the pi at /home/pi/reactor/storage, but I cannot find where the addon created the storage folder/rules on the HA Yellow.
I've searched the homeassistant directory and everything in root also for these files using various methods including *.json and rule but I cannot locate them.

I've a lot of rules and expressions, so need to copy rather than recreate them as I cant just export/import from reactor. I also want to be sure I can take reactor backups when needed on the HA Yellow

I posted an issue in the add-on maintainers github but they seem to respond to nothing posted there.

Any tips appreciated.

I also tried to just install using docker-compose in HAOS instead, but kept hitting the "Error response from daemon: error while creating mount source path '/home/': mkdir /home/reactordocker: read-only file system" error no matter what permissions were assigned to a folder or where I created it. I think perhaps because somethings containerised in that OS

[Pabla]

@Cadwizzard While I may not be able to help you out fully I can point you in the right direction. HA OS hides the docker containers that the add ons run in and the only way to access them is via the SSH add-on. In the SSH add-ons config you will need to turn off protection mode so you can navigate above the /homeassistant directory. From there you can run docker commands to get the container info and navigate each docker container. Importing MSR configs are a little passed my knowledge so we'll let Patrick or anyone else handle that.

[Cadwizzard]

Hi Pabla - Thanks for the tip. I disabled protection mode as I was trying to install my own docker container with MSR but kept hitting the write error above. (could only get the command to complete using \home and no subdirectories, but the files didnt appear still there and reactor looked running in cli but never presented the web page)

Update, found them for anyone else who needs them:
Inside the container at /data/reactor/storage/rules

Would be nice if there was a rules export/import option in the UI.

I suppose another question is, could I pull the updates to MSR in this container myself, the add-on maintainer isn't prompt to do that into the add-on.

(Once you have the rules into the addon initially, the HA plugin backup does handle them from there :
)

[therealdb]

Just copy the entire storage and config directories. Then modify your config if needed. That’s backup/restore in MSR world.

[Cadwizzard]

@therealdb That was the whole problem - I Couldn't find the storage directory in the destination

[Cadwizzard]

Be Warned:
Just in case anyone else is using the Add-on in HA, There is an issue where global expressions all get deleted on host reboot.
I had created a test rule when i first installed the add-on. and restarted the host with no issues. I did this because on github add-on page people have complained global expressions get deleted on host reboot and I wanted to test)
Since then i've created 20 or so more and needed to reboot the host. I was greeted with all the new expressions missing, and only the test one i had deleted since showing again.
Interestingly a backup of the addon i had taken when everything was fine last night, contained only the test rule. So something isn't saving the live configuration properly somewhere.
My solution: (luckily had a backup of expressions.json from my old host)
export a reactor add-on backup, replace just the expressions.json using z-zip, reupload the backup and restore it.
I've checked /data/reactor/storage/ and the correct file is now there.
Unsure if this is related to the addon being latest-23063-c464b685 or not.

If anyone knows if i can/a way to update this version inside the docker container the add-on creates, i'm all for trying it.
Maybe I could have copied this into the docker container, but I wanted to make sure I also had a backup that was restorable and copying into docker isn't as simple.

[toggledbits]

@Cadwizzard said in Location of the rules on filesystem (.json etc) in HA Add-On:

So something isn't saving the live configuration properly somewhere.

Any time you back up a running system, you run the risk that data hasn't been flushed from cache to non-volatile storage. This applies to everything, not just Reactor.

[Cadwizzard]

@toggledbits That I understand.
But in case I was unclear above. I had a lot of saved and working global expressions that were lost after a host restart. They referred back to one test expression I'd previously created. I then went to the backup to find that was the same

Is there a way I could update this docker container myself to the latest version of msr. The add-on maintainer seems absent and I don't want to fall behind.
Docker is not at all my speciality but I can follow guides/instructions.
I used docker-compose on the pi, but on the HA Yellow it looks like HA OS is is protected and wipes out anything not in the homeassistant directory or the installed plugin containers (including when I installed the docker-compose plugin).
Would it just be a case of logging into the container and running the normal docker pull commands, or running them outside the container but specifying the docker container in the command?

Many thanks!

[Pabla]

Don't mean to highjack this thread, but when I do a reactor backup, reactor is usually running when the backup occurs. Should I instead shutdown reactor then back it up?

[toggledbits]

@Cadwizzard said in Location of the rules on filesystem (.json etc) in HA Add-On:

Is there a way I could update this docker container myself to the latest version of msr.

I wouldn't know (I didn't create the HA Add-On and I don't provide support for it), but maybe @mrw298 can produce an updated version?

[Cadwizzard]

@Pabla I've been running Reactor for years across many platforms and never ever had an issue with backups doing weird things - until running the add-on install.
I never saw it with a docker install as per toggledbits installation guides.

This issue may be just related to the add-on as i've never heard of expressions disappearing before, until here and my install: https://github.com/mrw298/hassio-reactor-addon/issues

[Cadwizzard]

@toggledbits @mrw298 was last showing online here in March.
Thats why I was reaching out in this forum for wider experience

Do you or @Pabla happen to be able to answer a more generic question about Docker in the context of HA (and i'll run the risk myself of something going wrong).

Kind of forgetting that MSA was installed via the add-on, it currently exists in a docker container in HA yellow, which i can see and access via SSH
Should upgrading it mean running commands inside the container, or somehow specifying the addon container by name to update MSR inside it?

I'm really trying to learn more about how docker updates/linux inter-relate, but MSR is literally the only use case i have for it. I'm not a linux specialist, more of Windows sysadmin with vmware etc and only touch linux/docker for Home Assistant/MSR and PiHole, so there is a lot of trial an error with this for me It's not yet clear about the possibilities or the best way to interact with docker after someone/thing else has created containers like in the add-on.

@toggledbits btw your guides are actually great when installing MSR from scratch, thanks for taking the time with the documentation, it really helped previously move from Vera to Pi and keep MSR on the same hardware.

Any tips really appreciated.

[toggledbits]

You could attach to the running Reactor container (typically) using docker exec -it reactor /bin/sh. This assumes that the container name is reactor; you may need to use docker ps to establish the actual container name, or alternately, use its ID in the docker exec command.

From there, you should be able to go to the directory above the container's running Reactor: cd /opt

Then, you can download the latest Reactor:

    wget 'https://reactor.toggledbits.com/download/latest/<image-tarball-name>'

Then, un-tar the image in place: tar xvzf <image-tarball-name>. This will overwrite the Reactor install files native to the container with the files from the downloaded tarball.

Restart the container and it should be running on the installed/updated version.

A word of caution: This process only modifies the running docker container. It does not modify the image. I have no idea how HA handles the containers for add-ons. If the container is destroyed and recreated on every startup of HA or its add-ons, then your changes will go away, because the container is being rebuilt from the image, and the image has the original as-built Reactor files on it.

[Cadwizzard]

So, @toggledbits - thanks again.
It IS possible to update the docker container. It does survive a Home assistant restart..... but not a host reboot. The version reverts. So the container is rebuilt.

I've built a test Pi with HAOS on it for testing outside of Prod.
I created one test rule and one global expression while testing the updates.

Host reboot wiped out the global expressions again. @mrw298 There is definitely some fault with the add-on, this is a completely seperate install and hardware with no config, except the SSL and Reactor add-on installed.

I guess this leaves me to try to figure out where is safe to manually install reactor in HAOS and get round the "Error response from daemon: error while creating mount source path '[any path]': mkdir [any path]: read-only file system"
Update-
Actually the whole point of moving to HA Yellow was to be on a fully supported HAOS - So I'm going have to learn how to build my own local add-on to pull reactor into a supported container. My need is to run reactor on the same hardware as HA, and not have to rely on other plugin maintainers to stay in a supported MSR version

@toggledbits - have you considered building a Home assistant Add-on for Reactor at all?

Thanks again

[toggledbits]

@Cadwizzard said in Location of the rules on filesystem (.json etc) in HA Add-On:

have you considered building a Home assistant Add-on for Reactor at all?

No. There's more to it than building, as you are finding out. That also means I have to support it, including understanding how to do remote troubleshooting and debugging in that environment. It's not on my radar right now.

[Cadwizzard]

@toggledbits thanks.
I've modified the build.sh from the other add-on and set up docker hub. Despite not fully being able to understand what and where Linux read/write errors actually arise from, I've got the files built and pushed. Had to use a pi to build the arm and aarch, then Ubuntu in Windows to build the amd64 due to errors and warnings about architecture. Unsure if that has to be that way, or they should be buildable from one platform or not.
This was the better idea as when I tried to update the other add-on container, msr warned about node.js being out of date by doing it that way.
I've got my version loading from a local plug in and pulled/running from GitHub now on my test install. Only thing to work out
Only thing left to work out is why the alerts/entities aren't updating, but I'm thinking that's because the default in msr points to a dns name for HA. Which is likely resolving to my prod HA Yellow (incorrect login alerts on there also point to that). Was a rush before I needed to head away. I'll backup/remove and pull the new add-on to my prod. hardware once at a computer I can work properly from, then copy the rulesets etc into a backup for that and re-export it.
Appreciate the tips from you and @Pabla
It's a bit more work to build the files and pull into my own add-on but it's better I have the control and still run a supported HA install

Update: That was it, installed/updated Reactor on my HA Yellow, entities updating as expected