Advice on Storing and Backing Up Bind Mounts - Docker
-
Hello all, after seeing Catman's posts about their disaster recovery and move to Docker I took that as a sign to migrate everything (aside from HA) to Docker. After a small learning curve I had Docker+Portainer up and running in a few days.
Instead of using named Volumes I opted to use Bind Mounts so I can easily edit conf files and any other file needed. I do understand the nuances that come with bind mounts, such as migration to a different host may require changing file structures, the possibility of someone editing the bind mount files and permissions but to me those aren't too big of a deal.
My question is what is the best way to keep a back up of these bind mounts? I currently have them stored in the
/etc
directory in another directory named on a per container basis. I was thinking to move it all to a/home/user/docker/
directory so that I can use a simplecp
command to my mounted SMB share to backup all the container data files. Anyone else do it differently?Side note: I finally got to flex the benefits of Docker with updating Reactor.. it was dead simple. I had no idea what I was missing out on lol!
-
Yeah, containers are a game changer for manageability.
I just have a /home/casa/containers (where casa is the user) where I have my configs/storage. Easy to backup/mirror. In case of disaster recovery, just copy everything and I'm ready. For consistency, I save my docker-compose file in the same dir. Restoring a container is very easy.
-
@therealdb is spot on here. I do something very similar. His last point, having the docker-compose file with the volatile data, is particularly important to speeding recovery. Another tip: practice doing recovery, and write yourself a text file in the same directory with the steps to do it, so when it happens, you're not (a) doing it for the first time, or (b) trying to remember what worked two years ago.
As a side note, in a past life, I wrote a "high availability" subsystem for a routing product -- software that ran on a master system and a number of hot standbys that would bring up a standby quickly when the master failed. I have often thought about doing that for Reactor, but always conclude that our world doesn't really demand that level of response, no matter how "cool" it may be. It's so easy and quick to recover containers, I feel like lt's good enough, and my time is better spent on other features.
-
Yeah, containers are a game changer for manageability.
I just have a /home/casa/containers (where casa is the user) where I have my configs/storage. Easy to backup/mirror. In case of disaster recovery, just copy everything and I'm ready. For consistency, I save my docker-compose file in the same dir. Restoring a container is very easy.
@therealdb said in Advice on Storing and Backing Up Bind Mounts - Docker:
Yeah, containers are a game changer for manageability.
I just have a /home/casa/containers (where casa is the user) where I have my configs/storage. Easy to backup/mirror. In case of disaster recovery, just copy everything and I'm ready. For consistency, I save my docker-compose file in the same dir. Restoring a container is very easy.
Perfect thats exactly what I was thinking of doing! Since I am using Portainer their equivalent of Docker Compose is Stacks and Portainer has a handy built in GUI back up feature which backs all that up. Just need to figure out how to automate it and store it off the host for easy future retrieval.
@toggledbits said in Advice on Storing and Backing Up Bind Mounts - Docker:
@therealdb is spot on here. I do something very similar. His last point, having the docker-compose file with the volatile data, is particularly important to speeding recovery. Another tip: practice doing recovery, and write yourself a text file in the same directory with the steps to do it, so when it happens, you're not (a) doing it for the first time, or (b) trying to remember what worked two years ago.
As a side note, in a past life, I wrote a "high availability" subsystem for a routing product -- software that ran on a master system and a number of hot standbys that would bring up a standby quickly when the master failed. I have often thought about doing that for Reactor, but always conclude that our world doesn't really demand that level of response, no matter how "cool" it may be. It's so easy and quick to recover containers, I feel like lt's good enough, and my time is better spent on other features.
I did actually already kinda do a practice recovery to make sure Docker was the right choice for me. I used an old pi I had kicking around to tinker with to get familiar with Docker before doing a clean wipe and install of Ubuntu on my main driver. Transferred everything over from the old Pi to my main host (odroid N2+) and it was painless. On top of that I was also able to set up a few more containers that I wouldn't have otherwise set up because Docker is so damn easy.
In my lifetime with home automation I actually (*knocks on wood) haven't suffered a complete failure that isn't self inflicted. However I have moved hosts pretty often as my setup continues to grow in size and complexity so having Docker now will allow me to continue moving up without having to worry about spending countless hours trying to install packages via the CLI.
-
@therealdb said in Advice on Storing and Backing Up Bind Mounts - Docker:
Yeah, containers are a game changer for manageability.
I just have a /home/casa/containers (where casa is the user) where I have my configs/storage. Easy to backup/mirror. In case of disaster recovery, just copy everything and I'm ready. For consistency, I save my docker-compose file in the same dir. Restoring a container is very easy.
Perfect thats exactly what I was thinking of doing! Since I am using Portainer their equivalent of Docker Compose is Stacks and Portainer has a handy built in GUI back up feature which backs all that up. Just need to figure out how to automate it and store it off the host for easy future retrieval.
@toggledbits said in Advice on Storing and Backing Up Bind Mounts - Docker:
@therealdb is spot on here. I do something very similar. His last point, having the docker-compose file with the volatile data, is particularly important to speeding recovery. Another tip: practice doing recovery, and write yourself a text file in the same directory with the steps to do it, so when it happens, you're not (a) doing it for the first time, or (b) trying to remember what worked two years ago.
As a side note, in a past life, I wrote a "high availability" subsystem for a routing product -- software that ran on a master system and a number of hot standbys that would bring up a standby quickly when the master failed. I have often thought about doing that for Reactor, but always conclude that our world doesn't really demand that level of response, no matter how "cool" it may be. It's so easy and quick to recover containers, I feel like lt's good enough, and my time is better spent on other features.
I did actually already kinda do a practice recovery to make sure Docker was the right choice for me. I used an old pi I had kicking around to tinker with to get familiar with Docker before doing a clean wipe and install of Ubuntu on my main driver. Transferred everything over from the old Pi to my main host (odroid N2+) and it was painless. On top of that I was also able to set up a few more containers that I wouldn't have otherwise set up because Docker is so damn easy.
In my lifetime with home automation I actually (*knocks on wood) haven't suffered a complete failure that isn't self inflicted. However I have moved hosts pretty often as my setup continues to grow in size and complexity so having Docker now will allow me to continue moving up without having to worry about spending countless hours trying to install packages via the CLI.
@Pabla said in Advice on Storing and Backing Up Bind Mounts - Docker:
Perfect thats exactly what I was thinking of doing! Since I am using Portainer their equivalent of Docker Compose is Stacks and Portainer has a handy built in GUI back up feature which backs all that up. Just need to figure out how to automate it and store it off the host for easy future retrieval.
Portainer is just a nice GUI - I'm using it too to do basic stuff like restarting a container or see the logs, especially when I'm remote. But at the end of the day, it's just some GUI over docker/docker-compose CLI commands:
cd /home/casa/containers/zwavejs/ && sudo docker compose pull && sudo docker compose up -d
and you're good to go. Easier to run 5-6 times per container than clicking on some random web page, but I know it's personal. The only thing that matters is a disaster/recovery strategy that you'll be able to quickly follow.