Using overlayfs

Question

1

How can I switch a folder to read-only mode so I can backup, but letting the server to write transparently to a journal file

rated 0 times [ 1] [ 0] / answers: 1 / hits: 945 / 2 Years ago, sun, october 2, 2022, 3:37:11

I have a very large server (Minecraft) using tens of gigabytes of files. Every two days, I make a backup, but since the server is so big, I have to shut it down for 45 minutes until the backup is done.

If I backup while the server is running, the backup folder gets corrupted because not all files were backed up at the same time and there's duplicate data, missing data and other problems. Chmodding the server folder recursively to 440 (read-only) makes the server spam thousands of errors, and sometimes crashes it or corrupts data.

What I am trying to do is to switch the server to read-only mode, but to allow it to write to temporary "journal" files which will be written to the real files once the backup is finished.

Is this possible at all? If yes, what tool should I use to do this? I was thinking something such switching the server folder to a soft link to a folder or partition that simulates this.

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

anoalk

Add To Favorites

Follow

Total Points: 271

Total Questions: 90

Total Answers: 112

Location: Zambia

Member since Wed, Dec 16, 2020

3 Years ago

anoalk questions

1 Explanation of files under /etc/NetworkManager?

Wed, Mar 15, 23, 04:06, 1 Year ago

1 Can I install software from debian package without internet

Fri, Sep 24, 21, 02:59, 3 Years ago

1 Ubuntu Server VM (Hyper-V): formatting of sda2 partition (mkfs.ext4 -G 4096 /dev/sda2) before install

Tue, Oct 5, 21, 14:02, 3 Years ago

1 Start KeePassXC on boot

Sat, Mar 5, 22, 02:33, 2 Years ago

1 How do I change the date language in the terminal?

Tue, Mar 22, 22, 07:19, 2 Years ago

View All

answered 2 Years ago ateact · Accepted Answer

Using overlayfs

Use overlayfs on the root of your server (server folder).

Make a read-only link to the server folder, from which you can back up the data while the real directory is covered

mkdir -p /path/to/ReadOnlyServerRoot
sudo mount --bind /path/to/serverRoot /path/to/ReadOnlyServerRoot -o remount,ro

Shutdown the server so that files and databases are properly closed

Protect actual server root, /path/to/serverRoot, from write.

sudo mount -t overlayfs overlayfs /path/to/serverRoot -o rw,uppderdir=/path/to/changes,lowerdir=/path/to/serverRoot

The changed and newly added data is stored at /path/to/changes. When a file is modified a new copy is made on /path/to/changes/path/to/file and this file is updated.

Start the server

Now you can backup your data from /path/to/ReadOnlyServerRoot

Shutdown server after the backup is completed

Remove the cover
```
sudo umount /path/to/serverRoot
```

Use a synchronization software, such as rsync, to merge the change data into the server file.
```
rsync /path/to/changes /path/to/serverRoot

rm -rf /path/to/changes
```

Start the server

More information on on overlayfs's option can be found on this answer.

Steps 2, 4, 6, and 9 are optional, but highly recommended: At the time of creating and removing overlay on server folder it is highly recommended to shutdown your server (only seconds of downtime) for a successful backup as server may cache partial data on their memory and only a part of the data would be updated. As a result backup contains invalid/unusable data which in turn lead to lost data.

Using a file system snapshot

switch to a file system that support snapshot like btrfs or lvm

Update

If overlayfs is not available use aufs or unionfs

unionfs

    mount -t unionfs -o dirs=/branch_rw=rw:/branch_ro=rounionfs/union

creates a Union in directory /union with the branch directories /branch_rw(writable) and /branch_ro (read-only).

aufs

mkdir /tmp/dir1

mkdir /tmp/aufs-root

mount -t aufs -o br=/tmp/dir1:/tmp/dir2 none /tmp/aufs-root/

The first two commands created 2 new directories. The mount.aufs is the command to mount the filesystem as Union mount.
The mount command, specifies it is going to union mount /tmp/dir1 and /tmp/dir2 under /tmp/aufs-root. The directory /tmp/aufs-root will have the content of both /tmp/dir1 and /tmp/dir2.