Building a Resilient Proxmox Backup Server: From Standard LVM to RAID5 over Multipath

Setting up a Proxmox Backup Server (PBS) is usually straightforward, but when you introduce external storage arrays, multipath I/O, and the need for rock-solid redundancy, the terminal quickly becomes your best friend.

Recently, I went through the process of provisioning a new PBS instance connected to a SAN. What started as a standard LVM deployment quickly turned into a lesson on pivoting to LVM RAID5 on the fly. If you’re looking to build a resilient backup datastore, here is a breakdown of my workflow, the commands I used, and the thought process behind the pivot.

Phase 1: Taming the Storage Array (Multipath I/O)

When you attach a server to a storage array via multiple paths (like dual-port Fiber Channel or iSCSI), the OS sees the same disk multiple times. To prevent data corruption and enable path failover, you need multipath I/O.

First, I checked the lay of the land and installed the necessary tools:

lsblk
df -h
apt install multipath-tools multipath-tools-boot

Next came the tedious but necessary part: identifying the disks. I used scsi_id against /dev/sd* to pull the unique World Wide Identifiers (WWIDs) for each path:

/lib/udev/scsi_id -g -u -d /dev/sda
# ... repeated for sdb through sdz, etc.

Once I had my WWIDs, I mapped them into the multipath daemon and reloaded the configuration to bind the paths together into clean /dev/mapper/ devices:

multipath -a 350000396d85ba574
# ... added the rest of the WWIDs
multipath -r
multipath -ll
update-initramfs -u -k all

Phase 2: The Initial Setup (And the “Aha!” Moment)

With the storage paths secured, I moved on to creating the logical volumes. I initialized the multipath devices as Physical Volumes (PVs), grouped them into a Volume Group (VG) named backups, and carved out a 10TB Logical Volume (LV) named vms.

pvcreate /dev/mapper/350000396... # for all devices
vgcreate backups /dev/mapper/350000396... 
lvcreate -L 10T -n vms backups

Because this volume is specifically for Proxmox backups (which consist of massive chunks of data), I formatted it with ext4 using the -T largefile flag to optimize inodes, and bypassed the slow initialization process:

mkfs.ext4 -m 0 -T largefile -E lazy_itable_init=0,lazy_journal_init=0 /dev/backups/vms

I created a mount point at /mnt/datastore, grabbed the UUID via blkid, pinned it to /etc/fstab, and created the datastore in PBS:

proxmox-backup-manager datastore create main /mnt/datastore

The Pivot: It was right around here that I realized a massive single point of failure. I had a 10TB striped volume, but no host-level redundancy. If one of those LUNs dropped completely, the backup datastore would be toast. It was time to tear it down and do it right.

Phase 3: Rebuilding for Resilience with LVM RAID5

The beauty of LVM is how forgiving it is when you change your mind. I deleted the datastore in PBS, unmounted the drive, and nuked the logical volume.

proxmox-backup-manager datastore remove main
umount /mnt/datastore
lvremove /dev/backups/vms

Now for the real magic. I recreated the vms volume, but this time I explicitly instructed LVM to build it as a RAID5 array across 10 stripes with a 512KB stripe size:

lvcreate -L 10T --type raid5 -i 10 -I 512 -n vms backups

I repeated the same ext4 formatting process, updated /etc/fstab with the new UUID (since recreating the volume changes the identifier), mounted it, and successfully recreated my PBS datastore.

Phase 4: The Waiting Game

Software RAID doesn’t magically appear out of thin air; it has to calculate parity and sync across the disks. To keep an eye on the background synchronization without spamming the enter key, I used the watch command combined with lvs:

watch -n 1 lvs -a -o name,copy_percent,health_status,devices backups

While that chugged along, I dropped into btop to watch my CPU and I/O wait times, and installed some quality-of-life sysadmin tools like tmux (for terminal multiplexing) and mc (Midnight Commander).

Phase 5: Housekeeping

With the storage syncing happily, I noticed a minor annoyance: update-initramfs was likely throwing a warning about a missing resume device (swap).

To fix this, I grabbed the UUID of my swap partition and updated the initramfs config:

blkid /dev/mapper/pbs-swap
nano /etc/initramfs-tools/conf.d/resume
update-initramfs -u -k all

Finally, a server pushing this much I/O needs to be monitored thermally. After poking around for standard lm-sensors, I realized I needed to interface directly with the server’s BMC, so I installed ipmiutil to keep tabs on the hardware thermals.

The Takeaway

Building a backup server is about expecting the worst. While my first attempt would have worked for a homelab, moving to LVM RAID5 ensures that even if a LUN drops out, my backups remain intact. Measure twice, cut once—but if you have to cut twice, make sure you know your LVM commands!