Remember that no one cares if you can back up — only if you can restore.
— Amanda 2.5.2 DocumentationSo we’ve got a new file server in the middle of initial installation and configuration. The file server is one of our most mission-critical systems — if mail goes down, a half-dozen people care. If the web server goes down, a few more would care, but it’s not a life-or-death issue. But the file server? That’s important. Students I’ve never met, students who wouldn’t know the difference between a shell prompt and a hole in the ground, students who couldn’t care less about parallel computing or anything else I put effort into on this area — they’ll notice the file server being down.
And all things considered, I like it that way. I know that a student or faculty member is statistically more likely to lose data on their local hard drive, their flash drive, or their removable media of choice than I am to lose it on a RAID-5, hot-spare-ready, redundant power supplies drive array connected to a RAID-1, redundant power supplies server. We’ve had one data loss experience since 2001 when I started doing this. And we only lost data because of
- Human error in moving the external RAID in our server rack
- Having Amanda holding disk space on the external RAID in addition to what was on the system’s internal drives
and I’m not too keen to repeat it. I didn’t get more than 2 hours of sleep at a time for most of a week while I was constantly having to load a different tape in the changer. Thankfully, I didn’t have to camp out in the server room the whole time, since I could manipulate the changer via ssh. But it was both embarrassing and a major drag away from anything I’d have rather been doing at the time.
But the new file server is physically ready, and >90% ready as far as configuration and software are concerned. More details on this after the jump.
As far as the server specifications go, we’ve got
- Dell PowerEdge 2950 server with Energy Smart options
- 1 quad-core Xeon 1.6 GHz CPU
- 2 GB RAM
- 8 146 GB SAS drives (2.5 inch, 10K RPM) in a RAID-5 with hotspare
- redundant power supplies
- 4-year warranty
For the external disk array, we’ve got a Dell PowerVault MD1000 with 15 750 GB SATA drives in a RAID-5 with hotspare. The tape changer is a Dell PowerVault 124T LTO-3 with a barcode reader and capacity for eight 400 GB (native capacity) tapes.
My management goal with this system is to never type
apt-get install,crontab -e, orxemacson it. I want preseeding and puppet to handle all package installation, package configuration, and as many other administration duties as possible. That way, in the unlikely event of a physical disaster, I can get services back up and running as quickly as possible, and I can redeploy these services on to a future server if needed.My original preseed.cfg contains the vast majority of what I needed for this server. The biggest difference is in the partitioning scheme:
- I want the file server to use LVM in case I need to change how space is divided up
- I want separate partitions for /tmp and /var to help prevent users from sucking up all the space in the system areas
- I need a separate space for Amanda’s holding disk, since we don’t back up a full tape of data each day, and you don’t want to back up your holding directory into your holding directory each day
So my new preseeded partitioning instructions work out as
d-i partman-auto/disk string /dev/discs/disc0/disc d-i partman-auto/method string lvm d-i partman-auto/purge_lvm_from_device boolean true d-i partman-lvm/confirm boolean true d-i partman-auto/init_automatically_partition \\ select Guided - use entire disk and set up LVM d-i partman-auto/expert_recipe string \\ boot-root :: \\ 40 300 300 ext3 \\ $primary{ } $bootable{ } \\ method{ format } format{ } \\ use_filesystem{ } filesystem{ ext3 } \\ mountpoint{ /boot } \\ . \\ 500 10000 1000000000 ext3 \\ method{ format } format{ } $lvmok{ } \\ use_filesystem{ } filesystem{ ext3 } \\ mountpoint{ / } \\ . \\ 600000 600000 600000 ext3 \\ method{ format } format{ } $lvmok{ } \\ use_filesystem{ } filesystem{ ext3 } \\ mountpoint{ /opt/amanda } \\ . \\ 500 9000 5000 ext3 \\ method{ format } format{ } $lvmok{ } \\ use_filesystem{ } filesystem{ ext3 } \\ mountpoint{ /var } \\ . \\ 500 9000 5000 ext3 \\ method{ format } format{ } $lvmok{ } \\ use_filesystem{ } filesystem{ ext3 } \\ mountpoint{ /tmp } \\ . \\ 64 512 200% linux-swap $lvmok{ } \\ method{ swap } format{ } \\ . d-i partman/confirm_write_new_label boolean true d-i partman/choose_partition \\ select Finish partitioning and write changes to disk d-i partman/confirm boolean trueThis ends up giving me a system disk layout as follows:
Filesystem Size Used Avail Use% Mounted on /dev/mapper/ch208r-root 230G 707M 218G 1% / /dev/sda1 274M 17M 243M 7% /boot /dev/mapper/ch208r-opt+amanda 564G 422M 535G 1% /opt/amanda /dev/mapper/ch208r-tmp 4.7G 138M 4.4G 4% /tmp /dev/mapper/ch208r-var 4.7G 264M 4.2G 6% /varThe next thing I discovered is that fdisk, cfdisk, and anything using DOS-style partition tables has trouble with comically-large volumes like our 9.75 TB (pre-formatting) RAID volume. This document from Coraid gives enough information about using parted and GPT to let us partition the new array. Next, it turns out that the old reliable ext3 filesystem has an 8 TB size limit on it, so we went with xfs on the external array.