Unattended Debian Installations (or How I Learned to Stop Worrying and Love the preseed.cfg)

A CMR project recently bought 12 new Dell PowerEdge SC1435 servers to replace some of our aging compute cluster systems. In previous server rollouts, I’d generally get one system installed and configured, image it with SystemImager, and then PXE-boot the rest of the systems to pull that image. However, it’s tough to audit exactly what got installed, and how. It’s also arguably a waste of space to keep images of all the types of cluster systems we have (PowerEdge 2650, PowerEdge 1850, PowerEdge 1855, Dimension 9200, etc.). So enter Debian Preseeding. With preseeding, I can make a text file that describes what packages I want installed, debconf configurations that vary from the default, how I want the partitioning done, copy configuration files from web or ftp servers to the target drive, etc. It took a few days to get going, but the long-term payoff should be enormous.

The resulting installation timeline for a PowerEdge SC1435 (relative to power-on in HH:MM:SS):

• 00:00:25 – Power-on self tests have completed enough to allow me to hit F12 for PXE boot.
• 00:01:20 – Rest of power-on self tests have completed, PXE boot process starts. All installation parameters are passed in from the pxelinux.cfg file on the DHCP server or the preseed.cfg file on a nearby ftp server.
• 00:04:50 – Base and standard task packages installed. openssh-server and puppet packages are installed. puppet defaults file is downloaded. System starts formatting a 500GB SATA drive for /tmp space.
• 00:08:20 – System finishes formatting /tmp, and reboots since that was the last step of the unattended install.
• 00:09:30 – System is at login prompt. Root password works. puppet will take over installing miscellaneous packages and other post-installation configuration tasks. Once puppet has finished copying over root@adminserver’s public key, I can log in from adminserver without being prompted for a password.

Total number of keypresses from power-up to login: 1. Annotated preseed and pxelinux files after the jump.

PXELinux config file (amd64 version, i386 would be nearly identical — the contents of the “append” line should be all on one line. It’s broken here for readability.)

default preseed
label preseed
kernel debian-installer/amd64/linux
append initrd=debian-installer/amd64/initrd.gz ramdisk_size=10800 root=/dev/rd/0 rw auto
  url=http://REDACTED/./preseed.cfg console-keymaps-at/keymap=us locale=en_US interface=eth0
  hostname=x domain=x netcfg/dhcp_timeout=60 --
prompt 1
timeout 1

Changes versus the default configuration:

1. Removed display command for splash screen. At one point, I was running out of parameter space on the append line, and removing ‘vga=normal’ was one space saver. I didn’t expect the splash screen to work once that was removed, so I dumped it as well.
2. Removed F1, F2, … , F9, F0 commands for online help. Optional.
3. Changed ‘default install’ to ‘default preseed’.
4. Added preseed stanza, with default kernel, but much different append options.
5. At the time I started this, the default pxelinux config file had entries for the ramdisk size and root device; these do not appear in the current Debian config file, and I’ll remove mine once I verify that they’re unneeded.
6. Added append option ‘auto url=http://REDACTED/./preseed.cfg’ to define location of preseed configuration file.
7. Added append options ‘console-keymaps-at/keymap=us’ and ‘locale=en_US’ to define basic localization for the installation.
8. Addd append options ‘interface=eth0’, ‘hostname=x’, ‘domain=x’, and ‘netcfg/dhcp_timeout=60’ to get the networking configured. The ‘hostname=x’ and ‘domain=x’ parameters were one of the last tricky bits. I had wrongly assumed that specifying that I’d be using DHCP to get my address would make it pass over asking for a hostname and domain. Preseeding dummy values for each gets past the questions, and DHCP overrides the dummy values as soon as the DHCP negotiation is done.
9. Changed ‘timeout 0’ to ‘timeout 1’ to allow the installer to proceed without me manually selecting any configurations. Obviously, you may want to increase the timeout to some larger value if you need more time to hit a key to override something for testing.

preseed.cfg (the contents of the “d-i preseed/late_command” line should be all on one line. It’s broken here for readability.):

### References:
###  http://d-i.alioth.debian.org/manual/en.amd64/install.en.pdf
###  http://d-i.alioth.debian.org/manual/example-preseed.txt
###  http://www.hands.com/d-i/etch/classes/

### This file needs to be at
### http://REDACTED/preseed.cfg and
### referenced via boot arguments as
### http://REDACTED/./preseed.cfg

### Generic settings applicable to all CAE Debian systems

d-i debian-installer/locale string en_US
d-i console-tools/archs select at
d-i console-keymaps-at/keymap select American English
d-i debian-installer/keymap string us
d-i netcfg/choose_interface select eth0
d-i netcfg/dhcp_timeout string 60
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_hostname seen true
d-i netcfg/get_domain string unassigned-domain
d-i netcfg/get_domain seen true
d-i netcfg/dhcp_options select Retry network autoconfiguration
d-i mirror/codename string etch
d-i mirror/country string enter information manually
d-i mirror/http/countries select enter information manually
d-i mirror/http/directory string /debian/
d-i mirror/http/hostname string ftp.cae.tntech.edu
d-i mirror/http/proxy string
d-i clock-setup/utc boolean false
d-i time/zone string US/Central
d-i apt-setup/security_host string
d-i apt-setup/local0/repository string http://ftp.cae.tntech.edu/debian-security etch/updates main

# XXX -- TEMPORARY HACK UNTIL WE GET A SIGNED LOCAL REPOSITORY
d-i debian-installer/allow_unauthenticated string true

d-i passwd/make-user boolean false
d-i passwd/root-password-crypted password $1$REDACTED
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
tasksel tasksel/first multiselect standard
d-i pkgsel/include string puppet lsb-release openssh-server
d-i finish-install/reboot_in_progress note

popularity-contest popularity-contest/participate boolean false

### Less generic settings

# Partitioning:
# sda1: 40-100MB for /boot
# sda2: majority of remaining space for /
# sda3: 64MB-(2*RAM) for swap
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/expert_recipe string 
  boot-root :: 
    40 50 100 ext3 
      $primary{ } $bootable{ } 
      method{ format } format{ } 
      use_filesystem{ } filesystem{ ext3 } 
      mountpoint{ /boot } 
    . 
    500 10000 1000000000 ext3 
      method{ format } format{ } 
      use_filesystem{ } filesystem{ ext3 } 
      mountpoint{ / } 
    . 
    64 512 200% linux-swap 
      method{ swap } format{ } 
    .
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select Finish partitioning and write changes to disk
d-i partman/confirm boolean true

# Some systems may vary on /tmp -- many have separate disks for it
# (/dev/sdb1), some may have a separate partition on the sda
# drive. This needs to be split out into a system-specific file and
# chained in via class tmp_on_sdb1
# Only applicable for hosts with separate /tmp disk on /dev/sdb1 --
# PE1855 blades (ch226-11...-19), SC1435 servers (ch226-21...-32), old
# ch208a
d-i preseed/late_command string wget -O /target/etc/default/puppet http://REDACTED/puppet ; 
  wget -O /target/root/caeftp_key.asc http://REDACTED/caeftp_key.asc ;
  in-target apt-key add /root/caeftp_key.asc ; echo ';' | sfdisk /dev/sdb ; mke2fs -j -q /dev/sdb1 ;
  echo '/dev/sdb1 /tmp ext3 defaults 0 2' >> /target/etc/fstab; cp /var/log/syslog /target/root

I’m not going to run through every change made in the preseed.cfg file, since the options are well-documented and what’s useful in one environment won’t necessarily translate to another. But I am mildly proud of the late_command at the end: this does two things that were minor pains on a normal preseed installation:

1. Bootstraps a basic puppet defaults file over to the target drive. Grabbing /etc/default/puppet from a remote site before rebooting makes sure that when puppetd starts up, that it can contact the central server for key signing, and work normally afterwards.
2. Partitions and formats /dev/sdb to be my /tmp directory. Debian-installer’s partman can’t partition more than one disk automatically, but each of these new systems had a second drive I had planned on using for /tmp. Sfdisk comes to the rescue. The input format description in the sfdisk man page indicates that if you give it a line with no values specified, it will make a partition from the first nonassigned sector to the maximum size, and set the ID to 83 for a Linux partition. So the first time I run this, I go from a drive with an empty partition table to one with a giant partition 1, exactly what I wanted. Subsequent times I run this, it should leave the partition table unmodified, which is also fine. I can’t find the original hint I found for this use of sfdisk.

My /etc/default/puppet is:

# Puppet doesn't like starting before it can contact the puppetmaster.
# Wait up to 30 seconds before continuing.

for n in `seq 1 30`; do
  ping -c 1 REDACTED >& /dev/null && break
  echo -n " $n"
  sleep 1
done
DAEMON_OPTS="--server REDACTED --factsync"

You may be able to avoid putting in the 30-second sleep, but at one point, with some puppet version, it was helpful on DHCP-managed systems. The critical thing to override on Debian, though, is the -w 0 flag in the default DAEMON_OPTS in /etc/init.d/puppetd. Removing it ensures that puppetd will start up, contact the gold server for key signing, and wait for the signing to happen, rather than immediately exit.