HOWTOS

Installing Debian Jessie with encrypted LVM on SoftRAID (English)

Installing Debian Jessie with encrypted LVM and SoftRAID

A. Overview

I use the following system configuration at Hetzner:

  • CPU: Intel Core i7-4770
  • SSD: 2x 256 GB SATA
  • HDD: 2x HDD 3.0 TB SATA
  • RAM: 4x RAM 8192 MB DDR3

What we will setup:

  • Debian Stretch 9.0
  • Software-RAID 1
  • LUKS/DM-Crypt (Full Disk Encryption)
  • Logical Volume Manager
  • Remote unlock via SSH

1. Login to your server

First you need to login to your machine, at Hetzner I use the Linux x64 Rescue System:

ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no root@server.example.org

2. Partition

I will choose a simple layout with following configuration:

  • 32 MB: Bios boot partition
  • 500 MB: Unencrypted Boot partition (ext4)
  • Rest: Encrypted LVM

I will do this with gdisk, because it’s standard now, but if you have hard drives smaller than 2 TB you can also use fdisk.

2.1 Create new scheme

root@rescue ~ # gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.10

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): o
This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N): Y

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-500118158, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-500118158, default = 500118158) or {+-}size{KMGTP}: +32M
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): ef02
Changed type of partition to 'BIOS boot partition'

Command (? for help): n
Partition number (2-128, default 2):
First sector (34-500118158, default = 67584) or {+-}size{KMGTP}:
Last sector (67584-500118158, default = 500118158) or {+-}size{KMGTP}: +500M
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): fd00
Changed type of partition to 'Linux RAID'

Command (? for help): n
Partition number (3-128, default 3):
First sector (34-500118158, default = 1091584) or {+-}size{KMGTP}:
Last sector (1091584-500118158, default = 500118158) or {+-}size{KMGTP}:
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): fd00
Changed type of partition to 'Linux RAID'

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): Y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.

Now you have something like this:

root@rescue ~ # gdisk -l /dev/sda
GPT fdisk (gdisk) version 0.8.10

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 500118192 sectors, 238.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 54969DF5-E317-4DFE-B2B2-D04A08FFAB55
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 500118158
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           67583   32.0 MiB    EF02  BIOS boot partition
   2           67584         1091583   500.0 MiB   FD00  Linux RAID
   3         1091584       500118158   238.0 GiB   FD00  Linux RAID

2.2 Copy scheme

First copy the partition table:

root@rescue ~ # sgdisk -R=/dev/sdb /dev/sda
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.

And then randomize it:

root@rescue ~ # sgdisk -G /dev/sdb
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.

Reload partition table with: partprobe

3. Create /boot and LVM

We will now first setup the boot and LVM Partition.

3.1 /boot

The /boot partition is unencrypted so we only need to setup the RAID. Notice the option –metadata=0.90. It make sure that the bootloader can read the RAID Volume.

root@rescue ~ # mdadm -v --create /dev/md0 --metadata=0.90 --level=mirror --raid-devices=2 /dev/sda2 /dev/sdb2
mdadm: size set to 510912K
mdadm: array /dev/md0 started.

Then we need to create a Ext4 filesystem:

root@rescue ~ # mkfs.ext4 /dev/md0

Now is the partition /boot ready so we now gonna create the other partitions.

3.2 Excrypted LVM

So again we first create a RAID Volume:

root@rescue ~ # mdadm -v --create /dev/md1 --metadata=1.2 --level=mirror --raid-devices=2 /dev/sda3 /dev/sdb3
mdadm: size set to 2929622272K
mdadm: array /dev/md1 started.

On this RAID volume we are now going to setup an encrypted volume with cryptsetup:

root@rescue ~ # cryptsetup -s 512 -c aes-xts-plain64 luksFormat /dev/md1

WARNING!
========
This will overwrite data on /dev/md1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:
Verify passphrase:

root@rescue ~ # cryptsetup luksOpen /dev/md1 md1_crypt
Enter passphrase for /dev/md1:

Now we gonna to create the real LVM:

root@rescue ~ # pvcreate /dev/mapper/md1_crypt
  Writing physical volume data to disk "/dev/mapper/md1_crypt"
  Physical volume "/dev/mapper/md1_crypt" successfully created
root@rescue ~ # vgcreate vg0 /dev/mapper/md1_crypt
  Volume group "vg0" successfully created

3.2.1 Create the LVM volumes

On this LVM you can now create your volumes according to your needs, there is no right or wrong:

root@rescue ~ # lvcreate -L 32G -n root vg0
  Logical volume "root" created
root@rescue ~ # lvcreate -L 16G -n var vg0
  Logical volume "var" created
root@rescue ~ # lvcreate -L 8G -n home vg0
  Logical volume "home" created
root@rescue ~ # lvcreate -L 2G -n tmp vg0
  Logical volume "tmp" created
root@rescue ~ # lvcreate -L 4G -n swap vg0

After creating the volumes, we need to format everything:

root@rescue ~ # for i in home root tmp var; do mkfs.ext4 -L $i /dev/mapper/vg0-$i; done
root@rescue ~ # mkswap /dev/mapper/vg0-swap

4. Baseinstallation of your system

Now it’s time to mount everything of your new system:

root@rescue ~ # mount /dev/mapper/vg0-root /mnt
root@rescue ~ # mkdir /mnt/{boot,home,tmp,var}
root@rescue ~ # mount /dev/md0 /mnt/boot
root@rescue ~ # for i in home tmp var; do mount /dev/mapper/vg0-$i /mnt/$i; done
root@rescue ~ # swapon /dev/mapper/vg0-swap

Important: We should not forget to give /mnt/tmp the right permissions:

root@rescue ~ # chmod 1777 /mnt/tmp

4.1 Download debootstrap

Now we are going to download the newest debootstrap from debian.org to ensure that it’s safe:

root@rescue ~ # mount -t tmpfs none /tmp
root@rescue ~ # cd /tmp
root@rescue /tmp # wget http://ftp.de.debian.org/debian/pool/main/d/debootstrap/debootstrap_1.0.93_all.deb
--2018-01-02 04:00:05--  http://ftp.de.debian.org/debian/pool/main/d/debootstrap/debootstrap_1.0.93_all.deb
Resolving ftp.de.debian.org (ftp.de.debian.org)... 141.76.2.4
Connecting to ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 67262 (66K) [application/x-debian-package]
Saving to: ‘debootstrap_1.0.93_all.deb’

debootstrap_1.0.93_all.deb                                 100%[=========================================================================================================================================>]  65.69K  --.-KB/s   in 0.05s

2018-01-02 04:00:05 (1.34 MB/s) - ‘debootstrap_1.0.93_all.deb’ saved [67262/67262]

If you get a 404-error, then mostly there is a new version. You can go to ftp.de.debian.org/debian/pool/main/d/debootstrap/ to check for the new one.

Now we will extract this archive:

root@rescue /tmp # ar -xf debootstrap_1.0.93_all.deb
root@rescue /tmp # tar xzf data.tar.gz
root@rescue /tmp # tar xzf control.tar.gz

Then we will check if the the debootstrap is fine with the following command:

root@rescue /tmp # cat md5sums | cut -d " " -f 3 | xargs md5sum $1 > md5sums.local; diff md5sums md5sums.local

This command should not give any output. If it does, the file is corrupt.

Now that we can install our system, we need to make a small change in debootstrap. Please open the file with the following command:

root@rescue /tmp # nano usr/sbin/debootstrap

Search for this line:

DEBOOTSTRAP_DIR=/usr/share/debootstrap

and replace it with:

DEBOOTSTRAP_DIR=/tmp/usr/share/debootstrap

4.2 Run the installation

Before running debootstrap we need to import the gpg keys of debian. Thats very important that we can get sure that everything is right.

root@rescue /tmp # wget https://ftp-master.debian.org/keys/archive-key-8.asc
--2018-01-02 04:04:50--  https://ftp-master.debian.org/keys/archive-key-8.asc
Resolving ftp-master.debian.org (ftp-master.debian.org)... 138.16.160.17
Connecting to ftp-master.debian.org (ftp-master.debian.org)|138.16.160.17|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7012 (6.8K) [text/plain]
Saving to: ‘archive-key-8.asc’

archive-key-8.asc                                          100%[=========================================================================================================================================>]   6.85K  --.-KB/s   in 0s

2018-01-02 04:04:50 (353 MB/s) - ‘archive-key-8.asc’ saved [7012/7012]

root@rescue /tmp # gpg --import archive-key-8.asc
gpg: key 2B90D010: public key "Debian Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
gpg: no ultimately trusted keys found

Now we start the installation process and log the result to /installlog.txt in the new system:

root@rescue /tmp # usr/sbin/debootstrap --keyring=/root/.gnupg/pubring.gpg --arch amd64 jessie /mnt/ http://ftp2.de.debian.org/debian | tee /mnt/installlog.txt

This is now during some time. And we’re still not finished. After we need to go into the whole system and we still need to tighten a few screws manually. So let us go into the system:

root@rescue /tmp # mount -o bind /dev /mnt/dev
root@rescue /tmp # mount -t proc proc /mnt/proc
root@rescue /tmp # mount -t sysfs sys /mnt/sys
root@rescue /tmp # XTERM=xterm-color LANG=C.UTF-8 chroot /mnt /bin/bash

Now we are already logged in in our new system.

5. Configuration

When we are in the chroot, it’s time to configure everything now.

5.1 Set root password

The first thing we gonna to do is setting a password for root.

root@rescue:/# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

5.2 Set hostname and nameservers

root@rescue:/# echo "YOUR_HOSTNAME" > /etc/hostname
root@rescue:/# echo "YOUR_HOSTNAME" > /etc/mailname
root@rescue:/# echo "MY.HOST.IP.ADDRESS YOUR_HOSTNAME YOUR_HOSTNAME.example.org" >> /etc/hosts
root@rescue:/# cat > /etc/resolv.conf << "EOF"
search example.org
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF

Feel free to use other nameservers.

5.3 Set network address

Now we can set the network address. Please replace this with own data.

root@rescue:/# cat > /etc/network/interfaces << "EOF"
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.1.2
netmask 255.255.255.0
broadcast 192.168.1.255
gateway 192.168.1.1
pre-up /sbin/ip addr flush dev eth0 || true
EOF

5.4 Install base software

Now we should have connection to the internet and set the /etc/apt/sources.list.

root@rescue:/# cat > /etc/apt/sources.list << "EOF"
deb http://deb.debian.org/debian stretch main contrib non-free
deb-src http://deb.debian.org/debian stretch main contrib non-free

deb http://deb.debian.org/debian stretch-updates main contrib non-free
deb-src http://deb.debian.org/debian stretch-updates main contrib non-free

deb http://security.debian.org/ stretch/updates main contrib non-free
deb-src http://security.debian.org/ stretch/updates main contrib non-free
EOF

Now we need to update the cache of APT.

root@rescue:/# apt update

Then we can install and select the needed locales:

root@rescue:/# apt -y install locales && dpkg-reconfigure locales

You can now save the locales like this:

root@rescue:~# cat > /etc/environment << "EOF"
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
EOF

After this we need to setup the timezone:

root@rescue:/# dpkg-reconfigure tzdata

Now we can install the kernel:

root@rescue:/# apt -y install linux-image-amd64 firmware-realtek cryptsetup

5.5 Create /etc/crypttab

That debian knows that it is on an encrypted volume we first need to create a file called cryptab. For that we need the UUID of the partition. We get it with:

root@rescue:/# cryptsetup luksDump /dev/md1 | grep UUID:
UUID:          	xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Then we create the crypttab:

root@rescue:/# echo 'md1_crypt UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx none luks' > /etc/crypttab

5.6 Create /etc/fstab

Now we need to ensure that everything will be properly mounted. First we need the UUIDs of all the disks.

root@rescue:/# blkid /dev/md0
root@rescue:/# blkid /dev/mapper/vg0-*
/dev/mapper/vg0-home: LABEL="home" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4"
/dev/mapper/vg0-root: LABEL="root" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4"
/dev/mapper/vg0-swap: UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="swap"
/dev/mapper/vg0-tmp: LABEL="tmp" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4"
/dev/mapper/vg0-var: LABEL="var" UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4"

Create /etc/fstab with following content:

# file system                                   mount point     type    options                 dump    pass
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx       /               ext4    errors=remount-ro       0       1
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx       /boot           ext4    errors=remount-ro       0       1
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx       /tmp            ext4    rw,nosuid,nodev         0       2
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx       /var            ext4    rw                      0       2
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx       /home           ext4    rw,nosuid,nodev         0       2
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx       none            swap    sw                      0       0

Many programs depend on /etc/mtab so we need to set a symbolic link:

root@rescue:/# ln -sf /proc/mounts /etc/mtab

5.7 Install the rest

Install the rest base stuff:

root@rescue:/# apt -y install makedev lvm2 ssh dropbear busybox initramfs-tools bash-completion kbd console-setup pciutils psmisc grub-pc mdadm plymouth

When it comes to the boot device then select both RAID devices (/dev/sda,/dev/sdb).

If you want to install some standard stuff (recommended) enter:

root@rescue:/# tasksel install standard

6. Configure remote unlock

Now mostly everything is configured. But we can’t login from remote at the moment. We will setup this. First you need to add your ssh key. If you don’t know what this is, google for ssh-keygen. Add your key to _/root/.ssh/authorizedkeys.

Then you can copy your key into the initramfs:

root@rescue:~# cp /root/.ssh/authorized_keys /etc/initramfs-tools/root/.ssh/authorized_keys

Create _/etc/initramfs-tools/conf.d/networkconfig with contents like this:

export IP=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>

can be left blank.

or

export IP=dhcp

A possible example is:

export IP=192.168.1.2::192.168.1.1:255.255.255.0:myhostname:eth0

6.1 Add your Network Driver

Normally initramfs doesn’t load your network driver. So you first need to check which driver this is and then add it to the module file.

root@rescue:~# grep DRIVER /sys/class/net/eth0/device/uevent
DRIVER=r8169
root@rescue:~# echo "r8169" >> /etc/initramfs-tools/modules

6.2 Add the unlock hook

That you can unlock your system easily, we need to add a little script to initramfs.

Add this script to _/etc/initramfs-tools/hooks/mountcryptroot:

#!/bin/sh

# This script generates two scripts in the initramfs output,
# /root/mount_cryptroot.sh and /root/.profile


ALLOW_SHELL=0
# Set this to 1 before running update-initramfs if you want
# to allow authorized users to type Ctrl-C to drop to a
# root shell (useful for debugging, potential for abuse.)
#
# (Note that even with ALLOW_SHELL=0 it may still be possible
# to achieve a root shell.)
#

if [ -z ${DESTDIR} ]; then
    exit
fi

SCRIPT="${DESTDIR}/root/mount_cryptroot.sh"
cat > "${SCRIPT}" << 'EOF'
CMD=
while [ -z "$CMD" -o -z "`pidof askpass plymouth`" ]; do
  CMD=`ps -o args | grep cryptsetup | grep -i open | grep -v grep`
  sleep 0.1
done
while [ -n "`pidof askpass plymouth`" ]; do
  $CMD && kill -9 `pidof askpass plymouth` && echo "Success"
done
EOF

chmod +x "${SCRIPT}"

# Run mount_cryptroot by default and close the login session afterwards
# If ALLOW_SHELL is set to 1, you can press Ctrl-C to get to an interactive prompt
cat > "${DESTDIR}/root/.profile" << EOF
ctrl_c_exit() {
  exit 1
}
ctrl_c_shell() {
  # Ctrl-C during .profile appears to mangle terminal settings
  reset
}
if [ "$ALLOW_SHELL" == "1" ]; then
  echo "Unlocking rootfs... Type Ctrl-C for a shell."
  trap ctrl_c_shell INT
else
  echo "Unlocking rootfs..."
  trap ctrl_c_exit INT
fi
/root/mount_cryptroot.sh && exit 1 || echo "Run ./mount_cryptroot.sh to try unlocking again"
trap INT
EOF

Activate it with the following command:

chmod +x /etc/initramfs-tools/hooks/mount_cryptroot

6.3 Update initramfs

Now you need to save your changes with:

root@rescue:~# update-initramfs -u -k all
update-initramfs: Generating /boot/initrd.img-3.16.0-4-amd64

7. Finishing setup

We will no going to finish everything, that we can boot up our new system.

7.1 Update grub

To be sure, we now update grub before rebooting:

root@rescue:~# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.16.0-4-amd64
Found initrd image: /boot/initrd.img-3.16.0-4-amd64
done

7.2 Reboot everything

root@rescue:~# exit
exit
root@rescue:~# reboot

X. Additional Tipps

X.1 Remove old RAID Volumes

If you get errors like:

root@rescue ~ # mdadm -v --create /dev/md0 --level=mirror --raid-devices=2 /dev/sda1 /dev/sdb1
mdadm: super1.x cannot open /dev/sda1: Device or resource busy
mdadm: failed container membership check
mdadm: cannot open /dev/sda1: Device or resource busy

List your RAID Volumes like this:

root@rescue ~ # mdadm --examine --verbose --scan
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx
   devices=/dev/sdb1,/dev/sda1
ARRAY /dev/md/1 level=raid1 metadata=1.2 num-devices=2 UUID=xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx name=rescue:1
   devices=/dev/sdb2,/dev/sda2

You first need to stop these arrays:

root@rescue ~ # mdadm --stop /dev/md0
mdadm: stopped /dev/md0
root@rescue ~ # mdadm --stop /dev/md1
mdadm: stopped /dev/md1

Now you can remove all metadata:

root@rescue ~ # mdadm --zero-superblock /dev/sda1
root@rescue ~ # mdadm --zero-superblock /dev/sdb1
root@rescue ~ # mdadm --zero-superblock /dev/sda2
root@rescue ~ # mdadm --zero-superblock /dev/sdb2

Now the scan command should return nothing more:

root@rescue ~ # mdadm --examine --verbose --scan
root@rescue ~ #

X.2 Mount into system from rescue

root@rescue ~ # cryptsetup luksOpen /dev/md1 md1_crypt
Enter passphrase for /dev/md1:
root@rescue ~ # vgchange -ay vg0
  5 logical volume(s) in volume group "vg0" now active
root@rescue ~ # mount /dev/mapper/vg0-root /mnt
root@rescue ~ # mount /dev/md0 /mnt/boot
root@rescue ~ # for i in home tmp var; do mount /dev/mapper/vg0-$i /mnt/$i; done
root@rescue ~ # swapon /dev/mapper/vg0-swap
root@rescue ~ # mount -o bind /dev /mnt/dev
root@rescue ~ # mount -t proc proc /mnt/proc
root@rescue ~ # mount -t sysfs sys /mnt/sys
root@rescue ~ # XTERM=xterm-color LANG=C.UTF-8 chroot /mnt /bin/bash