install to emmc with luks full disk encryption

in case you want to install to emmc on a chromebook then maybe have a look at
install-to-emmc-on-arm-chromebooks.md in the same dir first as it might
explain the required steps in a somewhat easier to follow way.

these are some notes about how to install the system booted from sd card or
usb onto the internal emmc storage. this is just the most probably most
common use case, but the target can also be any other storage device, even
another sd card or usb drive. it is focussed on chromebooks, but many parts
can be used in the same or a similar way for other systems.

in case one want to have another default username than linux of the provided
bootable sd card images, just boot the image once from sd card, loging, set
a password for root (passwd root), reboot, switch to the first virtual console
(ctrl-alt-f1 - important: before logging into xorg) and login there as root
(mind the default us keyboard layout for the default password changem). now
run /scripts/rename-default-user.sh mypreferredusername and afterwards reboot.

IMPORTANT: in case the target is the internal emmc of a chromebook, it is
very recommended to set the gbb flags to 0x19 as otherwise one might end up
in a situation where the system gets unbootable when the usb boot flags get
lost due to a drained battery

IMPORTANT: better do not do this on amlogic or rockchip android tv boxes as
they usually have essential parts required for the boot process located at
certain places in the emmc and might get unbootable if they get overwritten.
there are ways to work around this in some situations, but they are far from
trivial and might not work in all situations. allwinner tv boxes should be
fine as allwinner socs always boot from sd card first by default (better
test it before starting to overwrite the emmc)


# chromebook special section - relevant for all chromebooks

# chromebook partition setup required for chromebooks - see scripts/create-image.sh
# this will create two chromeos kernel partitions
apt-get install cgpt
sgdisk -Z /dev/mmcblk0
partprobe /dev/mmcblk0
sgdisk -C -e -G /dev/mmcblk0
partprobe /dev/mmcblk0
cgpt create /dev/mmcblk0
partprobe /dev/mmcblk0
# the next two commands should work for all chromebooks except veyron - see below for those
cgpt add -i 1 -t kernel -b 8192 -s 65536 -l KernelA -S 1 -T 2 -P 10 /dev/mmcblk0
cgpt add -i 2 -t kernel -b 73728 -s 65536 -l KernelB -S 0 -T 2 -P 5 /dev/mmcblk0
# for veyron (= rk3288 based) chromebooks the following commands have to be used instead
# as for some strange reasons the beginning of the emmc cannot be written to which also
# results in the primary gpt partition table having to be ignored on those devices
# the workaround here is to simply create the kernel partitions a bit further into
# the emmc device - then everything is working fine - the extra cgpt repair is
# required due to the mentioned issues with the primary gpt partition table
cgpt add -i 1 -t kernel -b 73728 -s 32768 -l KernelA -S 1 -T 2 -P 10 /dev/mmcblk0
cgpt repair /dev/mmcblk0
cgpt add -i 2 -t kernel -b 106496 -s 32768 -l KernelB -S 0 -T 2 -P 5 /dev/mmcblk0
cgpt repair /dev/mmcblk0

# side note: in case of a dd copy a full disk and the 4th partition is too large
# for the target disk - just delete it while fixing the gpt table and recreate it
# afterwards with fdisk
# sgdisk -C -e -G -d=4 /dev/targetdisk

# now fdisk by hand to get to something like below - for non chromebooks a 512mb
# boot partitions and a large partition for the encrypted root as p1 and p2 might
# be good, for efi setups a 128mb efi partitions (partition type 1) as p1 and
# boot and root as p2 and p3 might work well - non chromebook and non efi setups
# usually do not need a gpt partition table and an empty mbr partition table
# (o in fdisk) might be a good start - as a result the partition numers might
# differ from the examples below

# chromebook example
fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 29.12 GiB, 31268536320 bytes, 61071360 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 44BBFDC7-6617-FF43-9DC7-5817789BF8A6

Device           Start      End  Sectors  Size Type
/dev/mmcblk0p1    8192    73727    65536   32M ChromeOS kernel
/dev/mmcblk0p2   73728   139263    65536   32M ChromeOS kernel
/dev/mmcblk0p3  139264  1187839  1048576  512M Linux filesystem
/dev/mmcblk0p4 1187840 61071326 59883487 28.6G Linux filesystem


# efi boot special section in case efi booting is used

# efi example
fdisk -l
Disk /dev/mmcblk1: 14.68 GiB, 15758000128 bytes, 30777344 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 06C88013-0E38-B140-9267-D85E97917781

Device           Start      End  Sectors  Size Type
/dev/mmcblk1p1    2048   264191   262144  128M EFI System (type 1)
/dev/mmcblk1p2  264192  1312767  1048576  512M Linux filesystem (default type)
/dev/mmcblk1p3 1312768 30777310 29464543   14G Linux filesystem


# mbr boot special section in case mbr  booting is used

# mbr example
fdisk -l /dev/sdb
Disk /dev/sdb: 14.84 GiB, 15931539456 bytes, 31116288 sectors
Disk model: Card  Reader
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x422081e2

Device     Boot   Start      End  Sectors  Size Id Type
/dev/sdb1  *      32768  1081343  1048576  512M 83 Linux (type 83)
/dev/sdb2       1081344 31116287 30034944 14.3G 83 Linux


# u-boot special section in case u-boot is used

# set the boot partition lecacy bootable in fdisk - only required really
# for u-boot as it uses this flag to find its extlinux.conf:
# mbr: a 1 w (for partition 1)
# gpt: x A 3 r w (for partition 3) - gpt example below:
/dev/mmcblk0p3  139264  1187839  1048576 0FC63DAF-8483-4772-8E79-3D69D8477DE4 41D13648-115C-AA4E-A49B-A41FB6968AE7      LegacyBIOSBootable


# generic section required for all systems - but of course the device names have to be adjusted!

# side note: for very low end armv7 systems adiantum might be a good idea for the crypto
# cryptsetup luksFormat -c xchacha12,aes-adiantum-plain64 -s 256 /dev/xyz
# luks2 - just for reference, not used or needed here
# cryptsetup luksFormat --type luks2 --sector-size 4096 -c xchacha12,aes-adiantum-plain64 -s 256 /dev/xyz

# create and ext4 fs without journal on boot, setup the crypt stuff and create
# a btrfs fs on it (ext4 example below) - partition numbers based on the chromebook
# example from above - might need to be adjusted
mkfs -t ext4 -O ^has_journal -m 0 -L bootemmc /dev/mmcblk0p3
cryptsetup luksFormat /dev/mmcblk0p4
cryptsetup open --type luks /dev/mmcblk0p4 encrypted
mkfs -t btrfs -m single -L rootemmc /dev/mapper/encrypted 
mount -o ssd,compress-force=zstd,noatime,nodiratime /dev/mapper/encrypted /mnt
# ext4 case without journal (to reduce writes to emmc etc.)
# mkfs -t ext4 -O ^has_journal -L rootemmc /dev/mapper/encrypted
# mount -o noatime,nodiratime,errors=remount-ro /dev/mapper/encrypted /mnt

# mount the fresh crypted root disk to /mnt and the other parts as well
cd /mnt
mkdir -p /mnt/boot
mount /dev/mmcblk0p3 /mnt/boot
# efi case
# mkdir -p /mnt/boot/efi
# mkfs.fat -F32 -n EFI /dev/your_efi_partition
# mount /dev/your_efi_partition /mnt/boot/efi

# copy over the currently running system - efi will be installed from scratch
rsync -axADHSX --no-inc-recursive --delete /boot/ /mnt/boot
rsync -axADHSX --no-inc-recursive --delete --exclude='/swap/*' / /mnt

# in case one wants to transfer the installation via a tar archive
# the "--numeric-owner" option is a good idea to make sure the
# uids/gids of the files are consistent independently of what system
# has been booted to do all the installation procedure
# tar --numeric-owner -xzf some-tar-archive.tar.gz

# create swap files or file - the brtfs cmds will give an error in the ext4
# case, but this should be no problem
rm -rf /mnt/swap
btrfs subvolume create /mnt/swap
chmod 755 /mnt/swap
chattr -R +C /mnt/swap
btrfs property set /mnt/swap compression none
cd /mnt/swap
# see https://blog.passcod.name/2020/jun/16/full-disk-encryption-with-btrfs-swap-and-hibernation
# single file - better for hibernation
truncate -s 0 ./file.0; btrfs property set ./file.0 compression none; fallocate -l 2G file.0; chmod 600 ./file.0; mkswap -L file.0 ./file.0
# alternative for multiple smaller swap files - more flexible to later remove some ifspace is needed
for i in 0 1 2 3 ; do truncate -s 0 ./file.$i; btrfs property set ./file.$i compression none; fallocate -l 512M file.$i; chmod 600 ./file.$i; mkswap -L file.$i ./file.$i; done

# adjust partition table and hostname (if wanted) - fstab example below
# in case the bootable image was using ext4 as root fs and now btrfs is chosen,
# this needs to be adjusted in the fstab accordinly (see example for both below)
cd ..
vi etc/fstab etc/hosts etc/hostname 

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# LABEL=rootemmc / ext4 defaults,noatime,nodiratime,errors=remount-ro 0 1
LABEL=rootemmc / btrfs defaults,ssd,compress-force=zstd,noatime,nodiratime 0 1
LABEL=bootemmc /boot ext4 defaults 0 2
/swap/file.0 none swap sw 0 0
#/swap/file.1 none swap sw 0 0
#/swap/file.2 none swap sw 0 0
#/swap/file.3 none swap sw 0 0

# adjust the crypttab file, so that initrd knows which disk to setup encrypted
vi etc/crypttab

# the uuid can be obtained via blkid cmd for the partition we used for cryptsetup above
encrypted UUID=26519515-9875-4bb3-b244-0e9d403bc794 none luks,discard

# enter chroot of the new system to do some required setup
chroot /mnt mount -t proc proc /proc 
chroot /mnt mount -t sysfs sysfs /sys 
mount --bind /dev /mnt/dev 
mount --bind /run /mnt/run 
chroot /mnt

# enable xz compression for initrd - saves space in /boot and makes initrd smaller
# for fit image in case that is needed - memlimit might need adjustment in case
# update-initramfs gives an error
# create /etc/initramfs-tools/conf.d/compress:
vi /etc/initramfs-tools/conf.d/compress
# content of the file:
COMPRESS=xz
XZ_OPT='-9 --check=crc32 –memlimit-compress=25%'

# rebuild initrd for all kernels to have the crypttab setup in it
# to be on the safe side: make sure cryptsetup-initramfs is installed (it looks like
# some older images did for some reason not have it installed
apt-get update
apt-get -y install cryptsetup-initramfs
update-initramfs -c -k all
# IMPORTANT: if that returns immediately use the kernel version found in /boot
# like 5.10.50-stb-mt7+ instead of all - this most probably will be the normal case
# for self built and non packaged kernels
update-initramfs -c -k 5.10.50-stb-mt7+

# 64bit aarch64 chromebook special section

# this section is required for chromebooks using fit images on aarch64 (gru, oak, kukui etc.)
# the below example needs to be adjusted accordingly (see comments) ... it can then be put into
# a script /boot/kernel-and-initrd.sh ... this needs to be run whenever a new kernel is being
# built ... vmlinux.kpart-initrd-${kver} will be the chromeos style kernel image for the
# chromeos kernel partition which contains the initrd with the cryptsetup info - without this
# it will not be possible to access the crypted root disk - so run it already once after
# creating, chmod a+x and and adjusting the script - afterwards the resulting 
# vmlinux.kpart-initrd-version file should be written to the first chromeos kernel partition
# IMPORTANT: the resulting vmlinux.kpart-initrd... files has to be smaller than 32mb for
#            aarch64 systems and smaller than 16mb for armv7l systems - thus its nearly
#            impossible to do this on a armv7l system as the initrd is usually too big
#            (for exynos armv7l devices the size limit seems to be even 8mb)
# it might be a good idea to remove the "quiet splash" kernel cmdline options in the echo line
# for a more verbose boot which makes debugging possible problems easier

kukui example (others as comments - for some the device is mmcblk0 and for others mmcblk1):
#!/bin/bash

# adjust to kernel version and name here
kver="6.1.0-stb-mt8+"

cd /boot
cp -v vmlinux.kpart-initrd-${kver} vmlinux.kpart-initrd-${kver}.old
cp Image-${kver} Image
lzma -9 -z -f -k -v Image
cp initrd.img-${kver} initrd.img.xz
# for cbq add: deferred_probe_timeout=30 clk_ignore_unused=1
echo "console=tty1 root=LABEL=rootemmc rootwait ro fsck.fix=yes fsck.repair=yes net.ifnames=0 ipv6.disable=1 quiet splash" > cmdline
dd if=/dev/zero of=bootloader.bin bs=512 count=1
# adjust to dtb names here:
# - cbg: dtb-${kver}/rk3399-gru-*.dtb
# - mt7: dtb-${kver}/mt8173-*.dtb
# - mt8: dtb-${kver}/mt8183-*.dtb
# - cbq: dtb-${kver}/sc7180-trogdor-*.dtb
ls dtb-${kver}/mt8183-*.dtb | xargs printf " -b %s" | xargs mkimage -D "-I dts -O dtb -p 2048" -f auto -A arm64 -O linux -T kernel -C lzma -a 0 -d Image.lzma -i initrd.img.xz kernel.itb
vbutil_kernel --pack vmlinux.kpart --keyblock /usr/share/vboot/devkeys/kernel.keyblock --signprivate /usr/share/vboot/devkeys/kernel_data_key.vbprivk --version 1 --config cmdline --bootloader bootloader.bin --vmlinuz kernel.itb --arch arm
cp -v vmlinux.kpart /boot/vmlinux.kpart-initrd-${kver}
rm -f Image Image.lzma initrd.img.xz cmdline bootloader.bin kernel.itb vmlinux.kpart
echo ""
echo "for single boot only testing:"
echo "IMPORTANT: please double check your mmcblk device name beforehand"
echo ""
echo "  dd if=/boot/vmlinux.kpart-initrd-${kver} of=/dev/mmcblk0p2"
echo "  cgpt add -i 2 -S 0 -T 1 -P 15 /dev/mmcblk0"
echo ""
echo "to always boot this kernel after successful testing:"
echo ""
echo "  dd if=/boot/vmlinux.kpart-initrd-${kver} of=/dev/mmcblk0p1"
echo ""

as mentioned above: after creating this script and making it executable, run it once


# 32bit armv7l chromebook special section

in case of 32bit armv7l chromebooks (snow, peach, nyan, veyron, etc.)
u-boot needs to be written to the emmc kernel partition like in the below
veyron example which is using the jerry u-boot binary - for each chromebook
the proper u-boot binary has to be selected from /boot/extra:

dd if=/boot/extra/uboot.kpart.cbr-jerry of=/dev/mmcblk0p1


# grub special section in case grub is used

# grub install - this section is required intel mbr or uefibooting systems
# non uefi - xyz = disk to install bootblock on (example /dev/sdb), apt-get install grub2-common grub-pc grub-pc-bin
grub-install /dev/xyz
# uefi 32bit - xyz = efi partition (example /dev/sdb1), apt-get install grub2-common grub-efi-ia32 grub-efi-ia32-bin
grub-install --target=i386-efi /dev/xyz --efi-directory=/boot/efi/ --boot-directory=/boot/
# uefi 64bit - xyz = efi partition (example /dev/sdb1), apt-get install grub2-common grub-efi-amd64 grub-efi-amd64-bin
grub-install --target=x86_64-efi /dev/xyz --efi-directory=/boot/efi/ --boot-directory=/boot/

# maybe update the grub config, especially the kernel cmdline - if for example the
# graphical password prompt for cryptsetup is perferred over the simple console
# text prompt the option "splash" should be added to the kernel cmdline (GRUB_CMDLINE_LINUX
# and/or GRUB_CMDLINE_LINUX_DEFAULT) - other potentially interesting options might be:
# "root=LABEL=rootemmc rootwait ro fsck.repair=yes net.ifnames=0 ipv6.disable=1 systemd.gpt_auto=0"
vi /etc/default/grub

update-grub

# on some systems (for example chrombooks with a flashed coreboot uefi bios) the
# following steps might be required in order to make them boot properly (this
# is required at least for debian bullseye on some systems - see example below)
mkdir -p /boot/efi/EFI/BOOT
cp /boot/efi/EFI/debian/grubx64.efi /boot/efi/EFI/BOOT/BOOTX64.EFI
# for 32bit uefi it would be
cp /boot/efi/EFI/debian/grubia32.efi /boot/efi/EFI/BOOT/BOOTIA32.EFI


# u-boot special section in case u-boot is used (also relevant for 32bit chromebooks)

# extlinux.conf adjustment - this section is required for systems using u-boot
# usually only the root disk should be adjusted to root=rootemmc and maybe the
# default boot should be adjusted in case of multiple options
# it might be a good idea to remove the "quiet splash" kernel cmdline options for
# a more verbose boot which makes debugging possible problems easier
vi /boot/extlinux/extlinux.conf


# generic section required for all systems

# exit chroot
exit
cd 

# unmount chroot dirs - the efi one might give an error in the non efi case as there is no such dir there
umount /mnt/dev
umount /mnt/proc
umount /mnt/sys
umount /mnt/run
umount /mnt/boot/efi
umount /mnt/boot
umount /mnt

# close the crypted device
cryptsetup close encrypted

# reboot in the hope that it will work :)
reboot


# all done here - below is some extra info which might be helpful maybe

# some cosmetic information to get rid of potential kernel messages during the
# graphical plymouth bootscreen (quie splash kernel cmdline options)
- add 'loglevel=3 vt.global_cursor_default=0' after "quiet" to kernel cmdline
- add 'echo "4 4 1 7" > /proc/sys/kernel/printk' and 'setterm -cursor on' to /etc/rc.local
# inspired by
- via: https://wiki.archlinux.org/title/silent_boot#Kernel_parameters
- and: https://serverfault.com/questions/724069/hide-kernel-messages-in-plymouth

# on some system bluetooth seems to be always enable on bootup even if disabled 
# in the last session - to disable it each time on boot automatically:
- add '( sleep 15 ; rfkill block 0 ) &' to /etc/rc.local
- check the id/number with 'rfkill list' beforehand

# some useful further cryptsetup info
# from https://alvinalexander.com/linux-unix/how-change-disk-encryption-password-luks
#  and https://www.lisenet.com/2013/luks-add-keys-backup-and-restore-volume-header/
# multiple passphrases - max 8
# get the used slots
cryptsetup luksDump /dev/sdxy
# add to some free slot - for example 1
cryptsetup luksAddKey /dev/sdxy -S 1
# remove a passphrase - IMPORTANT: do not remove the last one
cryptsetup luksRemoveKey /dev/sdxy

# home partition with the same key - apt-get install keyutils - kernel config: CONFIG_KEYS=y
# see: /usr/share/doc/cryptsetup/README.keyctl
cryptsetup luksFormat /dev/sdxy
cryptsetup open --type luks /dev/sdxy homecrypt
mkfs -t ext4 -O ^has_journal -m 0 -L home /dev/mapper/homecrypt

/etc/crypttab:
encrypted UUID=0e5bbb2c-0021-4e6d-adaa-de4bcd27d75b root_and_home luks,discard,keyscript=decrypt_keyctl
homecrypt UUID=94fd468c-abfd-4bfb-930c-18eee845daeb root_and_home luks,keyscript=decrypt_keyctl

# extending a luks crypt partition
# see: https://unix.stackexchange.com/questions/320957/extend-a-luks-encrypted-partition-to-fill-disk
# beforehand maybe extend the partition with fdisk without removing the signature from it
cryptsetup luksOpen /dev/somedevice crypt-volume
cryptsetup resize crypt-volume
e2fsck -f /dev/mapper/crypt-volume
resize2fs /dev/mapper/crypt-volume

# some read/write queue optimization
https://wiki.archlinux.org/index.php/Dm-crypt/Specialties#Disable_workqueue_for_increased_solid_state_drive_(SSD)_performance
via: https://forum.armbian.com/topic/17272-cryptsetup-supporting-no_read_workqueueno_write_workqueue-on-ssds/
