Self-installing Yocto Image from a USB drive

We are all familiar with the process of installing a desktop operating system onto a PC; simply insert the installation media, typically a USB stick, and boot the device. A minimal version of the OS then runs to perform the process of partitioning discs and installing the full OS.

Yocto also has the ability to generate a self-installing image, just like a desktop operating system. For an embedded system that can boot from a USB drive this gives some significant advantages:

• The WIC disc image is the minimal size needed to hold the root file system.
• Target partitions are created during installation so the install script can create additional partitions that are not in the WIC image, and customise sizes to use all available storage. This removes the need to create partitions and ‘finish off’ installation during first-boot.
• No additional tools are needed to load an image onto the target
• The installation process is quick and professional

This blog post shows how to configure Yocto to generate a self-installing image for a genericx86-64 based distribution. Further details may be found in the Yocto documentation and examples in the meta-amd and meta-intel layers.

Backgound to the approach described here

Yocto can build a variety of different types of ‘live’ images, so called because the image can come alive directly from a removable storage medium without needing to be installed first. This could be an ISO image, suitable for burning to a CD or DVD, or disk image suitable to writing with ‘dd’ directly to a USB drive. The Yocto class poky/classes/image-live.bbclass is the basis of all live images – whether for generating a self-installer or simply booting a full Yocto image from a removable medium.

In the earlier days of Yocto, the hddimg image type was the means of generating a self-booting / installing disc image, and was added by this line in local.conf:

IMAGE_FSTYPES += " hddimg"

It produces an .hddimg file, to be written to the USB drive, which contains a single FAT file system. The use of only FAT, needed to boot the system by EFI or BIOS, means hddimg is unable to hold a rootfs image file of >4GB; quite a limitation for some full-featured Yocto distributions.

More recently the Open Embedded Image Creator tool (a.k.a. WIC) became available, offering a far more flexible means of creating disk images. The alternative approach to hddimg, which we’re describing here, is to use WIC to generate an image (as a .wic file) containing both the FAT boot partition (where the 4GB limit is not an issue) and an EXT4 image partition (supporting files > 4GB) to hold the rootfs image to be installed.

Although the hddimg approach is still available in Yocto, it is effectively deprecated. Yocto will emit warnings if hddimg attempts to use a rootfs image file of >4GB.

Building and installing the image

First lets look at how Yocto builds the image, and then how this image installs itself on to the target system.

The build process

The Yocto distribution build process generates the live installation image through the following steps:

• A yocto build is started for the full image (e.g. core-image-weston)
• Yocto builds a bootloader (e.g. GRUB), Linux kernel and the target rootfs (e.g. from core-image-weston).
• The target rootfs (e.g. from core-image-weston) is written to an .ext4 file.
• Yocto builds an initrd RAM file system (rootfs.img) for the live image (e.g. from core-image-minimal-initramfs)
• The initrd contains a script (init-install-efi.sh, installed as install-efi.sh) which actually performs the installation.
• Yocto builds a WIC image containing two partitions:
  • VFAT named ‘install’: holding the live bootloader, kernel and initrd (.cpio.gz)
  • EXT4 name ‘image’: holding the bootloader, kernel and rootfs ext4 image for the target device
• The WIC image is named after the full distribution (e.g. core-image-weston-genericx86-64.wic)

Finally the WIC image file should be written to a USB drive using dd.

The USB boot and installation process

Once we have created a USB drive with the self-installing WIC image, the target device must be configured to boot from USB. For an x86 system this means BIOS settings to enable EFI boot and select the USB drive as the boot device. The live-boot and install process then proceeds as follows:

• The target machine boots from USB and loads the bootloader (e.g. GRUB)
• The bootloader loads the Linux kernel and initrd
• The Linux kernel starts the init process (e.g. init.d)
• The init process loads the setup-live script which does the following:
  • Waits for Linux to mount the USB media we booted from
  • Loads the install-efi.sh script
• The install-efi.sh script does the following:
  • Identifies all target storage devices (excluding the installation USB drive)
  • Selects the first fixed storage device
  • Creates the partitions required
  • Mounts the ‘image’ ext4 partition on the installation media
  • Copies the bootloader, bootloader config and Linux kernel to the target device
  • Patches the target bootloader config to boot the target rootfs
  • Mounts the rootfs ext4 partition of the target system
  • Loop mounts the rootfs.img file from the installation ext4 ‘image’ partition
  • Copies all files from within rootfs.img to the target rootfs partition
  • Syncs the filesystem to ensure all writes complete
  • Unmounts the target rootfs
• Optionally shuts down the system

Note: The partition layout is hard-coded into the install-efi script. it is NOT controlled by Yocto variables. To change it the install script itself must be modified.

Enable Self-installing

Yocto must be configured to create a self-installing image. This requires changes to local.conf and a custom WKS file. The configuration described here stores the root file system image in an EXT4 partition so it can be larger than the 4GB limit imposed by VFAT.

Items to add to layer.conf

Specify the OpenEmbedded Kickstart (WKS) file which defines how to generate the WIC file.

WKS_FILE = "image-installer.wks.in"

Generate an ext4 image file of the root file system.

IMAGE_FSTYPES:append = " ext4"
IMAGE_TYPEDEP_wic = "ext4"

The initrd will be built from the core-image-minimal-initramfs Yocto image.

INITRD_IMAGE_LIVE="core-image-minimal-initramfs"

Yocto executes multiple build steps in parallel therefore it important to specify build dependencies. In this case the WIC image needs both the initrd and target rootfs before it can build, and these are both dependent on the kernel being built first.

do_image_wic[depends] += "${INITRD_IMAGE_LIVE}:do_image_complete"
do_image_wic[depends] += "${IMAGE_BASENAME}:do_image_ext4"
do_rootfs[depends] += "virtual/kernel:do_deploy"

IMAGE_BOOT_FILES specifies the files that will be included into the boot image.

IMAGE_BOOT_FILES:append = "\
    ${KERNEL_IMAGETYPE} \
    \
    ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.ext4;rootfs.img \
    \
    ${@bb.utils.contains('EFI_PROVIDER', 'grub-efi', \
      'grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi', \
      '', d)} \
    \
    ${@bb.utils.contains('EFI_PROVIDER', 'grub-efi', \
      '${IMAGE_ROOTFS}/boot/EFI/BOOT/grub.cfg;EFI/BOOT/grub.cfg', \
      '', d)} \
    \
    \
    ${@bb.utils.contains('EFI_PROVIDER', 'systemd-boot', \
      'systemd-bootx64.efi;EFI/BOOT/bootx64.efi', '', d)} \
    \
    ${@bb.utils.contains('EFI_PROVIDER', 'systemd-boot', \
      '${IMAGE_ROOTFS}/boot/loader/loader.conf;loader/loader.conf', \
      '', d)} \
    \
    ${@bb.utils.contains('EFI_PROVIDER', 'systemd-boot', \
      '${IMAGE_ROOTFS}/boot/loader/entries/boot.conf;loader/entries/boot.conf', \
      '', d)} "

The kernel and target root file system (an .ext4 image file) are always added to the image. Notice the bb.utils.contains statements – this is a common helper function used throughout Yocto. It checks whether any of the values in argument $2 (e.g. ‘grub-efi’) appear in variable $1 (e.g. ‘EFI_PROVIDER’). If true then string $3 is inserted, otherwise $4 is inserted. In the above snippet this is used to select different boot files depending whether the image is configured to use the GRUB or systemd-boot bootloader.

We have only one boot menu option in GRUB (‘install’ for the live image, and ‘boot’ for the target image) so there is no point waiting for the user to press a key. Therefore set the timeout to zero and boot immediately.

GRUB_TIMEOUT = "0"

WKS file

The OpenEmbedded Kickstart file (WKS) defines the configuration passed to the OpenEmbedded Image Creator tool. This specifies the partition layout and content of the generated WIC file.

Explained: Why is it WKS and not OEKS? Well, Open Embeeded is abbreviated to OE, which is replaced by the similar phoneme W. The same applies for the OpenEmbedded Image Creator, abbreviated to OEIC, and then WIC.

The WKS file for self-installing defines two partitions:
1. VFAT named ‘install’: holding the live bootloader, kernel and initrd (.cpio.gz)
2. EXT4 name ‘image’: holding the bootloader, kernel and rootfs ext4 image for the target device

The WIC tool greatly simplifies the process of building disc images. Each partition is assigned a source type that tells WIC what it will be used for. There is configuration to override default source settings (e.g. through sourceparams) and also configuration generic to any partition regardless of source (e.g. ondisk or fstype)

# Create a live self-installer disk image
# populate content to install using IMAGE_BOOT_FILES
part /boot   \
     --source bootimg-efi \
     --sourceparams="loader=${EFI_PROVIDER}, \
                     title=install, \
                     label=install-efi, \
                     initrd=${INITRD_IMAGE_LIVE}-${MACHINE}.${INITRAMFS_FSTYPES}" \
     --ondisk sda --label install --align 1024 --use-uuid --active \

part /   \
     --source bootimg-partition \
     --ondisk sda --label image --align 1024 --use-uuid --fstype=ext4 \

bootloader --ptable gpt --timeout=0 --append=" rootwait "

Install script

The actual installation of EFI self-booting image is performed by a shell script.

Default install script:
poky/meta/recipes-core/initrdscripts/files/init-install-efi.sh

Installed to this location on the target:
/init.d/install-efi.sh

Added to the image by this file:
poky/meta/recipes-core/initrdscripts/initramfs-module-install-efi_1.0.bb

Although it works, and will install an image, it is best viewed as a starting point for creating your own custom install script. The script is well commented and easy to modify. Example changes might include:

• Run without user intervention
• Disable the swap partition
• Install to the first fixed disc
• Create additional partitions for A/B update
• Create partitions for read-only and overlay file systems.

To use your own custom install script, start by making a copy of the example script:

Copy the script from:
poky/meta/recipes-core/initrdscripts/files/init-install-efi.sh

...to the directory in your layer:
meta-mylayer/recipes-core/initrdscripts/module-install-efi/init-install-efi.sh

We then tell Yocto to use your script:

Create a bbappend file named:
meta-mylayer/recipes-core/initrdscripts/initramfs-module-install-efi_%.%.bbappend

...containing this one line:
FILESEXTRAPATHS:prepend := "${THISDIR}/module-install-efi:"

It it not usually necessary to modify the script that itself calls the install script. If you need to do so, it is located here:

Default install script:
poky/meta/recipes-core/initrdscripts/initramfs-framework/setup-live

Installed to this location on the target:
/init.d/80-setup-live.sh

Added to the image by this file:
poky/meta/recipes-core/initrdscripts/initramfs-module-setup-live_1.0.bb

Conclusion

The image installation process is one of the often overlooked areas of embedded systems design but, as this blog-post shows, Yocto makes it easy to add fast and professional self-installation to your build. Although more commonly used for x86, this process can be extended to support ARM and other platforms.

Please contact us if you would like us to help you customize your Yocto system.