Trying to get iPXE as the default method to netinstalls working

(based on http://ipxe.org/howto/chainloading and https://doc.rogerwhittaker.org.uk/ipxe-installation-and-EFI/)

The goal is to support legacy BIOS PXE boots as well as UEFI network boots both via PXE and HTTP. Each method will have its dedicated set of first steps but will all use common iPXE steps. For this, each node should always first boot via the network and then fall through to the locally installed OS. We will always boot iPXE first, which in turn will query FAI's pxelinug.cfg set-up to create a proper boot menu for the server.

client booting steps

BIOS/PXE boot

1. a system starts querying its DHCP server
2. it ought to receive its IP, the address of a TFTP server and a filename to download.
3. it should get the undionly.kpxe iPXE file via TFTP with an embedded "chain file" described below.
4. continue with iPXE below

UEFI network boot (both PXE and HTTP)

Upfront - as most servers still allow "secure boot" to be disabled, this HowTo is ignoring booting via signed "shims".

1. as before, the DHCP server is queried
2. in either UEFO boot case the file ipxe.efi should be downloaded, either via TFTP or via HTTP
3. from there on, this file should start and with the embedded chain load should load the installation menu

Preperations

Build iPXE images with embedded chain loader

1. install necessary stuff:

   apt -y install git build-essential liblzma-dev

2. get ipxe repo:

   git clone git://git.ipxe.org/ipxe.git

3. download both patch files (attached to this page) and apply those:

   cd ipxe/src
   git am 00*patch
   

4. edit chain.ipxe and fix the IP address as necessary
5. build for legacy PXE:

   make bin/undionly.kpxe EMBED=chain.ipxe

6. build for EFI:

   make bin-x86_64-efi/ipxe.efi EMBED=chain.ipxe

7. copy the resulting files bin/undionly.kpxe and bin-x86_64-efi/ipxe.efi to the fai server, i.e. undionly.kpxe to /srv/tftp/ipxe and ipxe.efi to both /srv/tftp/ipxe/ and /var/www/html/ipxe/.

DHCP set-up

TBD (above change plus small change to default filename for legacy PXE) To be refined...

Here, the DHCP server will offer the iPXE EFI file with a simple switch, e.g. for isc-dhcpd:

        class "pxeclients" {
          match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
          next-server 10.20.60.103;
          filename "ipxe/ipxe.efi";
        }
        class "httpclients" {
          match if substring (option vendor-class-identifier, 0, 10) = "HTTPClient";
          option vendor-class-identifier "HTTPClient";
          filename "http://10.20.60.103/ipxe/ipxe.efi";
        }

TFTP set-up

1. Install server:

   apt -y install tftpd-hpa

2. Create ipxe directory within the TFTP tree and copy files created in step one here, e.g.

   mkdir -p /srv/tftp/ipxe
   ls -l
   total 992
   -rw-r--r-- 1 root root 943264 Jul  9 09:37 ipxe.efi
   -rw-r--r-- 1 root root  67022 Jul  9 09:37 undionly.kpxe
   

Web server set-up

1. Install web server of choice, e.g.

   apt -y install lighttpd

2. Create download directory and copy ipxe.efi there as well (same file as put into tftp area above)L

   mkdir -p /var/www/html/ipxe
   ls -l ipxe.efi 
   -rw-r--r-- 1 root root 943296 Jul  9 11:34 ipxe.efi
   

3. We will use ruby for the generated menu file but any other scripting language ought to work here, Perl, Python, PHP, ... Therefore install the language of choice.
4. for lighttd we add this configuration:

   server.modules += ( "mod_cgi" )
   
   $HTTP["url"] =~ "^/ipxe/install.ipxe" {
           cgi.assign = ( ".ipxe" => "/usr/bin/ruby" )
           alias.url += ( "/ipxe/" => "/usr/lib/cgi-bin/" )
   }
   
   

An example script for /usr/lib/cgi-bin/install.ipxe is attached.

Findings/results

Problems

• We encountered problems with Proxmox's UEFI/PXE booting (OVHM BIOS) - it would get the ipxe.efi file from the TFTP server, but would not be able to run it. Thus, currently we need to stay with legacy PXe there.
• ISO booting seems to be very problematic and for example in iPXE forums, it is highly discouraged. memdisk itself could only run in non-EFI mode and is thus not taken into account.
• GRML booting still does not work: Trying to with fetch=IP/path but it will currently not boot.

Successes

• Asus nodes (P9D-M mainboards) were able to boot via UEFI/PXE
• X11DDW-NT based system was able to boot via UEFI/HTTP
• Proxmox VM with SeaBIOS work via legacy/PXE

-- CarstenAulbert - 09 Jul 2019