The first one is fun and you got to discover all the new options. During the second one you're still amazed by all the shiny new features. The excitement fades away one installation after the other. By the time you've reached the thirteenth it's been four hours and you're getting sick of the blue and red colors. Your mind starts to wander "if I could automate all of this, I could watch the latest episode of [name your favorite TV show here] instead of triggering an epileptic fit due to the red and blue stroboscopic effect of the installer"...
Virtualization, templating and scripting come to the rescue ! Here is an overview of the workflow I've developed and refined to conduct ubuntu-server iso testing at each milestone.
Generate one preseed for each test case
Although each test case has its own preseed file most of the content is identical in all of them. Only the package installation and the partition setup can differ. The mako template engine is used to generate all of the presseed files.
Each test case template inherits from a base template which has all of the content. The base template defines two functions - pkg_install and partition - that will be called in each test case template to define the correct content of the preseed file.
Remaster the iso for each test case
Once the preseed has been created the original server iso is remastered. The preseed is added to the iso and the isolinux configuration file is modified to use it. The installer is also booted with a debconf level of critical. All necessary debconf questions have preseeded answers either from the kernel command line (via the isolinux configuration line) or via the generated preseed file. The goal is to have the installation fully automated without any interaction required.
Create a guest for each test case
The next step is to define a virtual guest for each test case that is set to boot from the remastered iso. A qcow2 file is created to be used as the root partition. A template file is used to create the libvirt configuration file for the guest. It has a custom network address and correct paths to the root disk and remastered iso file.
Automate the whole setup phase
The three operations outlined above are automated for each ubuntu-server isos to be tested. In the end there are twenty guests ready to be started.
The directory looks like this:
intrepid-server-amd64-default/
intrepid-server-amd64-dns-server/
intrepid-server-amd64-lamp/
[...]
intrepid-server-i386-default/
intrepid-server-i386-dns-server/
[...]Each intrepid-server-* directory has the following structure:
preseed
remaster.iso
vm/libvirt.xml
vm/root.qcow2
vm/root.qcow2.orig
Run the installation phase of each test case
Once the guests are setup they are booted one after the other. However a guest installation requires IO on the host. Once installed the guest vm reboots and then idles. In order to not bring down the server, guests are only booted if the load is below a certain threshold.
Perform the test procedure and report the result
After a couple of hours all of the guests are installed, rebooted and ready to be tested. This part of the process boils down to login via ssh on each guest and follow the test case procedure from the ServerInstall wiki page. The outcome is then reported to the iso testing tracker.
This part is still manual. The next step is to automate it: run the test procedure according to the installation that has been done and report the result to a central location. Once implemented whenever a new ubuntu-server iso is available it can be automatically downloaded, configured, installed and tested. Continuous integration testing is just around the corner !
