Creating a custom Linux Live USB environment

Published: January 27, 2015

In the advanced C programming class that I assist in teaching, we are holding interactive programming exams. That means that every student is tasked with constructing a C program in a time frame of a about 2 hours. Students have access to GCC, Valgrind, as well as the Emacs and Vim text editors (we are staunch proponents of religious freedom).

One hurdle in holding programming exams is the problem of cheating: Nowadays, every computer at any computer lab is connected to some kind of network, which would make it easy for students to share files or look up hints and answers to problems on the internet. Our solution to this problem involves booting the lab computers from a USB drive which contains a customized, lock-down version of Linux, combined with the aforementioned tools and assignment files.

After shopping around for possible distributions to base my custom exam environment on (I looked at Debian Live and Fedora), I quicky settled on Arch Linux. The linked wiki provides great documentation about the process of installing Arch on a USB drive, and since I run Arch on my personal machine, I had all of the required tools already on my machine.

Whereas my attempts to build Debian and Fedora images never quite worked out, doing things "The Arch Way ©" was a pleasure. Installation on a USB drive works the same way as installing it on a regular hard drive, with only a few minor differences.

The USB drives I am using are your standard, run-of-of-the-mill 4 GB USB drives. My plan was to finish one USB drive, test and debug it on the lab computers, and then copy the 'master' image on as many drives that I need.

I partitioned my USB key the following way:

I configured the image to not use a display manager, but to instead automatically log in the user ee264 into a terminal session. This is archieved by changing a systemd configuration file. Then, as described here, the user ee264 automatically executes the startx command, which loads the desktop environment.

I chose MATE as the DE for my custom distribution, mainly because our lab's computers run RHEL 6, which uses GNOME 2.

To disable any kind of network connectivity, I made sure to uninstall tools like the DCHP client dhcpcd and remote access tools like ssh. Also, I configured iptables to drop any incoming or outgoing data packets. This means absolute radio silence. The configuration file for iptables looks like this: 

/etc/iptables/iptables.rules
----------------------------
filter :INPUT DROP [0:0] :FORWARD DROP [0:0] :OUTPUT DROP [0:0] COMMIT

After setting up basic configuration files for Emacs and Vim, installing the bootloader, and setting a long root password, the master USB drive was done. After testing it and confirming that it actually booted up on our computer's lab machines, I created my master image: A .img file that contains the entire drive, including the bootloader and all of the partitions. To do this, I used the GNU coreutils tool dd (Note that in oder to access /dev/* files, I needed to execute dd as root): 

dd if=/dev/sdd of=/data/ee264exam_usb.img

Now I had a working, bootable USB image on my harddrive. But, as it is the norm with many things, making the exam OS was an iterative process, and rebooting my computer every time I wanted to make a change to the image was a massive waste of time.

To shorten my feedback loop, I explored other options, and one of them was the utility kpartx which is included in the AUR package multipath-tools. With help from this blog post, I was able to mount the partitions of my .img file into my local filesystem. A combination of the tools arch-chroot (a nice spin on the regular chroot) and pacstrap allowed me to navigate my USB filesystem from my regular Arch Linux installation without rebooting.

Another cool trick to shorten the feedback loop was booting the .img file inside a virtual machine, without having to set up a new VirtualBox or VMware machine. This was done using QEMU: 

qemu-system-x86_64 -m 4096 -vga std -usbdevice tablet -hda /data/ee264exam_usb.img

Here, the -m 4096 switch gives the VM 4 GB of memory, -vga std enables video, -usbdevice tablet matches the host mouse cursor to the VM cursor, and -hda /data/ee264exam_usb.img loads my .img file as the VM's main harddrive.

When it was time to finalize the image and to make 20 identical copies, I was faced with a problem: The USB drives we ordered were of the 'painfully slow' kind. Using dd to write the image on to drive took 17 minutes at a rate of only 4 MB/s! In addition to that, dd is not the most verbose of tools, either: It does not output any status information until it's done copying bytes. My laptop has 3 USB ports, so to be as efficient as possible, I wanted to make use of all three ports at the same time. After searching for solutions, I stumbled across these two hints. The first link introduced my to the tool pv, or "pipe viewer", which can be used to monitor the status of things passing through its stdin and stdout, and the second link showed me a smart way of redirecting and splitting dd's output using the tee command. From this, I solved my problem of writing three USB drives from one image at the same time with one line of bash: 

dd if=ee264exam_usb,img | pv -petrb -s $(stat -c%s ee264exam_usb.img) | tee >(dd of=/dev/sdd) >(dd of=/dev/sde) | dd of=/dev/sdf

This wonderful line of bash uses dd to read my image file, pv to create a progress bar, and tee to create 3 output streams that are subsequently input to three calls to dd to write to the USB keys.

This is pure beauty.

It will be interesting to see how students perform during an interactive exam, but on thing is for sure: A lot of work has gone into their custom "Exam OS".