Writing an OS in Rust: Hardware Interrupts

In this post we set up the programmable interrupt controller to correctly forward hardware interrupts to the CPU. To handle these interrups we add new entries to our interrupt descriptor table, just like we did for our exception handlers. We will learn how to get periodic timer interrupts and how to get input from the keyboard.

This blog is openly developed on Github . If you have any problems or questions, please open an issue there. You can also leave comments.

Interrupts provide a way to notify the CPU from attached hardware devices. So instead of letting the kernel periodically check the keyboard for new characters (a process called polling ), the keyboard can notify the kernel of each keypress. This is much more efficient because the kernel only needs to act when something happened. It also allows faster reaction times, since the kernel can react immediately and not only at the next poll.

Connecting all hardware devices directly to the CPU is not possible. Instead, a separate interrupt controller aggregates the interrupts from all devices and then notifies the CPU:

                                    ____________             _____
               Timer ------------> |            |           |     |
               Keyboard ---------> | Interrupt  |---------> | CPU |
               Other Hardware ---> | Controller |           |_____|
               Etc. -------------> |____________|

Most interrupt controllers are programmable, which means that they support different priority levels for interrupts. For example, this allows to give timer interrupts a higher priority than keyboard interrupts to ensure accurate timekeeping.

Unlike exceptions, hardware interrupts occur asynchronously . This means that they are completely independent from the executed code and can occur at any time. Thus we suddenly have a form of concurrency in our kernel with all the potential concurrency-related bugs. Rust's strict ownership model helps us here because it forbids mutable global state. However, deadlocks are still possible, as we will see later in this post.

The Intel 8259 is a programmable interrupt controller (PIC) introduced in 1976. It has long been replaced by the newer APIC , but its interface is still supported on current systems for backwards compatibiliy reasons. The 8259 PIC is significantly easier to set up than the APIC, so we will use it to introduce ourselves to interrupts before we switch to the APIC in a later post.

The 8259 has 8 interrupt lines and several lines for communicating with the CPU. The typical systems back then where equipped with two instances of the 8259 PIC, one primary and one secondary PIC connected to one of the interrupt lines of the primary:

                     ____________                          ____________
Real Time Clock --> |            |   Timer -------------> |            |
ACPI -------------> |            |   Keyboard-----------> |            |      _____
Available --------> | Secondary  |----------------------> | Primary    |     |     |
Available --------> | Interrupt  |   Serial Port 2 -----> | Interrupt  |---> | CPU |
Mouse ------------> | Controller |   Serial Port 1 -----> | Controller |     |_____|
Co-Processor -----> |            |   Parallel Port 2/3 -> |            |
Primary ATA ------> |            |   Floppy disk -------> |            |
Secondary ATA ----> |____________|   Parallel Port 1----> |____________|

This graphic shows the typical assignment of interrupt lines. We see that most of the 15 lines have a fixed mapping, e.g. line 4 of the secondary PIC is assigned to the mouse.

Each controller can be configured through twoI/O ports, one “command” port and one “data” port. For the primary controller these ports are 0x20 (command) and 0x21 (data). For the secondary controller they are 0xa0 (command) and 0xa1 (data). For more information on how the PICs can be configured see the article on osdev.org .

The default configuration of the PICs is not usable, because it sends interrupt vector numbers in the range 0–15 to the CPU. These numbers are already occupied by CPU exceptions, for example number 8 corresponds to a double fault. To fix this overlapping issue, we need to remap the PIC interrupts to different numbers. The actual range doesn't matter as long as it does not overlap with the exceptions, but typically the range 32–47 is chosen, because these are the first free numbers after the 32 exception slots.

The configuration happens by writing special values to the command and data ports of the PICs. Fortunately there is already a crate called pic8259_simple , so we don't need to write the initialization sequence ourselves. In case you are interested how it works, check out its source code , it's fairly small and well documented.

To add the crate as dependency, we add the following to our project:

# in Cargo.toml

[dependencies]
pic8259_simple = "0.1.1"

// in src/lib.rs

extern crate pic8259_simple;

The main abstraction provided by the crate is the ChainedPics struct that represents the primary/secondary PIC layout we saw above. It is designed to be used in the following way:

// in src/interrupts.rs

use pic8259_simple::ChainedPics;
use spin;

pub const PIC_1_OFFSET: u8 = 32;
pub const PIC_2_OFFSET: u8 = PIC_1_OFFSET + 8;

pub static PICS: spin::Mutex<ChainedPics> =
    spin::Mutex::new(unsafe { ChainedPics::new(PIC_1_OFFSET, PIC_2_OFFSET) });

We're setting the offsets for the pics to the range 32–47 as we noted above. By wrapping the ChainedPics struct in a Mutex we are able to get safe mutable access (through the lock method ), which we need in the next step. The ChainedPics::new function is unsafe because wrong offsets could cause undefined behavior.

We can now initialize the 8259 PIC from our _start function:

// in src/main.rs

#[cfg(not(test))]
#[no_mangle]
pub extern "C" fn _start() -> ! {
    println!("Hello World{}", "!");

    blog_os::gdt::init();
    blog_os::interrupts::init_idt();
    unsafe { PICS.lock().initialize() }; // new

    println!("It did not crash!");
    loop {}
}

We use the initialize function to perform the PIC initialization. Like the ChainedPics::new function, this function is also unsafe because it can cause undefined behavior if the PIC is misconfigured.

If all goes well we should continue to see the "It did not crash" message when executing bootimage run .

Enabling Interrupts

Until now nothing happened because interrupts are still disabled in the CPU configuration. This means that the CPU does not listen to the interrupt controller at all, so no interrupts can reach the CPU. Let's change that:

// in src/main.rs

#[cfg(not(test))]
#[no_mangle]
pub extern "C" fn _start() -> ! {
    println!("Hello World{}", "!");

    blog_os::gdt::init();
    blog_os::interrupts::init_idt();
    unsafe { PICS.lock().initialize() };
    x86_64::instructions::interrupts::enable(); // new

    println!("It did not crash!");
    loop {}
}

The interrupts::enable function of the x86_64 crate executes the special sti instruction (“set interrupts”) to enable external interrupts. When we try bootimage run now, we see that a double fault occurs:

The reason for this double fault is that the hardware timer (the Intel 8253 to be exact) is enabled by default, so we start receiving timer interrupts as soon as we enable interrupts. Since we didn't define a handler function for it yet, our double fault handler is invoked.

Handling Timer Interrupts

As we see from the graphic, the timer uses line 0 of the primary PIC. This means that it arrives at the CPU as interrupt 32 (0 + offset 32). Therefore we need to add a handler for interrupt 32 if we want to handle the timer interrupt:

// in src/interrupts.rs

pub const TIMER_INTERRUPT_ID: u8 = PIC_1_OFFSET; // new

[…]

lazy_static! {
    static ref IDT: InterruptDescriptorTable = {
        let mut idt = InterruptDescriptorTable::new();
        idt.breakpoint.set_handler_fn(breakpoint_handler);
        […]
        idt[usize::from(TIMER_INTERRUPT_ID)]
            .set_handler_fn(timer_interrupt_handler); // new

        idt
    };
}

extern "x86-interrupt" fn timer_interrupt_handler(
    _stack_frame: &mut ExceptionStackFrame)
{
    print!(".");
}

We introduce a TIMER_INTERRUPT_ID constant to keep things organized. Our timer_interrupt_handler has the same signature as our exception handlers, because the CPU reacts identically to exceptions and external interrupts (the only difference is that some exceptions push an error code). The InterruptDescriptorTable struct implements the IndexMut trait, so we can access individual entries through array indexing syntax.

In our timer interrupt handler, we print a dot to the screen. As the timer interrupt happens periodically, we would expect to see a dot appearing on each timer tick. However, when we run it we see that only a single dot is printed:

The reason is that the PIC expects an explicit “end of interrupt” (EOI) signal from our interrupt handler. This signal tells the controller that the interrupt was processed and that the system is ready to receive the next interrupt. So the PIC thinks we're still busy processing the first timer interrupt and waits patiently for the EOI signal before sending the next one.

To send the EOI, we use our static PICS struct again:

// in src/interrupts.rs

extern "x86-interrupt" fn timer_interrupt_handler(
    _stack_frame: &mut ExceptionStackFrame)
{
    print!(".");
    unsafe { PICS.lock().notify_end_of_interrupt(TIMER_INTERRUPT_ID) }
}

The notify_end_of_interrupt figures out wether the primary or secondary PIC sent the interrupt and then uses the command and data ports to send an EOI signal to respective controllers. If the secondary PIC sent the interrupt both PICs need to be notified because the secondary PIC is connected to an input line of the primary PIC.

We need to be careful to use the correct interrupt vector number, otherwise we could accidentally delete an important unsent interrupt or cause our system to hang. This is the reason that the function is unsafe.

When we now execute bootimage run we see dots periodically appearing on the screen:

Configuring the Timer

The hardware timer that we use is called the Progammable Interval Timer or PIT for short. Like the name says, it is possible to configure the interval between two interrupts. We won't go into details here because we will switch to the APIC timer soon, but the OSDev wiki has an extensive article about the configuring the PIT .

We now have a form of concurrency in our kernel: The timer interrupts occur asynchronously, so they can interrupt our _start function at any time. Fortunately Rust's ownership system prevents many types of concurrency related bugs at compile time. One notable exception are deadlocks. Deadlocks occur if a thread tries to aquire a lock that will never become free. Thus the thread hangs indefinitely.

We can already provoke a deadlock in our kernel. Remember, our println macro calls the vga_buffer::print function, which locks a global WRITER using a spinlock:

// in src/vga_buffer.rs

[…]

pub fn print(args: fmt::Arguments) {
    use core::fmt::Write;
    WRITER.lock().write_fmt(args).unwrap();
}

It locks the WRITER , calls write_fmt on it, and implicitly unlocks it at the end of the function. Now imagine that an interrupt occurs while the WRITER is locked and the interrupt handler tries to print something too:

Timestep _start interrupt_handler 0 calls println! 1 print locks WRITER 2 interrupt occurs , handler begins to run 3 calls println! 4 print tries to lock WRITER (already locked) 5 print tries to lock WRITER (already locked) … … never unlock WRITER

The WRITER is locked, so the interrupt handler waits until it becomes free. But this never happens, because the _start function only continues to run after the interrupt handler returns. Thus the complete system hangs.

Provoking a Deadlock

We can easily provoke such a deadlock in our kernel by printing something in the loop at the end of our _start function:

// in src/main.rs

#[cfg(not(test))]
#[no_mangle]
pub extern "C" fn _start() -> ! {
    […]
    loop {
        print!("-");        // new
    }
}

When we run it in QEMU we get output of the form:

We see that only a limited number of hyphens is printed, until the first timer interrupt occurs. Then the system hangs because the timer interrupt handler deadlocks when it tries to print a dot. This is the reason that we see no dots in the above output.

The actual number of hyphens varies between runs because the timer interrupt occurs asynchronously. This non-determinism is what makes concurrency related bugs so difficult to debug.

Fixing the Deadlock

To avoid this deadlock, we can disable interrupts as long as the Mutex is locked:

// in src/vga_buffer.rs

/// Prints the given formatted string to the VGA text buffer
/// through the global `WRITER` instance.
pub fn print(args: fmt::Arguments) {
    use core::fmt::Write;
    use x86_64::instructions::interrupts;   // new

    interrupts::without_interrupts(|| {     // new
        WRITER.lock().write_fmt(args).unwrap();
    });
}

The without_interrupts function takes a closure and executes it in an interrupt-free environment. We use it to ensure that no interrupt can occur as long as the Mutex is locked. When we run our kernel now we see that it keeps running without hanging. (We still don't notice any dots, but this is because they're scrolling by too fast. Try to slow down the printing, e.g. by putting a for _ in 0..10000 {} inside the loop.)

We can apply the same change to our serial printing function to ensure that no deadlocks occur with it either:

// in src/serial.rs

pub fn print(args: ::core::fmt::Arguments) {
    use core::fmt::Write;
    use x86_64::instructions::interrupts;       // new

    interrupts::without_interrupts(|| {         // new
        SERIAL1
            .lock()
            .write_fmt(args)
            .expect("Printing to serial failed");
    });
}

Note that disabling interrupts shouldn't be a general solution. The problem is that it increases the worst case interrupt latency, i.e. the time until the system reacts to an interrupt. Therefore interrupts should be only disabled for a very short time.

The hlt Instruction

Until now we used a simple empty loop statement at the end of our _start and panic functions. This causes the CPU to spin endlessly and thus works as expected. But it is also very inefficient, because the CPU continues to run at full speed even though there's no work to do. You can see this problem in your task manager when you run your kernel: The QEMU process needs close to 100% CPU the whole time.

What we really want to do is to halt the CPU until the next interrupt arrives. This allows the CPU to enter a sleep state in which it consumes much less energy. The hlt instruction does exactly that. Let's use this instruction to create an energy efficient endless loop:

// in src/lib.rs

pub fn hlt_loop() -> ! {
    loop {
        x86_64::instructions::hlt();
    }
}

The instructions::hlt function is just a thin wrapper around the assembly instruction. It is safe because there's no way it can compromise memory safety.

We can now use this hlt_loop instead of the endless loops in our _start and panic functions:

// in src/main.rs

#[cfg(not(test))]
#[no_mangle]
pub extern "C" fn _start() -> ! {
    […]

    println!("It did not crash!");
    blog_os::hlt_loop();            // new
}


#[cfg(not(test))]
#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
    println!("{}", info);
    blog_os::hlt_loop();            // new
}

We can also use hlt_loop in our double fault exception handler as well:

// in src/interrupts.rs

use hlt_loop; // new

extern "x86-interrupt" fn double_fault_handler(
    stack_frame: &mut ExceptionStackFrame,
    _error_code: u64,
) {
    println!("EXCEPTION: DOUBLE FAULT\n{:#?}", stack_frame);
    hlt_loop(); // new
}

When we run our kernel now in QEMU, we see a much lower CPU usage.

Now that we are able to handle interrupts from external devices we are finally able to add support for keyboard input. This will allow us to interact with our kernel for the first time.

Like the hardware timer, the keyboard controller is already enabled by default. So when you press a key the keyboard controller sends an interrupt to the PIC, which forwards it to the CPU. The CPU looks for a handler function in the IDT, but the corresponding entry is empty. Therefore a double fault occurs.

So let's add a handler function for the keyboard interrupt. It's quite similar to how we defined the handler for the timer interrupt, it just uses a different interrupt number:

// in src/interrupts.rs

pub const KEYBOARD_INTERRUPT_ID: u8 = PIC_1_OFFSET + 1; // new

lazy_static! {
    static ref IDT: InterruptDescriptorTable = {
        let mut idt = InterruptDescriptorTable::new();
        idt.breakpoint.set_handler_fn(breakpoint_handler);
        […]
        // new
        idt[usize::from(KEYBOARD_INTERRUPT_ID)]
            .set_handler_fn(keyboard_interrupt_handler);

        idt
    };
}

extern "x86-interrupt" fn keyboard_interrupt_handler(
    _stack_frame: &mut ExceptionStackFrame)
{
    print!("k");
    unsafe { PICS.lock().notify_end_of_interrupt(KEYBOARD_INTERRUPT_ID) }
}

As we see from the graphic, the keyboard uses line 1 of the primary PIC. This means that it arrives at the CPU as interrupt 33 (1 + offset 32). We again create a KEYBOARD_INTERRUPT_ID constant to keep things organized. In the interrupt handler, we print a k and send the end of interrupt signal to the interrupt controller.

We now see that a k appears on the screen when we press a key. However, this only works for the first key we press, even if we continue to press keys no more k s appear on the screen. This is because the keyboard controller won't send another interrupt until we have read the so-called scancode of the pressed key.

Reading the Scancodes

To find out which key was pressed, we need to query the keyboard controller. We do this by reading from the from the data port of the PS/2 controller, which is theI/O port with number 0x60 :

// in src/interrupts.rs

extern "x86-interrupt" fn keyboard_interrupt_handler(
    _stack_frame: &mut ExceptionStackFrame)
{
    use x86_64::instructions::port::Port;

    let port = Port::new(0x60);
    let scancode: u8 = unsafe { port.read() };
    print!("{}", scancode);
    unsafe { PICS.lock().notify_end_of_interrupt(KEYBOARD_INTERRUPT_ID) }
}

We use the Port type of the x86_64 crate to read a byte from the keyboard's data port. This byte is called the scancode and is a number that represents the key press/release. We don't do anything with the scancode yet, we just print it to the screen:

The above image shows me slowly typing "123". We see that adjacent keys have adjacent scancodes and that pressing a key causes a different scancode than releasing it. But how do we translate the scancodes to the actual key actions exactly?

Interpreting the Scancodes

There are three different standards for the mapping between scancodes and keys, the so-called scancode sets . All three go back to the keyboards of early IBM computers: the IBM XT , the IBM 3270 PC , and the IBM AT . Later computers fortunately did not continue the trend of defining new scancode sets, but rather emulated the existing sets and extended them. Today most keyboards can be configured to emulate any of the three sets.

By default, PS/2 keyboards emulate scancode set 1 ("XT"). In this set, the lower 7 bits of a scancode byte define the key, and the most significant bit defines whether it's a press ("0") or a release ("1"). Keys that were not present on the original IBM XT keyboard, such as the enter key on the keypad, generate two scancodes in succession: a 0xe0 escape byte and then a byte representing the key. For a list of all set 1 scancodes and their corresponding keys, check out the OSDev Wiki .

To translate the scancodes to keys, we can use a match statement:

// in src/interrupts.rs

extern "x86-interrupt" fn keyboard_interrupt_handler(
    _stack_frame: &mut ExceptionStackFrame)
{
    use x86_64::instructions::port::Port;

    let port = Port::new(0x60);
    let scancode: u8 = unsafe { port.read() };

    // new
    let key = match scancode {
        0x02 => Some('1'),
        0x03 => Some('2'),
        0x04 => Some('3'),
        0x05 => Some('4'),
        0x06 => Some('5'),
        0x07 => Some('6'),
        0x08 => Some('7'),
        0x09 => Some('8'),
        0x0a => Some('9'),
        0x0b => Some('0'),
        _ => None,
    };
    if let Some(key) = key {
        print!("{}", key);
    }
    unsafe { PICS.lock().notify_end_of_interrupt(KEYBOARD_INTERRUPT_ID) }
}

The above code just translates keypresses of the number keys 0-9 and ignores all other keys. Now we can write numbers:

Translating the other keys could work in the same way, probably with an enum for control keys such as escape or backspace. Such a translation function would be a good candidate for a small external crate, but I couldn't find one that works with scancode set 1. In case you'd like to write such a crate and need mentoring, just let us know, we're happy to help!

Configuring the Keyboard

It's possible to configure some aspects of a PS/2 keyboard, for example which scancode set it should use. We won't cover it here because this post is already long enough, but the OSDev Wiki has an overview of possible configuration commands .

In this post we learned how to enable and handle external interrupts. We learned about the 8259 PIC and its primary/secondary layout, the remapping of the interrupt numbers, and the "end of interrupt" signal. We saw that the hardware timer and the keyboard controller are active by default and start to send interrupts as soon as we enable them in the CPU. We learned about the hlt instruction, which halts the CPU until the next interrupt, and about the scancode sets of PS/2 keyboards.

Now we are able to interact with our kernel and have some fundamental building blocks for creating a small shell or simple games.

As already mentioned, the 8259 APIC has been superseded by the APIC , a controller with more capabilities and multicore support. In the next post we will explore this controller and learn how to use its integrated timer and how to set interrupt priorities.