Chapter 7: Timers And Interrupts
You may already know what interrupts are by now. Typically in general computer science langauge, an interrupt simply means temporarily stopping the execution of some set of instruction and executing some instructions at a completely different location. Then returning back to the original set of instructions whose execution was interrupted.
Of course this is not a new concept to you at all. You have already seen this kind of behaviouor in earlier chapters. Which is hardware exceptions of course. In chapter 3 we set up a way to catch hardware exceptions, and once our handler is done handling the exception, it returns the PC back to where it was; right before it got sent to the exception handler. You are very familiar with this procedure by now.
Now as you may remember, we learned that there were four kinds of exceptions that can occur. There was the synchronous, IRQ, FIQ, and SError. These exceptions trigger the CPU to prompt it to do some sort of handling for them. We have also seen through the chapter 5 on syscalls, that it is possible to have manually triggered exceptions for the sake of some manual prompt to the kernel. To be exact we are talking about the svc instruction. Which upon execution, immediately causes an exception in the EL1 space under the type “Synchronous exception”. It is called a synchronous exception because it occured directly do to the execution of instructions in the EL0 space. Execution cannot conitnue without synchronous exceptions being handled. However in this chapter we will learn about a kind of exception of the type IRQ or FIQ.
IRQ Exceptions
Now, the full form of IRQ is “Interrupt Request”. These exceptions are typically caused whenever a hardware event occurs. So an hardware event that you may want to watch for and do something on is notified to the CPU by the hardware as an IRQ exception. It may not be something which needs handling for the current code execution to continue. An example of an hardware event is a USB device being plugged in. If your program is not trying to do anything with USB peripherals, then it likely can continue execution without caring about USB devices being connected or disconnected. Notice how these exceptions are categorzed as interrupt requests. This implies that somewhere in the chain of the hardware event occurring, and the CPU handling the event as a IRQ exception, there is an option presented. An option that lets you choose whether or not this interrupt is worth accepting by the CPU. We will talk about this more in detail later.
Hardware event to IRQ pipeline
Now, whenever an hardware event occurs on your machine, in some way, the electronics on your hardware recognize that event, identify that event, and send an appropriate exception to the CPU. Obviously it is not the CPU chip on the hardware which monitors every single peripheral and peripheral for possible events. There is a different part of the hardware which does this job. That part is called the Interrupt Controller.
The interrupt controller is a component on the hardware whose job is to be the central hub of recognizing hardware events and notifying the CPU by sending an appropriate signal for IRQ exception. Hardware events upon occuring are recognized by the interrupt controller (by signals received from the part of hardware where event occured). And then the interrupt controller records it, and triggers an Interrupt Request to the CPU. The CPU when upon receiving the interrupt request from the Interrupt Controller, it triggers an appropriate exception to the exception handler. There are two points in this pipeline which can be configured. First is the interrupt controller itself. It can be configured to choose which hardware events should cause an IRQ exception to the CPU, and which ones shouldn’t. Next, the CPU itself can also be configured on if certain category of exceptions should straight up be ignored by it.
So, the pipeline looks something like this:
┌─────────────────────────────┐
│ Hardware Event │
└──────────────┬──────────────┘
│ signal sent
▼
┌─────────────────────────────┐
│ Interrupt Controller │
└──────────────┬──────────────┘
│
Is the interrupt controller
configured to produce an IRQ
upon this hardware event?
┌─────┴───────┐
│ │
No│ │Yes
▼ ▼ (CPU receives IRQ signal)
Nothing happens ┌─────┐
│ CPU │
└─────┘
│
▼
is the CPU configured
to accept IRQ interrupts
right now?
┌────────┴──────────┐
│ │
No│ │Yes
▼ ▼
CPU ignores IRQ An IRQ Exception occurs
in our CPU
│
▼
┌───────────────────────┐
│ EL1 Exception Handler │
└───────────────────────┘
Our ultimate goal in this chapter is to set up something called a “timer interrrupt”. It will be an IRQ exception that you can “schedule” ahead of time to occur after a certain duration automatically. This is the core mechanism on which our scheduler will be built. However we will talk details about that later when we implement the scheduler. First let’s learn how to setup the first step of our IRQ pipeline.
A Scheduled Hardware Event
First of all we are going to attempt to setup the first ever step of our interrupt pipeline that we discussed above. Which is the step of actually causing the hardware event.
Our main goal is to make it so there is a certain exception caused into the CPU when T amount of time passes. Upon which, the timer should be reset and once again after passage of T amount of time, the exception must occur again. This will go infinitely.
Sure you can have a part in your kernel where you have a rust counter variable of type integer, and you could just increment it and check if it has reached some threshhold value. And when the threshhold value is reached, you could execute svc instruction. After which the kernel shall reset the variable to zero and start incrementing it again. This could technically work. But it is difficult to be precise with this solution. Since if you want the svc to trigger every lets say 20ms, then you would need to somehow figure out what threshhold value the counter variable would need exactly 20ms to reach. Another issue here is that this relies on the execution of the kernel code for the variable incrementing and svc call. So you have to somehow make sure that this piece of code does not get interrupted and always run exactly the same amount of time on the CPU everytime. It also wastes CPU cycles, costing time in executing instructions to just increment a variable.
There is in fact a better solution exactly for this scenario. Your CPU itself actually has features in it to cause an IRQ hardware event after a fixed amount of time. This feature is called the CPU TIMER. There is a component inside your CPU which actually has a register which it is continually incrementing, and when that register’s value reaches a threshold value in another register, it causes an hardware event signal to the interrupt controller.
It is quite literally a small component inside the CPU chip which is literally called the “timer”. There are four of this on a single CPU core in fact. Since we have 4 cores on the Raspberry Pi 3, it of course means we have total 16 of these timers. Four in each CPU. However there are other timers available also. Let’s list them.
- ARM Generic Timers: These are the timers we just discussed. We will learn about these in the next heading. There’s exactly four of these in each of the available cores. The four are as follows:
- Physical Timer
CNTP: This is the standard timer used typically by kernels running in EL1. - Virtual Timer
CNTV: This is typically used by a guest operating system running in EL1 under a EL2 hypervisor. - Hypervisor Physical Timer
CNTH: Used by a hypervisor at EL2. - Secure World Physical Timer
CNTPS: Used by secure world framework at EL3.
-
The BCM System Timer: This is one exists outside the CPU, on the overall broadcom SoC of the system. This timer is accessed by the CPU as a peripheral with MMIO. It has a 64 bit counter register being incremented at a fixed 1MHz. You can set a compare value to this in one of the four 32 bit compare value registers. When the lower 32 bits of the counter register are equal to any one of the compare value register values, the timer goes off. That is, an IRQ hardware event is generated that signifies the timer going off.
-
The QA7 Timer: This timer exists inside the Interrupt Controller component. Can be accessed within the Interrupt Controller’s MMIO. It is not generally used and only really exists for the sake of backwards compatibility on the Raspberry Pi. We will not discuss about this one.
For our project, we are going to utilize the ARM Generic Timer for scheduling. Specifically the Physical timer. Also called the Physical Non Secure Timer to explicitly differentiate it fromm the Secure World Physical Timer.
The Non Secure Physical Timer
Let’s first start by understanding the easiest and simplest timer among the ARM Generic Timers. The Physical Non Secure timer.
The Physical Non Secure timer works very similarly to how our hypothetical Rust solution would work. Let’s go register wise as we study the functioning of this timer.
CNTPCT_EL0
This is the register which serves the role of the counter register being incremented repeatedly. The name stands for “COUNTER PHYSICAL COUNT EL0”. It is a read only register of 64 bits. The 64 bit value in it is constantly being incremented by one at a certain frequency. That frequency is not fixed, as it can be configured and changed through registers which we will learn about next. Functionally speaking, this register just has a value which is being incremented at that frequency. Nothing more. The EL0 suffix in the name signifies that this register can be accessed from EL0 or higher levels (in read only of course).
CNTP_CVAL_EL0
The name of this register stands for “COUNTER PHYSICAL: COMPARE VALUE EL0”. This a 64 bit read and write register. It serves a very simple function. When the value in CNTPCT_EL0 is greater than or equal to the value inside this register, the Physical Timer goes off! Which means an IRQ hardware event signal is generated to the interrupt controller. So for example if you wanted a timer to go off after 2ms, when the CNTPCT_EL0 value is being increased at some f frequency, you could set this to (CNTPCT_EL0 + 0.002f). Where 0.002f reprenents the increase in the counter value after 2ms. However, you don’t have to actually do this calculation by yourself. There is a register which does this for you. Described next.
CNTP_TVAL_EL0
The name stands for “COUNTER PHYSICAL: TIMER VALUE EL0”. This register is signed 32 bit read and write. It has two behaviours that happen when you attempt to preform write or read to this register. First let’s discuss what happens if you try to write to this.
When you write some value to this register, it is interpreted as a signed 32 bit integer. And immediately the compare value register CNTP_CVAL_EL0 is updated and set exactly to: (CNTPCT_EL0 + CNTP_TVAL_EL0).
So this way, instead of reading the counter register, and then adding to it and updating the compare value register, you can simply set the change in count you want to wait for in this timer value register.
A passive behaviour of this register is that it is constantly being decremented at the same frequency as the counter register CNTPCT_EL0. This does not affect the hardware at all and is merely so that you could read it and get a value that represents how many counts are left before the counter register equals set compare value. Negative values representing that the timer has already gone off.
Of course the constant decrementing being a passive behaviour means it never stops. If the value inside it hits the largest 32 bit negative integer, it overflows and wraps arround to largest positive 32 bit integer.
CNTP_CTL_EL0
This one is very simple. It standards for “COUNTER PHYSICAL: CONTROL EL0”. It is the main control register for the Non Secure Physical counter/timer. It is a 3 bit read and wite register. Each of the three bits represents different fields.
-
Bit[0] "ENABLE": This bit controls if the timer is enabled at all. Set this to0for the timer to stop working. Set this to1to turn the timer on. Very simple. When the timer is off, the hardware event is no longer fired. The increment of counter register and decrement of timer value register still happen. Read and Writes will still work as usual. However the actual hardware event will never occur for the IRQ.Bit[2]also behaves differently which you will see in a bit. -
Bit[1] "MASK": This bit controls if the hardware event singal should be supressed or not. When set to1, the IRQ hardware event for the timer going off will be blocked from reaching the interrupt controller. Otherwise at0the interrupt controller will be delivered the hardware event signal when the timer goes off. Note the timer only “goes off” when the timer is enabled bybit[0]. -
Bit[2] "ISTATUS": Stands for Interrupt Status. It is a read only bit. It merely records if currently the timer’s hardware event signal should be going off to the interrupt controller or not. A value of1means that currently the conditionCNTPCT_EL0 >= CNTP_CVAL_EL0is true, and that the timer is enabled bybit[0].
Note that the IRQ hardware event signal is sent to the interrupt controller at EVERY single increment of the counter register, for which the timer expiration condition is true. So when CNTPCT_EL0 is incremented, and ISTATUS reads the value of 1, an hardware event signal is sent. So your code needs to make sure to turn off this timer upon the first signal otherwise it will keep sending IRQ signals to the interrupt controller repeatedly until CNTPCT_EL0 overflows and becomes smaller than compare value again.
CNTFRQ_EL0
This register actually doesn’t do anything at all. On paper the name stands for “COUNTER FREQUENCY EL0. You would think that this is the register that must be responsible for controlling the frequency at which the counter register is increased at. However that’s not true at all. The frequency is actually controlled by the interrupt controller. This design choice might seem strange or unintuitive to you, but as an OS developer all we can do is understand the design of the hardware and make use of it.
Functionally speaking the only use for this register is to serve as a place to store the current frequency of the timer. Not for any hardware function purpose, but simply so other programs running can read this register to immediately know what frequency the timer counter is being incremented at. It is more so like a dictionary in that regard rather than a register which causes some hardware behaviour.
We will talk about controlling or modifying the frequency later on in this chapter. Just know that on RPI3B+, by default it is 1 MHz.
Other ARM Generic Timers
There are three other timers similar to the one we discussed. However we will not use them in our project, and thus they will not be discussed in great detail. However, they are actually very similar to the one we just discussed.
Both the Hypervisor Physical, and Secure World Physical Timers function exactly the same way the Non Secure Physica Timer does. They all have their own compare value, timer value, and control registers. All prefixed with their own names instead of CNTP_. Which is CNTHP_ for Hypervisor Physical and CNTPS_ for the physical secure timer. The only difference is who owns them.
The Hypervisor timer is meant to be used by something running in EL2 level, called the “hypervisor”. All registers of this timer are also prefixed by _EL2. Thus they can only be accessed by EL2 or higher levels.
The Physical Secure timer is meant to be used by EL3 level software. Or by EL1 OS running in secure world controlled by something from EL3. Registers here are suffixed with _EL1. Because though mainly for EL3, it can be configured to allow usage from EL1.
Both of these are called “Physical” in their name. This is because both of these use the same incrementing counter register for their compare value conditions. Which is the CNTPCT_EL0 register.
The Virtual Timer is a little different. Unlike others, it does not rely on the incrementing physical counter register for comparisions. Instead, it uses the Virtual Counter Register, CNTVCT_EL0. Everything else is the same. It has it’s usual compare value, timer value, and control registers. All prefixed with CNTV_. Also suffixed with _EL0 similar to the non secure physical timer. It is essentially the non secure physical timer which replaces the Physical Counter register for the Virtual Counter register.
The value in virtual counter register “CNTVCT_EL0” is always exactly equal to CNTPCT_EL0 - CNTVOFF_EL2. Where CNTVOFF_EL2 is a 64 bit register defining some offset. Do the naming and formula give you any hints? The CNTVCT_EL0 register basically maintains a value at some offset from the CNTPCT_EL0 register. This offset is defined by EL2 level program since the CNTVOFF_EL2 register can only be accessed by EL2. The name stands for “COUNTER VIRTUAL OFFSET EL2”. If no EL2 program is running to set this offset, it defaults to zero. In which case the CNTVCT_EL0 is the same as CNTPCT_EL0.
Implementing the NS-Physical Timer
Now you understand well enough the ARM Generic Timers on the ARM CPU that our hardware has. And you also understand when those timers send off an IRQ hardware event signal to the interrupt controller. It is finally time to actually implement Rust abstractions that will allow you to interact with the timer easily in code. As discussed we will use the None secure physical timer. Which, is commonly just called the physical timer since the non secure part is redundant unless hypervisor or secure world timers are involved.
Well, the implementation itself is actually not complicated at all. We literally just create functions which will execute appropriate assembly to tamper the relevant timer registers.
Now then, let’s create our src/kernel/timer.rs.
use crate::println;
pub struct PhysicalTimer;
Now, before we implement functions to start timer or set timer to some seconds, let’s first wrap up relevant assembly into neat inline functions. This is because it is a good habit to reduce the amount of time you have to write unsafe code. asm! macro counts as unsafe code and thus we wrap up in functions so we can just call the functions instead of using asm! macro everytime.
impl PhysicalTimer {
#[inline(always)]
pub fn read_cnt() -> u64 {
let value: u64;
unsafe {
core::arch::asm!(
"mrs {val}, CNTPCT_EL0",
val = out(reg) value,
options(nostack, preserves_flags)
);
}
value
}
#[inline(always)]
pub fn read_frq() -> u64 {
let value: u64;
unsafe {
core::arch::asm!(
"mrs {val}, CNTFRQ_EL0",
val = out(reg) value,
options(nostack, preserves_flags)
);
}
value
}
#[inline(always)]
pub fn read_ctl() -> u64 {
let value: u64;
unsafe {
core::arch::asm!(
"mrs {val}, CNTP_CTL_EL0",
"isb",
val = out(reg) value,
options(nostack, preserves_flags)
);
}
value
}
#[inline(always)]
fn write_ctl(val: u64) {
unsafe {
core::arch::asm!(
"msr CNTP_CTL_EL0, {val}",
"isb",
val = in(reg) val,
options(nostack, preserves_flags)
);
}
}
#[inline(always)]
pub fn set_cval(new_cval: u64) {
unsafe {
core::arch::asm!(
"msr CNTP_CVAL_EL0, {val}",
"isb",
val = in(reg) new_cval,
);
}
}
#[inline(always)]
pub fn set_tval(new_tval: i32) {
unsafe {
core::arch::asm!(
"msr CNTP_TVAL_EL0, {val:w}",
"isb",
val = in(reg) new_tval,
);
}
}
}
Now we have functions we can call to read or write to registers. Notice we how we did not create writer function for CNTPCT_EL0 or CNTFRQ_EL0. This is because the first is read only, and the latter serves no purpose for writing to.
We can proceed to creating functions for enabling or disabling timer, or masking or unmasking the timer hardware event IRQ. Which is done using the bits[1:0] of the control register.
#[inline(always)]
pub fn enable() {
let mut ctl = Self::read_ctl();
ctl |= 1 << 0;
Self::write_ctl(ctl);
}
#[inline(always)]
pub fn disable() {
let mut ctl = Self::read_ctl();
ctl &= !1;
Self::write_ctl(ctl);
}
#[inline(always)]
pub fn mask_int() {
let mut ctl = Self::read_ctl();
ctl |= 1 << 1;
Self::write_ctl(ctl);
}
#[inline(always)]
pub fn unmask_int() {
let mut ctl = Self::read_ctl();
ctl &= !(1 << 1);
Self::write_ctl(ctl);
}
And now a function to set the timer to a certain amount of seconds:
pub fn set_seconds(seconds: u64) {
let freq = Self::read_frq();
let now = Self::read_cnt();
let ticks = freq
.checked_mul(seconds)
.expect("Physical Timer Seconds Overflow.");
let cval = now
.checked_add(ticks)
.expect("Physical Timer CVAL Overflow.");
Self::set_cval(cval);
}
The code here should be understandable. All we do is calculate seconds*frequency to calculate the change in counter value till expected time. And then we add that value to current counter value and set it to compare value.
We can do a same one for lesser amount of time like milliseconds
pub fn set_milliseconds(milliseconds: u64) {
let freq = Self::read_frq();
let now = Self::read_cnt();
let ticks = freq
.checked_mul(milliseconds)
.expect("Physical Timer Milliseconds Overflow.")
/ 1000;
let cval = now
.checked_add(ticks)
.expect("Physical Timer CVAL Overflow.");
Self::set_cval(cval);
dprintln!("[TIMER] p cval was set.. {} ms", milliseconds);
}
And now finally a function to start the timer after the compare value has been set through some function:
#[inline(always)]
pub fn start() {
Self::unmask_int();
Self::enable();
}
And with that our Physical Timer is now usable in our project!
You can now try it out in your rust main function:
use kernel::timer::PhysicalTimer
// testing timer.
PhysicalTimer::set_seconds(3);
PhysicalTimer::start();
loop {
if (PhysicalTimer::read_ctl() & 0b100) == 0b100 {
println!("Timer went off!").unwrap();
println!("The end. Make sure to power off your RPi before disconnecting it :)").unwrap();
loop { // if we don't enter loop here, it will keep printing that timer went off
delay(1_000_000);
}
}
}
Notice how in this test we check if the timer expired by checking the bit[2] of the control register value. Recall that this specific bit depicts if an hardware event IRQ is being fired to the interrupt register or not.
We check it this way for now because our timer may send hardware event IRQ signal to the interrupt controller, but we have not yet configured the interrupt controller to actually route it back to the CPU as an IRQ Exception. Thus our CPU currently will not receive any sort of exception or interrupt even if timer works correctly. Checking if the CTL value outputs after the correct duration is the only way to test for now.
However, that will change as now we will talk about configuring the interrupt controller. So our interrupt pipeline may progress further with the timer IRQ.
Interrupt Controller
So far in this chapter we’ve mentioned the interrupt controller many times, so you must already know that all it does is serve as the observer of hardware. All hardware event signals are first sent to it. Whereby the interrupt controller then decides if the hardware signal should cause an IRQ Exception to the CPU or not.
First you need to know is that the interrupt controller is just another component within the SoC like many others. Similar to how the mini UART is a component in the SoC. And so our way to communicate with the interrupt controller is going to be same as the way with which we communicate to the mini UART component. Which is through MMIO registers described in appropriate documentation.
This section we will talk about how to configure the interrupt controller on our hardware, such that upon receiving the hardware interrupt signal from the timer, it will forward an IRQ interrupt signal to the CPU.
The Quad-Core A7 Control Block
The Interrupt controller component on the RPi3B+ is often called the QA7. Because the official name of this component is the “Quad Core A7 Control”. The reason for the name is not relevant. Throughout this chapter and beyond we are going to call this interrupt controller as “QA7”.
Just like you’ve been already explained, the QA7 essentially will do the job of sending an Interrupt Request (IRQ) Signal to the CPU upon the timer-went-off hardware event. The QA7 has many other features as well. It has the capability to enable sending of data between different CPU cores, it even has it’s own timer component, which works separately from the four timers we’ve studied about so far. But all those features are irrelevant to us right now. Currently all we are trying to do is configure it so that it sends an IRQ request to the CPU upon the NSP timer going off.
You can find the official documentation for this component at https://github.com/Tekki/rasp(…)berrypi/bcm2836/QA7_rev3.4.pdf
You will see that it details all of its capabilities, and also how to configure them. However we’re only really interested in the section about Interrupts Router. It is the part of QA7 which manages hardware interrupts to IRQ/FIQ request pipeline. It is described in the afformentioned document at the 3.2 section “Interrupt Routing”.
(screenshot taken from aforementioned document)
First of all you will immediately notice this diagram when you scroll to the mentioned section. This depicts the hardware event to interrupt pipeline that we’ve been discussing. In the center is the QA7’s interrupt routing block/component. The arrows you see going towards it, represent different kind of hardware event signals that can be received by the interrupt router. You can see from the top left, firstly we have IRQ or FIQ hardware events that might come from the GPU. Then 16 “mailbox” type events which can come from other CPU cores trying to send some data to our current core. And then two more kind of events labelled “USB Timer” and “Spare” which could represent other events occurring across hardware.
Then on the right the block represents the CPU cores of the system. Raspberry Pi 3B+ has four ARM Cortex A53 CPU cores. So that’s what this block on the right represents. From the bottom right you can see four arrows going to the interrupt router. These four arrows represent the hardware events of timers going off for each of the four timers that we’ve learned so far. You can see each arrow as a “4” labelled on it. This is because each of the four CPU cores has their own four timers. So total there can be theoretically 4*4 => 16 different unique timers going off on the system, and QA7 interrupt router can receive a hardware event for all of these. There’s then also two last hardware events going labelled nPMUIRQ* and nAXIERRIRQ. We are not going to cover these.
Lastly, there’s only four arrows going out of the interrupt router block. These represent the four interrupt requests that the QA7 can send to the ARM CPUs. From top right we have IRQ and FIQ interrupt requests. If you don’t know yet, an FIQ is basically an IRQ which deserves a super high priority for handling. Then there’s two other arrows labelled VIRQ and VFIQ. These are related to virualization in EL2 level, which we are not going to discuss within our project scope.
You can read about all of this in the section 3.2 of the document.
Timer Interrupt Control Register
Now, let’s take a look at the MMIO registers which are useful in configuring the QA7 interrupt router. A comprehensive list of them is shown at the start of section 4. Of course we don’t have to learn about all of these. Only the registers relevant to our goal of setting up the NSP Timer Hardware Event -> IRQ Request to Core 0 pipeline.
For that purpose, the only register you have to take note of is the one mentioned at section 4.6, “Core timers interrupts”.
This section describes four registers:
- Address: 0x4000_0040 Core 0 Timers interrupt control
- Address: 0x4000_0044 Core 1 Timers interrupt control
- Address: 0x4000_0048 Core 2 Timers interrupt control
- Address: 0x4000_004C Core 3 Timers interrupt control
Each four of these is basically the same register. But each one is for a different core. We are only working with a single core, core 0 in our project. So that is what we’re going to look at.
This is what the description says:
There are four core timer control and four core timer status registers. The registers allow you to enable or disable an IRQ or FIQ interrupt. They cannot clear an pending interrupts.
This register is named the “Timers Interrupt Control” register.
If you read the first table under this heading, you will see that it decscribes the fields of this register. The first 8 bits are the only ones which have a function. bits[3:0] are used for timer to IRQ configuration. Each of these bits is one of the four timers. from bit 0, these bits are for the physical secure timer, physical non secure timer, hypervisor timer, and virtual timer. Writing a ‘1’ to any of these fields makes it so when corresponding timer goes off, the interrupt router will send an IRQ request to the CPU.
This is it! This is the setting we were looking for. In order to configure QA7 to route non-secure physical timer’s going-off hardware event to cause an IRQ to the core 0 CPU, this is the register we need to write to! We need to write the value of “1” to the bit[1] of this register.
Note that the bits[7:4] in this register serve the exact same purpose, except for FIQ instead of IRQ. They also supercede and override the corresponding IRQ bits. So if bit[5] is set to 1, then regardless of bit[1]’s state, an NSP timer going off will always cause an FIQ exception request to the CPU.
So finally, if we want our CPU to receive an IRQ request upon the going off of the non-secure physical timer, we simply need to make sure bit[1] in this register is set to ‘1’ and bit[5] is set to ‘0’.
Abstraction implementation
Now lets add a module in our Rust project which will have the job of configuring the hardware for interrupt handling. Let this module be called “interrupts.rs”, in src/kernel/. Let’s start it off with:
#![allow(unused)]
fn main() {
use core::ptr::{read_volatile, write_volatile}; // we're handling MMIO registers
/*
~~~~ THE DOCUMENTATION FOR THE QA7 COMPONENT CAN BE FOUND AT
https://github.com/Tekki/raspberrypi-documentation/blob/b1df6ea8e135254e5feb0c8bb036b2a18db8b859/hardware/raspberrypi/bcm2836/QA7_rev3.4.pdf
*/
// QA7 Base for BCM2837 (Raspberry Pi 3)
const QA7_BASE: usize = 0x4000_0000;
// Core timers interrupts
pub const CORE_0_CNT_INT_CTL: *mut u32 = (QA7_BASE + 0x40) as *mut u32;
}Remember to add this to our kernel/mod.rs!
#![allow(unused)]
fn main() {
pub mod interrupts;
}Now, we’re going to try an implement a function route_timer_interrupt. Which will take arguments for what timer’s hardware event to route to an IRQ or FIQ.
#![allow(unused)]
fn main() {
#[repr(u8)]
#[derive(Copy, Clone, Debug)]
pub enum TimerInterruptSource {
Physical = 1 << 0,
PhysicalNonSecure = 1 << 1,
Hypervisor = 1 << 2,
Virtual = 1 << 3,
}
#[repr(u8)]
#[derive(Copy, Clone, Debug)]
pub enum InterruptRoute {
IRQ,
FIQ,
}
pub struct Interrupts;
impl Interrupts {
pub fn route_timer_interrupt(source: TimerInterruptSource, route: InterruptRoute) {
let mut enable_bit: u32 = source as u32;
enable_bit = match route {
InterruptRoute::IRQ => enable_bit,
InterruptRoute::FIQ => enable_bit << 4,
};
unsafe {
let current_val: u32 = read_volatile(CORE_0_CNT_INT_CTL);
write_volatile(CORE_0_CNT_INT_CTL, current_val | enable_bit);
}
}
}
}This code looks verbose, but it is actually surprisingly simple. Firstly, in the register, bit[0] corresponds to the Secure Physical Timer, bit[1] to NSP, bit[2] to HV, bit[3] to Virtual timer. We already know that. So we’ve created an enum TimerInterruptSource which encodes this information correctly. Casting it to unsigned integer will give us a number where only the corresponding register’s field’s location’s bit is set to 1.
We also create an enum for InterruptRoute which signifies the two types of target routes an interrupt request can be sent. IRQ request or FIQ request. Our method then appropriately sets the corresponding bit in the MMIO register to 1. It’s working should be self explanatory.
Then we should also create a function to be able to clear the bit to break this routing.
#![allow(unused)]
fn main() {
pub fn deroute_timer_interrupt(source: TimerInterruptSource) {
let mut disable_bit: u32 = source as u32;
// disabling the FIQ as well
disable_bit = disable_bit | (disable_bit << 4);
unsafe {
let current_val: u32 = read_volatile(CORE_0_CNT_INT_CTL);
write_volatile(CORE_0_CNT_INT_CTL, current_val & !disable_bit);
}
}
}Now, for safety let’s also use this method to deroute the timer interrupt before setting it in the router method:
pub fn route_timer_interrupt(source: TimerInterruptSource, route: InterruptRoute) {
// first clearing the interrupt routes for the given source
Self::deroute_timer_interrupt(source);
/* (...rest is same as before) */
}
Now we have a safe and convenient abstraction to configure the QA7 for timer interrupts! Let’s review what we have got setup so far:
- We have the ARM Non-Secure Physical Timer on Core 0 abstracted. So we can configure it to fire off after some amount of time we like.
- We have the QA7 interrupt controller abstracted. We can configure it so when a timer goes off, QA7 will send an IRQ request to the CPU.
And now that we have those two things set up we have finally reached the final phase of our Interrupt Pipeline. The CPU accepting the request and it causing an IRQ type exception in the exception handler. Which the exception handler should be able to identify as hanving come from a timer going off.
Turning an IRQ Signal into IRQ Exception
Now, when a CPU receives an IRQ signal from the hardware event, by default it will cause an exception of type IRQ to occur. But sometimes the CPU might be configured to ignore these IRQ interrupt request signals. This is configured by the DAIF register.
First you should know what it is. Sometimes in CPU, you might want to purposefully ignore a certain kind of exception from occurring. This could be because you’re handling something really important so you don’t want anything to interrupt it. It might also be because you’ve already acknowledged some interrupt so you don’t want it to keep causing exceptions in the meantime. Whatever may be the case, the ARM CPU architecture provides a way just for that. It is called interrupt masking.
This is done through the DAIF register. The name of this register is short form for, Debug, Asynchronous abort, IRQ, FIQ. Those are the different kind of interrupt requests that the CPU can receive. You may look up more information about this on their own. The register only has four bits [9:6] with a function, the rest are reserved. Each of these four bits corresponds to one of the four types of interrupt requests. When a corresponding field is set to 1, said interrupt is said to be masked. In that case, the CPU will ignore all interrupt requests of that corresponding type. So if you set the bit for IRQ to 1, the CPU will ignore all incoming IRQ interrupt requests, and they will not cause an exception to the exception handler.
You can read about this register here.
In the above mentioned documentation you will also find that there are registers which exist just to make it easier to flip singular bits inside the DAIF register. These registers are DAIFSet and DAIFClr. In these registers the bits[3:0] corresponds to D,A,I,F respectively. In DAIFSet if you write some value, then in that case, whichever of the D,A,I,F bits was 1, is set to value 1 in the DAIF register, and whichever was 0 is ignored. Then in DAIFClr, it is the same, except writing a 1 to this register will set the corresponding interrupt type’s field to zero value in the DAIF register.
So to mask the IRQ interrupt requests, we can write 0b0010 to DAIFSet. And to unmask them, we can write same value to DAIFClr this time.
Now, the way to configure the CPU to accept our IRQ requests and let them trigger exception handler is simple. You just need to ensure that the IRQ bit in DAIF is set to zero. This can be done by writing 0b0010 to DAIFClr.
Let’s implement a basic method for doing this in our Interrupts abstraction.
Implemented in struct Interrupts
#![allow(unused)]
fn main() {
pub fn irq_disable() {
unsafe {
// Mask IRQ interrupts (Bit 1)
core::arch::asm!("msr DAIFSet, 0b0010",
options(nostack, preserves_flags));
}
}
pub fn irq_enable() {
unsafe {
// Unmask IRQ interrupts (Bit 1)
core::arch::asm!("msr DAIFClr, 0b0010",
options(nostack, preserves_flags));
}
}
}Also a function if we ever want to check the current state of it:
#![allow(unused)]
fn main() {
// Reads to check if IRQs are masked. Returns true if interrupts are
// currently enabled.
pub fn is_irq_enabled() -> bool {
let daif: u64;
unsafe {
core::arch::asm!("mrs {}, DAIF",
out(reg) daif,
options(nostack, preserves_flags));
}
// Bit 7 is A, Bit 8 is D, Bit 1 is I (IRQ Mask), Bit 0 is F
// If bit is 1, it means masked otherwise unmasked.
((daif >> 7) & 1) == 0
}
}NOTE
When an exception occurs, then right before sending control to the exception handler, the CPU automatically sets all DAIF fields to values of 1. Basically masking all possible interrupts that can be masked. This is a ARM Architecture Design feature which makes sure that while an exception is being handled, another interrupt is unlikely to interfere with the exception handling. The original values of DAIF are restored when exception handler returns through ERET. For restoration reference these values are stored in exception’s PSTATE data which is stored in SPSR_EL1 register.
I myself also implemented the following two wrapper methods to my struct Interrupts. Useful for masking all interrupts or unmasking all interrupts.
#![allow(unused)]
fn main() {
pub fn daif_unmask_all() {
unsafe {
core::arch::asm!("msr DAIFClr, 0b1111", options(nostack, preserves_flags));
}
}
pub fn daif_mask_all() {
unsafe {
core::arch::asm!("msr DAIFSet, 0b1111", options(nostack, preserves_flags));
}
}
}You can really use your own creativity and design decisions to decide for yourself– what all wrappers you want to be available in your abstractions.
Finally, for extra extra readability, I also added the IRQ routing method to the physical timer struct itself.
#![allow(unused)]
fn main() {
impl PhysicalTimer {
pub fn init_irq() {
Interrupts::route_timer_interrupt(TimerInterruptSource::PhysicalNonSecure, InterruptRoute::IRQ);
}
}
}And with that we are officially done! We successfully made sure that every part of the interrupt pipeline is implemented and configurable!
- We have a
kernel::timermodule which lets us configure a hardware timer to go off after a fixed amount of duration. - We have configured it to cause an IRQ hardware event when the timer expires/goes off.
- We then have a QA7 Interrupt Controller abstraction in
kernel::interruptswhich lets us configure the QA7. Such that:- The QA7 recognizes said timemr’s expiration hardware event.
- Upon getting it, immediately sends an IRQ signal to the CPU core 0 (the core our kernel is currently running on).
- Then we have implemented basic methods so we are able to configure the CPU such that:
- When an IRQ request signal is received from the QA7, it causes an Exception to occur of the type IRQ.
Testing
You can test out this entire pipeline by the following procedure:
#![allow(unused)]
fn main() {
PhysicalTimer::init_irq();
Interrupts::irq_enable();
PhysicalTimer::set_seconds(2);
PhysicalTimer::enable();
}Then you’ll see that after 2 seconds, you will get an exception! And the exception context being printed will show you that it is a IRQ type exception. Just according to plan! This exception is caused because of the timer going off after 2 seconds! You will likely keep getting IRQ exceptions repeatedly at a super high rate. This is just because of how we discussed earlier in this chapter, that the timer keeps emitting expiration hardware event to QA7 every count that the counter’s expiration condition is true. Wait long enough and the exceptions will stop the moment that the timer’s counter register overflows and wraps around to zero.
Normally, you are supposed to turn the timer OFF upon handling its IRQ request. Since IRQ interrupts can be caused by pretty much every single hardware event there can be, there is a dedicated procedure to query QA7 to ask which hardware event caused said exception. From there you can identify if the IRQ was due to a timer. If it was you can then turn off that timer as acknowledgement of it, so it doesn’t continue ‘buzzing’.
However this chapter has already grown very heavy and long. Thus I will conclude this chapter here and cover said concept in the next chapter, with Process Scheduling.
Final codes
You can find Timer implementation in the same repository snapshot that was linked in the end of previous chapter:
github.com/ZackyGameDev/AtOS/blob/ae(…)rnel/processes.rs
Please note, said snapshot also includes implementation of the scheduler. So ignore it for now. it also includes methods in struct Interrupts that help in identifying the IRQ source. You may ignore it as well.