Thanks to BeagleBoard.org, we’ve recently received a pre-production beta version of the BeagleV StarLight development board. It’s an affordable Linux platform that’s truly open source with it’s software, hardware design and RISC-V instruction set architecture all made available under free and open licenses.
When we booted the board for the first time we noticed some output on the console coming from something called ‘OpenSBI‘ – we were curious to find out what this is and why it’s needed. Read on for find out what we learnt.
We quickly learnt that OpenSBI is an open-source reference implementation of the RISC-V Supervisor Binary Interface (SBI) specification. In brief – it’s firmware running at a higher privilege level that initialises hardware and allows lower privilege levels, such as an operating system, to call into it to perform actions such as power-cycling the device or managing CPU cores. This draws parallels to something like Arm Trusted Firmware (ATF) in the ARM ecosystem – or even the BIOS/UEFI on PCs. Now that we have a high level view, let’s dig into this a little deeper…
The whole point of having an SBI is succinctly described in the specification: “The SBI allows supervisor-mode (S-mode or VS-mode) software to be portable across all RISC-V implementations by defining an abstraction for platform (or hypervisor) specific functionality.” – in other words it’s understood here that hardware platforms are all different and inevitably need platform specific software to make them work – yet it’s also recognised that there are great benefits to abstracting this away from higher level components such as operating systems – allowing them to be more portable and easier to write. Thus an abstraction layer is born. We touched upon similar issues in the ARM world with PSCI in a previous post.
As a side note – In the original description above the specification referred to “supervisor-mode (S-mode or VS-mode) software being portable”… You may be familiar with ARMv8-A privilege levels where userspace typically runs at ‘EL0’, the kernel at ‘EL1’ or ‘EL2’ and the firmware at the highest level ‘EL3’. In the RISC-V world this is comparable, with userspace at ‘U’, the kernel at ‘S’ (supervisor) and the firmware at ‘M’ (machine). Thus ‘S’ and ‘VS’ refer to the kernel (where ‘VS’ relates to virtual s-mode when virtualisation is in use).
Let’s take a look at our boot output from power on.
bootloader version:210209-4547a8d
ddr 0x00000000, 1M test
ddr 0x00100000, 2M test
DDR clk 2133M,Version: 210302-5aea32f
0 crc flash: 7ebedaa2, crc ddr: 7ebedaa2
crc check PASSED
bootloader.
OpenSBI v0.9
____ _____ ____ _____
/ __ \ / ____| _ \_ _|
| | | |_ __ ___ _ __ | (___ | |_) || |
| | | | '_ \ / _ \ '_ \ \___ \| _ < | |
| |__| | |_) | __/ | | |____) | |_) || |_
\____/| .__/ \___|_| |_|_____/|____/_____|
| |
|_|
Platform Name : StarFive VIC7100
Platform Features : timer,mfdeleg
Platform HART Count : 2
Firmware Base : 0x80000000
Firmware Size : 92 KB
Runtime SBI Version : 0.2
Domain0 Name : root
Domain0 Boot HART : 1
Domain0 HARTs : 0*,1*
Domain0 Region00 : 0x0000000080000000-0x000000008001ffff ()
Domain0 Region01 : 0x0000000000000000-0x0000007fffffffff (R,W,X)
Domain0 Next Address : 0x0000000080020000
Domain0 Next Arg1 : 0x0000000088000000
Domain0 Next Mode : S-mode
Domain0 SysReset : yes
Boot HART ID : 1
Boot HART Domain : root
Boot HART ISA : rv64imafdcsux
Boot HART Features : scounteren,mcounteren
Boot HART PMP Count : 16
Boot HART PMP Granularity : 4096
Boot HART PMP Address Bits: 36
Boot HART MHPM Count : 0
Boot HART MHPM Count : 0
Boot HART MIDELEG : 0x0000000000000222
Boot HART MEDELEG : 0x000000000000b109
U-Boot 2021.04-rc4-g5b63233bc6-dirty (Apr 08 2021 - 14:09:59 +0800)
CPU: rv64imafdc
DRAM: 8 GiB
MMC: sdio0@10000000: 0, sdio1@10010000: 1
Net: dwmac.10020000
Autoboot in 2 seconds
MMC CD is 0x1, force to True.
MMC CD is 0x1, force to True.
Card did not respond to voltage select! : -110
BeagleV #
There are actually four components here. The first two ‘secondBoot‘ and ‘ddrinit‘ are responsible for everything up to the OpenSBI output. These run at the ‘M’ privilege level and set up the DDR and basic hardware – perhaps some day these two components will be replaced with a single U-Boot’s SPL. Next we have openSBI, also running at ‘M’ privilege, which eventually launches U-Boot running at the lower ‘S’ privilege.
The OpenSBI software also acts as a Supervisor Execution Environment (SEE) as it’s the provider of SBI services to lower privilege levels. Though it should be noted, that an SEE could additionally be a hypervisor executing in ‘HS’ mode providing SBI services for it’s kernels running in ‘VS’ mode (Because if you’re a Hypervisor and you’re going to pretend to look like a machine then you need to act like one).
Software that wishes to make use of the functionality offered by OpenSBI interacts with it by calling functions. Under the hood these are implemented via the RISC-V ‘ecall‘ instruction which causes an exception to occur which moves execution into a higher privilege SEE environment where the call is processed before execution is eventually returned back to the caller. This is exactly how an ordinary system call works.
There are some interesting examples of where SBI is used in the Linux kernel so let’s take a look.
struct sbiret sbi_ecall(int ext, int fid, unsigned long arg0,
unsigned long arg1, unsigned long arg2,
unsigned long arg3, unsigned long arg4,
unsigned long arg5)
{
struct sbiret ret;
register uintptr_t a0 asm ("a0") = (uintptr_t)(arg0);
register uintptr_t a1 asm ("a1") = (uintptr_t)(arg1);
register uintptr_t a2 asm ("a2") = (uintptr_t)(arg2);
register uintptr_t a3 asm ("a3") = (uintptr_t)(arg3);
register uintptr_t a4 asm ("a4") = (uintptr_t)(arg4);
register uintptr_t a5 asm ("a5") = (uintptr_t)(arg5);
register uintptr_t a6 asm ("a6") = (uintptr_t)(fid);
register uintptr_t a7 asm ("a7") = (uintptr_t)(ext);
asm volatile ("ecall"
: "+r" (a0), "+r" (a1)
: "r" (a2), "r" (a3), "r" (a4), "r" (a5), "r" (a6), "r" (a7)
: "memory");
ret.error = a0;
ret.value = a1;
return ret;
}
EXPORT_SYMBOL(sbi_ecall);
Above you’ll see the implementation of sbi_ecall, this is the function in the Linux kernel that is used to invoke any of OpenSBI’s functions. On our board this will result in calling into OpenSBI running at the ‘M’ privilege level.
Below you’ll see that one of the SBI functions allows you to write characters to a console. You can see the benefit of abstraction here as the kernel doesn’t need to know about where the console is connected or how it has to drive it – it can simply trust the SEE to do the right thing.
/**
* sbi_console_putchar() - Writes given character to the console device.
* @ch: The data to be written to the console.
*
* Return: None
*/
void sbi_console_putchar(int ch)
{
sbi_ecall(SBI_EXT_0_1_CONSOLE_PUTCHAR, 0, ch, 0, 0, 0, 0, 0);
}
EXPORT_SYMBOL(sbi_console_putchar);You’ll also notice below that the kernel makes use of the console SBI service by wrapping it up in an early console driver:
static void sbi_putc(struct uart_port *port, int c)
{
sbi_console_putchar(c);
}
static void sbi_console_write(struct console *con,
const char *s, unsigned n)
{
struct earlycon_device *dev = con->data;
uart_console_write(&dev->port, s, n, sbi_putc);
}
static int __init early_sbi_setup(struct earlycon_device *device,
const char *opt)
{
device->con->write = sbi_console_write;
return 0;
}
EARLYCON_DECLARE(sbi, early_sbi_setup);
And finally, you’ll see below that there are SBI functions for stopping and starting CPUs, wrapped up inside the kernel’s cpu_operations structure. Due to the SBI abstraction, the implementation of these functions are very trivial.
static void sbi_cpu_stop(void)
{
int ret;
ret = sbi_hsm_hart_stop();
pr_crit("Unable to stop the cpu %u (%d)\n", smp_processor_id(), ret);
}
const struct cpu_operations cpu_ops_sbi = {
.name = "sbi",
.cpu_prepare = sbi_cpu_prepare,
.cpu_start = sbi_cpu_start,
#ifdef CONFIG_HOTPLUG_CPU
.cpu_disable = sbi_cpu_disable,
.cpu_stop = sbi_cpu_stop,
.cpu_is_stopped = sbi_cpu_is_stopped,
#endif
};
And if you’re wondering – the terminology ‘hart‘ refers to a CPU core. If you’re looking for more information, we’d recommend you read this presentation.
If you’re like to get updates about the BeagleV product launch later this year, feel free to join the public BeagleV forum.