Porting C++ to Rust by example with Yocto

Rust is an efficient system level language, in terms of performance it sits somewhere on par with C++ but lower than C. Similar to C++, it offers high level abstractions and types intended to help build system applications. Where Rust excels though is its focus on memory/thread safety and its data ownership model (enforced by its borrow checker at compile time). Alternative memory safe languages exist such as Go, however it has a garbage collector and is probably best suited for tooling and cloud services. Embedded Linux devices are our daily bread and butter here at The Good Penguin so performance of Rust is of interest. There are other languages to keep an eye out for too, for example Zig, but the memory safety there is required to be managed manually so it might be prone to the usual mistakes. Also, Zig’s release has also not reached v1.0 yet.

Rust is a fairly complex language with a lot of features to digest, we have (among others): mutable/immutable variables, structs, enums and patterns, traits, ownership, declarative and procedural macros, lifetimes, cargo crates and an in-built testing framework inside its compiler. There is a lot of information required to understand and hold in your head at any point in time to get the full picture of the Rust programming language and to use its features effectively.

Rust is large but one can just start writing code and use the best tool within the rich space of Rust features for the job at hand, as well as use a reference book opened next to your IDE to look up what other open-source projects did facing similar problems. Trying to internalise the whole of Rust in your head wont work well without practice, otherwise the concepts behind it wont be too memorable. At least this is what we have found, which brings us to the actual subject matter, porting a C++ application to Rust.

In 2023 we developed krill-kounter a C++ daemon for monitoring the health of flash devices, we even gave a talk about it and flash wear at the Embedded Linux Conference in Seattle in 2024. The daemon reads block stats to provide an accumulated number of bytes written to the device among other statistics and is targeted to run on deeply embedded devices (e.g. industrial in a factory). The daemon seemed like a good enough and simple candidate to see what will happen if we would port it to Rust.

The first step was to look at the existing C++ code base and catalogue all of the objects and data paths to have a model of the architecture and hierarchy – writing code in Rust is different than OOP C++ and so it is helpful to have a block diagram of how the existing application worked and then use that to write the Rust application.

Dependencies – crates

Thanks to the many crates that the Rust language provides, we were able to drastically simplify the boilerplate that needed to be written. Thus, using crates and doing it often is the first major change in the programming habit.

For example, data serialization/deserialization to JSON was handled by the serde crate – all we needed to do was to define how this process should be done in the exact same place where the struct was defined in the source code (which we liked):

#[derive(Deserialize, Serialize, Clone, Default, Debug)]
#[serde(rename_all = "camelCase")]
pub struct DeviceEntry {
    pub first_sithing_date: String,

    #[serde(rename = "path", skip_serializing)]
    pub previous_path: String,
    #[serde(rename = "path", skip_deserializing)]
    pub current_path: String,

    #[serde(rename = "stats")]
    pub stored_stats: BlockStats,

    pub total_bytes_written: u128,
    pub disk_seq: u64,

    // The below entries exist only in memory and are not stored.
    #[serde(skip)]
    pub previous_stats: BlockStats,
    #[serde(skip)]
    pub is_active: bool,
    #[serde(skip)]
    pub serial_number: String,
    #[serde(skip)]
    pub device_name: String,
    #[serde(skip)]
    pub stat_path: String,
}

As can be seen above we could handle camel-case-iation of the idiomatic Rust snake case variable naming, skipping items (on read and/or write) and renaming items in a compact and clear way – this was all done manually in our C++ code and it would be disadvantageous to not to use the Rust crate to get the boilerplate out of the way. However, with that approach we found that we have generated more dependencies in our Rust program than we would like and got a bit lazy which resulted in adding even more crates to solve the problems faced. That is the idiomatic way – to use the crates – but this approach could potentially lead to a supply chain attack – the type of npm is well known for.

This aspect needs extra care and consideration when writing Rust – one should check the crates being used and make sure they are in active development. It is useful to go and investigate the crate repositories and establish some sort of chain of trust. There are existing tools to aid you with the crate auditing and checking for existing vulnerabilities within your dependency chain like cargo-audit or the osv-scanner (driven by the RustSec Advisory Database). One can also use tools like cargo-crev which is a database of distributed peer code reviews of various crates – you can read more in-depth about the subject in a blog here. One can also use cargo-modules to visualise a crate’s internal structure or display the dependency tree with cargo tree, which we have found very useful.

Lastly, you should also read the documentation of the crates carefully to avoid miss-using them and also limit the amount of features that are pulled in into your application to reduce the bloat (and potentially an attack surface) – you can use cargo-bloat for this. We shall now give examples on how to use these audit tools below, it is usually a one-liner that could easily be a part of your CI job :

# cargo-crev
$ cargo install cargo-crev
$ cargo crev trust --level high https://github.com/dpc/crev-proofs
$ cargo crev repo fetch all
$ cargo crev verify --show-all

# cargo-audit
$ cargo install cargo-audit
$ cargo audit
Fetching advisory database from https://github.com/RustSec/advisory-db.git       Loaded 1160 security advisories (from /home/user/.cargo/advisory-db)
     Updating crates.io index
     Scanning Cargo.lock for vulnerabilities (98 crate dependencies)

# cargo-modules
cargo install cargo-modules
# display structure tree of the crate
cargo modules structure --lib
# display dependency graph (and store in SVG for viewing)
cargo modules dependencies --lib | dot -Tsvg > deps.svg

In summary, we had 12 explicit dependencies found in our Cargo.toml, but these crates themselves had their own dependencies so the actual number was up to 98. In comparison the Cmakelists for our C++ project had only 4 dependencies – we could reduce the number of dependencies in Rust by dropping all the crates of course and implement everything from scratch – but that is probably not the most efficient way to use the language.

Binaries and linking

The Rust crates are linked in statically by default (however dynamically linked crates are also supported). On the other hand, in C++ the dependencies are resolved at runtime by the dynamic linker. This translated directly into the release binary size: the C++ application was just 232 kB, but it needs other .so objects to exist on the disk that also have their size (but are reusable between different applications). The Rust application binary was 2.0MB so it is larger, but it had minimal runtime dependencies. This can be seen in the output from a ‘readelf -d’:

# Rust krill-kounter
0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

# C++ krill-kounter
0x0000000000000001 (NEEDED)             Shared library: [libglib-2.0.so.0]
0x0000000000000001 (NEEDED)             Shared library: [libkrillkounter.so.0]
0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

One thing we have also noted is that in the Rust binary we had a lot more relative relocations of type R_X86_64_RELATIVE (around 30 times more) as show by running ‘readelf -r’ and counting the relocation types (we note that we had to add ‘-fno-plt’ and eager binding to our C++ compile time options to match closer the rest of relocation types with the Rust app to reduce the diff). These are offsets (known as addends) into the binary itself relative to its actual base address – which is not known at compile time and so the address is resolved at runtime – but the relative offsets required are known by the compiler. This is due to how the Rust application itself was compiled – its crates are statically linked into a PIE binary and so the code has more pointers to local data and functions – and also these relocations are inferred by the features of the language itself that we have used in the code.

# Rust krill-kounter
    1482 R_X86_64_RELATIVE
     126 R_X86_64_GLOB_DAT
       2 Type
       2 R_X86_64_JUMP_SLO
       2 
       1 '.rela.plt'
       1 '.rela.dyn'

# C++ krill-kounter
     115 R_X86_64_GLOB_DAT
      45 R_X86_64_RELATIVE
      13 R_X86_64_64
       8 R_X86_64_COPY
       2 Type
       2 R_X86_64_JUMP_SLO
       2 
       1 '.rela.plt'
       1 '.rela.dyn'

Indeed if we look up to see to what these references resolve to in the output binary then we discover that nearly half of them are associated with trait object vtables – so these are effectively function pointers. The trait is an implementation of a generic interface for multiple object types and multiple objects can implement multiple traits, or otherwise one of the ways polymorphism is handled in the Rust language. Traits were inspired by Haskell’s typeclasses and the other way to do polymorphism in Rust are generics. The trait type can be known or unknown at compile time which results in either static or dynamic dispatch. When the dispatch is static the code can be inlined as we know the exact code to execute, when it is dynamic we need two pointers: data pointer to the object itself and a pointer to its vtable that contains function pointers for the object’s implementation of the trait (these two pointers wrapped in a fat pointer is a trait object). We use traits in our code mostly for printing logs and handling of errors which is aided by the anyhow crate. Some of these R_X86_64_RELATIVE relocations are used by the trait Drop – created for dropping objects when they go out of scope – as each object will have its own destruct routine. The other half of the relocations seems to indicate references to other data structures used in the code like strings for example.

Code structure

The next thing we noted was that the code structure was closer to something we would get from a C code base than from C++ i.e. we ended up with structs processed in modules as opposed to packing all of the functionality inside object method themselves. The main classes in C++ were: JSON parser/writer, statistics computer and statistics reader, on the other hand in Rust we have ended up with: block, device, config and daemon modules/structs – which is much more logical and clearly representing the actual data flow. What we have liked in particular is wrapping function return values in the Result type to aid error branch handling (using the already mentioned anyhow crate for convenience while we are at it) – which saves a lot of if/else type of code to validate values when writing C because the checks are simply replaced with a ‘?’ try operator in the code and the compiler does the rest.

So yes, Rust made us write better and cleaner code as we had to think about the flow of data a bit more (due to the ownership model) and the code actually felt more familiar to a C code base than C with classes, that is minus the boilerplate you would have to write and debug in C which is a bonus in our book. Yes – the syntax was awkward at the beginning but it makes sense after some actual legwork and actual typing.

Testing

The next aspect was testing which is provided by the rustc compiler – therefore we tried to write the application with testing in mind from day one. The workflow was to write a module, then a test rig for it and at the end write a test harness to simulate operation of the whole daemon. This has proven to be very useful and somewhat refreshing approach as it can give the developers more confidence into the code that they are writing – you shall know that a change made in a few months wont break something that was done a year ago – which is priceless. We do a lot of CI test workflows for whole Yocto distributions but doing that for a single application to a great level of detail is not that common. Thanks to this approach we have found bugs during development even before whole of the application was ready. However it is worth noticing that this can increase the development time.

Linting and docs

The last thing on the list we should talk about is clippy – the Rust linting tool that finds all 101 type of mistakes. Once we have finished all of our coding we did run clippy in pedantic mode on our source code – it found hundreds of issues that could have be done better and in more idiomatic way – in retrospective we should have used it as an integral part of the development workflow. That tool also forced us to document the source code accordingly, we just run it in the CI now. Another useful tool is rustfmt – replacing the .clang-format in the C realm and cargo-doc – which will generate documentation that one can put on the web.

So to summarize, in our case, apart form the memory safety, Rust comes with a lot of useful tools that can aid your workflow and speed up the development process and will force you to write cleaner, testable and documented code. The price to pay is that one has to be careful with the dependency trees as literally anything is available to link in to your crate via cargo without any duct taping work required, one will have to accept larger binaries and slower compilation times (on top of the steep learning curve).

Rust in Yocto: openembedded-core vs meta-rust-bin

With the application port and testing done, we then explored the ways of integrating the Rust application into a Yocto OE image, the current way of integrating Rust is to use openembedded-core layer or to use the meta-rust-bin layer, we shall evaluate working with both below. For the record, in the past we would have used meta-rust – but parts of it were merged into openembedded-core. You can also read more on that matter and history of Rust support in Yocto in the great article from our friends at Memfault.

openembedded-core

This layer builds whole of Rust tool-chain: cargo, rustc and std and other libraries from scratch, so it is optimized for the target hardware. The problem is that the rust version is tied to the Yocto version itself, so it might be out of date compared to upstream Rust. We also manage dependencies manually – you will know exactly what is being pulled in. Luckily we can just use bitbake directly to populate the Rust crate dependencies and store them in an include file that is then pulled in the actual bitbake recipe i.e. running:

bitbake -c update_crates krillkounter-rs

This will generate a krillkounter-rs-crates.inc file that shall append the SRC_URI with all of the dependencies in it, then we have to require that file in our bitbake recipe (for this to work we need to inherit the cargo-update-recipe-crates class that shall add do_update_crates task to your recipe’s task list), so our recipe looks like this:

inherit cargo cargo-update-recipe-crates systemd

DESCRIPTION = "Krillkounter-rs"
HOMEPAGE = "github.com/The-Good-Penguin/krill-kounter-rs"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=bcb3ac6803f69378f6a200eeddfa331a"

SRC_URI = "git://github.com/The-Good-Penguin/krill-kounter-rs.git;protocol=https;nobranch=1"

require krillkounter-rs-crates.inc

SRCREV = "be6aa88a1d6eca084eed6493fa3de17628b47559"
S = "${WORKDIR}/git"

SYSTEMD_SERVICE:${PN} = "krill-kounter-rs.service"

do_install:append () {
    install -d ${D}${systemd_unitdir}/system
    install -m 0644 ${S}/install/service/krill-kounter-rs.service ${D}${systemd_unitdir}/system
}

FILES:${PN} += "${systemd_unitdir}/system"

The above process is a bit manual, also the rust version found in openembedded-core at the scarthgap branch is at v1.75 while upstream is at v1.97.1. Since we have built the application with Rust edition 2024 – we had to explore updating the Rust version found in openebedded-core – which was done with the use of the layer meta-lts-mixins. This layer allowed to up the Rust version to v1.92 which was supporting the matching 2024 edition and with that in place our application did build. We also had to inherit the cargo class which will deal with boot strapping the cross compiler variables for the Rust toolchain and deal with matching the target hardware with a correct LLVM config and issue a cargo build. The crates are downloaded locally first into the ${WORKDIR}/cargo_home/bitbake during do_fetch and then this folder is used as the source of all crates during the build step. Interestingly since LLVM 18 the i128 variable is aligned to 16-byte boundary on x86 systems as seen in this PR. In practice it means that Rust < v1.77 might run into build errors with LLVM 18+ without some back-porting of patches, as we have discovered ourselves by experimenting: the ABI and data_layout were not matching and we hit a static assert in the rust_ast. In theory you should not run into this as rustc source does include a matching LLVM as a git module that is used by default – you can read more about this here.

meta-rust-bin

This layer is using pre-built binaries of the compiler and standard libraries from upstream of the Rust tool-chain – thus it might not be fully optimised for the target hardware that you will be using. The advantage though, is easy interoperability between Yocto versions. As we do not build any Rust binaries using Yocto backends, for the most part we are independent of the Yocto version (except the bitbake syntax of course). Another win are easy updates of the Rust version itself without resolving to external duct taping layers. The other convenience is that you do not have to manage the dependencies manually and check them out locally first as this will be just done by cargo. Therefore our Yocto recipe for integrating krillkounter to the build looks like this:

inherit cargo_bin systemd

DESCRIPTION = "Krillkounter-rs"
HOMEPAGE = "github.com/The-Good-Penguin/krill-kounter-rs"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=bcb3ac6803f69378f6a200eeddfa331a"

SRC_URI = "git://github.com/The-Good-Penguin/krill-kounter-rs.git;protocol=https;nobranch=1"
SRCREV = "f03ea717c104d1808bff771501999f01b9fde052"
S = "${WORKDIR}/git"

SYSTEMD_SERVICE:${PN} = "krill-kounter-rs.service"

# Enable network for the compile task allowing cargo to download dependencies
do_compile[network] = "1"

do_install:append () {
    install -d ${D}${systemd_unitdir}/system
    install -m 0644 ${S}/install/service/krill-kounter-rs.service ${D}${systemd_unitdir}/system
}

FILES:${PN} += "${systemd_unitdir}/system"

And that is all that there was too it, the recipe needs to inherit the cargo_bin class that will pull in the cross tool chain by appending your recipe’s DEPENDS variable, then create wrapper scripts required for expanding compiler/linker flags for cargo/rustc (similarly to openembedded-core layer) and finally just issue a cargo build command.

Conclusions

Using the memory safety that comes with Rust is not for free: one should take care about scrutinising the dependencies (which is really true for any language). However, it is easier to have latent dependencies in Rust as in theory any crate can be pulled in via cargo that could lead to a supply chain attack. The other aspect is the binary size due to the default static linking of the crates dependencies – lack of space is usually not a problem on embedded Linux machines nowadays and this might only be an issue for large code-bases. Though one could in theory then do more work to switch to dynamically linking some of the most commonly re-used crates. Lastly, the effort of writing Rust code has overheads due to the testing that usually goes with it – of course you could skip that bit, but perhaps that is bad practice. Regarding the coding experience itself, we have enjoyed writing Rust, hope to write more. We found the integration with Yocto at the time of writing to be very good with two main choices – with meta-rust-bin being the clear winner for a zero effort solution and rapid bring up – whereas we found the openembedded-core approach requires a bit more work but provides more explicit control over its dependencies. It’s also better optimised for the target hardware.

Please contact us if you would like us to assist you with your Linux Rust projects !