At The Good Penguin we like to keep up to date with the latest developments in our community, so that’s why we sent most of our team to the Embedded Linux Conference a couple of weeks ago. This year it was hosted as part of the wider Embedded Open Source Summit and located in Prague. Not only do these events give us the opportunity to stay up to date, they also allow us to collaborate and make connections with the open-source community. In this blog post we’ll share some of our highlights.
Status of Embedded Linux, Tim Bird, Sony Electronics
Every year Tim Bird provides an interesting and excellent snapshot of the status Embedded Linux. The talk covers major developments, noticeable trends and community/industry news and is by far the best way to keep up to date with everything embedded Linux and so we highly recommend you read the slides or watch the talk. There was a lot to fit in this year’s talk (75 slides) and too much to summarise here, however the slides mention some interesting developments such as wget support in U-Boot, snagboot, the ever increasing performance of io_uring, Rust in the kernel, PREEMPT_RT, nolibc and mailing list tools (b4). And finally, one of the trends Tim uncovered was ‘Linux in space’ – we too noticed this given the number of talks on the subject during the conference – our favourite was “VZLUSAT-2: CubeSat with a Linux Payload Computer” by Martin Sabol & Tomas Novotny (slides here).
Read the slides here.
Maintaining a Community BSP Layer: Updating Meta-Tegra Through Major Changes, Tim Orling, Konsulko Group & Ilies Chergui, Medtronic
We’ve recently worked on several projects that make use of NVIDIA’s impressive Jetson hardware – this hardware is supported by NVIDIA’s Ubuntu based JetPack SDK. Whilst a Ubuntu desktop based environment is helpful for prototyping and evaluating, we find that it’s not always suitable in deployment – as such our customers ask for a Yocto based distribution. Thankfully the meta-tegra project provides this, however as it’s a project not supported or funded by NVIDIA, the community are often playing catch-up with the features of the JetPack. For example recent changes to the JetPack have moved from Trusty to OP-TEE and from cboot to UEFI – such changes have knock on effects for features like secure boot and software update which can impact our customers. Therefore we were keen to attend this talk and speak to those working on the project.
We learnt that NVIDIA are rapidly advancing their JetPack software with a view to the 6.x release being targeted for Q3 ’23, and only security updates for the 5.x series going forward from that point. This could affect customers who are only now transitioning to Orin based systems due to availability issues with the lower end Orin NX devices. The community is also attempting to overcome some of the shortfalls of the Ubuntu based system with regard to update mechanisms and security when running a yocto derived distribution.
Read the slides here.
Evaluation of PREEMPT_RT in Virtualized Environments, Jan Altenberg, Open Source Automation Development Lab (OSADL)
rtla timerlat: Debugging Real-time Linux Scheduling Latency, Daniel Bristot de Oliveira, Red Hat
We’ve had some recent customer interest regarding optimisation of real-time performance on Linux so these talks by Jan and Daniel were of interest to us.
The talk by Jan focused on how the host or guest of a virtualized or containerised system may effect the real-time performance of both host and guest. It considered docker containers, KVM and Jailhouse. This talk was so interesting because it raised so many questions such as what is Jailhouse, why would anyone want real-time in a virtualized environment, how does virtualization work and what limitations are imposed by the hardware and software architecture. The talk introduced useful utiltiies such as blocksys, demonstrated ways of measuring latency, but crucially it also provided some practical guidance on optimising real time performance. And finally one of the take homes here was that guests can have real-time capabilities, and it’s really not a stupid thing to do.
The talk by Daniel provided a good overview of the Real-time Linux Analysis Toolset (RTLA), a userspace tool that can measure real-time latency and automatically analyse latencies by leveraging the kernel tracing framework. The tool also provides a way to measure and understand ‘osnoise‘.
Read the slides here.
The Yocto Project – Where We’re Going and What’s Next, Philip Balister, The Yocto Project
BoF: The Yocto Project and OpenEmbedded Organization, Philip Balister, OpenEmbedded & Josef Holzmayr, Mender.io
Given Yocto is the de-facto embedded Linux distribution, it’s no surprise that we were interested in attending talks from Philip and Josef on the state of the Yocto project.
Despite it’s widespread adoption and importance, the Yocto project remains a surprisingly under-resourced project as you can read in recent news articles. It’s no surprise then that the talk by Philip included a call to action to join the Yocto project and dedicate resources to tackle some of the bit-rotten features and backlog of bugs. Some of the areas they wish to focus on is patchtest (automated testing of patches), toaster (web UI for bitbake), core workflow (public sstate cache), project tooling (improve recipetool, devtool, fix bugs, support Go, Rust, etc), meta-openembedded (improved support), security (processes and tools), VSCode IDE integration and binary distro (analogous to apt-get). Fortunately, some funding has been found and as a result The Yocto Project have now issued an RFQ to find resource to do this work.
Other good news from the Yocto project was the announcement that support would be extended to 4 years by default for kirkstone and subsequent LTS releases going forward.
Read the slides here and here.
Fetching, Configuring and Building Your Bitbake Project with Just One Command, Jan Kiszka, Siemens AG
This was a talk about Kas, not the town in Turkey or the cheese, but the setup tool designed to build bitbake based projects. We’ve already started using this as an alternative to repo or git submodules, but the talk gave a good overview on all of the features offered by Kas and what’s to come.
Read the slides here.
Designing to the Worst Case Scenario – Practical System Call Filtering with Seccomp, Simon Goda, Doulos Ltd
This was an interesting talk that introduced the kernel’s seccomp feature (CONFIG_SECCOMP) and userspace library – libseccomp. The talk demonstrated how it can be used to start a process and limit the system calls that the process can make. Thus providing a useful tool in the pocket of anyone wishing to security harden a platform. Seccomp is already used by Docker and LXC, but it is also used by systemd, making it possible to easily limit what a systemd service can do simply by adding ‘SystemCallFilter‘ in the service file.
Read the slides here.
Testing and Remote Access to Embedded System DPI/LVDS Display Output, Marek Vasut
One of the trends we’ve observed recently is automated testing – there were many talks at the conference describing how companies are adding hardware testing to their CI systems. This talk described the journey of Marek in finding a way to automate the testing of digital displays by replacing those displays with customised test hardware. We were particularly interested in the ability to capture and analyse the entire video signal, including all the sync signals and non-displayed data. Most video capture systems only record the visible video.
Read the slides here.
Stateless V4L2 Video Encoding, Andrzej Pietrasiewicz, Collabora
And finally, we found this talk by Andrzej on stateless video encoding very interesting – it highlighted the differences between stateful and stateless encoding and also made as aware of the V4L2 interfaces available for video encoding.
Read the slides here.
The full schedule for the conference (along with the slides) can be found here. Videos are available on YouTube. We’re already looking forward to the next Embedded Linux Conference which will be April 15-19 in Seattle next year.