Kernel developers discuss Linux and Rust’s future in a roundtable.

Prominent Linux kernel developers gathered to share insights on kernel development’s current state and future at a recent roundtable discussion during the Open Source Summit Europe in Vienna.

The panel, moderated by Jonathan Corbet, Linux kernel developer and co-founder of Linux Weekly News (LWN), featured developers Alice Ryhl, a Google software engineer on the Android Rust team, Josef Bacik, tech lead for Meta‘s kernel file system team, Anna-Maria Behnsen, a Linux kernel developer at Linutronix who’s worked on real-time Linux, and Dan Williams, a member of Intel’s Linux core kernel architecture team.

Also: Linus Torvalds muses about maintainer gray hairs and the next ‘King of Linux’

The panel, in short, included people from several companies. In the Linux and open-source world, people work together across company borders all the time. These folks are also gainfully employed to work on Linux. Some say you can’t make a living working on open-source software and Linux. That’s not true. There’s real money in Linux and open-source jobs.

Much of the discussion centred on integrating Rust into the Linux kernel. At the Linux Kernel Maintainer Summit the day before, Corbet said: “There’s been some frustration in some quarters about just how long it was taking. I think some people wanted Rust to take over the world more quickly than it has.”

At the Maintainer Summit, Linus Torvalds, who likes having Rust in Linux, observed that some kernel features, such as modversions, are currently incompatible with Rust but are being worked on.

Ryhl, who works with Rust, expressed optimism about the progress despite acknowledging that it will take time for the integration to take place: “There are some things that I wish were already in and some things could have gone in faster, but things will always get better.”

Also: The Linux file system structure explained

One problem Rust in the Linux kernel faces, said Willams, is that “kernel maintainers tend to be very conservative.” They know C backward and forward, but they don’t know Rust. So, they “don’t know how to review this or debug that because they don’t understand the code.”

According to Linux and Rust developer Miguel Ojeda, who spoke at the Linux Plumbers Rust in Linux panel, the answer is for maintainers to work with Rust for Linux developers. The maintainers bring their understanding of the subsystem, while the Rust programmers bring their language expertise. By working together, the two groups can help bring Rust code into Linux.

Rhyl agreed: “A successful approach for me is to combine a kernel developer and a Rust expert. I had a call with the maintainer, and we went through the patch together. We could learn from each other.”

That was an example of what can happen when things go well. Sometimes, however, they don’t.

Wedson Almeida, a Microsoft software engineer and one of the Rust for Linux maintainers, recently quit the project in a note to the Linux Kernel Mailing List (LKML). He left because, “After almost four years, I find myself lacking the energy and enthusiasm I once had to respond to some of the nontechnical nonsense, so it’s best to leave it up to those who still have it in them.” Ojeda said that despite this, “The core Rust team is still growing.”

Also: The most popular programming languages in 2024

However, the panel agreed that a positive side effect of people working together is that it’s led to changes in existing C APIs and documentation practices. By making people think about how Rust can help Linux, they’ve also had to dig deep into their own code and explain it.

For example, Al Viro, the Linux Virtual Filesystem (VFS) maintainer, said Josef Bacik has reworked 65 pages of documentation. This, in turn, has made the associated C code much simpler for future developers to convert to. The documentation now gives clear rules on what the code does and why.

Yet the panel also pointed out a critical issue: despite thousands of paid kernel developers, no one is funded specifically for documentation work. This is an all-too-well-known but often ignored problem with Linux and open-source documentation.

Also: These Linux distributions are best for developers – here’s why

Corbet also observed that some of the tension around documentation comes from Rust and C’s fundamentally different ways of dealing with things:

If you look at how a C API in the kernel designs calls it this way, you have to make sure to call it this way, and that pointer better not be null, and don’t touch that button over there, or the whole thing will explode. The Rust approach is that we will design the API, so your code will probably be correct if the compiler lets you call it. C just doesn’t let you do that. But when we’re trying to design Rust APIs for the kernel, they have to sit on top of these C APIs. And so there’s a fundamental mismatch in the approach being taken, and I see that it’s pushing back into the C APIs and trying to force changes there. I think the changes are good, but I think it creates some resistance from the maintainers of the existing APIs, who put decades into creating and using those APIs, know where the pitfalls are and can instinctively avoid them.

Still, while not everyone is thrilled with Rust in Linux, progress is being made. At the Maintainers Summit, Torvalds said maintainers don’t need to understand Rust to let it into a subsystem. After all, he pointed out, half ironically, “Nobody understands the memory-management subsystem, but everybody can work with it.”

Also: Linux and open-source documentation is a mess: Here’s the solution

Ojeda also remarked that some Linux distros, notably Debian and Ubuntu, are adopting Rust. In short, he concluded, “Rust is now in a good position.” Rust may not have gotten into Linux as fast as some people wanted, but it’s progressing well.