We are pleased to announce that the Real-time Microconference has been accepted into the 2021 Linux Plumbers Conference. Since 2004, the project that has become known as PREEMPT_RT, formally the real-time patch, has improved the real-time and low-latency features of the Linux kernel. Over the past decade, many parts of PREEMPT_RT have been included into the official Linux codebase. Examples include: mutexes, high-resolution timers, lockdep, ftrace, RT scheduling, SCHED_DEADLINE, RCU_PREEMPT, generic interrupts, priority inheritance futexes, threaded interrupt handlers, and more. The number of patches that need integration has been significantly reduced, and the rest is mature enough to make their way into mainline Linux.
The following accomplishments have been made as a result of last year’s microconference:
This year’s topics to be discussed include:
- New tools for PREEMPT_RT analysis.
- How do we teach the rest of the kernel developers how not to break PREEMPT_RT?
- Stable maintainers tools discussion & improvements.
- The usage of PREEMPT_RT on safety-critical systems: what do we need to do?
- Make NAPI and the kernel-rt working better together.
- Migrate disable and the problems that they cause on rt tasks.
- It is time to discuss the “BKL”-like style of our preempt/bh/irq_disable() synchronization functions.
- How do we close the documentation gap
- The status of the merge, and how can we resolve the last issues that block the merge.
- Invite the developers of the areas where patches are still under discussion to help to find an agreement.
- How can we improve the testing of the -rt, to follow the problems raised as Linus tree advances?
- What’s next?
Come and join us in the discussion of controlling what tasks get to runon your machine and when.
We hope to see you there.
We are pleased to announce that the Toolchains and Kernel Microconference has been accepted into the 2021 Linux Plumbers Conference. Toolchains are the main part of any development, as they create the executables from the code a developer writes. In order to run efficiently on the operating system, there needs to be a strong understanding of the interface between the application and the kernel it runs on. This microconference is focused on the integration of toolchains and the Linux kernel.
Since last year’s meet up, the following has been accomplished:
- Linux-toolchains mailing list and archive created.
- Rust-for-linux Github org established. Patches move from out of tree module building, to in tree module building.
- CI for kernel builds with LLVM moved to tuxbuild after an unexpected “no more free lunch” from TravisCI.
- LTO support landed in mainline.
- PGO patches sent upstream.
- At least one bugfix sent found via clang-tidy/clang-analyzer, discussions around driving tree wide cleanups via clang-tidy.
- GCC implemented support for asm goto with outputs
- Support for auto-initialized automatics in GCC is being worked out in GCC upstream. This is one of the security features that were deemed as desirable by the kernel last year. Work on the other missing desired security features is WIP.
This year’s topics to be discussed include:
Come and join us in the discussion of making the toolchains work better with the Linux kernel.
We hope to see you there.
We are pleased to announce that the Tracing Microconference has been accepted into the 2021 Linux Plumbers Conference. Tracing in the Linux kernel is constantly improving. Tracing was officially added to Linux in 2008. Since then, more tooling has been constantly added to help out with visibility. The work is still ongoing, with Perf, ftrace, Lttng, and eBPF. User space tooling is expanding and as the kernel gets more complex, so does the need for facilitating seeing what is going on under the hood.
Since the last tracing meetup at Linux Plumbers in 2019, a few accomplishments have come out of it:
- The final design of bootconfig came out, which enables kernel command lines be attached to the init ramdisk.
- Discussion on how to simplify the interface to ftrace histograms from user-space resulted in a SQL like utility (still being worked on, but almost finished). This came from the help of the database folks.
- After several rounds of trying to have perf share PMUs (beyond the hardware limit), another approach was taken to use a BPF based solution that does not need any kernel changes. Now perf can use BPF to aggregate counters.
- Work to natively incorporate ftrace into the babeltrace library has moved forward, although more still needs to be done for it to be completed.
This year’s topics to be discussed include:
- Tracepoints that allow faults. It may be necessary to read user space address, but currently because tracepoints disable preemption, it can not sleep, nor fault. And then there’s the possibilities of causing locking issues.
- Function parameter parsing. Now that on x86 function tracing has full access to the arguments of a function, it is possible to record them as they are being traced. But knowing how to read the parameters may be difficult, because it is necessary to know the prototype of the function to do so. Having some kind of mapping between functions and how to read their parameters would be useful. Using BTF is a likely candidate.
- Consolidating tracing of return of a function. Currently there’s three use cases that hook to the return of a function, and they all do it differently. kretprobes, function graph tracer, and eBPF.
- User space libraries. Now that libtraceevent, libtracefs, and libtracecmd have been released, what tooling can be built around them. Also, improving the libtraceevent API to be more intuitive.
- Improving the libtracefs API to handle kprobes and uprobes easier.
- Python interface. Working on getting the libraries a python interface to allow full tracing from within python scripts.
- Tracing containers. What would be useful to expose on creating and running containers.
Come and join us and not only learn but help direct the future progress of tracing inside the Linux kernel and beyond!
We hope to see you there!
We are pleased to announce that the IoThree’s Company Microconference has been accepted into the 2021 Linux Plumbers Conference. As everyday devices start to become more connected to the internet, the infrastructure around it constantly needs to be developed. Linux is showing up more in products that are not normally considered to be computers, but now need to interact with a central location (cloud). This brings new challenges that need to be addressed.
Last’s years meetup produced the following:
This year’s topics to be discussed include:
Come and join us in some heated but productive discussions in making your everyday devices communicate with the world around them.
We hope to see you there.
We are pleased to announce that the Performance and Scalability Microconference has been accepted into the 2021 Linux Plumbers Conference.
All parts of the Linux ecosystem, kernel and userspace, should account for performance and scalability. The purpose of this microconference is for developers from different projects to meet and collaborate, as the entire stack must perform well for the user to see good results. Because performance and scalability are very generic topics, this microconference focuses on issues that may also be addressed in other, more specific sessions.
The structure will be similar to what was followed in previous years, including topics such as synchronization primitives, bottlenecks in memory management, testing/validation, lockless algorithms and RCU, among others.
Here are some of the outcomes from the last time the event was held in 2018:
- With feedback from the audience, Daniel Jordan was able to get the first steps of his project, formerly known as ktask to parallize CPU-intensive work, merged in mainline as part of the padata parallel execution mechanism. Deferred struct page init is now parallelized on x86 systems.
- During Boqun Feng’s topic on workqueues and CPU hotplug, the audience concluded that the problem under discussion was actually not an issue and a stale comment had misled the community, so a follow-on patch removed part of the initial fix.
- In response to a question from the audience, Mike Kravetz proposed aligning addresses returned from mmap(MAP_ANONYMOUS) calls of at least THP size on THP boundaries, resulting in an RFC patch discussion.
This year’s topics tentatively include:
Come and join us in the discussion of improving performance and scalability of your system.
We hope to see you there.
We are pleased to announce that the Scheduler Microconference has been accepted into the 2021 Linux Plumbers Conference! The scheduler is an important functionality of the Linux kernel, deciding what process gets to run when, where and for how long. With different topologies and workloads, it is no easy task to give the user the best experience possible. Schedulers are one of the most discussed topics on the Linux Kernel Mailing List, but many of these topics need further discussion in a conference format. Indeed, the scheduler microconference is responsible for many topics to make progress.
At last year’s meet up, the Scheduler microconference achieved the following results:
Not only were enhancements made, but the meetup also helped prove that some topics were not feasible and we do not need to spend more time on them.
This year’s topics to be discussed include:
- Cgroup interface and other updates for core-scheduling
- Cgroup and SCHED_DEADLINE
- Capacity Awareness – For busy systems
- Interrupt Awareness
- Load Balancing:
Come and join us in the discussion of controlling what tasks get to run on your machine and when. We hope to see you there!
We are pleased to announce that the Confidential Computing Microconference has been accepted into the 2021 Linux Plumbers Conference! In this microconference we will discuss how Linux can support encryption technologies which protect data during processing on the CPU. Examples are AMD SEV, Intel TDX, IBM Secure Execution for s390x and ARM Secure Virtualization. These are recent additions compared to technologies which protect data while in transit (SSL, VPNs) and at rest (disk encryption).
The Linux kernel recently gained support for SEV-ES and support for Intel TDX is upcoming. AMD SEV will be further enhanced by Secure Nested Paging (SNP). Support for these technologies requires intrusive changes to the Linux kernel for memory integrity and secure interrupt delivery to virtual machines. Designing these changes in a way that works for different confidential computing technologies is one goal of this microconference.
Topics to be included, but not limited to, are:
Please come and join us in the discussion for solutions to the open problems for supporting these technologies.
We hope to see you there!
We took a look at all the events that were announced at the same time as OSS, including KVM Forum. The dates 20-24 September still seem to be clear of conference overlaps so we thought we’d grab them for Plumbers before someone else does. We also thought the timezone last year (Atlantic, 1h ahead of US Eastern and 5h behind central European) worked well, so we’ll plan to hold the conference mostly in that timezone (Although Microconference sessions can vary this if participants need. Our conference architecture will be available 24h)
We are pleased to announce that the Containers and Checkpoint/Restore Microconference has been accepted into the 2021 Linux Plumbers Conference! The Containers and Checkpoint/Restore micro-conference brings together kernel developers, runtime maintainers, and developers working on container- and sandboxing related technologies in general to discuss current problems and agree on new features.
Last year’s meetup resulted in:
This year’s edition of the Containers and Checkpoint/Restore micro-conference will focus on a variety of topics that are in need of discussion. The list of ideas is constantly evolving and we expect even more topics to pop up during the coming months as past experience has shown. Here is an excerpt:
- How to best use CAP_CHECKPOINT_RESTORE in CRIU to make it possible to run checkpoint/restore as non-root (with CAP_CHECKPOINT_RESTORE)
- Extending the idmapped mount feature to unprivileged containers, i.e. agreeing on a sane and safe delegation mechanism with clean semantics.
- Porting more filesystems to support idmapped mounts.
- Making it possible for unprivileged containers and unprivileged users in general to install fanotify subtree watches.
- Discussing and agreeing on a concept of delegated mounts, i.e. the ability for a privileged process to create a mount context that can be handed of to a lesser privileged process which it can interact with safely.
- Fixing outstanding problems in the seccomp notifier to handle syscall preemption cleanly. A patchset for this is already out but we need a more thorough understanding of the problem and its proposed solution.
- With more container engines and orchestrators supporting checkpoint/restore there has come up the idea to provide an optional interface with which applications can be notified that they are about to be checkpointed. Possible example is a JVM that could do cleanups which do not need to be part of a checkpoint.
- Discussing an extension of the seccomp API to make it possible to ideally attach a seccomp filter to a task, i.e. the inverse of the current model instead of caller-based seccomp sandboxing enabling full supervisor-based sandboxing.
- Integration of the new Landlock LSM into container runtimes.
- Although checkpoint/restore can handle cgroupv1 correctly the cgroupv2 support is very limited and there is a need to figure out what is still missing to have v2 supported just as good as v1.
- Isolated user namespaces (each with full 32bit uid/gid range) and easier way for users to create and manage them.
- Figure out what is missing on the checkpoint/restore level and maybe the container runtime level to support optimal checkpoint/restore integration on the orchestration level. Especially the pod concept of Kubernetes introduces new challenges which have not been part of checkpoint/restore before (containers sharing namespaces for example).
Come join us and participate in the discussion with what holds “The Cloud” together.
We hope to see you there!
You may have noticed that the Linux Foundation has announced moving OSS+ELC from Dublin to Seattle, WA due to survey results and vaccination rates in Europe. Since we agreed to co-locate with OSS+ELC this year, we’ve been debating following suit or going virtual. Unfortunately, the safety protocols imposed by event venues in the US require masks and social distancing, making it impossible to hold the interactive part of Plumbers (the Microconferences). Since Microconferences are a differentiating feature of plumbers, we felt that rather than lose such an essential element we’d move the entire conference on-line and hope to be back in-person next year.
As with last year, we’ll be using BigBlueButton for the main video interactions, but, following the example of FOSDEM, we’ll be using Matrix for the chat portion (and following feedback, we’ll be trying to integrate the matrix chat into the BBB chat window).
OSS+ELC in Seattle is now across our original dates, so we’ll try to find new ones to not clash with existing events, stay tuned for an update.
