We are pleased to announce that the Tracing Microconference has been accepted into the 2021 Linux Plumbers Conference. Tracing in the Linux kernel is constantly improving. Tracing was officially added to Linux in 2008. Since then, more tooling has been constantly added to help out with visibility. The work is still ongoing, with Perf, ftrace, Lttng, and eBPF. User space tooling is expanding and as the kernel gets more complex, so does the need for facilitating seeing what is going on under the hood.

Since the last tracing meetup at Linux Plumbers in 2019, a few accomplishments have come out of it:

• The final design of bootconfig came out, which enables kernel command lines be attached to the init ramdisk.
• Discussion on how to simplify the interface to ftrace histograms from user-space resulted in a SQL like utility (still being worked on, but almost finished). This came from the help of the database folks.
• After several rounds of trying to have perf share PMUs (beyond the hardware limit), another approach was taken to use a BPF based solution that does not need any kernel changes. Now perf can use BPF to aggregate counters.
• Work to natively incorporate ftrace into the babeltrace library has moved forward, although more still needs to be done for it to be completed.

This year’s topics to be discussed include:

• Tracepoints that allow faults. It may be necessary to read user space address, but currently because tracepoints disable preemption, it can not sleep, nor fault. And then there’s the possibilities of causing locking issues.
• Function parameter parsing. Now that on x86 function tracing has full access to the arguments of a function, it is possible to record them as they are being traced. But knowing how to read the parameters may be difficult, because it is necessary to know the prototype of the function to do so. Having some kind of mapping between functions and how to read their parameters would be useful. Using BTF is a likely candidate.
• Consolidating tracing of return of a function. Currently there’s three use cases that hook to the return of a function, and they all do it differently. kretprobes, function graph tracer, and eBPF.
• User space libraries. Now that libtraceevent, libtracefs, and libtracecmd have been released, what tooling can be built around them. Also, improving the libtraceevent API to be more intuitive.
• Improving the libtracefs API to handle kprobes and uprobes easier.
• Python interface. Working on getting the libraries a python interface to allow full tracing from within python scripts.
• Tracing containers. What would be useful to expose on creating and running containers.

Come and join us and not only learn but help direct the future progress of tracing inside the Linux kernel and beyond!

We hope to see you there!

We are pleased to announce that the Performance and Scalability Microconference has been accepted into the 2021 Linux Plumbers Conference.

All parts of the Linux ecosystem, kernel and userspace, should account for performance and scalability. The purpose of this microconference is for developers from different projects to meet and collaborate, as the entire stack must perform well for the user to see good results. Because performance and scalability are very generic topics, this microconference focuses on issues that may also be addressed in other, more specific sessions.

The structure will be similar to what was followed in previous years, including topics such as synchronization primitives, bottlenecks in memory management, testing/validation, lockless algorithms and RCU, among others.

Here are some of the outcomes from the last time the event was held in 2018:

• With feedback from the audience, Daniel Jordan was able to get the first steps of his project, formerly known as ktask to parallize CPU-intensive work, merged in mainline as part of the padata parallel execution mechanism. Deferred struct page init is now parallelized on x86 systems.
• During Boqun Feng’s topic on workqueues and CPU hotplug, the audience concluded that the problem under discussion was actually not an issue and a stale comment had misled the community, so a follow-on patch removed part of the initial fix.
• In response to a question from the audience, Mike Kravetz proposed aligning addresses returned from mmap(MAP_ANONYMOUS) calls of at least THP size on THP boundaries, resulting in an RFC patch discussion.

This year’s topics tentatively include:

• Seamless hypervisor update with IOMMU-type-agnostic, directly-attached devices and virtual functions.  Related projects are VMM Fast Restart, PKRAM, and MMU enabled kexec relocation for arm64.
• Performance characteristics of RT spinlocks.
• Accounting CPU-intensive kernel threads in the CPU controller via remote charging (thread 1, thread 2, thread 3).
• Design discussion and performance characteristics of Maple Tree (lwn article).
• mmap_sem contention in procfs (test code, gitweb).
• NUMA-aware spinlocks: (series, lwn article).
• futex2: attempts to tackle the performance limitations of the single NUMA node hash table.
• Batching optimizations in the internals of get_user_pages() and put_user_pages().
• Fast kdump for embedded devices.

Come and join us in the discussion of improving performance and scalability of your system.

We hope to see you there.

We are pleased to announce that the Confidential Computing Microconference has been accepted into the 2021 Linux Plumbers Conference! In this microconference we will discuss how Linux can support encryption technologies which protect data during processing on the CPU. Examples are AMD SEV, Intel TDX, IBM Secure Execution for s390x and ARM Secure Virtualization. These are recent additions compared to technologies which protect data while in transit (SSL, VPNs) and at rest (disk encryption).

The Linux kernel recently gained support for SEV-ES and support for Intel TDX is upcoming. AMD SEV will be further enhanced by Secure Nested Paging (SNP). Support for these technologies requires intrusive changes to the Linux kernel for memory integrity and secure interrupt delivery to virtual machines. Designing these changes in a way that works for different confidential computing technologies is one goal of this microconference.

Topics to be included, but not limited to, are:

• Live Migration of Confidential VMs
• Lazy Memory Validation
• APIC emulation/interrupt management
• Debug Support for Confidential VMs
• Required Memory Management changes for memory validation
• Safe Kernel entry for TDX and SEV exceptions
• Requirements for Confidential Containers
• Trusted Device Drivers Framework and driver fuzzing
• Remote Attestation

Please come and join us in the discussion for solutions to the open problems for supporting these technologies.

We hope to see you there!

We are pleased to announce that the Containers and Checkpoint/Restore Microconference has been accepted into the 2021 Linux Plumbers Conference! The Containers and Checkpoint/Restore micro-conference brings together kernel developers, runtime maintainers, and developers working on container- and sandboxing related technologies in general to discuss current problems and agree on new features.

Last year’s meetup resulted in:

• The overlayfs to be mountable inside unprivileged containers
• The idmapped mount patchset that adds the ability to dynamically alter filesystem permissions for unprivileged containers (which was never solved from a decade of mailing list discussions)
• A new checkpoint restore capability was merged
• Hardening of procfs mount points
• Better integration of systemd-udevd with unprivileged containers.

This year’s edition of the Containers and Checkpoint/Restore micro-conference will focus on a variety of topics that are in need of discussion. The list of ideas is constantly evolving and we expect even more topics to pop up during the coming months as past experience has shown. Here is an excerpt:

• How to best use CAP_CHECKPOINT_RESTORE in CRIU to make it possible to run checkpoint/restore as non-root (with CAP_CHECKPOINT_RESTORE)
• Extending the idmapped mount feature to unprivileged containers, i.e. agreeing on a sane and safe delegation mechanism with clean semantics.
• Porting more filesystems to support idmapped mounts.
• Making it possible for unprivileged containers and unprivileged users in general to install fanotify subtree watches.
• Discussing and agreeing on a concept of delegated mounts, i.e. the ability for a privileged process to create a mount context that can be handed of to a lesser privileged process which it can interact with safely.
• Fixing outstanding problems in the seccomp notifier to handle syscall preemption cleanly. A patchset for this is already out but we need a more thorough understanding of the problem and its proposed solution.
• With more container engines and orchestrators supporting checkpoint/restore there has come up the idea to provide an optional interface with which applications can be notified that they are about to be checkpointed. Possible example is a JVM that could do cleanups which do not need to be part of a checkpoint.
• Discussing an extension of the seccomp API to make it possible to  ideally attach a seccomp filter to a task, i.e. the inverse of the current model instead of caller-based seccomp sandboxing enabling full supervisor-based sandboxing.
• Integration of the new Landlock LSM into container runtimes.
• Although checkpoint/restore can handle cgroupv1 correctly the cgroupv2 support is very limited and there is a need to figure out what is still missing to have v2 supported just as good as v1.
• Isolated user namespaces (each with full 32bit uid/gid range) and easier way for users to create and manage them.
• Figure out what is missing on the checkpoint/restore level and maybe the container runtime level to support optimal checkpoint/restore integration on the orchestration level. Especially the pod concept of Kubernetes introduces new challenges which have not been part of checkpoint/restore before (containers sharing namespaces for example).

Come join us and participate in the discussion with what holds “The Cloud” together.

We hope to see you there!