Speaker
Description
My employer relies heavily on livepatch to rapidly experiment with new kernel features without interrupting production workloads. Our use cases include:
-
Case 1: Deploying a livepatch function as a stable BPF hook.
For example, some proposals for such a use case has already been submitted upstream but has not yet been accepted:
https://lwn.net/Articles/1054030/
https://lwn.net/Articles/1043548/
We also have another internal use case that we do not plan to upstream:
https://lore.kernel.org/live-patching/CALOAHbDnNba_w_nWH3-S9GAXw0+VKuLTh1gy5hy9Yqgeo4C0iA@mail.gmail.com/ -
Case 2: Combining fleet-wide cumulative livepatches with workload-specific livepatches.
For example, consider the VFS cache adjustment feature that we previously upstreamed:
https://lore.kernel.org/linux-fsdevel/20250511083624.9305-1-laoar.shao@gmail.com/
We initially deployed this feature as a livepatch before upstreaming it. In the livepatch version, we could not introduce a sysctl interface for per-workload tuning, so we used a hard-coded default value and deployed it only to specific servers.
In Case 1, the livepatched BPF hook must remain stable. Otherwise, existing BPF programs may become invalid and need to be reloaded, which could introduce operational risks.
In Case 2, we currently need to release different cumulative livepatches for different workloads. However, we would like to have a generic cumulative livepatch combined with individual workload-specific livepatches. This approach would significantly reduce the maintenance burden of managing multiple livepatch variants.
Based on these requirements, we proposed a hybrid livepatch mode, which allows multiple livepatches to coexist with different scopes. This proposal was later refined into the replace_set support proposal, which has already been discussed on the livepatch mailing list:
https://lore.kernel.org/live-patching/20260607131659.29281-1-laoar.shao@gmail.com/
However, several implementation details and design decisions remain open. We would like to discuss them at LPC and determine the best path forward.
In this presentation, I will describe how we use livepatch across our large fleet of production servers and discuss the improvements needed to make livepatch suitable for a broader range of use cases.