Conveners
Live Update MC
- Mike Rapoport
- Alexander Graf
- Pasha Tatashin
- David Matlack (Google)
Description
Live Update is a specialized reboot process where selected devices are kept operational and kernel state is preserved and recreated across a kexec. For devices, DMA and interrupts may continue during the reboot.
The primary use-case of Live Update is to enable hypervisor updates in cloud environments with minimal disruption to running virtual machines. During a Live Update, a VM can pause and its state is stored to memory while the hypervisor reboots. PCIe devices attached to those VMs (such as GPUs, NICs, and SSDs), are kept running during the Live Update. After the reboot, VMs are recreated and restored from memory, reattached to devices, and resumed. The disruption is limited to the time it takes to complete this entire process.
With Live Update infrastructure in place, other use-cases may emerge, like for example preserving the state of GPU doing LLM, freezing running containers with CRIU, and preserving large in-memory databases.
The Live Update and state persistence functionality touch on different parts of the kernel and this microconference aims to bring together people from different subsystems. Upstream support for Live Updates is still in its infancy and there are a lot of unsolved aspects that will benefit from direct communication.
Key problems that will be discussed:
Support for memfd/guest_memfd/hugetlb/tmpfs
Preserving the state of VFIO, IOMMUFD, and IOMMU drivers.
Kernel <-> userspace interaction during Live Update
Integration of Live Update with PCI and Device Model
Persistence of movable memory
Leveraging suspend/resume functionality for device state preservation
Optimizing kernel shutdown and boot times
Automated Testing of Live Updates
Key attendees:
Pasha Tatashin
David Matlack
David Rientjes
Chris Li
Bjorn Helgaas
Samiullah Khawaja
Vipin Sharma
Josh Hilke
Changyuan Lyu
Alex Graf
David Woodhouse
James Gowans
Pratyush Yadav
Jason Gunthorpe
Mike Rapoport
Alex Williamson
Hello, I'd like to propose discussion about KSTATE
as a solution for [de]serializing kernel's state.
Thanks,
Andrey
In this talk, we'll discuss our work to support Live Update in VFIO for PCI devices.
Live Update is a mechanism to quickly update the kernel while running virtual machines using kexec. VFIO is a kernel module to allow devices to be controlled by userspace and virtual machines.
During our talk we will cover the problems that need to be solved to support Live Updates in VFIO, and our...
Restarting a node running a stateful workload for an infrastructure software upgrade can be an extremely costly operation. Modern infrastructure software upgrades must also account for applications which are using accelerators such as GPUs, RDMA NICs and NIC stateful flow accelerators. While these workloads may typically run in isolated VMs, a hypervisor reboot for a kernel update can lead to...
At scale, virtualization uncovers hidden bottlenecks, including the cost of PCI configuration space accesses. In SR-IOV deployments with thousands of VFs, each configuration read triggers a hardware transaction. As VFs increase, these accesses scale linearly, leading to longer VM boot times, heavier bus contention, and noticeable startup delays.
The PCI subsystem today treats every...