Speakers
Description
High-performance storage environments increasingly rely on direct data movement between PCIe endpoints, such as NVMe Controller Memory Buffers (CMB) or BAR memory. Today, NFS relies on standard page references (get_page) and doesn't support pinning pages for DMA, making the subsystem pin-unaware. Furthermore, the legacy iterator APIs used in NFS are incapable of passing the required capability flags mandated by the GUP subsystem. Thus, NFS remains incompatible with P2PDMA due to this lack of pin-awareness.
This talk walks through the design challenges in enabling P2PDMA for the NFS Direct I/O path using a two-phased approach. The first phase involves an architectural modernization, i.e. migrating the Direct I/O path to use the modern iov_iter_extract_pages API. This transition introduces pin-awareness to the NFS request lifecycle. Additionally, it migrates the NFS Direct I/O from pages to folios. We have an on-going series posted upstream for the first phase:
https://lore.kernel.org/all/20260603053033.3300318-1-praan@google.com
Plumber's Discussion: Transport-Level P2PDMA Capability Detection
The second phase of this modernization addresses the detection and propagation of P2PDMA capabilities based on the underlying transport. Since NFS connections can dynamically migrate across different hardware interfaces or utilize multipathing (nconnect), P2PDMA support is a property of the active connection rather than a static server capability. This session focuses on the following architectural challenges:
- Detection & Propagation: Discuss & align on the proposed design for transport-level capability discovery. This would allow NFS to conditionally signal hardware P2PDMA support (via ITER_ALLOW_P2PDMA) to the GUP subsystem, potentially on a per-request basis.
- Architectural Implications: Discuss the cross-subsystem impact of this transport-centric approach. Key areas include managing state consistency across transport failover, the implications of dynamic device rebinding during re-connections, and the challenge of transport selection in multipath (nconnect) environments.
The talk is backed by the ongoing NFS Modernization and P2PDMA series upstream:
- RFC: https://lore.kernel.org/all/20260401194501.2269200-1-praan@google.com/
- On-going Series: https://lore.kernel.org/all/20260616134000.2733403-1-praan@google.com/