The sched_ext Microconference was very active this year at Linux Plumbers in Prague. It featured
six talks, preceded by an opening discussion on the future direction of sched_ext.
(Written by Andrea Righi)

Three main topics dominated the discussions

• Priority inversion and how to handle it in sched_ext: lock holder and
  lock waiter preemption, time slice extension, and making proxy execution
  work together with BPF schedulers.
• Paravirtualized scheduling: letting the host and guest schedulers
  cooperate across the VM boundary, with sched_ext as the policy engine on
  the host.
• Device and topology-aware scheduling: using hardware feedback (Intel
  HFI), device/accelerator locality and cache topology to drive task
  placement.

Intro: where sched_ext is going

The opening discussion outlined three directions for sched_ext development:

• Move more into BPF. With BPF arenas, queuing and idle CPU management can
  be implemented entirely in BPF, and several schedulers are already
  converging on this model. Once they all move, a significant amount of
  in-kernel sched_ext code can be removed.
• Composability. Long discussed, now a good time to tackle it: reusable
  BPF/user-space building blocks (load balancer from one scheduler, idle CPU
  selection from another, etc.) that can be combined into new schedulers.
• Generate BPF code from Rust. Most schedulers today are Rust in
  user space plus C for the BPF part. The trend, aligned with the wider
  BPF community, is to generate the BPF code from Rust as well, so a
  single language and shared abstractions can be used to access
  scheduling data structures from both sides. Prototypes exist but this
  is not there yet.

Priority inversion

Making proxy execution compatible with sched_ext
(Andrea Righi and John Stultz)

Proxy execution lets a mutex-blocked task lend its scheduling context to the
lock owner. The work presented makes this compatible with sched_ext, with
support enabled explicitly by each BPF scheduler. Dispatching a blocked task
triggers the core proxy machinery, keeping the BPF scheduler in control.

The compatibility changes and follow-up improvements are queued for Linux v7.4.
Distributions will be able to enable both `CONFIG_SCHED_PROXY_EXEC` and
`CONFIG_SCHED_CLASS_EXT` in the same kernel, removing their previous mutual
exclusion. The follow-up changes will also make proxy migrations transparent to
BPF, without exposing intermediate bookkeeping through extra enqueue callbacks.

Discussion points:

• User-space locks: futexes are not covered since `FUTEX_WAIT` does not
  identify the owner. A new futex op that carries ownership is being explored;
  passing user-space owner hints (e.g., via BPF maps or futex flags) is
  inherently racy. A “directed yield” (timed proxy to a specific task) was
  mentioned as future work.
• rwsems / semaphores: writer boosting is supported via a patch; boosting
  multiple readers is harder and has overhead. Plain semaphores have no owner
  and cannot be proxied.
• Observability: there are no upstream tracepoints for proxy execution
  yet, which makes traces confusing (the owner runs under the donor’s
  context). Something similar to the existing PI tracepoints was requested.

Taming latency spikes caused by lock holder and lock waiter preemption
(Changwoo Min)

On the Steam Frame VR headset, camera processing must finish within 33ms but
needs only about 7–8ms of CPU time. Under heavy gaming load, preempting a lock
holder or delaying a woken lock waiter can stretch the pipeline enough to miss
that deadline.

LAVD addresses both cases with time slice extension and boosting of lock
waiters. The mitigations reduced scheduling delays above 2ms across four games,
with no missed camera frames observed during an hour of gameplay in the tested
titles.

A common interface for lock ownership and waiting information, covering kernel
and user-space locks, would help replace scheduler-specific heuristics.

Discussion points:

• Abuse of the extension is bounded by fairness: a task that constantly
  extends its slice is deprioritized.
• Why not just rely on proxy execution? Proxy is reactive (owner scheduled
  out, donor scheduled in, owner scheduled in again), with extra overhead.
  The consensus was that the two approaches are complementary: slice
  extension is like a spinlock (cheap, good for short critical sections),
  proxy execution is like a mutex (heavier, but handles long ones).

Paravirtualized scheduling

Bridging the VM Boundary: Scheduling Passthrough via pvsched and sched_ext
(Josh Don and Vineeth Remanan Pillai)

A host scheduler normally sees opaque vCPU threads, without knowing whether
they are running urgent guest work or background jobs. pvsched provides a
bidirectional channel: guests communicate priorities and preemption hints,
while hosts report which vCPUs they can run concurrently.

The proposed v3 design moves negotiation into the VMM through a virtual PCI
device, keeping policy outside KVM. On the host, sched_ext can implement bounded
vCPU boosting and gang scheduling. Guests can concentrate work on the vCPUs the
host can actually run, reducing the cost of double scheduling.

The next revision is still to be posted upstream.

Discussion points:

• BPF arenas as an alternative to a dedicated device: a host scheduler
  could allocate an arena, hand out per-guest chunks and get zero-copy
  communication with no kernel changes, needing only a way to pin arena
  memory. Someone from the audience pointed out that allocating memory on
  the host and mapping it into the guest on the fly is slow (memslot
  creation, guest notification), which is why guest-initiated allocation was
  chosen. However, this cost can be avoided: the host BPF scheduler can
  create the arena once when it is started and hand out per-guest chunks
  from the preallocated arena, mapping each chunk when the guest is set up
  rather than on the fly, which keeps the fast path cheap. It was also
  noted that the device model keeps the handshake arch-agnostic and
  evolvable without kernel changes.
• There is related work from others in this area; the agreement was that
  the various efforts should converge on a single kernel module, possibly
  alongside a BPF-arena-based path.
• Standardization: post the kernel module first, then the QEMU device, then
  other VMMs.
• The preferred-CPU framework (steal governor) is related, but reactive,
  while pvsched allows an explicit policy.

Device and topology-aware scheduling

Automatic placement of accelerator workloads
(Balbir Singh and Lee Trager)

CPU and memory placement can have a large effect on accelerator performance,
but manual pinning requires applications or administrators to know the topology
and device usage in advance. The proposed alternative combines hints about
which device a task is using, sched_ext task placement, and memory-access
telemetry to guide page placement. PMU counters were also suggested as a source
of remote-access and bandwidth telemetry to inform those decisions.

Two interfaces are needed: a generic process-to-device association and BPF
control over NUMA placement preferences. A series enabling NUMA balancing under
sched_ext had recently been posted, together with a kfunc for requesting a
preferred memory node.

Discussion points:

• Exposing PMU telemetry to non-root applications raises side-channel
  concerns; IPC is fairly safe, cache/TLB misses are a gray area. An
  allowlist of audited events was suggested.
• Is a BPF map the right interface, or is a common user-space library needed
  so all frameworks can provide hints?
• Migrating network-heavy tasks across nodes may interact badly with per-CPU
  queues and TCP flows.
• NUMA balancing needs improvements, including asynchronous migration and
  charging migration work to the right task. Enabling it under sched_ext
  reuses the existing mechanism, so that work can proceed in parallel with
  those fixes. Telemetry-driven `move_pages()` can serve as an interim solution.

HFI plug-in for sched_ext
(Srinivas Pandruvada)

Intel’s Hardware Feedback Interface supplies per-CPU performance and efficiency
hints that reflect the platform’s changing power budget. Integrating these
hints into LAVD improved Shadow of the Tomb Raider performance by about three
frames per second on a hybrid Intel handheld at a 10W power limit.

The prototype took less than a month. A Rust user-space thread receives HFI
updates over netlink and passes them to BPF through a map; LAVD then adjusts CPU
selection and effective capacity.

Discussion points:

• Why is HFI not used by the in-kernel scheduler? Older generations were too
  noisy, and the firmware was tuned mostly on Windows workloads; the last two
  generations treat Linux on equal footing, so it is worth revisiting.
• Qualcomm’s WALT has a similar CPU isolation feature (thermal and
  low-utilization cases), implemented downstream via Android vendor hooks.
• The preferred-CPU framework (steal governor) could be a natural in-kernel
  hook for HFI exclusion. The thermal pressure signal filter currently only allows
  slowing down the reaction time, and patches to make it configurable in both
  directions may be revisited given how fast firmware can now report changes.
• The HFI code is structured as a library and was proposed for inclusion in
  the scx library so other schedulers can reuse it.

Stickiness: keeping tasks cache-warm in the work-conserving LAVD scheduler
(Gavin Guo)

Keeping a task on a cache-warm CPU can improve performance, but waiting for that
CPU competes with the benefit of running immediately elsewhere. The session
compared cache-warmth modelling, CPU reservation, and look-ahead dispatch from a
shared queue.

Results depended on the workload and approach. Warmth-based placement improved
P99 latency for short waits, while look-ahead dispatch showed no noticeable
memcached latency improvement under light or moderate load and added overhead
under heavier load.

Discussion points:

• Staying on a busy CPU is a short-term gain, while an idle CPU may offer
  long-term uninterrupted execution, so the right answer is very workload-
  and topology-dependent (shared vs non-shared L2/L3, snoop efficiency).
• Suggestions included using PMU counters (as scx_cosmos already does) or Intel
  RDT/CMT for cache occupancy (limited by the number of monitoring IDs),
  tracking only latency-critical tasks, and broader workloads such as
  stress-ng and video encoders.