Speaker
Description
Work-conserving schedulers prefer running tasks on idle CPUs
immediately, ensuring no processing capacity is wasted while work is
waiting. In the lavd select_cpu process, when a task wakes, the
scheduler will do its best to seek an idle core and run the task over
there. However, this idle-oriented CPU selection generally prioritizes
idle cores over the cache-warm cores, leading to more migrations,
making the tasks cache-cold, and requiring L1/L2 caches and TLBs to be
refilled frequently.
The work-conserving mechanism works well in some scenarios. However,
there are cache-sensitive workloads such as edge routers using routing
tables as in-memory KV stores. The migrations to the cold CPU have a
global impact, and the cache refilling evicts the other cache lines
and pulls the new ones in, making the cache contention worse and further
impacting the tail latency.
We will present our measurements and open the floor on topics
including:
-
Warmth estimation: How long does L1/L2/TLB state realistically
survive on a CPU, and could the kernel expose hardware signals
(e.g., PMU counters, cache-occupancy registers) usable at
scheduling-decision frequency? -
The wait-or-migrate decision: Waiting requires predicting when
a busy CPU will become available (remaining slice + queue service
time, or queue load?). Which metrics should a scheduler maintain to
make that prediction accurate? -
The scheduler/userspace interface: Userspace structures (such
as allocator per-CPU caches) suffer heavily from core migrations.
Should applications hint their locality needs to the scheduler, or
should the scheduler expose warmth and migration state to userspace?