Speaker
Description
KCOV's trace-pc provides edge coverage feedback for kernel fuzzers, but captures no data-flow context. Two syscalls hitting identical basic blocks with different argument values (e.g., vfs_open with O_RDONLY vs O_WRONLY|O_TRUNC) are indistinguishable to the fuzzer. This "semantic gap" causes coverage saturation on value-dependent state transitions in complex subsystems (binder, io_uring, ksmbd).
ftrace function graph tracer support arg/ret values tracing during runtime. But their is a couple of challenge(FTRACE_REGS_MAX_ARGS 6) regarding architecture calling convention from register, and currently can't support structure and their members on the function.
On the other hands, Rust For Linux kernel modules can't supported by current tracing framework.
We need consensus on extending LLVM's SanitizerCoverage to capture function arguments and return values, including automatic struct field decomposition from DWARF metadata, with a kernel-resident per-task ring buffer.
What I Implemented
Two new SanitizerCoverage modes in LLVM:
-
Emits LLVM RFC submitted
__sanitizer_cov_trace_args(pc, arg_idx, arg_size, ptr, offsets, num_fields)at function entry for each parameter. -
Emits LLVM RFC submitted
__sanitizer_cov_trace_ret(pc, ret_size, ptr, offsets, num_fields)before each return.
The pass uses DICompositeType metadata to extract struct field offsets/sizes at compile time and emits them as global constant arrays. At runtime, the kernel backend reads fields via copy_from_kernel_nofault() into a per-task lock-free mmap'd buffer at /sys/kernel/debug/kcov_dataflow.
A native rustc path (rustc built against the custom LLVM) enables Rust kernel module instrumentation without a post-compilation pipeline — the only known method for capturing Rust function arguments at runtime given -O2 DWARF elision.
Discussion Topics Needing Agreement
-
Callback ABI stability
Should__sanitizer_cov_trace_args/retbe considered stable SanitizerCoverage API, or experimental? What's the path from-fsanitize-coverage=trace-argsto an accepted upstream flag? -
DWARF metadata dependency
The pass requires-gfor struct expansion. Currently it gracefully degrades: without-g,F.getSubprogram()returns null and the function is silently skipped. With-gbut missing type info for a param, it records as scalar (offsets=NULL, num_fields=0). Should this behavior be documented as the contract? -
Interaction with optimizations
At -O2, scalar arguments are spilled toallocaso the kernel callback receives a uniform pointer. Should the pass mark these as side-effecting to prevent DSE, or accept the current behavior (compiler preserves them because the call itself is a side effect)? -
Rust integration path
Currently requires building rustc against the custom LLVM (native path) or using a wrapper (rustc emit-llvm-ir then opt). Is there appetite for native-Zsanitizer-coverage=trace-argssupport in upstream rustc? The kernel's Kbuild does not expose intermediate IR for external instrumentation, making the native path mandatory for Rust kernel modules. -
Kernel-side callback design
The kernel backend usescopy_from_kernel_nofault()for safe pointer reads and a bit-31 recursion guard (needed becausecopy_from_kernel_nofaultitself is instrumented under INSTRUMENT_ALL). Should the LLVM pass remain fully target-agnostic, or emit hints for the runtime? -
Scope control
Per-module opt-in (KCOV_DATAFLOW_file.o := y) vs. global (CONFIG_KCOV_DATAFLOW_INSTRUMENT_ALL). The pass currently instruments all functions with DISubprogram. Should it support function-level filtering via attributes (e.g.,__attribute__((no_sanitize("dataflow"))))? -
Performance implications
With INSTRUMENT_ALL: +9.5% .text, +133% syscall latency. Per-module: +8.3% on instrumented paths. The per-callback cost is ~27ns dominated bycopy_from_kernel_nofault. Is this acceptable for the SanitizerCoverage framework, or should it be a separate pass?
Why Rust Kernel Modules Need This
Existing observability tools completely fail on Rust kernel code:
- ftrace: Cannot hook Rust functions (no
__fentry__prologue — rustc doesn't emit-mfentry) - kprobes/fprobe: Rust symbol mangling makes targeting impractical; even with correct names, only captures raw register values (no struct expansion)
- drgn/vmcore:
rustc -O2elidesDW_AT_locationfor all parameters —frame.locals()returns{} - eBPF: Requires manual struct layout specification; Rust
#[repr(Rust)]types have no stable ABI
Our LLVM pass operates on Rust-generated IR before codegen — the only point where both debug metadata and argument values coexist.
Verified: Rust eight_struct_args_rust selftest produces correct output at -O2 on CI result: https://github.com/yskzalloc/kcov-dataflow/actions/runs/29659128011/job/88119068007
do_el0_svc({0x4, 0x4, 0xffffffff, 0x0, 0x0, 0x0})
0x0 = B...
sb_start_write()
file_start_write()
0x0 = _RNvCsdfZGIOKgjaD_22eight_struct_args_rust13write_handler() [eight_struct_args_rust]
0x11 = rsf_1(0x11) [eight_struct_args_rust]
0x33 = rsf_2(0x11, {0x11, 0x22}) [eight_struct_args_rust]
...
0x30c = rnsf_8({0xa8, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11}) [eight_struct_args_rust]
0xaa = rsf_fwd_inner(0x11, {0x11, 0x22}, {0x11, 0x22, 0x33}, {0x11, 0x22, 0x33, 0x44}) [eight_struct_args_rust]
0xaa = rsf_fwd(0x11, {0x11, 0x22}, {0x11, 0x22, 0x33}, {0x11, 0x22, 0x33, 0x44}) [eight_struct_args_rust]
rsf_ret_struct(0x0, {0x11, 0x11}, 0x11) [eight_struct_args_rust]
0xfffffdffc0356e80 = rust_helper_krealloc_node_align(0x0, 0x8, 0x8, 0xcc0, 0xffffffff)
0x6f00000009 = rust_helper_krealloc_node_align(0x0, 0x10, 0x8, 0xcc0, 0xffffffff)
0xf200006e00000011 = rust_helper_krealloc_node_align(0x0, 0x18, 0x8, 0xcc0, 0xffffffff)
0x2500006e00000011 = rust_helper_krealloc_node_align(0x0, 0x20, 0x8, 0xcc0, 0xffffffff)
0xaa = rsf_heap(0x11, {0x11, 0x22}, {0x11, 0x22, 0x33}, {0x11, 0x22, 0x33, 0x44}) [eight_struct_args_rust]
0x0 = rust_helper_krealloc_node_align(0x11, 0x0, 0x1, 0x0, 0xffffffff)
0x0 = rust_helper_krealloc_node_align(0x11, 0x0, 0x1, 0x0, 0xffffffff)
0x0 = rust_helper_krealloc_node_align(0x11, 0x0, 0x1, 0x0, 0xffffffff)
0x0 = rust_helper_krealloc_node_align(0x11, 0x0, 0x1, 0x0, 0xffffffff)
0x1 = _RNvCsdfZGIOKgjaD_22eight_struct_args_rust13write_handler(0xffffa64d1231, 0x1, 0x0) [eight_struct_args_rust]
...
Implementation vs. ftrace funcgraph-args
ftrace's funcgraph-args (available since 6.x) captures 6 raw register values but:
- Requires -mfentry (no Rust support)
- No struct field expansion (just pointer addresses)
- Shared per-CPU ring buffer with preempt_disable per event
- Under load: events silently overwritten/dropped
- Stack-passed args (> 6) not captured
kcov-dataflow:
- Per-task private mmap buffer (no contention, no preempt_disable)
- Automatic struct field expansion from DWARF
- All args including stack-passed (alloca'd at compile time)
- Works on both C and Rust identically
- Zero-copy consumer via mmap
Current Status
- Repository (kernel + LLVM + rustc + selftests + CI):
https://github.com/yskzalloc/kcov-dataflow - CI (x86_64 + arm64, 6 selftests):
https://github.com/yskzalloc/kcov-dataflow/actions - Tested on linux-next 7.2.0-rc3 with custom clang/LLVM 23 and rustc 1.99-nightly
- All selftests pass on both x86_64 (KVM) and arm64 (native) Github CI
Why This Needs Face-to-Face Discussion
- This work crosses 3 communities (LLVM, kernel, Rust-for-Linux) that rarely meet simultaneously.
- The callback ABI, optimization interaction, and Rust pipeline decisions require input from all 3.
- LPC Toolchains Track is the only venue where clang/LLVM developers, kernel maintainers, and Rust-for-Linux contributors are in the same room.
References
- LLVM RFC: https://discourse.llvm.org/t/rfc-sanitizercoverage-add-fsanitize-coverage-trace-args-trace-ret/91026
- LLVM PR: https://github.com/llvm/llvm-project/pull/201410
- Kernel patch (v2): https://lore.kernel.org/all/20260611-b4-kcov-dataflow-v2-v2-0-0a261da3987c@est.tech/
- arXiv paper: https://arxiv.org/pdf/2606.00455
- LWN: https://lwn.net/Articles/1077606/