Description
The Tracing microconference focuses on improvements of the Linux kernel tracing infrastructure. Ways to make it more robust, faster and easier to use. Also focus on the tooling around the tracing infrastructure will be discussed. What interfaces can be used, how to share features between the different toolkits and how to make it easier for users to see what they need from their systems.
-
Steven Rostedt22/09/2021, 07:00
Short introduction to the Tracing Microconference
Go to contribution page -
Kris Van Hees (Oracle USA)22/09/2021, 07:05
The topic aims to present various challenges we have ran into during the implementation of DTrace on top of Linux tracing facilities such as BPF. We hope to have open discussion on how we can get around some of these challenges because they are likely to be things other projects will run into as well. In addition, we want to share some of the workarounds we came up with, and hopefully spark...
Go to contribution page -
Beau Belgrave22/09/2021, 07:30
Summary
Go to contribution page
We have many user processes today that run in various locations and control groups. To know
every binary location for each version becomes a challenge. We also have common events that
trace out among many processes. This makes using uprobes a challenge, but not impossible.
However, having a way for user processes to publish data directly to trace_events enables a
much easier... -
Yordan Karadzhov (VMware)22/09/2021, 07:55
Providing adequate observability in containerized workloads is getting more important. Sophisticated instruments are being developed to understand what is really going on, but most of the effort approaches the problem from top to bottom, operating at the abstraction layers of container orchestration.
What if we take the opposite approach and use Linux system tracing to unfold the container...
Go to contribution page -
Mathieu Desnoyers (EfficiOS Inc.)22/09/2021, 08:35
When invoked from system call enter/exit instrumentation, accessing user-space data is a common use-case for tracers. However, tracepoints currently disable preemption around iteration on the registered tracepoint probes and invocation of the probe callbacks, which prevents tracers from handling page faults.
Discuss the use-cases enabled by allowing system call entry/exit tracepoints to...
Go to contribution page -
Mathieu Desnoyers (EfficiOS Inc.)22/09/2021, 09:00
Upstreaming the LTTng kernel tracer [1] (originally created in 2005) into the Linux kernel has been a long-term goal of the LTTng project.
Today, various tracing technologies are available in the Linux kernel: instrumentation with tracepoints, kprobes, kretprobes, function tracing, performance counters through perf, as well as user-visible ABIs, namely Ftrace, Perf, and eBPF. There are...
Go to contribution page -
Ajay Kaher22/09/2021, 09:25
Problem Statement:
Linux tracing provides mechanism to have multiple instances of 'tracing' and each instance of tracing have individual events directory which is known as 'Eventfs Tracing Infrastructure' i.e. '/sys/kernel/debug/tracing/events'.'Eventfs Tracing Infrastructure' contains lot of files/directories although depending upon the Kernel config, still the number of...
Go to contribution page -
Steven Rostedt, Jiri Olsa22/09/2021, 10:05
With the new DYNAMIC_FTRACE_WITH_ARGS feature that x86 (and hopefully soon other archs have), the function tracer callback gets all the registers needed to see the arguments by default (but not all registers). In theory, we can use something like BTF, which can describe the arguments of every function, and use it to trace them.
Currently, BPF can do this on a function by function basis,...
Go to contribution page -
Steven Rostedt22/09/2021, 10:30
Currently there's three infrastructures that can trace the exit of the function.
kretprobes
function_graph
BPF direct trampolinesEach one does it differently, and they can stumble over each other when they trace the same function call return. There should be a way that all three can somehow use the same infrastructure. At least maybe two of them?
There's been prototypes to do this,...
Go to contribution page -
Tracing in the Linux kernel is constantly improving. Since 2008 tracing has been add to Linux, and more tooling is constantly being added to help out with visibility. The work is still ongoing, with Perf, ftrace, Lttng, and eBPF. User space tooling is expanding and as the kernel gets more complex, so does the need for facilitating seeing what is going on under the hood.
The last tracing...
Go to contribution page -
Tracing in the Linux kernel is constantly improving. Since 2008 tracing has been add to Linux, and more tooling is constantly being added to help out with visibility. The work is still ongoing, with Perf, ftrace, Lttng, and eBPF. User space tooling is expanding and as the kernel gets more complex, so does the need for facilitating seeing what is going on under the hood.
The last tracing...
Go to contribution page
