Enable kprobe-multi for kmods by default #470
Conversation
|
Observation: Running with kprobe-multi backend: @brb Do the respective numbers we observe for attached and ignored functions make sense? I'm having a hard time coming up with a meaningful way to test if enabling the kprobe-multi backend breaks any attachments. |
There was a problem hiding this comment.
I think we can compare what's been ignored by kprobe to kprobemulti. For example, we can print(fn) at https://github.com/cilium/pwru/blob/v1.0.10-pre/internal/pwru/kprobe.go#L180.
My feeling is this is fine, since kprobemulti ignored less targets (20) than kprobe (115).
There was a problem hiding this comment.
Thanks for the PR!
Unfortunately, I don't see kmods in a pwru traces.
The way I tested.
- Block 1.1.1.1 with
iptables -t filter -I OUTPUT 1 -m tcp --proto tcp --dst 1.1.1.1/32 -j DROP. - Run pwru with your changes:
KMODS=$(ls /sys/kernel/btf/ | grep -v vmlinux | grep -E '(ip|xt|nf)' | tr '\n' , | sed 's/.$//')
./pwru --kmods="$KMODS" 'host 1.1.1.1 and port 80'
- Observe that
curl 1.1.1.1does not show any function calls from netfilter modules. - Run
./pwru --kmods="$KMODS" --backend=kprobe 'host 1.1.1.1 and port 80'and see the modules, e.g., thenf_natmodule:
59:36298->1.1.1.1:80(tcp) nf_nat_ipv4_local_fn[nf_nat]
0xffff9feec55880e8 4 <empty>:0 4026531840 0 0 0x0800 1500 60 192.168.1.159:36298->1.1.1.1:80(tcp) sk_skb_reason_drop(SKB_DROP_REASON_NETFILTER_DROP)
My gut feeling is that the kmod functions are being ignored due to not exposed in the available_filter_functions (see #147 (comment)).
|
Yes agreed, it definitely needs some troubleshooting. Thanks for the test example, I'll start by taking a look at which funcs get ignored for the respective backend. However, on my machine the |
This commit ensures that we remove potential suitors of functions from the set of functions we want to attach to if they are not available in /sys/kernel/debug/tracing/available_filter_functions regardless of the backend that will be used. While this isn't technically only necessary for the kprobe-multi backend, this improves the output of pwru telling the user how many functions it attached to and how many were ignored. Signed-off-by: Robin Gögge <[email protected]>
This commit enables using the kprobe-multi backend for tracing functions in kernel modules by default if supported. Previously that was only possible by specifying the --backend=kprobe-multi option. Fixes cilium#147 Signed-off-by: Robin Gögge <[email protected]>
The needed log size is now automatically determined by the cilium/ebpf library. Signed-off-by: Robin Gögge <[email protected]>
rgo3
force-pushed
the
pr/rgo3/enable-kmods-kprobe-multi
branch
from
e4c11a7 to
92bf0ca
Compare
|
I think I fixed it, had to make sure we parse out functions from kmods in |
This commit enables using the kprobe-multi backend for tracing functions in kernel modules by default if supported. Previously that was only possible by specifying the
--backend=kprobe-multioption.Fixes #147