Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Program Load Failure When Mixing Rate-Limited and Non-Rate-Limited kprobes #3251

Open
justin0u0 opened this issue Dec 23, 2024 · 0 comments
Open
Labels
kind/bug Something isn't working

Comments

@justin0u0
Copy link
Contributor

What happened?

After upgrading to v1.3.0, I observed that applying kprobes with both rate-limited and non-rate-limited actions in a single kprobe TracingPolicy fails.

The tracing policy is defined as follows:

description: "Detects privileges raising operations"
kprobes:
  - call: "__sys_setuid"
    message: "Privileged operation setuid to root"
    syscall: false
    args:
    - index: 0
      type: "int"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "0"
      matchActions:
      - action: Post
        rateLimit: "1m"
  - call: "__sys_setgid"
    message: "Privileged operation setgid to root"
    syscall: false
    args:
    - index: 0
      type: "int"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "0"
      matchActions:
      - action: Post
        # rateLimit: "1m"

By either setting rateLimit for both or removing rateLimit from both matchActions, the program loads successfully.

Tetragon Version

v1.3.0

Kernel Version

Linux 5.15.0-118-generic x86_64

Kubernetes Version

v1.30.3

Bugtool

No response

Relevant log output

time="2024-12-23T03:59:05Z" level=info msg="adding tracing policy" info="privileges-raise (object:1/7fc86724-664b-437d-8580-8bb01820ec23) (type:/)" name=privileges-raise
time="2024-12-23T03:59:06Z" level=info msg="Added kprobe" function=__sys_setuid override=false return=false
time="2024-12-23T03:59:06Z" level=info msg="Added kprobe" function=__sys_setgid override=false return=false
time="2024-12-23T03:59:06Z" level=info msg="Added generic kprobe sensor: /var/lib/tetragon/bpf_generic_kprobe_v511.o -> __sys_setuid" override=false
time="2024-12-23T03:59:06Z" level=info msg="Added generic kprobe sensor: /var/lib/tetragon/bpf_generic_kprobe_v511.o -> __sys_setgid" override=false
time="2024-12-23T03:59:06Z" level=info msg="BTF file: using metadata file" metadata=/sys/kernel/btf/vmlinux
time="2024-12-23T03:59:06Z" level=info msg="Loading sensor" name=generic_kprobe
time="2024-12-23T03:59:06Z" level=info msg="Loading kernel version 5.15.143"
time="2024-12-23T03:59:09Z" level=warning msg="incompatible map found" error="expected max entries 1, got 32768: map spec is incompatible with existing map" map-name=ratelimit_map path=/sys/fs/bpf/tetragon/privileges-raise/generic_kprobe/ratelimit_map
time="2024-12-23T03:59:09Z" level=warning msg="will delete and recreate" map=ratelimit_map
time="2024-12-23T03:59:09Z" level=warning msg="adding tracing policy failed" error="sensor generic_kprobe from collection privileges-raise failed to load: failed prog /var/lib/tetragon/bpf_generic_kprobe_v511.o kern_version 331663 loadInstance: opening collection '/var/lib/tetragon/bpf_generic_kprobe_v511.o' failed: using replacement map ratelimit_map: MaxEntries: 1 changed to 32768: map spec is incompatible with existing map"

Anything else?

Upon investigation, I found that in PR #2128's commit d7a30b3, the ratelimit_map was changed to a per-sensor map. This modification causes the ratelimit_map to be shared among programs, but rate-limited and non-rate-limited programs expect different MaxEntries values. As a result, the program load fails due to map specification incompatibility.

@justin0u0 justin0u0 added the kind/bug Something isn't working label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant