cpu fullload's --cpu-percent flag doesn't work in K8s/CRI runtime

[spencercjh]

Issue Description

Type: bug report

Describe what happened (or what feature you want)

When using the chaosblade tool to simulate CPU load in a Kubernetes Pod, specifying the CPU load percentage (range 0-100, default 100) has no effect. The CPU usage always reaches 100%, regardless of the specified rate.

Describe what you expected to happen

When specifying a CPU load percentage (e.g., 50%), the CPU usage of the target Pod should reflect the specified percentage. For example, if the CPU limit of the Pod is 1 core and the CPU load percentage is set to 50%, the CPU usage should be around 50%.

How to reproduce it (as minimally and precisely as possible)

1. Create a CPU load exp refer to the docs
2. Observe Pod CPU usage with any tools you can use.

Tell us your environment

• ChaosBlade Version: v1.7.3
• Kubernetes Version: v1.20
• Operating System: Linux x86_64
• Node CPU Cores: 48

Anything else we need to know?

The issue arises because the chaosblade tool incorrectly sets the number of CPU cores to the total number of cores available on the host machine (runtime.NumCPU()). This leads to the CPU load being calculated based on the total host CPU cores, which often exceeds the CPU limit of the target Pod.

To resolve this issue, the tool should correctly determine the CPU limit of the target Pod and use that value to calculate the CPU load. The CPU limit can be obtained from the cgroup configuration of the Pod.

Additionally, the following issues have been reported and are related to this problem:

chaosblade-operator/issues/102
chaosblade/issues/463
chaosblade/issues/805
chaosblade/issues/928
chaosblade/issues/801

Detailed Analysis and Solution

Use the cgroups information to determine the CPU limit of the target Pod.

The CPU limit can be obtained from the cgroup files /sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_quota_us and /sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_period_us.
In the privileged chaosblade-tool daemonset pod, the host's /sys is mounted to /host-sys. Therefore, the correct path to the cgroup files would be /host-sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_quota_us and /host-sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_period_us.

[spencercjh]

问题描述

类型: bug report

问题描述

使用 chaosblade 工具在 Kubernetes Pod 中模拟 CPU 负载时，指定的 CPU 负载百分比（范围 0-100，默认为 100）无效。无论指定的百分比是多少，CPU 使用率始终达到 100%。

预期行为

当指定 CPU 负载百分比（例如 50%）时，目标 Pod 的 CPU 使用率应反映指定的百分比。例如，如果 Pod 的 CPU 限制为 1 核心且 CPU 负载百分比设置为 50%，则 CPU 使用率应约为 50%。

复现步骤（尽量简洁和精确）

1. 根据 文档 创建一个 CPU 负载实验。
2. 使用任何可用的工具观察 Pod 的 CPU 使用率。

环境信息

• ChaosBlade 版本: v1.7.3
• Kubernetes 版本: v1.20
• 操作系统: Linux x86_64
• 节点 CPU 核心数: 48

其他需要了解的信息

问题出现的原因是 chaosblade 工具错误地将 CPU 核心数设置为宿主机上的总核心数（runtime.NumCPU()）。这导致 CPU 负载是基于宿主机的总 CPU 核心数计算的，通常会超过目标 Pod 的 CPU 限制。

为了解决这个问题，工具应该正确确定目标 Pod 的 CPU 限制，并使用该值来计算 CPU 负载。CPU 限制可以从 Pod 的 cgroup 配置中获取。

此外，以下问题已被报告并与此问题相关：

• chaosblade-operator/issues/102
• chaosblade/issues/463
• chaosblade/issues/805
• chaosblade/issues/928
• chaosblade/issues/801

详细分析与解决方案

使用 cgroups 信息来确定目标 Pod 的 CPU 限制。

CPU 限制可以从 cgroup 文件 /sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_quota_us 和 /sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_period_us 中获取。在特权的 chaosblade-tool daemonset pod 中，宿主机的 /sys 被挂载到 /host-sys。因此，正确的 cgroup 文件路径为 /host-sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_quota_us 和 /host-sys/fs/cgroup/cpu/${pod-container-cgroup-dir}/cpu.cfs_period_us。

[spencercjh]

reference:

• 重学容器06: 容器资源限制背后的技术cgroups
• 重学容器29: 容器资源限制之限制容器的CPU
• https://github.com/uber-go/automaxprocs