Run-time failure on ppc64le: invalid input constraint 'YZ<>' in asm

[vt-alt]

jfyi, test run of cpudist from 0.31.0 started to fail on Linux v6.12.5 on ppc64le with:

[00:02:05] ++ /usr/src/tmp/bcc-buildroot/usr/share/bcc/tools/cpudist 1 1
[00:02:08] In file included from /virtual/main.c:2:
[00:02:08] In file included from include/uapi/linux/ptrace.h:145:
[00:02:08] In file included from arch/powerpc/include/asm/ptrace.h:141:
[00:02:08] In file included from arch/powerpc/include/asm/paca.h:25:
[00:02:08] arch/powerpc/include/asm/atomic.h:183:53: error: invalid input constraint 'YZ<>' in asm
[00:02:08]   183 |                 __asm__ __volatile__("ld%U1%X1 %0,%1" : "=r"(t) : DS_FORM_CONSTRAINT (v->counter));
[00:02:08]       |                                                                   ^
[00:02:08] arch/powerpc/include/asm/asm-compat.h:43:28: note: expanded from macro 'DS_FORM_CONSTRAINT'
[00:02:08]    43 | #define DS_FORM_CONSTRAINT "YZ<>"
[00:02:08]       |                            ^
[00:02:08] In file included from /virtual/main.c:2:
[00:02:08] In file included from include/uapi/linux/ptrace.h:145:
[00:02:08] In file included from arch/powerpc/include/asm/ptrace.h:141:
[00:02:08] In file included from arch/powerpc/include/asm/paca.h:25:
[00:02:08] arch/powerpc/include/asm/atomic.h:194:44: error: invalid output constraint '=YZ<>' in asm
[00:02:08]   194 |                 __asm__ __volatile__("std%U0%X0 %1,%0" : "=" DS_FORM_CONSTRAINT (v->counter) : "r"(i));
[00:02:08]       |                                                          ^
[00:02:08] 2 errors generated.
[00:02:08] Traceback (most recent call last):
[00:02:08]   File "/usr/src/tmp/bcc-buildroot/usr/share/bcc/tools/cpudist", line 210, in <module>
[00:02:08]     b = BPF(text=bpf_text, cflags=["-DMAX_PID=%d" % max_pid])
[00:02:08]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[00:02:08]   File "/usr/src/tmp/bcc-buildroot/usr/lib64/python3/site-packages/bcc/__init__.py", line 480, in __init__
[00:02:08]     raise Exception("Failed to compile BPF module %s" % (src_file or "<text>"))
[00:02:08] Exception: Failed to compile BPF module <text>

This test was working OK on Linux v6.10.6 (I'm not implying that kernel version difference is the cause of failure though).

This architecture is low priority for us, thus I'm just disabling build of bcc package on ppc64le for ALT Linux, but this failure might interesting for you, so I'm reporting it here.

[yonghong-song]

Thanks for reporting.
I briefly checked the source. The reason is due to the following static inline function in atomic.h

static __inline__ s64 arch_atomic64_read(const atomic64_t *v)
{
        s64 t;

        /* -mprefixed can generate offsets beyond range, fall back hack */
        if (IS_ENABLED(CONFIG_PPC_KERNEL_PREFIXED))
                __asm__ __volatile__("ld %0,0(%1)" : "=r"(t) : "b"(&v->counter));
        else
                __asm__ __volatile__("ld%U1%X1 %0,%1" : "=r"(t) : DS_FORM_CONSTRAINT (v->counter));

        return t;
}

static __inline__ void arch_atomic64_set(atomic64_t *v, s64 i)
{
        /* -mprefixed can generate offsets beyond range, fall back hack */
        if (IS_ENABLED(CONFIG_PPC_KERNEL_PREFIXED))
                __asm__ __volatile__("std %1,0(%2)" : "=m"(v->counter) : "r"(i), "b"(&v->counter));
        else
                __asm__ __volatile__("std%U0%X0 %1,%0" : "=" DS_FORM_CONSTRAINT (v->counter) : "r"(i));
}

They are supposed to be removed during arch specific compilation (powerpc) since nobody uses those functions but apparently they are still there for later bpf backend code generation.

I do not have environments to debug this and hopefully somebody else could help take a look if possible.

[dubeyabhishek]

In arch/powerpc/include/asm/asm-compat.h, we have ::
#ifdef CONFIG_CC_IS_CLANG
#define DS_FORM_CONSTRAINT "Z<>"
#else
#define DS_FORM_CONSTRAINT "YZ<>"
#endif

Can we compile the kernel headers in bcc setting CONFIG_CC_IS_CLANG, we get correct DS_FORM?

[yonghong-song]

You can add -DCONFIG_CC_IS_CLANG in cflags of BPF constructor like below.

b = BPF(text=bpf_text, cflags=['-DMAX_CPUS=%s' % str(num_cpus)])

[hramrach]

These are kernel-internal headers which are configured for use with the compiler that was used to build the kernel, and for that compiler YZ<> is the correct constraint.

However, bcc pulls that kernel-internal header into a different project, built with a different compiler. That's why the build fails. This is outside of what the kernel build system supports.

[shunghsiyu]

Would it make sense to define CONFIG_CC_IS_CLANG as part of KBuildHelper::get_flags()? There's quite a few Clang-specific workaround wrapped with such macro, and defining it once instead of per effect scripts seems like the easier way out.