Rust LLVMIR causes errors after LTO on x86-32

[jrmuizel]

#[repr(C)]
pub struct UrlpPattern(pub *mut c_void);

#[unsafe(no_mangle)]
pub unsafe extern "C" fn square(num: UrlpPattern) -> *mut c_void {
    num.0
}

generates

define noundef ptr @square(ptr noalias nocapture noundef readonly byval([4 x i8]) align 4 dereferenceable(4) %num) unnamed_addr {
start:
  %_0 = load ptr, ptr %num, align 4
  ret ptr %_0
}

but

struct wrapper {
    void *p;
};


void *square(wrapper p) {
    return p.p;
}

generates

define dso_local i8* @square(wrapper)(i8* readnone returned %p.0) local_unnamed_addr {
entry:
  ret i8* %p.0
}

It seems like this mostly works because it compiles to the same thing x86-32 code:

square(wrapper):                      # @square(wrapper)
        mov     eax, dword ptr [esp + 4]
        ret

However, if the rust code is inlined into C++ with LTO we end up with an extra dereference.

We ran into this in Firefox here: https://bugzilla.mozilla.org/show_bug.cgi?id=1973805

rustc --version --verbose:

rustc 1.88.0 (6b00bc388 2025-06-23)

[workingjubilee]

@jrmuizel hello there are like 5 different x86-32 ABIs, which one do you mean? which OS specifically

[workingjubilee]

( 5 may be slightly high, but there's at least 3 distinct variations. )

[workingjubilee]

@jrmuizel Are you using lto="thin" or lto="fat"?

[workingjubilee]

@jrmuizel ...What version of clang are you using to generate that comparison? That style of LLVMIR, with the i8* pointer instead of the untyped pointer, hasn't been emitted by LLVM since clang 14.

[jrmuizel]

Sorry, I meant to paste the godbolt links:

https://rust.godbolt.org/z/9nhPnEozs
vs
https://godbolt.org/z/xe8Mh9q8a

[jrmuizel]

The godbolt link above is from clang 10. (not sure why that version)

We actually ran into this problem in the wild with what was likely clang 19:
https://godbolt.org/z/nf5eP78cb

We saw this on Linux-x86-32.

We also saw crashes on Windows32 but I didn't investigate so I'm not sure it was the same issue.

[jrmuizel]

We ran into this with thin LTO

[workingjubilee]

cc @dianqk Hello you've gotten some time to look at linking-related concerns.

@jrmuizel Is there any chance you can test to see if this repros with a recent nightly and our latest version of LLVM? The one in rust-lang/llvm-project, I mean, which is a submodule in this repo. Using the clang that builds from that, of course. We have recently had to land many patches revolving around linkage-related optimizations.

I am... not entirely sure there is a calling convention disagreement here. Or rather, I kind of want to believe Rust is emitting LLVMIR that follows the calling convention, here. All arguments are expected to be passed on the stack, and a stack-placed argument like this is usually indicated using byval ptr. A more bizarre example:

struct wrapper {
    void *a;
    void *b;
    void *c;
    void *d;
};


void *return_one_of_four_ptrs(wrapper ptrs) {
    return ptrs.a;
}

Clang is choosing to represent this as an arity-4 function:

define dso_local noundef ptr @return_one_of_four_ptrs(wrapper)(ptr readnone returned %ptrs.0, ptr nocapture readnone %ptrs.1, ptr nocapture readnone %ptrs.2, ptr nocapture readnone %ptrs.3) local_unnamed_addr {
entry:
  ret ptr %ptrs.0
}

Meanwhile Rust represents this like so:

#[repr(C)]
pub struct Wrapper {
    a: *mut c_void,
    b: *mut c_void,
    c: *mut c_void,
    d: *mut c_void,
}

#[unsafe(no_mangle)]
pub unsafe extern "C" fn return_one_of_four_ptrs(ptrs: Wrapper) -> *mut c_void {
    ptrs.a
}

Which results in this IR, basically the same as before:

define noundef ptr @return_one_of_four_ptrs(ptr noalias nocapture noundef readonly byval([16 x i8]) align 4 dereferenceable(16) %ptrs) unnamed_addr {
start:
  %_0 = load ptr, ptr %ptrs, align 4
  ret ptr %_0
}