Pr/fasan multithreading fixes upstream #2955

mkravchik · 2025-02-08T19:14:03Z

This PR makes it possible to use Frida ASAN in realistic Windows multithreaded applications. On Windows, an application is created by default with a thread pool, and many system APIs invoke COM behind the scenes as a part of their normal functionality. COM can initiate a lot of activity on these thread pool threads without any direct request from the harness.
The main highlights of the PR are:

Mutex-protected ASAN allocator.
More rigorous hook reentrancy protection, including environments without TLS (which happens very early and very late in the thread lifecycle).
Correct initialization of Stalker, which excludes the fuzzer itself from being Stalkered (which is dangerous and completely unnecessary).
Correct hooking of some required C runtime functions (e.g., memcpy).
Static linking of the harness with C runtime, required for preventing Frida to hook its own calls to C runtime (e.g., memcpy, memcmp, etc.).
Proper handling of Map/UnmapViewOfFile
Unit test fix
Removing Frida hooks and ASAN upon crashing before running other observers. This is necessary as collecting module symboling information for minibsod can easily cause OOM under ASAN. For this purpose, a new observer was introduced that should be the first in the chain.

Known limitations:

No extensive testing on Linux/MacOS. The existing fuzzers and unit test work on Linux.
I don't know how to link statically with C runtime on Linux (and haven't looked at MacOS, because I don't have one). glibc can't really be linked statically (because its TLS functionality is not in the static library) and musl has no cpp...
In order to have correct coverage in the multithreaded environment, more changes are required (such as per-thread coverage map). This goes beyond my current scope of work....

…fFile

…o add the observer, clippy, fmt, and at least linux compilation

…b functions. Only frida_windows_gdiplus tested. Linux not tested

…k on Linux.

…ub.com/mkravchik/LibAFL into pr/fasan-multithreading-fixes-upstream

addisoncrump · 2025-02-11T17:19:14Z

Okay, I think I figured it out but it required digging around a lot in the guts of libafl_targets. Hopefully still works...

addisoncrump · 2025-02-11T18:00:04Z

I see, CI doesn't run if conflicts are present 🤦

addisoncrump · 2025-02-11T18:43:04Z

Seems to work. @mkravchik see diff when you get a chance, the trick was to just not use msvc but instead gnullvm and not specify this in the .cargo/config.toml.

mkravchik · 2025-02-11T19:11:21Z

@addisoncrump - thanks a lot! Indeed, lots of black magic involved.

tokatoka · 2025-02-12T10:18:43Z

.github/workflows/fuzzer-tester-prepare/action.yml

@@ -12,7 +12,7 @@ runs:
      with: { shared-key: "${{ runner.os }}-shared-fuzzer-cache" }
    - name: Install fuzzers deps
      shell: bash
-      run: sudo apt-get update && sudo apt-get install -y nasm nlohmann-json3-dev gcc-aarch64-linux-gnu g++-aarch64-linux-gnu gcc-mipsel-linux-gnu g++-mipsel-linux-gnu gcc-powerpc-linux-gnu g++-powerpc-linux-gnu libc6-dev-i386-cross libc6-dev libc6-dev-i386 lib32gcc-11-dev lib32stdc++-11-dev libgtk-3-dev pax-utils python3-msgpack python3-jinja2 
+      run: sudo apt-get update && sudo apt-get install -y nasm nlohmann-json3-dev gcc-aarch64-linux-gnu g++-aarch64-linux-gnu gcc-mipsel-linux-gnu g++-mipsel-linux-gnu gcc-powerpc-linux-gnu g++-powerpc-linux-gnu libc6-dev-i386-cross libc6-dev libc6-dev-i386 lib32gcc-11-dev lib32stdc++-11-dev libgtk-3-dev pax-utils python3-msgpack python3-jinja2 g++-mingw-w64


@addisoncrump
we need this?
but it is strange that it fixed your problem. this action file is not used by windows vm at all

(including frida_gdi_plus vm, the ci that failed before this change of yours)

now i understand that build is also running on ubuntu but it's useless.
we removed it in #2949

can you delete it?

and after we merge #2949 then that should be gone.
or you can just ignore that error we'll remove that anyway

tokatoka · 2025-02-12T15:27:50Z

libafl_targets/src/libfuzzer.c

@@ -47,7 +47,7 @@ EXT_FUNC_IMPL(main, int, (int argc, char **argv), false) {
  #endif
 }

-  #if defined(_WIN32)
+  #if defined(_WIN32) && !defined(__clang__)


not sure if this doesn't break libfuzzer_windows_asan.
i added the tests in #2961 let's see

mkravchik · 2025-02-13T10:09:15Z

Is there anything you need on my side to merge this?

tokatoka · 2025-02-13T10:31:15Z

no but for now i want #2949 to be merged and bring those tests in that PR to see if it works on this branch too

tokatoka · 2025-02-13T10:40:37Z

in my opinion probably better to change !defined(__clang__) to defined(_MSC_VER) instead. i don't think current code in libafl-targets works with other compilers

tokatoka · 2025-02-13T11:32:54Z

On which platform are you building this frida-gdiplus stuff with? You are using windows no?

tokatoka · 2025-02-13T11:33:00Z

@mkravchik

tokatoka · 2025-02-13T11:39:38Z

@mkravchik

Can you delete g++-mingw-w64 dependency that was added in .github/workflows/fuzzer-tester-prepare/action.yml in your branch?

Can you revert this
https://github.com/mkravchik/LibAFL/blob/pr/fasan-multithreading-fixes-upstream/libafl_targets/src/libfuzzer.c#L50
back to #if defined(_MSC_VER) and see if it works for you?

Also please test frida_windows_gdiplus on windows, you should not run or test this on Linux

mkravchik · 2025-02-13T12:04:49Z

Hi, I test on Windows, of course. The failure on Linux was fuzzer_test that ran clippy. only.
These were the changes added by @addisoncrump, if you get rid of clippy on Linux they are not necessary, I guess.
I will check today

This reverts commit 706c272.

This reverts commit 1d7c5d4.

tokatoka · 2025-02-14T10:33:54Z

libafl_targets/src/cmplog.c

@@ -15,7 +15,7 @@ void *__libafl_asan_region_is_poisoned(void *beg, size_t size) {
  return NULL;
 }

-  #if defined(__clang__)
+  #if defined(__clang__) && defined(_MSC_VER)


for this you don't have to revert

tokatoka · 2025-02-14T10:35:01Z

@mkravchik
for common.h and cmplog.c don't revert it.. (i.e. don't make any change)

tokatoka · 2025-02-14T11:33:41Z

it's not reverted.
can you just run git checkout origin/main -- libafl_targets/src ?

mkravchik · 2025-02-14T12:28:40Z

Here you go.

mkravchik added 30 commits January 10, 2025 06:24

Fixing the test_harness library name

766c908

Fasan works, but testing of all features is pending

ea9a3ab

Tests pass, before fixing clippy and fmt

715d6d8

CLippy+fmt

d2c675d

CLippy+fmt+tests running on linux

03940d8

Clippy

f8929c0

Not stalkering the fuzzer. In the correct way

e425797

Removing the instrumentation upon crash. Proper hooking of UnmapViewO…

20a7516

…fFile

Fixes after the merge from the upstream (before 0.15.0). Still need t…

edaf21f

…o add the observer, clippy, fmt, and at least linux compilation

Adding the helper observer and using it in the test

523aa3b

Removing the observer from the wrong location

c1f1d61

Adapting to the new helper ownership model

bda1c68

Adding an observer to shut down instrumentation upon crash

599ea9f

Clippy + fmt

19b6a43

Merge with upstream/main at 0.15.0. Dependent on the fix of frida gli…

dfe7f90

…b functions. Only frida_windows_gdiplus tested. Linux not tested

Using mimalloc everywhere

b3897fb

Deactivating before activating with the harness. Otherwise, gets stuc…

2b61856

…k on Linux.

Fixing imports for windows

c1fe491

Using the new way of passing the handler

3df9ccd

Using frida_helper_shutdown_observer

53ff249

Clippy+fmt

bd5e193

no-std, clippy

7013144

Merge branch 'pr/fasan-multithreading-fixes-upstream' of https://gith…

f882683

…ub.com/mkravchik/LibAFL into pr/fasan-multithreading-fixes-upstream

Fmt

1033f0a

Merge branch 'pr/fasan-multithreading-fixes-upstream' of https://gith…

be7daf0

…ub.com/mkravchik/LibAFL into pr/fasan-multithreading-fixes-upstream

Stable thread_id

2ba593d

Clippy 18

a5eb759

More clippy

55609bd

Merge remote-tracking branch 'upstream/main' into main

bb48783

Formatting toml

313a287

addisoncrump added 2 commits February 11, 2025 18:20

lots of digging around, pray for us

706c272

Merge branch 'main' into pr/fasan-multithreading-fixes-upstream

8f91ac1

fixup?

1d7c5d4

s1341 mentioned this pull request Feb 12, 2025

Frida windows thread local #2433

Closed

tokatoka reviewed Feb 12, 2025

View reviewed changes

Merge branch 'main' into pr/fasan-multithreading-fixes-upstream

2640ddc

Merge branch 'main' into pr/fasan-multithreading-fixes-upstream

61908b8

mkravchik added 3 commits February 14, 2025 01:41

Revert "lots of digging around, pray for us"

185d5e0

This reverts commit 706c272.

Revert "fixup?"

1110f6f

This reverts commit 1d7c5d4.

Revert artifact

0c2a6c3

tokatoka reviewed Feb 14, 2025

View reviewed changes

mkravchik added 2 commits February 14, 2025 12:46

Revert fixups

8f716aa

Removing unused

a4be351

Reverting to upstream/main

c8089ad

tokatoka merged commit b3fe744 into AFLplusplus:main Feb 14, 2025
106 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pr/fasan multithreading fixes upstream #2955

Pr/fasan multithreading fixes upstream #2955

mkravchik commented Feb 8, 2025

addisoncrump commented Feb 11, 2025

addisoncrump commented Feb 11, 2025

addisoncrump commented Feb 11, 2025

mkravchik commented Feb 11, 2025

tokatoka Feb 12, 2025

tokatoka Feb 12, 2025 •

edited

Loading

tokatoka Feb 12, 2025 •

edited

Loading

tokatoka Feb 12, 2025

mkravchik commented Feb 13, 2025

tokatoka commented Feb 13, 2025

tokatoka commented Feb 13, 2025 •

edited

Loading

tokatoka commented Feb 13, 2025

tokatoka commented Feb 13, 2025

tokatoka commented Feb 13, 2025 •

edited

Loading

mkravchik commented Feb 13, 2025

tokatoka Feb 14, 2025

tokatoka commented Feb 14, 2025

tokatoka commented Feb 14, 2025

mkravchik commented Feb 14, 2025

Pr/fasan multithreading fixes upstream #2955

Pr/fasan multithreading fixes upstream #2955

Conversation

mkravchik commented Feb 8, 2025

addisoncrump commented Feb 11, 2025

addisoncrump commented Feb 11, 2025

addisoncrump commented Feb 11, 2025

mkravchik commented Feb 11, 2025

tokatoka Feb 12, 2025

Choose a reason for hiding this comment

tokatoka Feb 12, 2025 • edited Loading

Choose a reason for hiding this comment

tokatoka Feb 12, 2025 • edited Loading

Choose a reason for hiding this comment

tokatoka Feb 12, 2025

Choose a reason for hiding this comment

mkravchik commented Feb 13, 2025

tokatoka commented Feb 13, 2025

tokatoka commented Feb 13, 2025 • edited Loading

tokatoka commented Feb 13, 2025

tokatoka commented Feb 13, 2025

tokatoka commented Feb 13, 2025 • edited Loading

mkravchik commented Feb 13, 2025

tokatoka Feb 14, 2025

Choose a reason for hiding this comment

tokatoka commented Feb 14, 2025

tokatoka commented Feb 14, 2025

mkravchik commented Feb 14, 2025

tokatoka Feb 12, 2025 •

edited

Loading

tokatoka Feb 12, 2025 •

edited

Loading

tokatoka commented Feb 13, 2025 •

edited

Loading

tokatoka commented Feb 13, 2025 •

edited

Loading