M1 Support

[shepting]

This tracking ticket can state the current issues blocking builds on M1 (Apple Silicon) machines.

Currently, when running on an M1 machine we will get output like this:

╰─ ~/.buck/buck build //ios:AirbnbApp
Unable to connect to Buck daemon, restarting it...
Not using buckd because daemon failed to start.
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.facebook.buck.cli.bootstrapper.ClassLoaderBootstrapper.main(ClassLoaderBootstrapper.java:55)
Caused by: java.lang.UnsatisfiedLinkError: /Users/stevenhepting/workspace/apps/buck-out/tmp/buck_run.82_1dr8k/jna-1265810726/jna2060014269842800161.tmp: dlopen(/Users/stevenhepting/workspace/apps/buck-out/tmp/buck_run.82_1dr8k/jna-1265810726/jna2060014269842800161.tmp, 0x0001): tried: '/Users/stevenhepting/workspace/apps/buck-out/tmp/buck_run.82_1dr8k/jna-1265810726/jna2060014269842800161.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/jna2060014269842800161.tmp' (no such file)

I believe the relevant snippet is (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e'))

[OleksiyA]

Workaround is to install buck using Rosetta terminal. Afterwards buck can be used from any terminal as usual.

Follow section Configuring Rosetta for ARM64 MacBook https://github.com/valhalla/valhalla#configuring-rosetta-for-arm64-macbook .
After that use Rosetta terminal with ibrew and follow section Build from Source from https://buck.build/setup/getting_started.html .

[EvenChang]

Excuse me, I have follow your step and also have same error msg.
Does anyone knows?

MacBook-Pro:buck even$ arch
i386
MacBook-Pro:buck even$ ./bin/buck build
Unable to connect to Buck daemon, restarting it...
Not using buckd because daemon failed to start.
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.facebook.buck.cli.bootstrapper.ClassLoaderBootstrapper.main(ClassLoaderBootstrapper.java:55)
Caused by: java.lang.UnsatisfiedLinkError: /Users/even/Library/Caches/JNA/temp/jna1285587138928287604.tmp: dlopen(/Users/even/Library/Caches/JNA/temp/jna1285587138928287604.tmp, 0x0001): tried: '/Users/even/Library/Caches/JNA/temp/jna1285587138928287604.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/jna1285587138928287604.tmp' (no such file)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
	at java.lang.Runtime.load0(Runtime.java:810)
	at java.lang.System.load(System.java:1088)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:1018)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:988)
	at com.sun.jna.Native.<clinit>(Native.java:195)
	at com.facebook.buck.cli.MainRunner.<clinit>(MainRunner.java:287)
	at com.facebook.buck.cli.AbstractMain.prepareMainRunner(AbstractMain.java:147)
	at com.facebook.buck.cli.MainWithoutNailgun.main(MainWithoutNailgun.java:56)
	... 5 more

[gabrieldonadel]

Excuse me, I have follow your step and also have same error msg. Does anyone knows?

That's because you should also use an x64 JDK to build

[hendych]

Using rosetta installation to build buck.pex, will it able to run on native arm64 CPU or we still need to use rosetta to run?

I'm building using rosetta and when using native arm64, it's still having error (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e'))

Is there any way we can define in the compiler to also build specific CPU arch?

[marcesengel]

I succeeded doing a local build after updating a few dependencies, will create a PR later today 👍

[marcesengel]

@ajanuar I published my PR, feel free to try it :) but be aware, Java 8 support is dropped due to upstream changes.

[hendych]

Hi @marcesengel, were you able to build using iphonesimulator-arm64?
I got this error when running it in some of my appex

Error scrubbing non-deterministic metadata from /Users/xxx/Widget#iphonesimulator-arm64
com.facebook.buck.io.file.FileScrubber.ScrubException: String table does not end at end of file
com.facebook.buck.io.file.FileScrubber$ScrubException: String table does not end at end of file
	at com.facebook.buck.cxx.toolchain.objectfile.OsoSymbolsContentsScrubber.scrubFile(OsoSymbolsContentsScrubber.java:41)
	at com.facebook.buck.step.fs.FileScrubberStep.execute(FileScrubberStep.java:58)
	at com.facebook.buck.step.StepRunner.runStep(StepRunner.java:58)
	at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.executeCommands(CachingBuildRuleBuilder.java:1417)
	at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.runWithDefaultExecutor(CachingBuildRuleBuilder.java:1394)
	at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submit$2(WeightedListeningExecutorService.java:100)
	at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submitWithSemaphore$0(WeightedListeningExecutorService.java:74)
	at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:206)
	at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:195)
	at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:115)
	at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:999)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

[marcesengel]

Hi @hendych, actually you getting this error is good news for me 😅 I got the same error building react-native and assumed it was a problem with their repo. Seems like one of the version bumps broke the FileScrubber 🤔 I will try to look into it next week, do you think you could have a look as well?

Edit: to be completely honest with you I haven't ran the tests and assumed all was fine when I was able to build Buck on my M1. In case there are failing tests this could make fixing all of this a lot easier. If there aren't any I suppose we should add one.

[hendych]

Hi @hendych, actually you getting this error is good news for me 😅 I got the same error building react-native and assumed it was a problem with their repo. Seems like one of the version bumps broke the FileScrubber 🤔 I will try to look into it next week, do you think you could have a look as well?

Edit: to be completely honest with you I haven't ran the tests and assumed all was fine when I was able to build Buck on my M1. In case there are failing tests this could make fixing all of this a lot easier. If there aren't any I suppose we should add one.

I saw someone having the same issue in Buck slack channel without any solution though 😅. This issue should have nothing to do with the version bump of the third party libraries because this issue exist since building buck with rosetta and iphonesimulator-arm64.

Did check it before. I think there is something should be changed in the FileScrubber code. I think this related to the symbol generation in linking phase or something. Tried to use FileScrubber on the dev branch but I think the changes is too far with master. Didn't have the chance to look further due to another project. But will do after the project.

[Solace-Studios]

@marcesengel @hendych Any further luck using buck on M1?

[gabrieldonadel]

@marcesengel @hendych Any further luck using buck on M1?

@Solace-Studios I got this working on my M1, the most important part is to make sure you're using an x64 JDK and brew running with roseta

[hendych]

@marcesengel @hendych Any further luck using buck on M1?

@Solace-Studios I had applied code changes by @marcesengel, now I'm using Buck on M1 but cannot get it working with iphonesimulator-arm64. It can run buck CLI using native M1 but cannot compile and run tests using arm64 simulator. As workaround, I don't use buck test CLI, but using xctestrunner with iponesimulator-x86_64. And now we're considering whether to continue using Buck or migrate to Bazel 🤔 so I stopped looking for the appropriate solution atm.

@Solace-Studios I got this working on my M1, the most important part is to make sure you're using an x64 JDK and brew running with roseta

@gabrieldonadel Can you run tests on arm64 simulator?

[gabrieldonadel]

@gabrieldonadel Can you run tests on arm64 simulator?

@hendych I only use buck build TBH but it runs without any issues

[marcesengel]

Hi @marcesengel, were you able to build using iphonesimulator-arm64? I got this error when running it in some of my appex

Error scrubbing non-deterministic metadata from /Users/xxx/Widget#iphonesimulator-arm64
com.facebook.buck.io.file.FileScrubber.ScrubException: String table does not end at end of file
com.facebook.buck.io.file.FileScrubber$ScrubException: String table does not end at end of file
	at com.facebook.buck.cxx.toolchain.objectfile.OsoSymbolsContentsScrubber.scrubFile(OsoSymbolsContentsScrubber.java:41)
	at com.facebook.buck.step.fs.FileScrubberStep.execute(FileScrubberStep.java:58)
	at com.facebook.buck.step.StepRunner.runStep(StepRunner.java:58)
	at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.executeCommands(CachingBuildRuleBuilder.java:1417)
	at com.facebook.buck.core.build.engine.impl.CachingBuildRuleBuilder$BuildRuleSteps.runWithDefaultExecutor(CachingBuildRuleBuilder.java:1394)
	at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submit$2(WeightedListeningExecutorService.java:100)
	at com.facebook.buck.util.concurrent.WeightedListeningExecutorService.lambda$submitWithSemaphore$0(WeightedListeningExecutorService.java:74)
	at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:206)
	at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:195)
	at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:115)
	at com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:999)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

IIRC I got the same error after the build was successful...

@marcesengel @hendych Any further luck using buck on M1?

@Solace-Studios I got this working on my M1, the most important part is to make sure you're using an x64 JDK and brew running with roseta

My goal was to actually have it run natively, but no luck as of now.