RJS-2101: "Build optimizations" for iOS and Android

[kraenhansen]

What, How & Why?

Important

With this change, our SDK is no longer pinned to a specific ABI of JSI and we expect the next release of our SDK (after merging this PR) to be backwards and forwards compatible with many past versions of React Native.

This is my friendly takeover of #6396, by reverting the compilation of Realm Core from source and instead distributing prebuilds via our NPM package:

• For React Native Apple (npm run prebuild-apple --workspace realm):
  • Prebuild format is an XCFramework of Realm Core
  • Deferred compilation on the developers machine is our binding (between Realm Core and React Native Core & JSI) driven by CocoaPods & XCode.
• For React Native Android (npm run prebuild-android --workspace realm):
  • Prebuilds format is a CPack install directory (containing CMake config files)
  • Deferred compilation on the developers machine is our binding (between Realm Core and React Native Core & JSI) driven by Gradle calling into CMake.
• For Node.js we're able to rely on the ABI stability of N-API and we produce prebuilds containing both Realm Core and our binding (between Realm Core and N-API) (npm run prebuild-node --workspace realm).

This also moves all binding related code (prebuilt or not), including that previously in packages/realm/react-native into packages/realm/binding in platform specific directories.

We won't support compiling our prebuilds from from the NPM archive as users still need to git clone our mono-repository to produce a build from source.

It's worth mentioning that this PR introduce an internal CLI incapsulating the complexity of invoking the CMake scripts for Android and Apple platforms. I imagine we could extend this to wrap cmake-js to provide a single cohesive experience for building our prebuilds in the future: This is a replacement for the packages/realm/scripts for Android and iOS that we currently use.

---

Here's a bit more context on this change and why we choose the approach we did:

As we started implementing out "React Native SDK Build Optimizations" project we realized a few things:

• Although we are generating 11K lines of C++ from the binding generator, it’s very fast to compile - most likely because it’s a single translation unit.
• This generated binding layer is the only code dependent on the React Native headers / ABI.
• We need Cmake to generate the Xcode project to compile Realm Core, which might not be installed on an app developers machine (in the correct version) but we can compile the binding layer without it.
• If we choose to leave out the binding layer from the prebuilt binary (and we can because of ☝️) we don’t need to produce separate prebuilds for each React Native version.
• Because of ☝️ we actually don’t have to delay the download of the prebuilt until “pod install” nor build-time. We could actually include it as an xcframework into the archive that we upload to NPM, like we do now. The main drawback of this approach is the download time for users as they “npm install” our package, but that won’t be a regression in DX to what we currently have and it doesn’t seem to be a deal-breaker to anyone as-is.
• For Android, the situation is slightly more complicated, since Realm Core is dependent on ABI of the C++ Standard Library brought in by the NDK. Since the NDK version is configurable by the developer, this becomes a bit more complicated. Luckily the React Native template app (and Expo) declares a default for this NDK version, which we could rely on for the prebuild we upload to NPM, as long as we take great care to error if developers try using this prebuild with an unexpected NDK version and provide an easy way to build Realm Core from source, for those developers choosing a different NDK than the default.

In the context of the above, if we choose to include prebuild binaries for iOS and Android of Realm Core in the NPM package, which will be ABI independent on the React Native version (except for the caveat of NDK version, which has a default that I'd expect 95+% to leave unchanged) ... do we actually need to support compiling from source when installing the realm package via NPM as the original project scope suggests? I'd expect the need for that to be very rare (as it is right now) and it does come with drawbacks:

• We'll have to avoid hoisting shared dev-dependencies to the root package of our mono-repo
• Depending on a decision of forcing developers that wants to use this to have to npm install in their node_modules/realm we might need to include these dev-dependencies as actual dependencies of realm. Which would add to the install time of our package.
• We'll most likely have to run tests to ensure all relevant dependencies are actually available to build from source when the realm package is installed outside of the mono-repo.
• We'll have to include all of the Realm Core source-code into the repository, not a huge deal - but it all adds up.

The alternative is to ask developers that want to consume Realm JS and Realm Core from source to perform a checkout of the repository, run the relevant build script and npm pack the SDK package themselves.

☑️ ToDos

• 📝 Changelog entry
• Include before / after size of the NPM package.
• 📝 Compatibility label is updated or copied from previous entry
• 📝 Update COMPATIBILITY.md
• 🚦 Tests
• 📦 Updated internal package version in consuming package.jsons (if updating internal packages)
• 📱 Check the React Native/other sample apps work if necessary
• 💥 Breaking label has been applied or is not necessary

[kraenhansen]

This was moved up: https://github.com/realm/realm-js/pull/6650/files#diff-63a513d26453362aa63e7f2a51f5e83c112a18561d1de82473faa5cf2bcbf0d8R448-R453

[kraenhansen]

These were moved into a separate Android specific prebuild-android job.