Limitations of es4x?

[santo74]

What are the limitations of es4x, if any, in regards to compatibility with both node.js/npm and Vert.x?

In other words, can I do everything with es4x that can be done with node.js and can I install any node.js library in es4x or are there any known limitations to be aware of?
Regarding Vert.x, does es4x provide 100% of the official Vert.x functionality or are there any limitations to be aware of?
What about the different modules such as Vert.x Web, clustering, authentication/authorization, monitoring, service proxies, etc?

The reason I'm asking this is because we started with Vert.x (java) in the past but after some time we had to (re)write more and more services in node.js because of limitations or incompatibilities we encountered with our Vert.x java services at that time.
As a result we currently have some kind of a hybrid environment where about 50% is written in javascript (node.js) and the other 50% in java (Vert.x), all communicating over a customized NATS implementation because the eventbus bridge wasn't usable in this particular setup.
The java part is stuck at Vert.x v3.5 because of some changes in later versions that are incompatible with the customizations we had to implement in the NATS integration.

Therefore it would be a real lifesaver if we could migrate everything to es4x but before we can do that we have to know how compatible it is with both node.js and Vert.x

[pmlopes]

Hi @santo74, the answer to your question is not a yes or no. Let me start breaking down your questions:

What are the limitations of es4x, if any, in regards to compatibility with both node.js/npm and Vert.x?

In other words, can I do everything with es4x that can be done with node.js and can I install any node.js library in es4x or are there any known limitations to be aware of?

Short answer, no. es4x uses graalvm JS engine underneath. Just like graalvm-node does. The difference is that graaljs engine is purely an ECMAScript engine, think of it, like v8. There are no node specific features included. No native modules, no fs, net, http, etc... which means many npm modules will not work out of the box. Unless they are not depending on node specifics.

For example, modules that are meant to run on the browser have a high chance of working in es4x.

Regarding Vert.x, does es4x provide 100% of the official Vert.x functionality or are there any limitations to be aware of?

Yes. es4x relies on vert.x codegen feature to generate the .d.ts files. For the runtime, the exact same objects are exposed to the js engine, so all you can do in vert.x with say java you can do with es4x javascript.

What about the different modules such as Vert.x Web, clustering, authentication/authorization, monitoring, service proxies, etc?

There aren't any differences, except es4x enhances the vert.x Future type. This means that any method returning a future, in es4x can be seen as an async method. So you can use the nice features of ES2015 where you can await and keep your code simple and readable.

You don't have to. You can still go fill vert.x with the .onSuccess() / .onFailure() alternative if you so wish.

[santo74]

@pmlopes thanks for the detailed answer.
So on the side of Vert.x compatibility there don't seem to be any reasons for us to not switch to es4x.

Regarding the node.js compatibility however, I'm a bit confused tbh.
On the one hand I completely understand your explanation and the comparison with v8, but on the other hand the GraalVM Reference Manual states that

GraalVM provides an ECMAScript-compliant runtime to execute JavaScript and Node.js applications

and a bit further on the same page:

GraalVM is capable of executing unmodified Node.js applications. Applications can import npm modules, including native ones.

And since es4x runs / can run on top of GraalVM I would actually expect the compatibility is the same or at least similar.

Can you elaborate a little bit more on this?

[pmlopes]

Sure, let me try to keep it simple, when embedding graaljs into a java project, you get only the ECMA runtime. This means that there is no require or modules. So it's the project responsibility to define how modules are loaded. This is why es4x has a custom require function that emulates the behavior of nodejs.

Of course, we could have decided to invert the ownership. Let's say that we would bootstrap vertx from graalvm node then the compatibility would be maximized, however, there are a few caveats.

1. Your application would now have 2 event loops, 1 from nodejs and the second from vert.x
2. The interop between java and js would not be trivial. Es4x is doing some extra configuration to graal to make it possible for vert.x json types to be seen as JS json object, same for vert.x Futures to become JS Promises.
3. With multiple event loops, we open the door to run code on the wrong thread, which will crash Graaljs (crash because gralljs has a very strict single thread policy and will stop if a wrong thread is detected)
4. The event loop of node runs on libuv which is significantly slower than vert.x one.
5. Vert.x event loop scales across all available CPUs while node is always single core.

For all these reason we only use the engine.

Maybe as a cool project, we could try to do some research on inverting this. Some things would be needed. For example, vert.x event loop should be locked to a single core, this would avoid the threading concerns. Second we would need to work the way types are converted, but the current graal API only allows conversion hooks from polyglot to native, and we would need the reverse.

If all these features would be available, then yes, integration with NPM would be flawless.

To conclude, I'd say, you should look at es4x more like, deno, than node.

[santo74]

Ok, thanks for the clarification, that makes totally sense now.
Hopefully one day those missing features will be implemented so that full npm integration can become a reality ;-)

Anyway, despite those "limitations", es4x definitely seems like a great initiative and since it implements the full Vert.x feature set we will certainly evaluate it further in order to - hopefully - replace some of our current node.js / old Vert.x services.

I have one additional question though, since it's still a 0.x version:
Is es4x already used in production environments that you know of / is es4x production ready?

[pmlopes]

I'm still using a 0.x version because I was hoping to get the "missing" graal features in. However, it has been a couple of years and not that much progress has done in that area.

So, I'm pretty sure that our code is very stable, so I could do a 1.x release.

Regarding production usage. Sadly, I don't have any numbers or names. Like you, I get many reports of people considering it as an alternative to using java with vert.x, but I haven't heard about anyone publicly stating that their are using it in production.

[santo74]

Ok, sounds fair enough.
As said we'll definitely do some further evaluation.
Thanks!