Thoughts on PrimJS

[ammarahm-ed]

So Bytedance open sourced their fork of QuickJS called PrimJS.

• It has template based interpreter (optionally enabled)
• Tracing GC
• Debugging support with Chrome DevTools

I ran some benchmarks to see how fast it is on my macbook m4 and I see it is 9-10% faster. What are your thoughts on it.

• It does not support as many es features as quickjs-ng by the way. I am wondering if we can back port stuff here. They did change a lot of things but fundamentally it's quickjs.
• Some of the things like Debugger, NAPI bindings are really nice

[saghul]

Oh I hadn't seen it, I'll check it out. Thanks for sharing!

[bnoordhuis]

I had a Chinese guy ask me probing questions about quickjs internals a while ago. Guess I know why now :-)

See discussion about template JITs in #659. tl;dr not dead set against but not a huge game changer without further work.

Their RC vs. GC docs lay out the pros and cons fairly. I'm not a huge fan of RC (error prone) but constrained environments are a design consideration for quickjs and RC beats out GC handily there.

napi and devtools: if we can crib that from them, let's, although it's probably an uphill battle if they're relying on GC.

[ammarahm-ed]

I ran some benchmarks to check how fast it gets compared to NG and I saw about 9-10% improvement on octane. The score was around 4700 on my M4 Macbook Pro. I get the same score if I use mimalloc with NG.

• it's using 2019 version of bellard/quickjs so missing a lot of es6 apis, the column number support is kinda buggy which is fixed in NG.
• The NAPI implementation is nice. The Napi layer is not spec compliant though. I internally wrote node-api bindings for quickjs btw located here: https://github.com/NativeScript/napi-android/blob/main/test-app/runtime/src/main/cpp/napi/quickjs/quickjs-api.c that are fairly complete but not well tested.
• Devtools is something actually useful though and worth taking from them.
• Most features they have added are behind feature flags which makes it easy to track what to do

It's too new I guess, let's see where they take it to and whether are open to fixing and upgrading it.

Comparing all of it to Static Hermes which has a score of 8000+, and Hermes having a score of 6500 something without JIT.

[bnoordhuis]

Their readme says this about octane:

[..] PrimJS outperforms QuickJS by approximately 28% [..]

Which sounds about right for bytecode dispatch overhead. If it's really just 10%, we may be able to close the gap simply by making the interpreter loop easier to optimize for compilers.

I've noticed (and been meaning to investigate) that in the not too distant past something happened that caused the generated release mode machine code for JS_CallInterval to grow from approx. 37k to 64k (edit: turned out to be a buggy cc wrapper, pfew)

I'll go track down if that's caused by an internal (e.g. code change) or external change (e.g. different compiler.)

[ammarahm-ed]

i just ran the benchmarks again (full). It indeed is 27-28% faster than QuickJS-NG. QuickJS-NG also appears to be slower than QuickJS-Bellard by 200 points.

NG:

Richards: 1546
DeltaBlue: 1448
Crypto: 1827
RayTrace: 1099
EarleyBoyer: 2328
RegExp: 385
Splay: 2521
SplayLatency: 9960
NavierStokes: 3461
PdfJS: 5688
Mandreel: 1536
MandreelLatency: 10837
Gameboy: 12354
CodeLoad: 20643
Box2D: 5970
Typescript: 19662
----

NG-Mimalloc

Richards: 1566
DeltaBlue: 1653
Crypto: 1834
RayTrace: 2349
EarleyBoyer: 4087
RegExp: 497
Splay: 4922
SplayLatency: 16607
NavierStokes: 3458
PdfJS: 6901
Mandreel: 1535
MandreelLatency: 10909
Gameboy: 12447
CodeLoad: 31842
Box2D: 6733
Typescript: 24305
----
Score (version 9): 4651

Bellard

Richards: 1629
DeltaBlue: 1427
Crypto: 2060
RayTrace: 1122
EarleyBoyer: 2361
RegExp: 397
Splay: 1995
SplayLatency: 8367
NavierStokes: 4360
PdfJS: 5791
Mandreel: 2030
MandreelLatency: 15339
Gameboy: 14352
CodeLoad: 22045
Box2D: 5987
Typescript: 19910
----
Score (version 9): 3866

PrimJs

Richards: 1770
DeltaBlue: 1826
Crypto: 3345
RayTrace: 3548
EarleyBoyer: 4508
RegExp: 543
Splay: 5521
SplayLatency: 2445
NavierStokes: 5948
PdfJS: 9859
Mandreel: 2049
MandreelLatency: 13925
Gameboy: 15710
CodeLoad: 26167
Box2D: 7881
Typescript: 29488
----
Score (version 9): 5089

It is indeed quite fast.

[bnoordhuis]

QuickJS-NG also appears to be slower than QuickJS-Bellard by 200 points

Yes, that's known. We include a couple of correctness fixes that unfortunately slow down the interpreter a bit.