Implementing debugger

[moretea]

I'm willing to take a stab at implementing debugger support as tracked in #119.

Are the maintainers interested in accepting a PR implementing a debugger? If so, I'd appreciate early feedback on the approach I plan to take, as my C is a bit rusty (pun intended, write most systems code in rust).

Overall plan

• Look at the existing implementation in https://github.com/koush/quickjs
• Get consensus on few questions up front-ish to ensure that the PR's will be accepted
• Submit a series of PR's

Existing implementation by koush

I have taken a look at the implementation at https://github.com/koush/quickjs, which seems reasonable; it generates two dispatch tables, and if the debugger is active, switches to the dispatch table that. The transport code (e.g. that allows an external debugger to open a TCP connection and control the quickjs debugger) seems to not be 100% compatible with the Debug Adapter Protocol. It also seems that no automated tests are included.

Upfront question

1. Determine if the debugger should always be included, or should be behind a feature toggle
2. Would the approach of using two dispatch tables be acceptable?
3. I would prefer to add a set of tests. I started out writing a C test program, but am considering using the quickjs javascript test harnass (tests/assert.js). Given that these tests would want to manipulate the runtime, this would entail adding a C module to expose those functions to quickjs itself. Would that be an acceptable route for writing debugger tests? If so, I would propose to create a new C module, qjs:qjs.

Questions to answer later
I'll do some prototyping and more investigation. Later I would like to get consensus on:
5. The API between QuickJS and a debugger stub
6. The debugging semantic model (options); per runtime, per context; should we do the same as koush w.r.t showing multiple threads for each context?

[bnoordhuis]

Determine if the debugger should always be included, or should be behind a feature toggle

I think it should be behind a compile-time feature flag, for people that want a minimal build (ex. quickjs-libc.c)

Would the approach of using two dispatch tables be acceptable?

Only if it doesn't regress the common case, which I suspect it will. If it does, what could work is breaking out JS_CallInternal to a separate source file and include it twice in quickjs.c, one with the regular dispatch table, and one with the debug table.

I would prefer to add a set of tests [..] I would propose to create a new C module, qjs:qjs

I'd rather not add new public modules without careful consideration. It's no problem to add a C module for use in your JS tests though.

The debugging semantic model (options); per runtime, per context

Per context, right? That's the unit of execution.

[sponge]

Was curious if anyone had messed with porting over koush changes to ng directly. I was able to port over the changes made for debugger support fairly easily, but I believe I'm running into issues with line number calculation preventing breakpoints from working, and some odd behavior in stepping.

[saghul]

I haven't tried it myself, but I think it would be cool to see how we could integrate it.

[sponge]

Here's what it looks like if you're curious!

sponge@73b5d0b

I am using a vendored version of the library, so I chopped out the changed files and added it to a branch. Didn't do any testing on this specific branch yet (I'm also using Premake and not CMake, but I don't think there are any build system changes needed besides the new files.)

I've also only tested this on MacOS so far, but when I was using this patch on a different QuickJS fork, it was working on Windows. This is all very much hacked together to just work for my tinkering level of quality :)

I extracted the changes by diffing the original debugger against that version of QuickJS, and then moved those changes over to quickjs-ng change by change, fixing up a handful of compiler issues along the way. The added files were brought over directly, minus a couple of other compiler changes.

Some thoughts:

• The core changes are pretty minimal overall. Most of it is just new functions at the bottom of quickjs.c, and the swapped out opcode table as mentioned.
• Hitting pause and stepping through functions seems to work. Variable and stackframe displays seem to work.
• Breakpoints don't work. As far as I can tell this statement never returns true. last_line_num always seems to be the line number the function starts on: https://github.com/sponge/quickjs-ng/blob/debugger/quickjs.c#L56066
  • This is where I start to hit the edges of my experience, not understanding terribly how language interpreters work, and QuickJS itself. I've tried to step through where I think the relevant points are, and I think this is where the breakage is happening.
• I don't love how the only interface to the debugger seems to be streams of JSON strings. I don't love the VSCode extension, it seems to be slow for breakpoints to start triggering, and I have problems getting it to connect when I launch the debugger and the host at the same time. I was hoping that I'd just have a bunch of plain functions and structs to let me hack up a little debugger UI in Dear ImGui or something.
  • I'm sure the extension could be improved, but considering Debugger Support #119 has been sitting around for a year, it's probably not a super in demand feature :)
• There were a couple of comments in the original version talking about how columns weren't tracked. They seem to be now, so I was able to fix those-- as long as I did it correctly!
• The debugger code seems to rely on reusing JSValue and such as generic containers for native debugger state. A bit weird, not sure if the runtime does this anywhere else, but it seems to work so vOv