Implement `read-string` & `read-file` by shantanu-sardesai · Pull Request #673 · jank-lang/jank

shantanu-sardesai · 2026-01-22T15:07:45Z

Implement read and it's siblings along with reader options.

shantanu-sardesai · 2026-01-22T15:58:14Z

My current implementation cause jank to crash for the following code:

(binding [clojure.core/*read-eval* true] (read-string {:eof false :read-cond :preserve} "#?(:jank)"))

This is because we throw parse_invalid_reader_conditional error which I think is not playing well with the error reporter.

Backtrace

Process 88216 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00000001838c2388 libsystem_kernel.dylib`__pthread_kill + 8
libsystem_kernel.dylib`__pthread_kill:
->  0x1838c2388 <+8>:  b.lo   0x1838c23a8    ; <+40>
    0x1838c238c <+12>: pacibsp
    0x1838c2390 <+16>: stp    x29, x30, [sp, #-0x10]!
    0x1838c2394 <+20>: mov    x29, sp
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00000001838c2388 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x00000001838fb848 libsystem_pthread.dylib`pthread_kill + 296
    frame #2: 0x00000001838049e4 libsystem_c.dylib`abort + 124
    frame #3: 0x00000001004d6d28 jank`jtl::do_assertion_panic(msg=0x000000016fdfcd58) at assert.cpp:12:5
    frame #4: 0x000000010057f7bc jank`jank::error::snippet::add(this=0x000000010db09870, body_source=0x000000010db6df30, n=0x000000010db6df10) at report.cpp:155:5
    frame #5: 0x0000000100582a90 jank`jank::error::plan::add(this=0x000000016fdfd348, body_source=0x000000010db6df30, n=0x000000010db6df10) at report.cpp:369:17
    frame #6: 0x000000010057f2ac jank`jank::error::plan::add(this=0x000000016fdfd348, n=0x000000010db6df10) at report.cpp:354:5
    frame #7: 0x000000010057f1e4 jank`jank::error::plan::plan(this=0x000000016fdfd348, e=jank::error_ref @ 0x000000016fdfcf00) at report.cpp:100:7
    frame #8: 0x000000010057f308 jank`jank::error::plan::plan(this=0x000000016fdfd348, e=jank::error_ref @ 0x000000016fdfcf38) at report.cpp:95:3
    frame #9: 0x0000000100582bf0 jank`jank::error::report(e=jank::error_ref @ 0x000000016fdfd3a0) at report.cpp:537:16
    frame #10: 0x000000010053cf7c jank`jank::util::print_exception(e=jank::error_ref @ 0x000000016fdfd4c8) at try.cpp:112:5
    frame #11: 0x0000000100005598 jank`jank::repl() at main.cpp:277:7
    frame #12: 0x000000010000356c jank`main::$_0::operator()(this=0x000000016fdfe21f, argc=2, argv=0x000000016fdfea30) const at main.cpp:414:9
    frame #13: 0x0000000100003138 jank`main::$_0::__invoke(argc=2, argv=0x000000016fdfea30) at main.cpp:361:60
    frame #14: 0x00000001004fe4d4 jank`jank_init_with_pch(argc=2, argv=0x000000016fdfea30, init_default_ctx='\0', pch_data=0x0000000000000000, pch_size=0, fn=(jank`main::$_0::__invoke(int, char const**) at main.cpp:361)) at c_api.cpp:1077:14
    frame #15: 0x00000001004fe318 jank`jank_init(argc=2, argv=0x000000016fdfea30, init_default_ctx='\0', fn=(jank`main::$_0::__invoke(int, char const**) at main.cpp:361)) at c_api.cpp:1026:12
    frame #16: 0x0000000100002af4 jank`main(argc=2, argv=0x000000016fdfea30) at main.cpp:361:10
    frame #17: 0x000000018355ab98 dyld`start + 6076

shantanu-sardesai · 2026-01-22T17:24:59Z

Probably caused due to #647.

Shall I just repackage our parser (compile-time) error into a runtime error (std::runtime_error) instead? Since anyways there will be no source info available, at least at the moment.

jeaye

Nice work, Shantanu. The overall direction is sane, but the implementation has a way to go to fit into the idioms of jank's C++.

compiler+runtime/src/cpp/jank/read/parse.cpp

compiler+runtime/src/cpp/jank/runtime/core.cpp

compiler+runtime/src/jank/clojure/core.jank

jeaye · 2026-01-22T19:00:44Z

Probably caused due to #647.

Shall I just repackage our parser (compile-time) error into a runtime error (std::runtime_error) instead? Since anyways there will be no source info available, at least at the moment.

This is from a failed assertion. Why did the assertion fail? What were the values? You are suggesting hacky workarounds, but we want to first understand the problem and then consider robust solutions. Hacky workarounds are a last resort.

shantanu-sardesai · 2026-01-25T12:37:06Z

@jeaye

#673 (comment)

The crash was also fixed, now it gives the same output as was reported in this issue (#647).

user=> (read-string {:eof false :read-cond :preserve} "#?(:jank)")
─ parse/invalid-reader-conditional ─────────────────────────────────────────────────────────────────
error: Invalid reader conditional.

─────┬──────────────────────────────────────────────────────────────────────────────────────────────
     │ /var/folders/xd/9v1dcg0d46l7d19830vq0lhc0000gn/T/jank-repl-WQl59K
─────┼──────────────────────────────────────────────────────────────────────────────────────────────
  1  │ (read-string {:eof false :read-cond :preserve} "#?(:jank)")
     │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Used here.
─────┴──────────────────────────────────────────────────────────────────────────────────────────────
╭ nil ─────────────────────────────────────────────────────────────────────────────────────────────╮
│Unable to map file: Unable to find module 'user'.                                                 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

shantanu-sardesai · 2026-01-25T12:42:43Z

Although part of the CI is busted.

shantanu-sardesai · 2026-01-25T13:00:04Z

Also, is there a way to know when read_string is being invoked by the jank compiler for C++ codegen and when it's being used by the end user? At the moment, clojure.core/*read-eval* could be disabled by binding the var and that could hinder the behaviour of codegen.

Or perhaps it won't matter? Since the clojure.core/*read-eval* var can be overwritten by the end user only after our compiler is done with codegen 🤔. Although the issue of initialising the clojure.core/*read-eval* with :unknown on start-up still remains (currently we don't support that, I've just placed a TODO there, perhaps I'll add a note about the potential for this issue there as well).

…est case

shantanu-sardesai · 2026-02-22T09:57:08Z

There is a new limitation encountered due to the :preserve mode in read-string and jank's choice on reader suppression.

Here is my summary of the problems with the complete context consolidated in a document.

shantanu-sardesai · 2026-02-28T08:21:06Z

I've updated the read-string implementation to adapt to our decision now. Things look great so far!

jeaye

Nice work, Shantanu.

book/src/differences-from-clojure.md

jeaye · 2026-02-28T18:36:44Z

compiler+runtime/src/cpp/jank/read/parse.cpp

+      return error::parse_invalid_reader_conditional(
+        { start_token.start, latest_token.end },
+        "Conditional read is not allowed by default. Set the :read-cond reader option to either "
+        ":preserve or :allow. Found a nil instead.");


I don't think Found a nil instead. is helpful here. I'm also not entirely sure if it's accurate, since we're working with a bool and not a runtime object.

The allow_reader_conditional boolean is set using the provided reader option (:read-cond) so for the end user this message is more actionable. What do you think?

I don't think Found a nil instead. is helpful here.

Removed.

compiler+runtime/src/cpp/jank/read/parse.cpp

compiler+runtime/src/cpp/jank/runtime/obj/reader_conditional.cpp

compiler+runtime/src/cpp/jank/runtime/context.cpp

jeaye · 2026-02-28T19:01:12Z

compiler+runtime/src/cpp/jank/runtime/context.cpp

+  object_ref context::read_string(jtl::immutable_string const &code,
+                                  object_ref const reader_opts,
+                                  u64 const nth_form)


There's a lot of code in here now, with very few comments. Some of the code is quite dense. Please do a comment pass for your new code, to see where we can explain more.

Not much to comment. Added in the few places I saw something worth adding.

jeaye

Nice job, Shantanu.

jeaye · 2026-03-05T16:04:49Z

compiler+runtime/src/cpp/jank/read/parse.cpp

+  processor::processor(lex::processor::iterator const &b,
+                       lex::processor::iterator const &e,
+                       object_ref const &extended_features,
+                       bool const &allow_reader_conditional,


We should not take bool by const &, since it's a primitive.

jeaye · 2026-03-05T16:05:32Z

compiler+runtime/src/cpp/jank/runtime/obj/reader_conditional.cpp

+  {
+  }
+
+  reader_conditional::reader_conditional(persistent_list_ref f, boolean_ref s)


These can be const.

shantanu-sardesai mentioned this pull request Jan 22, 2026

Reworked tagged literals #650

Closed

shantanu-sardesai force-pushed the issues/415-test-suite branch 3 times, most recently from 0699168 to 2c0fb6e Compare January 22, 2026 15:57

jeaye requested changes Jan 22, 2026

View reviewed changes

shantanu-sardesai force-pushed the issues/415-test-suite branch from e978520 to 0320155 Compare January 24, 2026 11:31

shantanu-sardesai marked this pull request as ready for review January 25, 2026 12:39

shantanu-sardesai requested a review from jeaye January 25, 2026 12:41

shantanu-sardesai changed the title ~~Implement read & read-string with opts~~ Implement read & siblings with opts Jan 25, 2026

shantanu-sardesai force-pushed the issues/415-test-suite branch from d401904 to 16df002 Compare January 27, 2026 14:04

shantanu-sardesai added 13 commits February 1, 2026 14:42

Implement read-string with opts

9293e94

Refactor to jank idiomatic code

974b786

Refactor existing read_string

10d9035

Add read_string overload for C++ codegen

46804c7

Minor clean-up

4a1edd2

Use runtime functions to access values

43c487c

Move *read-eval* to clojure.core/*read-eval*

74615ec

Fix eof behaviour

13027ac

Resolve clojure.core/*read-eval* at runtime instead

60b6df5

Minor clean-up

25f0e17

Add a CLI flag to disable read

632dbd2

Update codegen with forcefully_read_string

184ecc5

Merge remote-tracking branch 'origin/main' into issues/415-test-suite

f56992d

shantanu-sardesai force-pushed the issues/415-test-suite branch from 16df002 to 9285548 Compare February 13, 2026 16:28

shantanu-sardesai force-pushed the issues/415-test-suite branch from 393f31e to c55fad4 Compare February 15, 2026 07:19

shantanu-sardesai marked this pull request as draft February 15, 2026 07:20

jeaye mentioned this pull request Feb 16, 2026

[WIP] Add nREPL server and relevant supporting features #698

Merged

Add runtime::reader_conditional

5c2ea30

shantanu-sardesai force-pushed the issues/415-test-suite branch from 7f91c4b to 5c2ea30 Compare February 18, 2026 18:35

Fix splice non-sequencable values in unsupported reader conditional t…

85b8f9e

…est case

shantanu-sardesai added 3 commits February 27, 2026 19:50

Merge remote-tracking branch 'origin/main' into issues/415-test-suite

efe36bf

Construct the runtime::reader_conditional object

37092be

Preserve tagged literals in preserve mode

aeac301

shantanu-sardesai marked this pull request as ready for review February 28, 2026 08:21

shantanu-sardesai changed the title ~~Implement read & siblings with opts~~ Implement read-string & read-file Feb 28, 2026

Search for the reader opts provided features

e1ceb90

shantanu-sardesai force-pushed the issues/415-test-suite branch from 404d11c to 934d393 Compare February 28, 2026 10:30

shantanu-sardesai requested a review from jeaye February 28, 2026 11:19

Move reader opts validation to read-string

345e6a6

shantanu-sardesai force-pushed the issues/415-test-suite branch from 934d393 to 345e6a6 Compare February 28, 2026 11:50

Fix the reader_conditional behaviours

b996afa

jeaye requested changes Feb 28, 2026

View reviewed changes

shantanu-sardesai force-pushed the issues/415-test-suite branch 4 times, most recently from d1644ab to 263f239 Compare March 4, 2026 15:03

Address review comments

d38d5c9

shantanu-sardesai force-pushed the issues/415-test-suite branch from 263f239 to d38d5c9 Compare March 4, 2026 15:22

shantanu-sardesai requested a review from jeaye March 4, 2026 15:22

jeaye approved these changes Mar 5, 2026

View reviewed changes

jeaye merged commit 16ace19 into jank-lang:main Mar 5, 2026
15 checks passed

shantanu-sardesai deleted the issues/415-test-suite branch March 5, 2026 18:18

Uh oh!

Conversation

shantanu-sardesai commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shantanu-sardesai commented Jan 22, 2026

Backtrace

Uh oh!

shantanu-sardesai commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jeaye left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeaye commented Jan 22, 2026

Uh oh!

shantanu-sardesai commented Jan 25, 2026

Uh oh!

shantanu-sardesai commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shantanu-sardesai commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shantanu-sardesai commented Feb 22, 2026

Uh oh!

shantanu-sardesai commented Feb 28, 2026

Uh oh!

jeaye left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jeaye Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

shantanu-sardesai Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeaye Feb 28, 2026

Choose a reason for hiding this comment

Uh oh!

shantanu-sardesai Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

jeaye left a comment

Choose a reason for hiding this comment

Uh oh!

jeaye Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

jeaye Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shantanu-sardesai commented Jan 22, 2026 •

edited

Loading

shantanu-sardesai commented Jan 22, 2026 •

edited

Loading

shantanu-sardesai commented Jan 25, 2026 •

edited

Loading

shantanu-sardesai commented Jan 25, 2026 •

edited

Loading

shantanu-sardesai Mar 2, 2026 •

edited

Loading