Remove lexbuffer in favor of sedlex #369
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
w.i.p: unify lexers
Yes, in my switch v3.2 is installed (which is the latest in opam). |
|
Can you add it on the .opam file? |
Done in #375 |
lubegasimon
force-pushed
the
remove-Lexbuffer-in-favor-of-sedlex
branch
from
cba1f90 to
fcc8741
Compare
I guess this is resolved here fcc8741 |
lubegasimon
force-pushed
the
remove-Lexbuffer-in-favor-of-sedlex
branch
2 times, most recently
from
5c049ed to
bc043cd
Compare
lubegasimon
force-pushed
the
remove-Lexbuffer-in-favor-of-sedlex
branch
from
a3a519b to
c0150eb
Compare
lubegasimon
force-pushed
the
remove-Lexbuffer-in-favor-of-sedlex
branch
from
c0150eb to
e81ae0c
Compare
lubegasimon
force-pushed
the
remove-Lexbuffer-in-favor-of-sedlex
branch
from
e81ae0c to
3e40059
Compare
| let escape = [%sedlex.regexp? | ||
| unicode | ('\\', Compl('\r' | '\n' | '\012' | hex_digit)) | ||
| ]; | ||
| let escape = [%sedlex.regexp? '\\']; |
There was a problem hiding this comment.
The other parts were redundant and caused issues while starting with escape chars
| let char = uchar_of_int(char_code); | ||
| let _ = consume_whitespace(buf); | ||
| char_code == 0 || is_surrogate(char_code) | ||
| // U+FFFD is a character used as a substitute for an uninterpretable character from another encoding |
There was a problem hiding this comment.
Where this U+FFFD is being treated?
There was a problem hiding this comment.
It is referenced as a string on the next two lines. Though I am not so convinced with it.
| | Error((loc, msg)) => | ||
| let pos = loc.Css_types.loc_start; | ||
| let curr_pos = pos.pos_cnum; | ||
| let lnum = pos.pos_lnum; | ||
| let pos_bol = pos.pos_bol; | ||
| let err = | ||
| Printf.sprintf( | ||
| "%s on line %i at position %i", | ||
| msg, | ||
| lnum, | ||
| curr_pos - pos_bol, |
There was a problem hiding this comment.
Do you think is worth it add this level of detail on the error reporting by the ppx?
There was a problem hiding this comment.
Yes, I think it is for clarity.
Changes made:
Lex_buffer.rebecause we no longer have to call from it, instead we rely onsedlex-defined functions and a customlatin1function