Rewrite Lexer & Parser

[keithamus]

This PR rewrites the lexer to:

• Half the size of tokens from 32B to 16B.
• Skip whitespace & comments in the lexer, not the parser.
• Simplify some tokens (e.g a Percentage token is not needed).
• Move span data from the lexer to the parser.

It also rewrites the parser to:

• Move all individual parsing logic to hdx_ast, colocating it with the AST nodes
• Make use of more macros & Derives to massively DRY up parsing logic

The aim is to:

• Speed up overall parsing
• Make it easier to develop
• Make the whole thing more robust.

[keithamus]

Lexer benchmark comparison:

Old

lexer/960
                        time:   [53.217 µs 53.278 µs 53.341 µs]
                        thrpt:  [178.63 MiB/s 178.84 MiB/s 179.04 MiB/s]
lexer/animate.4.1.1
                        time:   [772.47 µs 773.50 µs 774.62 µs]
                        thrpt:  [117.48 MiB/s 117.65 MiB/s 117.81 MiB/s]
lexer/blueprint.1.0.1
                        time:   [108.56 µs 108.78 µs 109.05 µs]
                        thrpt:  [152.35 MiB/s 152.74 MiB/s 153.05 MiB/s]
lexer/bootstrap.4.6.2
                        time:   [1.3628 ms 1.3638 ms 1.3649 ms]
                        thrpt:  [140.01 MiB/s 140.12 MiB/s 140.23 MiB/s]
lexer/bootstrap.5.3.0
                        time:   [1.7953 ms 1.7989 ms 1.8035 ms]
                        thrpt:  [148.49 MiB/s 148.87 MiB/s 149.17 MiB/s]
lexer/font-awesome-all.5.15.4
                        time:   [463.86 µs 464.59 µs 465.58 µs]
                        thrpt:  [150.71 MiB/s 151.03 MiB/s 151.27 MiB/s]
lexer/foundation.6.7.5
                        time:   [1.3646 ms 1.3660 ms 1.3675 ms]
                        thrpt:  [138.61 MiB/s 138.76 MiB/s 138.90 MiB/s]
lexer/foundation.6.7.5.min
                        time:   [1.0481 ms 1.0490 ms 1.0499 ms]
                        thrpt:  [122.46 MiB/s 122.57 MiB/s 122.68 MiB/s]
lexer/inuitcss.6.0.0
                        time:   [189.41 µs 189.63 µs 189.86 µs]
                        thrpt:  [266.88 MiB/s 267.21 MiB/s 267.53 MiB/s]
lexer/mini.css.3.0.1
                        time:   [361.28 µs 361.82 µs 362.55 µs]
                        thrpt:  [145.40 MiB/s 145.69 MiB/s 145.91 MiB/s]
lexer/open-props.1.5.10.min
                        time:   [174.38 µs 174.62 µs 174.92 µs]
                        thrpt:  [107.06 MiB/s 107.25 MiB/s 107.40 MiB/s]
lexer/pure.2.0.3
                        time:   [149.10 µs 149.36 µs 149.65 µs]
                        thrpt:  [183.81 MiB/s 184.16 MiB/s 184.48 MiB/s]
lexer/reset.2.0
                        time:   [6.9250 µs 6.9410 µs 6.9608 µs]
                        thrpt:  [149.75 MiB/s 150.18 MiB/s 150.52 MiB/s]
lexer/tailwind.2.2.19.min
                        time:   [23.446 ms 23.477 ms 23.515 ms]
                        thrpt:  [118.99 MiB/s 119.19 MiB/s 119.34 MiB/s]

New

lexer/960 
                       time:   [46.043 µs 46.146 µs 46.276 µs]
                        thrpt:  [205.90 MiB/s 206.48 MiB/s 206.94 MiB/s]
lexer/animate.4.1.1
                        time:   [637.93 µs 639.03 µs 640.15 µs]
                        thrpt:  [142.16 MiB/s 142.41 MiB/s 142.66 MiB/s]
lexer/blueprint.1.0.1
                        time:   [92.470 µs 92.547 µs 92.635 µs]                                                                                                                            
                        thrpt:  [179.36 MiB/s 179.53 MiB/s 179.68 MiB/s]
lexer/bootstrap.4.6.2
                        time:   [1.2340 ms 1.2352 ms 1.2368 ms]
                        thrpt:  [154.51 MiB/s 154.71 MiB/s 154.87 MiB/s]
lexer/bootstrap.5.3.0
                        time:   [1.6490 ms 1.6531 ms 1.6578 ms]
                        thrpt:  [161.54 MiB/s 162.00 MiB/s 162.40 MiB/s]
lexer/font-awesome-all.5.15.4
                        time:   [428.82 µs 429.36 µs 429.98 µs]
                        thrpt:  [163.19 MiB/s 163.43 MiB/s 163.63 MiB/s]
lexer/foundation.6.7.5
                        time:   [1.2253 ms 1.2265 ms 1.2278 ms]
                        thrpt:  [154.37 MiB/s 154.54 MiB/s 154.70 MiB/s]
lexer/foundation.6.7.5.min
                        time:   [982.61 µs 983.17 µs 983.87 µs]
                        thrpt:  [130.68 MiB/s 130.78 MiB/s 130.85 MiB/s]
lexer/inuitcss.6.0.0
                        time:   [175.43 µs 175.64 µs 175.87 µs]
                        thrpt:  [288.12 MiB/s 288.49 MiB/s 288.85 MiB/s]
lexer/mini.css.3.0.1
                        time:   [333.29 µs 333.56 µs 333.85 µs]
                        thrpt:  [157.90 MiB/s 158.04 MiB/s 158.16 MiB/s]
lexer/open-props.1.5.10.min
                        time:   [153.09 µs 153.28 µs 153.56 µs]
                        thrpt:  [121.96 MiB/s 122.17 MiB/s 122.33 MiB/s]
lexer/pure.2.0.3
                        time:   [137.57 µs 137.69 µs 137.81 µs]
                        thrpt:  [199.60 MiB/s 199.78 MiB/s 199.95 MiB/s]
lexer/reset.2.0
                        time:   [6.2455 µs 6.2580 µs 6.2724 µs]
                        thrpt:  [166.18 MiB/s 166.57 MiB/s 166.90 MiB/s]
lexer/tailwind.2.2.19.min
                        time:   [21.954 ms 21.977 ms 22.007 ms]
                        thrpt:  [127.15 MiB/s 127.32 MiB/s 127.45 MiB/s]

This is a decent speed up in the lexer.

[keithamus]

Parser:

Old

parser/960
                        time:   [238.14 µs 239.17 µs 239.98 µs]
                        thrpt:  [39.704 MiB/s 39.838 MiB/s 40.012 MiB/s]
parser/animate.4.1.1
                        time:   [1.8714 ms 1.8779 ms 1.8832 ms]
                        thrpt:  [48.324 MiB/s 48.462 MiB/s 48.629 MiB/s]
parser/blueprint.1.0.1
                        time:   [376.84 µs 378.47 µs 380.12 µs]
                        thrpt:  [43.710 MiB/s 43.900 MiB/s 44.090 MiB/s]
parser/bootstrap.4.6.2  time:   [3.4237 ms 3.4373 ms 3.4490 ms]
                        thrpt:  [55.408 MiB/s 55.596 MiB/s 55.818 MiB/s]
parser/bootstrap.5.3.0
                        time:   [5.6988 ms 5.7163 ms 5.7347 ms]
                        thrpt:  [46.699 MiB/s 46.849 MiB/s 46.993 MiB/s]
parser/bootstrap.5.3.0.min
                        time:   [5.4250 ms 5.4368 ms 5.4512 ms]
                        thrpt:  [40.747 MiB/s 40.856 MiB/s 40.945 MiB/s]
parser/font-awesome-all.5.15.4
                        time:   [1.2422 ms 1.2471 ms 1.2509 ms]
                        thrpt:  [56.096 MiB/s 56.267 MiB/s 56.489 MiB/s]
parser/foundation.6.7.5
                        time:   [3.2582 ms 3.2672 ms 3.2745 ms]
                        thrpt:  [57.886 MiB/s 58.016 MiB/s 58.176 MiB/s]
parser/foundation.6.7.5.min
                        time:   [2.7389 ms 2.7446 ms 2.7490 ms]
                        thrpt:  [46.771 MiB/s 46.847 MiB/s 46.944 MiB/s]
parser/inuitcss.6.0.0
                        time:   [432.22 µs 433.98 µs 435.35 µs]
                        thrpt:  [116.39 MiB/s 116.76 MiB/s 117.24 MiB/s]

New

parser/960
                        time:   [235.08 µs 236.44 µs 237.53 µs]
                        thrpt:  [40.113 MiB/s 40.299 MiB/s 40.532 MiB/s]
parser/animate.4.1.1
                        time:   [4.1180 µs 4.1238 µs 4.1307 µs]
                        thrpt:  [21.515 GiB/s 21.551 GiB/s 21.581 GiB/s]
parser/blueprint.1.0.1
                        time:   [472.47 µs 475.41 µs 477.69 µs]
                        thrpt:  [34.781 MiB/s 34.949 MiB/s 35.166 MiB/s]
parser/bootstrap.4.6.2
                        time:   [3.3183 ms 3.3243 ms 3.3305 ms]
                        thrpt:  [57.380 MiB/s 57.487 MiB/s 57.591 MiB/s]
parser/bootstrap.5.3.0
                        time:   [4.1224 µs 4.1316 µs 4.1430 µs]
                        thrpt:  [63.125 GiB/s 63.299 GiB/s 63.440 GiB/s]
parser/font-awesome-all.5.15.4
                        time:   [1.2166 ms 1.2312 ms 1.2448 ms]
                        thrpt:  [56.369 MiB/s 56.992 MiB/s 57.677 MiB/s]
parser/foundation.6.7.5.min
                        time:   [4.1225 µs 4.1281 µs 4.1348 µs]
                        thrpt:  [30.367 GiB/s 30.416 GiB/s 30.457 GiB/s]
parser/inuitcss.6.0.0 
                        time:   [4.1154 µs 4.1244 µs 4.1364 µs]
                        thrpt:  [11.963 GiB/s 11.998 GiB/s 12.024 GiB/s]
parser/mini.css.3.0.1
                        time:   [4.1268 µs 4.1334 µs 4.1412 µs]
                        thrpt:  [12.431 GiB/s 12.455 GiB/s 12.474 GiB/s]
parser/open-props.1.5.10.min
                        time:   [341.44 µs 342.10 µs 342.84 µs]
                        thrpt:  [54.623 MiB/s 54.741 MiB/s 54.848 MiB/s]
parser/pure.2.0.3
                        time:   [365.75 µs 366.51 µs 367.11 µs]
                        thrpt:  [74.929 MiB/s 75.050 MiB/s 75.206 MiB/s]
parser/tailwind.2.2.19.min
                        time:   [60.206 ms 60.488 ms 60.805 ms]
                        thrpt:  [46.018 MiB/s 46.259 MiB/s 46.476 MiB/s]

I think these GiB/s readings are failing fast and swallowing the rest of the file, but the speedup from lexing to parsing is far more modest than the lexer gains.