Improve CI times

[meooow25]

Let's see if this affects CI times...

[meooow25]

For GHC 9.10,

master:
total: 11m 38s
build w/o tests: 4m 41s
build: 2m 56s

here:
total: 9m 27s
build w/o tests: 1m 25s
build: 4m 22s

Other steps seem unaffected.

Looks like a compilation of vector is gone, saving ~2m.

[meooow25]

It seems that another time sink is building text, because of tasty -> optparse-applicative -> prettyprinter-ansi-terminal -> prettyprinter. prettyprinter has a flag for disabling the text dependency but it breaks prettyprinter-ansi-terminal: quchen/prettyprinter#250

Edit: I missed that optparse-applicative depends on text directly since 0.18, making the prettyprinter issue moot.