feat: smooth and combine token output

[Saghen]

I noticed while using the Anthropic endpoint that the output would jump by ~10 tokens, pause for 200ms, and repeat. I've implemented a smoothing function for the streamed output that attempts to estimate the token/s and output at that rate instead.

I also added a chunking function that attempts to concatenate tokens like do and n't. It made it a bit easier to read, especially for long words and code which are split quite heavily.

Some stats for chunking (Anthropic Claude 3 Opus):
~10% fewer tokens in regular text
~25% fewer tokens in mix of 80% code and 20% text

Also, to make these changes easily, I moved the logic into a separate utils file. But I'm pretty sure that's not the right place for this :D

[nsarrazin]

Works quite well! It's quite a lot of new code so I want to make sure I understand it all but just from a usage point of view I think it's quite nice.

[gary149]

I like the UX of it - I'll let @nsarrazin check the code.

[nsarrazin]

I tested this PR a bit more and seems like the "Stop generating" feature doesn't work anymore, do you think you could take a look? @Saghen

[Saghen]

@nsarrazin fixed! More correct solution might be to expose the abort controller to the stop event and abort directly from there but that seemed hacky

[nsarrazin]

Seems to work quite well! Thanks for the contribution, will merge and deploy soon 🚀

[gary149]

You are awesome @Saghen 🥳