V2

[LoggeL]

This issue tracks my development progress and ideas for v2. Feel free to add ideas :)
The development will happen in the v2 Branch.

Planned Features:

• Retain all existing processing steps
• Use litellm to integrate and handle all providers
• Provide a WebUI for an easier User Experience
• More structured inputs for the story (characters, genre, setting, ...)
• The option to halt generation and edit a chapter or provide a custom prompt to regenerate from that point on

[jypma]

This is a cool project! I think your final point (halt generation and resume) might be the most important one for a V2, since it will help drive an on-disk representation of current generation progress. It will also help us mortals doing CPU/GPU-assisted generation with only 7B models getting a story forward a few runs at a time 😉

For example, the current functionality of writing progress to Logs could be used to more structurally store the current story's outline, chapter summaries, etc., and read those back in JSON on subsequent resume runs.

[LoggeL]

Yeah the idea of getting everying in a stateful configuration is a leading motive in the restructuring. But I'm unhappy with the current structure and will have to do a few more iterations till I'm happy.

[8ullyMaguire]

If it's possible I would like to be able to set the approx output length, for example 5k or 100k words.

[LoggeL]

Not sure if I can really restrict that accurately. I can try to add it to the prompt.