Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profile the gpt-2 module and the tortoise-tts gpt-2 module and try to improve the gpt-2 module's performance #6

Open
balisujohn opened this issue Apr 1, 2024 · 0 comments
Labels
good first issue Good for newcomers

Comments

@balisujohn
Copy link
Owner

This task is blocked by the gpt-2 forward pass test being added since this could introduce regressions. #5

The task is as follows:

measure the runtime of the autoregressive model all the way from inputs to the full sequence of tokens and last layer latents being generated (as checked by the test in #5 ), and the time taken for the corresponding batch of 4 token sequences and final layer latents in tortoise-tts.

Then, try improving the efficiency of the tortoise.cpp forward pass. Some suggestions are as follows:

try removing seemingly redundant ops
change ops to in place where possible

Feel free to ask questions here or in the discord.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant