Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse the jobs for exporting llama3.3.2-3b model. #153

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

codereba
Copy link

@codereba codereba commented Jan 21, 2025

I try to export the llama to Snapdragon 8 Elite X by following the guidance:
https://github.com/quic/ai-hub-apps/tree/main/apps/android/ChatApp

I run the command:
python -m qai_hub_models.models.llama_v3_2_3b_chat_quantized.export --context-length 2048 --device "Snapdragon 8 Elite QRD" --output-dir genie_bundle
After waiting many hours, the error happened, please refer to:
#154

I rerun this command:
python -m qai_hub_models.models.llama_v3_2_3b_chat_quantized.export --context-length 2048 --device "Snapdragon 8 Elite QRD" --output-dir genie_bundle

That will redo all the completed steps, that consumes a lot of time and many resources to redo the works.(many hours for 10Mbps upload speed).

This patch will check whether the jobs exist, if that's true then let user to choose whether to use them directly (because the jobs may be different in the detail, but that's not a common case).

Please note:
I think the script can ignore the quantization steps if all the jobs exist, but that changes more, I need to get your suggestion firstly before I try to do it.

The patch is tested locally, it used the submitted jobs, and download the linked models, please refer to the screen shot:
image

@codereba codereba force-pushed the gocode/topic/reuseLastJobsForExportingLlama3_3_2_3b branch from 32d0416 to 705c66e Compare January 21, 2025 23:38
@bhushan23
Copy link
Contributor

Thank you very much @codereba for this change.

We are internally also working on general caching for all LLM export and will release in next release.
Loved how you have used job summaries to cache at export level. Amazing work :)

We look forward to seeing more contributions from you :)

@codereba
Copy link
Author

codereba commented Jan 25, 2025

Thank you very much @codereba for this change.

We are internally also working on general caching for all LLM export and will release in next release. Loved how you have used job summaries to cache at export level. Amazing work :)

We look forward to seeing more contributions from you :)

Yes, I try to contribute more :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants