Skip to content

Llama3.2 vision model AOTI integration #1497

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
larryliu0820 opened this issue Feb 21, 2025 · 1 comment
Open

Llama3.2 vision model AOTI integration #1497

larryliu0820 opened this issue Feb 21, 2025 · 1 comment
Labels
Compile / AOTI Issues related to AOT Inductor and torch compile enhancement New feature or request triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@larryliu0820
Copy link
Contributor

larryliu0820 commented Feb 21, 2025 •
edited
Loading

🚀 The feature, motivation and pitch

Need work to enable:

python3 torchchat.py export llama3.2-11B --output-aoti-package-path exportedModels/llama3_2_artifacts.pt2

Features to be added:

  • Run AOTI export on preprocess, vision encoder and text decoder separately.
    • This requires dependency on torchtune's exportable modules and perform module transformation needed.
  • Package them into a single pt2 file.

Alternatives

No response

Additional context

No response

RFC (Optional)

No response
@larryliu0820 larryliu0820 mentioned this issue Feb 21, 2025
Closed
@Jack-Khuu Jack-Khuu added Compile / AOTI Issues related to AOT Inductor and torch compile triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Feb 21, 2025
@Jack-Khuu Jack-Khuu added the enhancement New feature or request label Feb 23, 2025
@HonestDeng
Copy link

I want to be a long-time contributor. This issue looks very interesting and challenging for me. Can I get any help and solve this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Compile / AOTI Issues related to AOT Inductor and torch compile enhancement New feature or request triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
[torchchat] Looking for Contributors
Status: Staging
Milestone
No milestone
Development

No branches or pull requests
3 participants
@larryliu0820 @Jack-Khuu @HonestDeng