[FEATURE REQUEST] Auto-detect llama2/llama3 from tokenizer.model in runner-aoti/runner-et

[metascroy]

Remove -l 2 and -l 3 flags and auto-detect model architecture from tokenizer class.

Issue warning if user supplied -v does not match the tokenizer.model inferred vocab size.

[Jack-Khuu]

AOTI support added, ET support flagged in #1484

[larryliu0820]

AOTI support added, ET support flagged in #1484

There's some remaining integration work needs to be done so maybe we should keep this open

[larryliu0820]

Basically right now if we do torchchat export with llama3.2 vision model tag and aoti, there's some breakage. Was hoping someone can enable that path

[larryliu0820]

Oh uh, this issue is not talking about aoti I guess?

[Jack-Khuu]

The original task was pretty old, so context has slightly changed as well

I took it to mean removing the need for manually specifying the tokenizer type during aoti_runs for the Text-Only models (Like angela's AOTI PR and the followup issue #1484)

torchchat/runner/run.cpp

Line 856 in 062dd87

ModelType model_type = get_model_type(std::stoi(aoti_metadata["tokenizer_type"]));

[Jack-Khuu]

Basically right now if we do torchchat export with llama3.2 vision model tag and aoti, there's some breakage. Was hoping someone can enable that path

Got it, mind spinning up an issue for the 3.2 aoti integration?

[larryliu0820]

Basically right now if we do torchchat export with llama3.2 vision model tag and aoti, there's some breakage. Was hoping someone can enable that path

Got it, mind spinning up an issue for the 3.2 aoti integration?

Here you go #1497