Return logits probabilities for NeuronX TGI generated tokens

[dacorvo]

Feature request

The current NeuronX TGI server does not return logits probabilities for generated tokens.

Motivation

This makes the best_of parameter useless (the first sequence is always selected).

Your contribution

Token Selector should be modified to return the next token logit probability.

[HuggingFaceDocBuilderDev]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

[HuggingFaceDocBuilderDev]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

[HuggingFaceDocBuilderDev]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!

[HuggingFaceDocBuilderDev]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Thank you!