[DRAFT] Set vllm version limit to avoid upstream API incompatibility

[vicoooo26]

According to vllm upstream v0.6.2 release note, beam search has been soft deprecated and attribute use_beam_search in SamplingParams has been removed in v0.6.3. To keep vllm backend available without refactoring, setting vllm max version limit will help.

[IlyasMoutawwakil]

Hi, I don't think limiting the version and all the new models/fixes that come with it is justifiable for something like a deprecated argument. I prefer compatibility with the latest version because that's more "fair" when it comes to benchmarks.

[vicoooo26]

Hi, I do agree with you about 'fairness'. To make it compatible with the latest vllm, maybe we could leverage the log probabilities and replace use_beam_search just like vllm beam search implementation, or we could use some higher API instead of vllm core. What do you think?

[IlyasMoutawwakil]

Yes I think just removing the beam_search argument from the generate methods for makes total sense, and using the new beam_search method with log probs if ever someone passes a beam search argument, since we don't care whether the output is correct, all we care about is that the model ran the correct amount of forward passes.
Unfortunately I don't have much bandwidth to tackle this, but would appreciate a PR.

[vicoooo26]

Hi, I remove use_beam_search param and let logprobs = 2 * num_beams if a user passes num_beams that greater than 1. If it looks good to you, I will create a new PR or reopen this PR.