[Bug]: OpenAI LogProbs format for Chat-Completion is incorrect about vllm HOT 5 CLOSED

@DarkLight1337 you were quicker by one hour, but you still have failing tests, so I win 😜

from vllm.

It's not a competition xD We can combine our solutions in your PR if need be.

from vllm.

I know! I was being (not so) funny.

from vllm.

I have updated my PR with more test cases. From my understanding, the behaviour of disabling logprobs and specifying zero top logprobs should be distinct. In particular:

• Disabling logprobs should return no logprobs at all:
  • Completions API: Input logprobs=None should result in output top_logprobs==None
  • Chat Completions API: Input logprobs=False should result in output len(top_logprobs)==0
• Specifying zero top logprobs should return the logprob for the output token only for Completions API (if it exists):
  • Completions API: Input logprobs=0 should result in output len(top_logprobs)<=1
  • Chat Completions API: Input logprobs=True,top_logprobs=0 should result in output len(top_logprobs)<=k len(top_logprobs)==0
• Specifying k top logprobs should return the top k items, plus the logprob for the output token only for Completions API (if it exists):
  • Completions API: Input logprobs=k should result in output len(top_logprobs)<=k+1
  • Chat Completions API: Input logprobs=True,top_logprobs=k should result in output len(top_logprobs)<=k+1 len(top_logprobs)==k

Edit: Thanks @br3no for the correction!

from vllm.

My PR #5026 now passes all tests as well.

from vllm.