Azure OpenAI: TypeError: Cannot read properties of undefined (reading 'delta') about openai-partial-stream HOT 7 OPEN

Hello thank you for reporting, do you have the actual raw content in msg to help debugging?

from openai-partial-stream.

For me it happens when enabling the pretty recently introduced stream_options.include_usage option. The final msg content looks like this:

{
   "id":"chatcmpl-<id here>",
   "object":"chat.completion.chunk",
   "created":1718756652,
   "model":"gpt-4-0613",
   "system_fingerprint":null,
   "choices":[
      
   ],
   "usage":{
      "prompt_tokens":308,
      "completion_tokens":68,
      "total_tokens":376
   }
}

from openai-partial-stream.

Are there plans to address this?

from openai-partial-stream.

@thomashibbard thank you for the details, this should help with the investigation.

In the JSON payload you attached, there are no choices. Is that the complete payload?

If stream_options.include_usage triggers this, it could be due to some versions that need updating. I will have to investigate.

from openai-partial-stream.

@st3w4r Yeah exactly, there are no choices—I think—because the final received chunk is just token usage information, and so it doesn't have any of the choices/delta/etc included in the rest of the stream for composing responses.

from openai-partial-stream.

I see, indeed if the last chunk have no choices we could just skip it.
I’m wondering if there is something else that could help differentiate this chunk.

A curl command could confirm this behavior it.

from openai-partial-stream.

I think the last chunk still needs to be processed so the usage can be accessed. Maybe it could emit a done or usage event. I'm getting a little out of my depth here, but would something like this be possible

function process() {
  // Create an instance of the handler
  const openAiHandler = new OpenAiHandler(mode);
  // Process the stream. 
  const entityStream = openAiHandler.process(stream);
  // Create an entity with the schema to validate the data
  const entityPostcode = new Entity("postcodes", PostcodeSchema);
  // Parse the stream to an entity, using the schema to validate the data
  const postcodeEntityStream = entityPostcode.genParseArray(entityStream);

  postcodeEntityStream.on('usage', saveUsageToDb)

  return postcodeEntityStream
}

from openai-partial-stream.