Rewrite the parser by more closely following spec

[nezhyborets]

What

Rewrite Server Sent Events parser

Why

Our parser doesn't follow the spec closely which may lead to bugs like in the connected issue

Affected Areas

Streaming Interpreter

[sonarqubecloud]

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[nezhyborets]

@JackuXL could you try to switch package to this branch to see if it works for you?

315-streaming-session-is-ended-before-any-output

[JackuXL]

@JackuXL could you try to switch package to this branch to see if it works for you?

315-streaming-session-is-ended-before-any-output

I have tried, but it still does not work. The output is the same as before.

[nezhyborets]

Omg, crazy. Thanks for trying. Will try to find how to configure Volcano on my side to reproduce

[nezhyborets]

It could be very helpful if you'd put a breakpoint in StreamingSession and print the raw response data. It returns data so you'd have to print String(data: data, encoding: .utf8). The response may come multiple times, the first one may be enough

[nezhyborets]

On line 62 or 63 should work, at the start of the delegate method that provides response data

[JackuXL]

On line 62 or 63 should work, at the start of the delegate method that provides response data

(lldb) po String(data:data, encoding: .utf8)
▿ Optional<String>
  - some : "data: [DONE]\n\n"

[nezhyborets]

Is it the first and the only response in streaming session? If so, then there would basically be nothing to parse. I am not sure why a platform would respond in such a manner 🤔 interesting

[nezhyborets]

Can you print the query?

UPD: sorry you've already printed it, no need

[JackuXL]

Is it the first and the only response in streaming session? If so, then there would basically be nothing to parse. I am not sure why a platform would respond in such a manner 🤔 interesting

The platform enables developers to bring websearch to any LLM, and use it through standard OpenAI API.

Later I would try to stream using official OpenAI Python SDK and see if it could work.

[nezhyborets]

It would be great, thanks!

[JackuXL]

It seems that the latest version works, and the previous error is due to volcengine itself. Thanks!

[nezhyborets]

Yay! Thanks for checking!

[nezhyborets]

Hi @JackuXL, it turned out that there had been multiple issues with the new parser. I've release the fixes in 0.4.1. So maybe it will work better for you too

[JackuXL]

Hi @JackuXL, it turned out that there had been multiple issues with the new parser. I've release the fixes in 0.4.1. So maybe it will work better for you too

Ok, thanks for the reminder!