• Add OpenAI Stream Client implementation.
  • Add openAiStreamClient test in ClientIT.

Resolves #116

Comment From: making

This PR is very specific to the openai-java implementation and does not resolve #116 .

Comment From: JM-Lab

I apologize for the confusion. The intention of this pull request was to add stream support to spring-ai's AiClient, inspired by the implementation in openai-java. This differs from the request in issue #116, which was specifically for an implementation not dependent on openai-java.

To meet the requirements of #116, it seems necessary to manually write all subclasses related to ChatCompletionsRequest, response classes, and SSE response classes. Accordingly, I will submit a new pull request with the necessary modifications.

If you have any further questions or suggestions, please feel free to let me know.

Comment From: tzolov

@JM-Lab as @making pointed out we can not expose proprietary libraries throughout the Spring AI public API.

Subclassing is not an option either. We need to define a new generic chunk interface or interfaces. Also the Spring projects is somewhat biased toward Flux instead of rx.
Please share your design thoughts, apologies for not providing any guidelines so far.

Comment From: JM-Lab

Regarding the feedback from @tzolov, the development was revisited. OpenAiStreamClient has been re-implemented to directly adhere to the latest Request/Response specifications of OpenAI's Chat Completion API. Using Flux, the existing 'generate' method was re-implemented, and the 'generateStream' method was added.

The commit details are as follows:

  • Add the latest Request and Response Classes of OpenAI's Chat Completion API
  • Add the spring-boot-starter-webflux dependency
  • Implement the AiStreamClient interface extending AiClient
  • Implement OpenAiStreamClient using Reactor Flux

If you have any additional inquiries or recommendations, please don't hesitate to reach out.

Comment From: XYWENJIE

Sorry, I didn't see this issue before. Just finished reading the code, thank you for sharing.

I recently encountered this issue with SSE requests when accessing other LLM products using Spring AI. Just implement the function yourself.

After reading this code, I think it has great reference value

Comment From: tzolov

Rebased, squashed and merged at 8139af468e9e3e8ba5f1d7f81f7e6e56c1c17fe0 @JM-Lab thank you for the contribution and expect a follow-up changes as I'm heavily refactoring the chat generation and embedding clients.