Spring-ai Feature Request: Add the capability to add name-spaced ChatClient configurations

Please do a quick search on GitHub issues first, the feature you are about to request might have already been requested.

Expected Behavior

Just as existing open issues touch on being able to override the underlying HTTP clients for models, there are other configurations such as temperature that would be nice to configure.

Ideally, this could be done using a namespace for each ChatClient in externalized configuration (application.properties/application.yml).

An alternative could be to document the idiomatic way of defining several different ChatClient beans.

Current Behavior

Right now, to override timeouts I've resorted to doing something like the following:

@Bean
public RestClientCustomizer restClientCustomizer() {
    return restClientBuilder -> restClientBuilder
        .requestFactory(ClientHttpRequestFactories.get(
              ClientHttpRequestFactorySettings.DEFAULTS
                  .withConnectTimeout(Duration.ofSeconds(5))
                  .withReadTimeout(Duration.ofSeconds(60))
    ));
}

Context

I've started work on an Agent framework similar to Crew AI using Spring AI and have a need to configure ChatClient on a per-agent basis. This would allow temperature or even different LLM per agent to be defined.

Comment From: ThomasVitale

@thesurlydev both the ChatClient API and the ChatModel API let you pass runtime options to customize settings like model and temperature, using the ChatOptions abstraction. Would that solve your problem?

Comment From: thesurlydev

@ThomasVitale The use case was to be able to inject various ChatClients configured with different LLM implementations or the same LLM but different temperatures, etc. If this is possible now, then please close this issue.

Comment From: ThomasVitale

You can use different LLM providers in the same application and define different ChatClient objects for them, or use the specific ChatModel implementation directly.

    @Bean
    @Qualifier("openai")
    ChatClient chatClientOpenAi(OpenAiChatModel openAiChatModel) {
        return ChatClient.builder(openAiChatModel).build();
    }

    @Bean
    @Qualifier("mistral")
    ChatClient chatClientMistralAi(MistralAiChatModel mistralAiChatModel) {
        return ChatClient.builder(mistralAiChatModel).build();
    }

Given a single ChatClient, you can use it to make calls with different models and settings as shown in the following, without the need to define multiple beans.

@RestController
class ChatController {

    private final ChatClient chatClient;

    ChatController(@Qualifier("openai") ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @GetMapping("/chat")
    String chat(@RequestParam(defaultValue = "What did Gandalf say to the Balrog?") String message) {
        return chatClient
                .prompt()
                .user(message)
                .call()
                .content();
    }

    @GetMapping("/chat/options")
    String chatWithOptions(@RequestParam(defaultValue = "What did Gandalf say to the Balrog?") String message) {
        return chatClient
                .prompt()
                .user(message)
                .options(OpenAiChatOptions.builder()
                        .withModel("gpt-4-turbo")
                        .withTemperature(1.0f)
                        .build())
                .call()
                .content();
    }

}

I hope that helps.

Comment From: thesurlydev

Extremely helpful. Thanks very much