This PR represents the works I've been doing for what I originally called "AiThing". I renamed it to "AiOperator", but I'm not sold on that name, either.

Regardless of naming, the idea is that using a fluent builder you can define an AiOperator that is equipped for common AI patterns. A few examples follow:

    @Bean
    AiOperator aiOperator(AiClient aiClient) {
        return AiOperator.builder()
                .aiClient(aiClient)
                .promptTemplate("Why is the sky blue?")
                .build();
    }

This is the simplest (and least useful) case, which really doesn't do much more than wrap AiClient. Given this AiOperator you can call generate() with no arguments and get your answer back.

The next example is uses parameters in the prompt:

    @Bean
    AiOperator aiOperator(AiClient aiClient) {
        return AiOperator.builder()
                .aiClient(aiClient)
                .promptTemplate("Tell me a joke about {subject}.")
                .build();
    }

With this, you can call generate(Map.of("subject", "cows")) (for example) to get a joke about cows.

For RAG style operations, you can configure this AiOperator:

    @Bean
    AiOperator aiOperator(AiClient aiClient, VectorStore vectorStore) {
        return AiOperator.builder()
                .aiClient(aiClient)
                .promptTemplate(DefaultPromptTemplateStrings.RAG_PROMPT)
                .vectorStore(vectorStore)
                .build();
    }

In this case you can call generate(Map.of("input", "How are cities scored at the end of the game?")) which will be answered based on the documents loaded into the given vector store. Note that the prompt given is a suggested prompt constant for RAG interactions, but can be overridden if the developer wants to use their own prompt.

For conversational operations, you can configure the AiOperator with memory:

    @Bean
    AiOperator aiOperator(AiClient aiClient, Memory memory) {
        return AiOperator.builder()
                .aiClient(aiClient)
                .promptTemplate(DefaultPromptTemplateStrings.CHAT_PROMPT)
                .conversationMemory(memory)
                .build();
    }

As with RAG, you can call something like generate(Map.of("input", "Where is it located?")) to ask questions based on the conversational context. In this case, "it" will be determined by conversational memory. And, again, a chat prompt has been provided for ease of use, but can be overridden.

Finally, for RAG+Conversational operations:

    @Bean
    AiOperator aiOperator(AiClient aiClient, VectorStore vectorStore, Memory memory) {
        return AiOperator.builder()
                .aiClient(aiClient)
                .promptTemplate(DefaultPromptTemplateStrings.RAG_PROMPT)
                .conversationMemory(memory)
                .vectorStore(vectorStore)
                .build();
    }

What's special about this case is that since both a vector store and conversation memory are provided, the AiOperator will first use the conversational history to ask the model to produce a standalone question. This weeds out any conversational history that isn't relevant to the current question. Then it submits the standalone question to the vector store for document context before sending the standalone question and document context to the model for generation of the final answer.

There are certainly other use-cases not (yet) covered by AiOperator (function invocation comes to mind), but it covers a great deal of common use cases with what is there now.

Comment From: jxblum

Hi @habuma - What about AiOperations or AiTemplate in the same spirit as JdbcTemplate or JmsTemplate?

In Spring Data, we also have a bunch of templates for each data store like Redis, in Spring Data Redis, RedisTemplate, and for MongoDB in Spring Data MongoDB, MongoTemplate, and so on. The Template class usually implements an Operations interface by the same name prefix (e.g. RedisOperations).

I like your DSL. Really cool!

Comment From: habuma

Hi @habuma - What about AiOperations or AiTemplate in the same spirit as JdbcTemplate or JmsTemplate?

My thinking was that this approach is more akin to things like Spring's JdbcClient, WebClient, or RestClient and less like Spring's various template classes. But I couldn't name it AiClient because that name is already taken.

Ultimately, I'm open to any name (even the template name). My ultimate goal here is to capture common patterns of use in such a way that a Spring developer using Spring AI in their project doesn't need to build up those patterns themselves. They're common enough and as I like to say "If everyone has to do it, then why does everyone have to do it?"

I like your DSL. Really cool!

I know that Mark has other ideas in this area. But for lack of anything materializing in Spring AI like this yet, I took a stab at it. And I rather like how it turned out. I'm glad that you do, too!

Comment From: markpollack

There is a branch here - https://github.com/spring-projects/spring-ai/tree/aitemplate and I put some thoughts in the other PR that were closed, I'll find that link in moment. This is a very important area for us to design/discuss.

Here are some thoughts on the AiTemplate, though in the context of thinking through 'chains'. https://github.com/spring-projects/spring-ai/pull/73

I have a bunch of issues in a google doc to copy into github issues, one of them in removing the chain package. I'd like to do that cut-n-paste today.

Comment From: markpollack

Closing as we have ChatClient