Description

There are two main issues with the current implementation of the createRequest method (see additional context below) in the OpenAiChatModel class:

  1. Inability to Pass tool_calls or tool_call_id: The ChatCompletionMessage constructor called in the createRequest method does not allow passing tool_calls or tool_call_id. These are always initialized as null.

java public ChatCompletionMessage(Object content, Role role) { this(content, role, null, null, null); }

  1. Discrepancy in Message Roles: There is a discrepancy between the MessageType enum in AbstractMessage and the Role enum in OpenAI messages.

MessageType in AbstractMessage: java public enum MessageType { USER("user"), ASSISTANT("assistant"), SYSTEM("system"), FUNCTION("function"); }

Role in OpenAI messages: java public enum Role { @JsonProperty("system") SYSTEM, @JsonProperty("user") USER, @JsonProperty("assistant") ASSISTANT, @JsonProperty("tool") TOOL } This results in in an java.lang.IllegalArgumentException

Steps to Reproduce

Passing a FunctionMessage in a prompt results in an error:

@Test
public void testPromptWithTool() throws JsonProcessingException {
    Prompt prompt = new Prompt(new FunctionMessage(""));
    openAiChatModel.call(prompt);
}

Error:

java.lang.IllegalArgumentException: No enum constant org.springframework.ai.openai.api.OpenAiApi.ChatCompletionMessage.Role.FUNCTION

    at java.base/java.lang.Enum.valueOf(Enum.java:293)
    at org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage$Role.valueOf(OpenAiApi.java:488)
    at org.springframework.ai.openai.OpenAiChatModel.lambda$createRequest$9(OpenAiChatModel.java:267)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
    at java.base/java.util.Collections$2.tryAdvance(Collections.java:5073)
    at java.base/java.util.Collections$2.forEachRemaining(Collections.java:5081)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
    at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
    at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
    at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
    at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
    at org.springframework.ai.openai.OpenAiChatModel.createRequest(OpenAiChatModel.java:268)
    at org.springframework.ai.openai.OpenAiChatModel.call(OpenAiChatModel.java:140)
    at it.ai.foundation.ServiceChatModelTest.testPromptWithTool(ServiceChatModelTest.java:143)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)

Expected Behavior

The current implementation of the createRequest method does not cover all possible prompts that a user could define. The expected behavior should be as follows:

  1. Support for Tool Calls:

  2. The createRequest method should allow passing tool_calls and tool_call_id to the ChatCompletionMessage constructor. This will enable the proper handling of tool calls within the chat prompt.

  3. Consistent Message Roles:

  4. The MessageType enum in AbstractMessage should be aligned with the Role enum in OpenAI messages. Alternatively, a mapping should be provided to ensure that all message types are correctly translated and no errors occur due to missing enum constants.

Additional Context

Here is the current implementation of the createRequest method:

ChatCompletionRequest createRequest(Prompt prompt, boolean stream) {
    Set<String> functionsForThisRequest = new HashSet<>();
    List<ChatCompletionMessage> chatCompletionMessages = prompt.getInstructions().stream().map(m -> {
        // Add text content.
        List<MediaContent> contents = new ArrayList<>(List.of(new MediaContent(m.getContent())));
        if (!CollectionUtils.isEmpty(m.getMedia())) {
            // Add media content.
            contents.addAll(m.getMedia()
                .stream()
                .map(media -> new MediaContent(
                    new MediaContent.ImageUrl(this.fromMediaData(media.getMimeType(), media.getData()))))
                .toList());
        }
        return new ChatCompletionMessage(contents, ChatCompletionMessage.Role.valueOf(m.getMessageType().name()));
    }).toList();
    // rest of the code...
}

Comment From: ThomasVitale

Thanks for reporting this issue. The function calling APIs have been recently refactored and improved. The changes should have handled both points you reported.

  1. The ChatCompletionMessage in the OpenAiApi class accepts both toolCallId and toolCalls. See: https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/api/OpenAiApi.java#L519
  2. The MessageType enum uses tool as the role name instead of function. See: https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/messages/MessageType.java#L51

Can you confirm this issue is fixed?

Comment From: csterwa

@RikJux please let us know if this is resolved. Will close in 7 days if no response. Thank you.

Comment From: csterwa

Closing for now.