Bug description
zhipu multiple modal api doc
The image_url. url
here can be a base64 string but should not include the prefix like this data:image/png;base64,
Environment Spring AI version:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-zhipuai-spring-boot-starter</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
Java version:17
Steps to reproduce
final Media recourceMedia = new Media(MimeTypeUtils.IMAGE_PNG, new FileSystemResource(imageFile));
String content = chatClient.prompt()
.user(userSpec -> userSpec
.text(message)
.media(recourceMedia)
)
.call()
.content();
return
org.springframework.ai.retry.NonTransientAiException: 400 - {"error":{"code":"1210","message":"图片输入格式/解析错误"}}
Expected behavior request success
Minimal Complete Reproducible example see Steps to reproduce
Comment From: viewhang
success like this,but @Deprecated(since = "1.0.0 M1", forRemoval = true)
String chatWithImage(String message, File imageFile) throws IOException {
final String base64Str = Base64.getEncoder().encodeToString(Files.readAllBytes(Paths.get(imageFile.getPath())));
final Media recourceMedia = new Media(MimeTypeUtils.IMAGE_PNG, base64Str);
String content = chatClient.prompt()
.user(userSpec -> userSpec
.text(message)
.media(recourceMedia)
)
.call()
.content();
logger.info(content);
return content;
}
Comment From: mxsl-gr
hi, I haven't added multi-modal (GLM-4V) support yet. i will take a look at this issue later.
Comment From: mxsl-gr
i have add multi-model support in PR #859
Comment From: mxsl-gr
@viewhang hi, i noticed the PR has been merged, you can pull the code and give it try 😊
Comment From: viewhang
it works well Thanks