Bug description When trying to connect to LM studio on http://localhost:1234 for a chat completion request the program hangs, if you exit from LMStudio Spring AI complains that to bytes are received, which makes me thinks that it is waiting for a response. Please note that using the official OpenAI endpoint works correctly
I tried to send a chat completion request using curl and it works after loading the model with lms load
curl http://localhost:1234/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -H "Authorization: Bearer YOUR_OPENAI_API_KEY" ^ More? -d "{ \"model\": \"llama-3.2-1b-instruct\", \"messages\": [ { \"role\": \"user\", \"content\": \"hello\" } ], \"temperature\": 0.7 }" { "id": "chatcmpl-qjmrt2dzv5plaf7w22f8p8", "object": "chat.completion", "created": 1741679687, "model": "llama-3.2-1b-instruct", "choices": [ { "index": 0, "logprobs": null, "finish_reason": "stop", "message": { "role": "assistant", "content": "Hello! How can I assist you today?" } } ], "usage": { "prompt_tokens": 11, "completion_tokens": 9, "total_tokens": 20 }, "system_fingerprint": "llama-3.2-1b-instruct" }
Environment I am using Windows 11 24H2, Java 23.0 (amazon corretto), spring AI M6 (but M4 shows the same behaviour) and LM Studio 0.3.8 (but 0.3.12 shows the same issue) I tried with both CORS enabled or disabled, and both serving the LMStudio on the local network and not serving it on the local network, it made no difference.
Steps to reproduce in the attached zip file there's a minimal spring application built with spring initializr that incorporates Spring AI M6 with the OpenAI module Just run SpringBootApplication.java Expected behavior Main.java is a CommandLineRunner where I build a ChatClient and send the prompt hello to the model, I expect the program to exit once the generation is complete but the program hangs forever.
Minimal Complete Reproducible example See the attached zip file demo.zip
Comment From: cjjava
I also tried to connect LM studio through Spring AI today, but it is always in a suspended state.