Expected Behavior
When using Function Calling with OpenAI and other models - the function gets invoked in a new thread.
The function being called in my application requires context, so it can persist the response data to the correct user / session etc.
My application uses Spring Security - however, I anticipate there could be other requirements to pass context to this thread.
It would be ideal to be able to store the context before the thread is called and then retrieve the context e.g. using:
SecurityContext context = SecurityContextHolder.getContext();
and
SecurityContextHolder.setContext( context );
Perhaps you could register listeners / functions that can be called to pass this information.
Current Behavior
The calling thread does not have any context about the session / user that invoked the call to the AI model.
Context
The only workaround is to request that the model pass an identified to the function via the prompt.
Comment From: miltonhit
I've opened a similar issue 3 weeks ago: https://github.com/spring-projects/spring-ai/issues/864
A possible workahound for it is passing a function "instance" in each interaction. Then you can pass some user/session in constructor.
Comment From: tzolov
Thanks you highlighting this use case @dolukhanov, @miltonhit
Can you please elaborate on why the function information should be kept in Thread context rather, for example, being returned with the chat response metadata as suggested here?
Comment From: markpollack
Hi! Can you see if https://github.com/spring-projects/spring-ai/pull/1458 fixes this issue for you. Here are some minimal docs - https://docs.spring.io/spring-ai/reference/api/chat/functions/openai-chat-functions.html#_tool_context_support see the test here as well https://github.com/spring-projects/spring-ai/blob/eb2deba9cb8d0bc22fb354ae3af59335eb4d740d/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/openai/tool/FunctionCallbackWithPlainFunctionBeanIT.java#L60
please reopen if you think something is not addressed.