Per #2356, I'm trying to prevent @Tool-annotated methods (or even Function beans from being invoked using Spring Security's @PreAuthorize annotation if the current security context doesn't have proper authority.

But what would be even better is if there were some pre-prompt filtering to prevent tools for which authority hasn't been granted from even making it into the prompt and therefore the LLM wouldn't even ask to invoke them.

Comment From: habuma

I spent a little time trying to create an implementation of CallAroundAdvisor to do this. In theory, it should work like this:

  • The custom advisor would inspect the available tools in the AdvisedRequest...
  • It would use an AuthorizationManager's authorize() to determine if the method in question is permitted.
  • If not, it would remove the tool from the request.

The challenges, however, are that ...

  • The FunctionCallbacks you get from the AdvisedRequest do not expose the underlying toolMethod property, so there's no way to know which class and method are associated with the tool. Even if you cast to MethodFunctionCallback that method isn't exposed
  • I'm not entirely sure how to inject an AuthorizationManager into the advisor, as there's not one in the application context. There may very well be a way to get one, but I gave up before figuring it out.
  • The functionCallbacks() method from AdvisedRequest is fine for @Tool-annotated methods. But if the tool is defined as a Function bean, then you have to call functionNames() to get the names of the functions and use those names to look up the beans and from that get a reference to the apply() method to test permissions on. Not impossible. Just not straightforward.

The good news is that it is quite easy to remove a tool if you decide it's not desired in the prompt.

In short, I don't have a solution to this (yet). But I wanted to raise it as a desired capability that would be useful.