The class org.springframework.ai.chat.prompt.PromptTemplate configures the stringtemplate renderer ST with non default delimiters { and }. In my opinion there is no need to change the default behavior. Here are the arguments that, for me, speak against deviating from the standard:

  • With curly brackets you have to escape JSON examples in the prompt (for example if you provide an expected JSON response)
  • The Spring AI documentation does not state this changed behavior (except in the examples) and the stringtemplate documentation uses < and >.

In my opinion, Spring AI should either use the stringtemplate standard, make delimiters configurable or mention the changed delimiters explicitly in the documentation.

If this issue is accepted I am happy to provide a pull request with the desired changes.

Comment From: markpollack

Interesting, I had done this in order to maintain some compatibility with prompts that come from python and their use of F-String. I did not anticipate the side effects you mention, in particular the JSON conflict.

So, I'm inclined to change it for sure, but wait for a 0.9.0 release as it is a going to break a great deal of existing code, but that is what the early releases are all about, collecting feedback to improve! Thanks for the issue, if you issue a PR, please wait until after the 0.8.1 release.

Comment From: pherklotz

Hello Mark, Thank you for your answer. I can provide the PR. Just to make sure that we are on the same page. I will remove the custom delimiters in Code an samples and stick with the default delimiters of stringtemplate?!

Comment From: markpollack

Hi. Sorry for the delay. Yes, that sounds great, an example with JSON would be appreciated.

i’ll schedule this for the next release after 0.8.1 and we will have to highlight it as a breaking change.

Comment From: ThomasVitale

@pherklotz @markpollack I appreciated a lot the templating compatibility of Spring AI with popular frameworks from the Python world. It helps a lot when getting started with Spring AI and when doing prompt design, since it's easy to re-use the many prompts available and used in Python applications. Even outside Python, the curly brackets syntax is the most used one for LLM prompts, making it straightforward to collaborate and share across languages and frameworks.

Would it be an idea to make it configurable? The default could be "{}" (which is what is expected in LLM prompts at this point), but an application property could be used to replace it with something else, such as "<>" or "$", so to accommodate specific use cases like the ones mentioned above.

Comment From: pherklotz

Hi @ThomasVitale, we have another similar discussion in the PR. I think it's easier if you join there :)

Comment From: youngmoneee

For Python's F-String, injecting JSON requires the use of double brackets. Focused on this functionality, I've developed a prototype for implementing F-Strings.

Even when using a delimiter injection method, I anticipate issues arising in templates mixed with JSON, HTML, and XML. Although it may be a challenging approach, this prototype allows for injecting and utilizing JSON in F-Strings, similar to its original functionality.

The test code written includes:

  1. Injecting JSON into the template.
  2. Injecting JSON into the template and using template variables within JSON.
  3. Injecting a nested JSON structure into the template and utilizing variables within it.

I believe that with more sophisticated test code, it would be possible to use templates safely and conveniently.

Thanks

Comment From: markpollack

The other option that I think may solve this is to allow for swapping out the templating engine implementation. This way if one wants to use a templating engine that is more suitable for I really wanted to avoid this and favor simplicity over flexibility, but i see the tension and there have been several requests along this line. I am not sure how much validation each templating engine offers, but that is the price one pays for flexibility.

Comment From: leni-kirilov

+1 from me about the issue.

I'll leave the best decision for you but I'm building a prompting app that I expect to return JSONs but sometimes the data that I'm passing to it is Json, XML, etc and sometimes the template engine fails...

⛔ I've tried escaping some values but I can't escape every possibility

Interestingly when I build the prompt myself 

new Prompt(

                    List.of(
                            systemPrompt,
                            new UserMessage(userPrompt)
                    ),

                    new OpenAiChatOptions.Builder()...

vs using the builder pattern

aiClient.prompt()
                    .system(systemPrompt.getContent()))
                    .user(userPrompt)

the templating engine doesn't complain even though I have placeholders in the strings, but... if I have added advisors to the client, they don't work (as the prompt is already built...)

❓ Is this by design or an oversight? tbh, I expected whichever way I build a prompt, that Spring AI will moderate it with an Advisor (for the RAG pattern for example)

In a way I HAVE to use the template engine if I want RAG (maybe functions, too, I'm yet to try them out)

Comment From: markpollack

@leni-kirilov regarding

the templating engine doesn't complain even though I have placeholders in the strings, but... if I have added advisors to the client, they don't work (as the prompt is already built...)

did you try to call the model after setting it up via

aiClient.prompt() .system(systemPrompt.getContent())) .user(userPrompt)

since only when the call is made the Prompt is created and the variables are replaced.

Comment From: markpollack

@pherklotz Thanks for your thoughts, I will document the deviation from stringtemplate defaults.

Comment From: leni-kirilov

@leni-kirilov regarding

the templating engine doesn't complain even though I have placeholders in the strings, but... if I have added advisors to the client, they don't work (as the prompt is already built...)

did you try to call the model after setting it up via

aiClient.prompt() .system(systemPrompt.getContent())) .user(userPrompt)

since only when the call is made the Prompt is created and the variables are replaced.

yes, and I had issues with advisors hooking to the prompt as explained

Comment From: johnsonr

I think compatibility with the Python ecosystem is important.

If I understand correctly, we presently have "not quite StringTemplate": allowing { syntax instead of ST <. So we have a unique solution based on a niche technology, which isn't compatible with anything.

While we're making StringTemplate look like mainstream templating systems for the trivial case of variable interpolation, anything more complex like looping (necessary in many prompts) will still look odd. See #1687 for examples.

I think the solution has to be to allow standard use of more popular technologies, with StringTemplate (preferably standard StringTemplate) as one of a number of pluggable solutions.

Comment From: markpollack

I don't disagree that more options would be nice, but haven't had time to prioritize it.

Also, I'd disagree that it is a 'unique solution that is based on a niche technology that isn't compatabile with anything' it's purposely by design compatible with the convention in the python echosystem and in fact there is no single choice of delimiter that everyone can some to a simple agreement upon.

Not against choices.