This is similar to https://github.com/spring-projects/spring-ai/pull/948, but applied to EvaluatorRequest
. Aside from being about to use the evaluator at runtime, this makes it possible to write a test like this:
@Test
public void testAsk() {
String question = "Why is the sky blue?";
Answer answer = askService.ask(new Question(question));
RelevancyEvaluator evaluator = new RelevancyEvaluator(chatClientBuilder);
EvaluationResponse response = evaluator.evaluate(
new EvaluationRequest(
question,
List.of(),
"Scattering of light causes the sky to be blue."));
Assertions.assertThat(response.getScore()).isEqualTo(1.0f);
}
Comment From: markpollack
updated docs merged in b5cc9db88a89356917fc876955ab9a80b5674e5f