Bug description

When running the mvn package or mvn clean package command, the package process fails on the "Spring AI Core" module, due to several test failures in the "BeanOutputConverterTest" class.

Note:

  • I have not touched any code here at the time of reproducing this. In fact, this is my first time using this repo, and I ran into this issue while following instructions on the README.md for how to set it up.
  • As of now, unfortunately the only workaround to this is for me to skip tests while performing the maven processes (i.e. mvn clean package -DskipTests).
  • The failures appear to be due to differences in spaces, newlines, and/or indentations between the expected vs actual results

Environment

  • Spring AI version: Sorry, not 100% sure how Spring AI versions are determined here. However, this is reproducible on the latest version of the main branch (currently commit 5a35213 at the time of writing this), if this helps.
  • Java version: Oracle Open JDK version 17.0.8
  • IDE: IntelliJ IDEA 2023.2 (Ultimate Edition); Build #IU-232.8660.185, built on July 26, 2023
  • Operating System: Windows 11 Pro (Build 22631.4169)

Steps to reproduce

Option A: 1. From the terminal, perform mvn clean package. Observe the test failures

Option B: 1. Run mvn clean (this will return successful build logs) 2. Run mvn package (this will return the test failures)

The main thing is running mvn package in some way, to produce the failures.

Expected behavior

  • The tests should pass. Should be able to successfully perform mvn package without having to skip tests

Minimal Complete Reproducible example

  • Follow the Steps to reproduce section above

Other notes

  • When investigating each of the failing tests individually, it appears the failures occur due to unexpected spaces, newlines, and/or indentations, when comparing the expected vs actual results
  • While I am able to consistently produce failures from the same tests, unfortunately the root cause of the failure itself is not always consistent, nor does it always make sense either.
  • For example, when running the formatTypeReferenceArray test (located in spring-ai-core/src/test/java/org/springframework/ai/converter/BeanOutputConverterTest.java), I got a failed result due to the following error: 20240913_formatTypeReferenceArray_IndividualTestFailure.txt
  • Note in the logs from the previous bullet point - on line 48 - where it says "Click to see difference". When clicking this link, what I see is what's shown in the attached image below. As you can see here, there are no differences at all. Yet the tests fail anyway. formatTypeReferenceArray_IndividualTestFailure_ExpectedVsActualResultComparison
  • In other attempts however, I have seen test failures due to something like an extra space, newline and/or indentation, specifically on the actual result.

The test failures seem bizarre, and there's no immediate reason that I can see as to why these issues are happening. But unfortunately its happening, hence this issue being filed. I am still actively investigating this issue as well, just in case this turns out being a silly "me" issue in some way.

Comment From: Sinclo

While looking at one of the sample snippets provided in my previous comment (20240913_formatTypeReferenceArray_IndividualTestFailure.txt), I did observe a difference in indentations between the expected vs actual result (even though my IDE did not point this out).

Indentation_from_logs

As seen in the image, the closing bracket under the actual results is indented by 2 characters (column 3). Meanwhile the closing bracket under the expected results is under column 1.

Comment From: Sinclo

Further investigation shows that

  1. The tests specifically in the "BeanOutputConverterTest" class are failing, due to only expecting the use of \n as part of formatting, but actually using \r\n in its actual format output. Example of "actual results" can be seen in the following screenshot. jsonSchema_debug_mode

  2. I've also discovered that the reason for mvn package failing goes well beyond the BeanOutputConverterTest class. This process failing is actually caused by numerous test failures and errors from other test classes as well (i.e. ContentFormatterTests.java, OllamaRuntimeHintsTests.java, PagePdfDocumentReaderTests.java, etc). . In other words, even if all of the test failures in the "BeanOutputConverterTest" class are addressed as part of the initial reported issue, mvn package will continue to fail until further effort is put into several additional classes & tests across the project as well.

Comment From: Sinclo

Closing this issue. After further digging, it appears the test failures I'm facing are specific to the IDE that I'm using (IntelliJ IDEA), and seem to be specific to the line separator settings.

I've tried several steps to resolve the issue in the IDE with no luck. That said, while I haven't yet found a fix to the problem, I do know enough to say that the failures seem to be a problem with the IDE, not with the tests.