The issue

When the JobLauncherApplicationRunner is used to start a Spring Batch job with a JobParametersIncrementer, parameter keys that have once been assigned a value cannot be unset without manipulating the history of previous job executions. If a parameter is not explicitly set to something, its value will be recovered from the last job execution (of the job, not the job instance).

This may be desirable for identifying parameters of the job instances but in my opinion not for non-identifying parameters.

A minimal example

I've written a minimal example. Originally, I've set up the repository for an analogous issue with Spring Cloud Task: https://github.com/spring-cloud/spring-cloud-task/issues/770 The branch without-task uses only Spring Batch and Spring Boot.

First, I start a MySQL container and build the code:

$ docker run -p 3306:3306 --name mysql -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=task -d mysql:5.7.25
$ ./gradlew build

Then, I start the application twice:

$ java -jar build/libs/task-0.0.1-SNAPSHOT.jar foo=bar -yada=yada
$ java -jar build/libs/task-0.0.1-SNAPSHOT.jar foo=bar

From the logs, one can see that the second job execution is still started with the non-identifying parameter:

o.s.b.a.b.JobLauncherApplicationRunner   : Running default command line with: [foo=bar, -yada=yada]
o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=job]] launched with the following parameters: [{run.id=1, yada=yada, foo=bar}]
...
o.s.b.a.b.JobLauncherApplicationRunner   : Running default command line with: [foo=bar]
o.s.b.c.l.support.SimpleJobLauncher      : Job: [SimpleJob: [name=job]] launched with the following parameters: [{run.id=2, yada=yada, foo=bar}]

It seems to me that there is no way to ever unset the yada parameter without manually removing the corresponding row in the database table for the last job execution which effectively means rewriting history.

Comment From: snicoll

Thanks for the PR.

paging @benas for a review

Comment From: fmbenhassine

Identifying and non-identifying job parameters are specific to Spring Batch, and Spring Boot should not be concerned with that. As far as Spring Boot is concerned, it should call the JobParametersIncrementer if one is provided by the user and that's it. It should not start to interpret parameters and remove non-identifying ones. This would go against the contract of a JobParametersIncrementer which is expected to get all previous parameters and not only identifying ones. Changing the user's input in any way would be a mistake in my opinion.

@hpoettker In your minimal example, if you remove Spring Boot and use the CommandLineJobRunner from Spring Batch, you will see the same behaviour when starting the next job instance with -next. So changing this in boot would be inconsistent with batch and a breaking change for those who are migrating to boot.

If you want to filter non-identifying job parameters, you can use a custom JobParametersIncrementer. You can find an example here.

It seems to me that there is no way to ever unset the yada parameter without manually removing the corresponding row in the database table for the last job execution

As mentioned previously, this should be done in your custom incrementer, and the non-identifying parameter will be unset in the subsequent execution automatically, see here.

Comment From: snicoll

Thanks a lot for the review @benas. I am going to close this PR now, thanks for your contribution in any case.

Comment From: hpoettker

@beans Thanks for the detailed answer. Makes total sense.

But it still seems a bit surprising to me that JobParametersIncrementer has this responsibility. It should be considered to extend the java doc of JobParametersIncrementer or JobParametersBuilder::getNextJobParameters accordingly.