Spring Request data binding fails for params with an empty array "[]" suffix

Hello, It seems like a bug.

org.sprinframework:spring-beans:5.2.8.RELEASE

We use filterDto in our controller.

data class FilteDto(
var types: MutableList<String>
)

But when we build our request like this types[]=first_type&types[]=second_type. Tokens keys in propertyTokenHolder are wrong. They are just one empty string "". And then we get such output Illegal request output.

We got such stacktrace.

java.lang.NumberFormatException: For input string: ""
    at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[na:na]
    at java.base/java.lang.Integer.parseInt(Integer.java:662) ~[na:na]
    at java.base/java.lang.Integer.parseInt(Integer.java:770) ~[na:na]
    at org.springframework.beans.AbstractNestablePropertyAccessor.processKeyedProperty(AbstractNestablePropertyAccessor.java:322) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.beans.AbstractNestablePropertyAccessor.setPropertyValue(AbstractNestablePropertyAccessor.java:275) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.beans.AbstractNestablePropertyAccessor.setPropertyValue(AbstractNestablePropertyAccessor.java:266) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:97) ~[spring-beans-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.validation.DataBinder.applyPropertyValues(DataBinder.java:848) ~[spring-context-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.validation.DataBinder.doBind(DataBinder.java:744) ~[spring-context-5.2.8.RELEASE.jar:5.2.8.RELEASE]
    at org.springframework.web.bind.WebDataBinder.doBind(WebDataBinder.java:197) ~[spring-web-5.2.8.RELEASE.jar:5.2.8.RELEASE]

I think problem in AbstractNestablePropertyAccessor. In getPropertyNameTokens, see 928 line in AbstractNestablePropertyAccessor.

Maybe in this line.

    if (key.length() > 1 && (key.startsWith("'") && key.endsWith("'")) ||
                            (key.startsWith("\"") && key.endsWith("\""))) {
                        key = key.substring(1, key.length() - 1);
                    }
                    keys.add(key);

P.S. When we search like this types=first_type&types=second_type all is fine.

Comment From: sbrannen

But when we build our request like this types[]=first_type&types[]=second_type.

Why are you attempting to build the query like that?

What happens if you use types[0]=first_type&types[1]=second_type?

Comment From: kostya05983

1)Our axis library from frontend build queries like that with parameter[]. 2)With your query It's working Also, I forget, that we add @RequestParam(name="types[]")

Maybe when it's empty it should not add keys in array

Comment From: sbrannen

When binding multiple values to a list, we only support two syntax variants.

excluding brackets
including brackets and an index

Thus, for your use case types=first_type&types=second_type and types[0]=first_type&types[1]=second_type are supported, but types[]=first_type&types[]=second_type is not.

In light of that, I am closing this issue.

Comment From: kostya05983

Thanks, @sbrannen can you support the third version with empty brackets? I can make a pr.

Comment From: sbrannen

can you support the third version with empty brackets?

To the best of my knowledge, you might be the first person to ever request support for that syntax for web binding in Spring.

When you say "Our axis library from frontend", are you referring to something in your control or something that is at least configurable?

Please elaborate on that to provide us more context.

I can make a pr.

Thanks for the offer, but there is no need for a PR at this time. We would first need to assess the concrete need to support a third syntax variant for such use cases.

Comment From: kostya05983

Thank you for reopening. There is the library https://github.com/axios/axios. There is one tricky moment, web application is configurable, but it's standalone and our architecture is microservices in backend. Earlier, we write code like this

fun getAllEntities(@RequestParam(name="types[]") types: List<String>) {}

And this is working as expected, we use types[]=first_type&types[]=second_type and all is good. After we update one of api and we decide use ParameterObject, such as filterDto

data class FilterDto(
@RequestParam(name="types[]") var types: MutableList<String>,
)

And when we use types[]=first_type&types[]=second_type it's not working... Also, I can make mini application with working variant of controller and variant which is not worked.

Comment From: rstoyanchev

Is it possible to configure this library to exclude brackets, i.e. "types=first_type&types=second_type"? From a quick look I see a paramsSerializer config option under https://github.com/axios/axios#request-config.

Comment From: sbrannen

Good catch, @rstoyanchev .

Indeed, the QS documentation seems to imply that one of the following should work.

You may use the arrayFormat option to specify the format of the output array:

qs.stringify({ a: ['b', 'c'] }, { arrayFormat: 'indices' })
// 'a[0]=b&a[1]=c'
qs.stringify({ a: ['b', 'c'] }, { arrayFormat: 'brackets' })
// 'a[]=b&a[]=c'
qs.stringify({ a: ['b', 'c'] }, { arrayFormat: 'repeat' })
// 'a=b&a=c'
qs.stringify({ a: ['b', 'c'] }, { arrayFormat: 'comma' })
// 'a=b,c'

brackets seems to be the default.

And either indices or repeat should work with Spring.

So based on the link @rstoyanchev provided, it looks like the following Axios configuration should work for you.

paramsSerializer: function (params) {
    return Qs.stringify(params, {arrayFormat: 'repeat'})
}

Please give that a try and let us know how it works out.

Comment From: kostya05983

Thank you, yes, it's working with axios from js side.

But I think it will be great if variant with brackets will be working with parameterObject, not only without it, and I think it can work without a lot of work.

Yaahoo I understood the reason, while I wrote the example for you. There is an example. Please see controller. There is working example by http://localhost:8080/test/working?types[]=ONE&types[]=TWO and unworking by http://localhost:8080/test/not_working?types[]=ONE&types[]=TWO

When I use filter with object with RequestParam, it's not working. It seachers types, instead of types[] and when types is unrequired, beanUtils init bean and types is null, but then AbstractNestablePropertyAccessor fails in 322 line, I'm not sure why... It would be more clearly if it runs without fail and than I will understand that types are just null and types was not mapped, but it just fails.

P.S. RequestParam is not working in filter object, 'cause kotlin classes required @field modifier, but @RequestParam isn't targetable to value parameter, but it is another story ;)

Comment From: rstoyanchev

Given the property name "types[]", BeanWrapperImpl sees the property key prefix "[" and suffix "]" and matches that to a field called types. As the field is a List, AbstractNestablePropertyAccessor tries to parse the value between "[" and "]" as an index but of course that's empty and it fails with a NumberFormatException.

BeanWrapperImpl could deal with "[]" by adding to a List but that wouldn't be ideal in contexts outside of request data binding where "[]" is less likely to be intentional. It might also have to be inconsistent with fields of type array where it's not as easy to decide at what index to add.

A more likely potential solution is for WebDataBinder and WebExchangeDataBinder to look for such parameters and drop the "[]" from the name, essentially adapting to what we would expect in the case of index-less array/list values. If "[]" is a common client-side convention then making this change for request data binding makes more sense. As for existing applications, a parameter named "xyz[]" where the target class has is a field xyz of any type, would fail today because it will try to treat as array, list, or map, or fail otherwise. So there should be no side effects for existing apps I think.

Comment From: rstoyanchev

Team Decision: We'll drop such a "[]" suffix for web data binding purposes before passing the input to the bean binding infrastructure. The suffix cannot have any other sensible meaning and it would fail currently anyway.

Comment From: quaff

Thank you for reopening. There is the library https://github.com/axios/axios. There is one tricky moment, web application is configurable, but it's standalone and our architecture is microservices in backend. Earlier, we write code like this

kotlin fun getAllEntities(@RequestParam(name="types[]") types: List<String>) {}

And this is working as expected, we use types[]=first_type&types[]=second_type and all is good. After we update one of api and we decide use ParameterObject, such as filterDto

kotlin data class FilterDto( @RequestParam(name="types[]") var types: MutableList<String>, )

And when we use types[]=first_type&types[]=second_type it's not working... Also, I can make mini application with working variant of controller and variant which is not worked.

@RequestParam only works on method parameter, It fails if you extract parameters to DTO, am I wrong? @rstoyanchev

Comment From: rstoyanchev

@RequestParam you mean? Yes it's only on a method argument. That said the goal of the issue is to make data binding work when parameter names follow this empty index [] convention.

Comment From: quaff

@RequestParam you mean? Yes it's only on a method argument. That said the goal of the issue is to make data binding work when parameter names follow this empty index [] convention.

Yes, I mean @RequestParam, I remember somebody open an issue requests such support.

Comment From: jonhiga

Today, I learned that jQuery.ajax() by default will serialize data: {a: ['x', 'y', 'z']} as 'a[]=x,a[]=y,a[]=z', leading to the issue that the original poster raised. Of course, jQuery.ajax() also provides the traditional: true option to force serialization as 'a=x,a=y,a=z', which I prefer anyway. I was disappointed that jQuery and Spring ('a[0]=x,a[1]=y,a[2]=z') have taken different directions when extending the traditional format of array serialization, but I'm glad they are able to find common ground by both respecting the twentieth-century standard.

Comment From: panovst

@rstoyanchev Hello!

How would you like to add support for empty array "[]" suffix to RequestParamMethodArgumentResolver? I have described an solution here #32577

Comment From: crux

When binding multiple values to a list, we only support two syntax variants.

excluding brackets

including brackets and an index

Thus, for your use case types=first_type&types=second_type and types[0]=first_type&types[1]=second_type are supported, but types[]=first_type&types[]=second_type is not.

In light of that, I am closing this issue.

what about arrays with structures data? like class Person { String name; }

person.name: foo person.name: bar

doesnt work.

person[].name: foo person[].name: bar

is not supported.

person[0].name: foo person[1].name: bar

puts the load on the client side to count the entries prio to request. not acceptable.