Since SpringBoot version 3.2 Kafka autoconfiguration is deprecated and marked for removal in version 3.4 https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/autoconfigure/kafka/KafkaProperties.html#buildAdminProperties() and it is recommended to use SSLBundles.

As we are connecting to Kafka cluster using SASL_PLAINTEXT with SCRAM-SHA-512


  kafka:
    security:
      protocol: SASL_PLAINTEXT
    bootstrap-servers: adress_of_the_server
    properties:
      sasl:
        mechanism: SCRAM-SHA-512
        jaas:
          config: org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password";
    jaas:
      enabled: true

how should I configure SslBundle for SASL_PLAINTEXT with SCRAM-SHA-512?

Comment From: wilkinsona

Unfortunately, I don't think I understand the question and how it connects to your snippet of YAML above.

If you don't have any code that's using KafkaProperties then you don't need to do anything. If you're calling buildAdminProperties(), you can call buildAdminProperties(null) instead, but you should note that methods on Boot's @ConfigurationProperties classes aren't really intended for use as public API.

Can you please clarify exactly how you're configuring things today and why you think the deprecation of buildAdminProperties() will affect you?

Comment From: radu-barbu-sage

Sorry for the late response. We need KafkaProperties for building consumer and producer properties. Properties are altered based on the needs of different producers and consumers.

To clarify my question: calling buildAdminProperties(null) is the equivalent of the deprecated buildAdminProperties()?

Comment From: wilkinsona

Yes, it is:

https://github.com/spring-projects/spring-boot/blob/40c188f8f73262e10b618efd5d27c56f08aabae2/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/kafka/KafkaProperties.java#L250-L252

Comment From: radu-barbu-sage

Ok, thanks for clarifications. Just a side question, why deprecating buildAdminProperties? What is the benefit between buildAdminProperties(null) and buildAdminProperties. Same for buildConsumerProperties and buildProducerProperties.

Comment From: wilkinsona

As I said above "methods on Boot's @ConfigurationProperties classes aren't really intended for use as public API" and buildAdminProperties() is no longer needed as Boot will always call buildAdminProperties(SslBundle) either with null or an SslBundle instance.

I'd still like to know why you're calling buildAdminProperties (and buildConsumerProperties and buildProducerProperties by the sounds of it). There may be a better way to do what you're doing or it may be an indicator that we need to provide a truly public API to meet your needs.

Comment From: radu-barbu-sage

We are working on a framework that provide Kafka connectivity to other teams. So we have our 'base' yaml configuration file. These teams will have their own yaml config file which override the base one. As for some framework internal kafka consumers and producers we need to have the specific properties we want during the execution, as other teams will have their properties required for their kafka consumers and produces, etc. build on top of the framework.

Thanks for the fast responses, issue can be closed.

Comment From: steinarb

As I said above "methods on Boot's @ConfigurationProperties classes aren't really intended for use as public API" and buildAdminProperties() is no longer needed as Boot will always call buildAdminProperties(SslBundle) either with null or an SslBundle instance.

I'd still like to know why you're calling buildAdminProperties (and buildConsumerProperties and buildProducerProperties by the sounds of it). There may be a better way to do what you're doing or it may be an indicator that we need to provide a truly public API to meet your needs.

@wilkinsona a note Re: public API for this

We are in the situation where we have both consumers and producers in the code that all use avro, and using config only (in YAML files) worked fine for this.

But now we have a single producer where the topic's consumer expect JSON objects serialized as strings, and we would like to be able to send messages to this topic without throwing out everything we have set up for avro to the other topics we communicate on.

Ideally we would have like to be able to use config to e.g. set a different serializer for a particular record type (key/message type combination) or topic or something. But that's not possible to do today.

So we have to configure producers in code.

But we still would prefer not to configure everything (bootstrap server, schema registry, the SSL config) in code. We would like to use as much as possible of the existing config for this.

Googling found us this stack overflow answer to someone wanting to do the same thing back in 2018 https://stackoverflow.com/a/50310843

But the API used in this example has been deprecated and removed, so we have googled on and landed on this Jira.

So or wish is basically: keep all config but be able to set custom serializers (and possibly deserializers, but we don't need that (yet))

Comment From: wilkinsona

@steinarb thanks for the details. Please open a new issue so that we can track your request. It's likely to get lost/forgotten as a comment on this closed issue.

Comment From: steinarb

@steinarb thanks for the details. Please open a new issue so that we can track your request. It's likely to get lost/forgotten as a comment on this closed issue.

Here is the issue: https://github.com/spring-projects/spring-boot/issues/43363

Comment From: wilkinsona

Thanks.