In spring-cloud-sleuth there was a property spring.zipkin.service.name which could be used to set span sender name that will appear in Zipkin. After sleuth decommission and upgrading to Spring Boot 3.1, the actuator autoconfiguration module always sets it to spring.application.name.
https://github.com/spring-projects/spring-boot/blob/3.1.x/spring-boot-project/spring-boot-actuator-autoconfigure/src/main/java/org/springframework/boot/actuate/autoconfigure/tracing/BraveAutoConfiguration.java#L121
I solved this issue by creating a custom TracingCustomizer bean in my configuration, but configuration via properties looks more convenient. What do you think?
Comment From: wilkinsona
I assume that you can't set spring.application.name for some reason. Can you please explain why that's not an option? Without such an explanation it will be hard to justify introducing a second property.
Comment From: chillb0nes
In our dev environment there are multiple k8s namespaces. We used a single Zipkin instance to collect traces from all namespaces. To quickly filter traces from service in a specific namespace, we added the namespace name to the microservice name (spring.application.name) using spring.zipkin.service.name. However, it would be inconvenient to include namespace information into spring.application.name directly, because it is used in other parts of the codebase.
Comment From: wilkinsona
Thanks.
In addition to BraveAutoConfiguration, there's also similar usage with Otel:
https://github.com/spring-projects/spring-boot/blob/7e5bfc4b2e88eb4a841df3108ee4344d2e486817/spring-boot-project/spring-boot-actuator-autoconfigure/src/main/java/org/springframework/boot/actuate/autoconfigure/tracing/OpenTelemetryAutoConfiguration.java#L105-L108
spring.application.name is also used with Wavefront but only as a fallback if management.wavefront.application.service-name hasn't been set:
https://github.com/spring-projects/spring-boot/blob/7e5bfc4b2e88eb4a841df3108ee4344d2e486817/spring-boot-project/spring-boot-actuator-autoconfigure/src/main/java/org/springframework/boot/actuate/autoconfigure/wavefront/WavefrontAutoConfiguration.java#L49-L54
Lastly, on the metrics side of things, there's also some usage with Prometheus:
https://github.com/spring-projects/spring-boot/blob/7e5bfc4b2e88eb4a841df3108ee4344d2e486817/spring-boot-project/spring-boot-actuator-autoconfigure/src/main/java/org/springframework/boot/actuate/autoconfigure/metrics/export/prometheus/PrometheusMetricsExportAutoConfiguration.java#L149-L153
Wavefront feels like an odd one out here and I think it would be good to remove that inconsistency. We'll need to decide how we want to do that. Should we:
- Remove the Wavefront-specific property and focus on
spring.application.name? - Add one new property that's used everywhere including in place of the Wavefront property?
- Add several new properties, one for each place where
spring.application.nameis used?
I've asked the observability team to take a look.
Comment From: jonatan-ivanov
Hi @chillb0nes,
I'm quite sure I don't have all the context but based on the information above I don’t think introducing a new property for this is a good idea. The application name is used for logs and can be used in other places like metrics, info endpoint, etc. Easily enabling to make them different does not seem like a good idea to me because it can cause confusion when different outputs have different application names.
Based on this information, if I were you, I would try to add a namespace KeyValue using an ObservationFilter so you can search by serviceName=demo namespace=dev123 (in metrics, traces, logs, you can also make this information available on the info endpoint, etc). Doing this should be possible via a single property (from Boot 3.2), right now it is a "one-liner" @Bean (example), see the connected issue: https://github.com/spring-projects/spring-boot/issues/33241. Also, adding it as a separate KeyValue is more explicit and flexible than encoding the namespace in the name, e.g.: you don't need to know the format and you can also filter them separately.
I would be curious what are you using the app name for in other parts of the codebase that prevents you adding whatever app name you want.
Comment From: wilkinsona
@shakuzen made a good point on Slack about Wavefront that explains what I described as an inconsistency above:
Wavefront is a bit of an odd one in that it has two distinct concepts of application AND service, whereas everything else tends to just have one concept (although it may variously be called one or the other).
Given this, I don't think we need to do anything for consistency's sake.
Comment From: chillb0nes
Hi @jonatan-ivanov,
That's a nice option too, thanks. The application name in our code is most often used together with the namespace, but they are provided as separate fields, parameters, etc. Providing the namespace both in the service name and in a separate field just because it is more handy to search traces in Zipkin seems superfluous.
Comment From: jonatan-ivanov
Providing the namespace both in the service name and in a separate field just because it is more handy to search traces in Zipkin seems superfluous.
I might not be perfectly clear on this, I would not encode the namespace in the app name, I would only add a tag with the namespace and leave the app name being only the name of the app. To me encoding the namespace into the app name seems quite hacky, confusing and it does not reflect reality. Adding the namespace separately as a tag does reflect reality (every app has a name and a namespace). That way it is easier to search for the app name or the namespace or both together. Also, you don't need to know the specific format, e.g.: if one sees abc:xyz, which one is the namespace/app name?
This is true both for tracing and metrics, please take a look at Micrometer docs and check the section about dimensional vs. hierarchical backends (it's about metrics but please bear with me). Dimensional systems add data as separate key-value pairs (~like a hashmap) while hierarchical systems are trying to encode everything into the name (~like a single string). Zipkin has key-values (tags), additional metadata should be added as such and not appended into a single string.
Comment From: chillb0nes
I got it, I answered this question:
I would be curious what are you using the app name for in other parts of the codebase that prevents you adding whatever app name you want.
Adding namespace as a tag is a nice option. I guess it just happened historically that the namespace was encoded in the service name for Zipkin in our code because spring-cloud-sleuth provides a property for that and it came in handy.
I feel that property-based customization is easier, and it also could make configurability of OTel and Brave consistent with Wavefront. But too many properties is not good too. If using different names for the spring application and service in the monitoring system is not a common pattern maybe, it might be better to leave it as it is.
Comment From: wilkinsona
Thanks for the discussion and suggestions. It sounds like we should leave this as it is.