I had created a Spring-boot micro-service using spring-cloud-sleuth for distributed-tracing. I had a otel-collector which was receiving logs and traces from the application and it was exporting it to Datadog. The flow was working properly.
However, starting from Spring-boot 3.0.5, support of sleuth is removed and observability has been split into two parts some in Spring-boot itself and some part in micrometer. I upgraded my spring-boot version and made changes to export traces to otel-collector . But the application is not exporting the traces to otel-collector. However, if I simply add the maven dependency for Zipkin, the application tries to export traces to Zipkin and shows error since Zipkin is not running at default expected address.
I enabled the Spring debug logs and I can see that the bean for OpenTelemetryAutoConfiguration is being created. In fact, I can see all the beans defined in this class are created. I have created a small reproducer with a very few files and can be found out here. I am posting the debug logs related to this class from the application:-
OpenTelemetryAutoConfiguration matched:
- @ConditionalOnClass found required classes 'io.micrometer.tracing.otel.bridge.OtelTracer', 'io.opentelemetry.sdk.trace.SdkTracerProvider', 'io.opentelemetry.api.OpenTelemetry' (OnClassCondition)
- @ConditionalOnProperty (management.tracing.enabled) matched (OnPropertyCondition)
OpenTelemetryAutoConfiguration#micrometerOtelTracer matched:
- @ConditionalOnMissingBean (types: io.micrometer.tracing.Tracer; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#openTelemetry matched:
- @ConditionalOnMissingBean (types: io.opentelemetry.api.OpenTelemetry; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelContextPropagators matched:
- @ConditionalOnMissingBean (types: io.opentelemetry.context.propagation.ContextPropagators; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelCurrentTraceContext matched:
- @ConditionalOnMissingBean (types: io.micrometer.tracing.otel.bridge.OtelCurrentTraceContext; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelPropagator matched:
- @ConditionalOnMissingBean (types: io.micrometer.tracing.otel.bridge.OtelPropagator; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelSampler matched:
- @ConditionalOnMissingBean (types: io.opentelemetry.sdk.trace.samplers.Sampler; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelSdkTracerProvider matched:
- @ConditionalOnMissingBean (types: io.opentelemetry.sdk.trace.SdkTracerProvider; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelSlf4JEventListener matched:
- @ConditionalOnMissingBean (types: io.micrometer.tracing.otel.bridge.Slf4JEventListener; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelSpanCustomizer matched:
- @ConditionalOnMissingBean (types: io.micrometer.tracing.SpanCustomizer; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelTracer matched:
- @ConditionalOnMissingBean (types: io.opentelemetry.api.trace.Tracer; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration#otelTracerEventPublisher matched:
- @ConditionalOnMissingBean (types: io.micrometer.tracing.otel.bridge.OtelTracer$EventPublisher; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration.BaggageConfiguration matched:
- @ConditionalOnProperty (management.tracing.baggage.enabled) matched (OnPropertyCondition)
OpenTelemetryAutoConfiguration.BaggageConfiguration#otelSlf4JBaggageEventListener matched:
- @ConditionalOnProperty (management.tracing.baggage.correlation.enabled) matched (OnPropertyCondition)
- @ConditionalOnMissingBean (types: io.micrometer.tracing.otel.bridge.Slf4JBaggageEventListener; SearchStrategy: all) did not find any beans (OnBeanCondition)
OpenTelemetryAutoConfiguration.BaggageConfiguration#w3cTextMapPropagatorWithBaggage matched:
- @ConditionalOnProperty (management.tracing.propagation.type=W3C) matched (OnPropertyCondition)
Comment From: jonatan-ivanov
Duplicate of https://github.com/spring-projects/spring-boot/pull/34508.
I'm a little bit confused since the title is about logs but the description is mostly about tracing.
For logs, you will need an extra property, e.g.:
logging.pattern.level=%5p [${spring.application.name:},%X{traceId:-},%X{spanId:-}]
See: sample
For tracing and OTLP, you will need this to be merged: https://github.com/spring-projects/spring-boot/pull/34508.
Right now, you can add io.opentelemetry:opentelemetry-exporter-otlp to your classpath and create an OtlpHttpSpanExporter @Bean as you see in the PR above.
I took a look at the reproducer too, there are a few issues you might want to fix:
- Do not define io.opentelemetry.version, let the BOMs do it for you
- You don't need spring-cloud-starter-config (nor the properties for config-server)
- micrometer-observation, opentelemetry-sdk-trace, opentelemetry-sdk, opentelemetry-extension-trace-propagators, opentelemetry-instrumentation-annotations, micrometer-tracing are not necessary
- You only need: web, actuator, micrometer-tracing-bridge-otel, and opentelemetry-exporter-otlp (see above)
- You don't need @ComponentScan
As I told you in email and on Micrometer Slack about the Micrometer Samples, e.g.: https://github.com/micrometer-metrics/micrometer-samples/tree/main/micrometer-samples-boot3-web
This sample will use opentelemetry-exporter-zipkin but as I mentioned above, you should add io.opentelemetry:opentelemetry-exporter-otlp right now to your classpath and create an OtlpHttpSpanExporter @Bean as you see in https://github.com/spring-projects/spring-boot/pull/34508. Hopefully this will not be needed in Boot 3.1.
Please let us know if I misunderstood something, we can reopen this issue.