SpringBoot Metrics percentiles throws 500 when running as native

Example application to reproduce:

@SpringBootApplication
class DemoApplication

fun main(args: Array<String>) {
    runApplication<DemoApplication>(*args)
}

@RestController
@RequestMapping("/")
class TestController {

    @GetMapping
    fun hello(): String {
        return "Hello world"
    }

}

Next we add actuator and prometheus dependencies to build:

plugins {
    id("org.springframework.boot") version "3.0.2"
    id("io.spring.dependency-management") version "1.1.0"
    id("org.graalvm.buildtools.native") version "0.9.18"
    kotlin("jvm") version "1.7.22"
    kotlin("plugin.spring") version "1.7.22"
}

group = "com.example"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_17

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-webflux")
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
    implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
    implementation("org.jetbrains.kotlin:kotlin-reflect")
    implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")

    implementation("org.springframework.boot:spring-boot-starter-actuator")
    implementation("io.micrometer:micrometer-registry-prometheus")

    testImplementation("org.springframework.boot:spring-boot-starter-test")
    testImplementation("io.projectreactor:reactor-test")
}

tasks.withType<KotlinCompile> {
    kotlinOptions {
        freeCompilerArgs = listOf("-Xjsr305=strict")
        jvmTarget = "17"
    }
}

tasks.withType<Test> {
    useJUnitPlatform()
}

and enable percentiles reporting for incoming requests in application.yml

application.yml:

management:
  metrics:
    distribution:
      percentiles:
        http:
          server:
            requests: 0.98
  endpoints:
    web:
      exposure:
        include: prometheus

Run with: ./gradlew clean nativeRun

Hit http://localhost:8080/ and next http://localhost:8080/actuator/prometheus to get registered metrics values. Response is 500

An exception in logs can be found:

2023-02-15T13:18:42.546+02:00 ERROR 77491 --- [oundedElastic-1] a.w.r.e.AbstractErrorWebExceptionHandler : [22d47a73-3]  500 Server Error for HTTP GET "/actuator/prometheus"

java.lang.IllegalArgumentException: java.lang.NoSuchMethodException: org.HdrHistogram.ConcurrentHistogram.<init>(org.HdrHistogram.AbstractHistogram)
        at org.HdrHistogram.DoubleHistogram.<init>(DoubleHistogram.java:220) ~[demo:2.1.12]
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
        *__checkpoint ⇢ Handler org.springframework.boot.actuate.endpoint.web.reactive.AbstractWebFluxEndpointHandlerMapping$ReadOperationHandler#handle(ServerWebExchange) [DispatcherHandler]
        *__checkpoint ⇢ org.springframework.web.filter.reactive.ServerHttpObservationFilter [DefaultWebFilterChain]
        *__checkpoint ⇢ HTTP GET "/actuator/prometheus" [ExceptionHandlingWebHandler]
Original Stack Trace:
                at org.HdrHistogram.DoubleHistogram.<init>(DoubleHistogram.java:220) ~[demo:2.1.12]
                at org.HdrHistogram.DoubleHistogram.<init>(DoubleHistogram.java:230) ~[demo:2.1.12]
                at org.HdrHistogram.ConcurrentDoubleHistogram.<init>(ConcurrentDoubleHistogram.java:92) ~[demo:2.1.12]
                at org.HdrHistogram.DoubleRecorder$InternalConcurrentDoubleHistogram.<init>(DoubleRecorder.java:322) ~[na:na]
                at org.HdrHistogram.DoubleRecorder$InternalConcurrentDoubleHistogram.<init>(DoubleRecorder.java:306) ~[na:na]
                at org.HdrHistogram.DoubleRecorder.performIntervalSample(DoubleRecorder.java:275) ~[demo:2.1.12]
                at org.HdrHistogram.DoubleRecorder.getIntervalHistogramInto(DoubleRecorder.java:254) ~[demo:2.1.12]
                at io.micrometer.core.instrument.distribution.TimeWindowPercentileHistogram.accumulate(TimeWindowPercentileHistogram.java:73) ~[na:na]
                at io.micrometer.core.instrument.distribution.AbstractTimeWindowHistogram.accumulateIfStale(AbstractTimeWindowHistogram.java:181) ~[demo:1.10.3]
                at io.micrometer.core.instrument.distribution.AbstractTimeWindowHistogram.takeSnapshot(AbstractTimeWindowHistogram.java:171) ~[demo:1.10.3]
                at io.micrometer.core.instrument.AbstractTimer.takeSnapshot(AbstractTimer.java:247) ~[demo:1.10.3]
                at io.micrometer.prometheus.PrometheusTimer.takeSnapshot(PrometheusTimer.java:137) ~[na:na]
                at io.micrometer.prometheus.PrometheusMeterRegistry.lambda$addDistributionStatisticSamples$16(PrometheusMeterRegistry.java:448) ~[demo:1.10.3]
                at io.micrometer.prometheus.MicrometerCollector.collect(MicrometerCollector.java:75) ~[na:na]
                at io.prometheus.client.Collector.collect(Collector.java:45) ~[demo:na]
                at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:204) ~[na:na]
                at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:219) ~[na:na]
                at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:152) ~[na:na]
                at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:71) ~[na:na]
                at org.springframework.boot.actuate.metrics.export.prometheus.TextOutputFormat$1.write(TextOutputFormat.java:45) ~[na:na]
                at org.springframework.boot.actuate.metrics.export.prometheus.PrometheusScrapeEndpoint.scrape(PrometheusScrapeEndpoint.java:62) ~[demo:3.0.2]
                at java.base@17.0.6/java.lang.reflect.Method.invoke(Method.java:568) ~[demo:na]
                at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:281) ~[na:na]
                at org.springframework.boot.actuate.endpoint.invoke.reflect.ReflectiveOperationInvoker.invoke(ReflectiveOperationInvoker.java:74) ~[na:na]
                at org.springframework.boot.actuate.endpoint.annotation.AbstractDiscoveredOperation.invoke(AbstractDiscoveredOperation.java:60) ~[demo:3.0.2]
                at org.springframework.boot.actuate.endpoint.web.reactive.AbstractWebFluxEndpointHandlerMapping$ElasticSchedulerInvoker.lambda$invoke$0(AbstractWebFluxEndpointHandlerMapping.java:245) ~[na:na]

It works when it's run as JVM application with ./gradlew clean bootRun.

Comment From: wilkinsona

Thanks for the report.

This needs to be addressed with some improvements to the reachability metadata for org.hdrhistrogram:HdrHistogram library to allow reflective calls to the org.HdrHistogram.ConcurrentHistogram.ConcurrentHistogram(AbstractHistogram) constructor. Please open a reachability metadata issue and comment here with a link to it.

In the meantime, you can provide some reflection config in your app by adding to src/main/resources a META-INF/native-image/reflect-config.json with the following contents:

[
    {
        "name": "org.HdrHistogram.ConcurrentHistogram",
        "methods": [
            {
                "name": "<init>",
                "parameterTypes": [
                    "org.HdrHistogram.AbstractHistogram"
                ]
            }
        ]
    }
]

Comment From: gzeskas

Created a ticket in graalvm reacabilty: https://github.com/oracle/graalvm-reachability-metadata/issues/234

Comment From: wilkinsona

Thanks, @gzeskas.