Commit 7b85a5d
authored
* Add BatchSplittingMetricExporter to prevent OTLP gRPC export failures
When high-cardinality metrics (consumer_group x topic) produce OTLP export
payloads exceeding the gRPC 32MB limit or SLS per-RPC processing limit,
all metrics fail to export. This adds a MetricExporter decorator that:
- Splits large batches of MetricData objects into smaller sub-batches
- Splits single oversized MetricData objects by their internal data points
into multiple smaller MetricData objects (supports all 7 MetricDataType)
- Configurable via BrokerConfig.metricsExportBatchMaxDataPoints (default 1000)
- Fast path with zero overhead when data points are within threshold
- Logs failed batch details for debugging
* fix(metrics): snapshot MetricData points before export to prevent AIOOBE
The OTel SDK's NumberDataPointMarshaler.createRepeated allocates an
array based on points.size() then iterates. If callback threads
concurrently add data points between size() and iteration, an
ArrayIndexOutOfBoundsException occurs. This adds a defensive snapshot
of all data point collections at the start of export(), ensuring
the delegate exporter always receives immutable point collections.
* test(metrics): add unit tests for snapshot defensive copy
- testSnapshotCreatesNewMetricData: verify delegate receives
snapshotted MetricData, not the original reference
- testSnapshotFallsBackToOriginal: verify catch block falls
back to original when snapshot fails (e.g., mock without type)
- testSnapshotPointsAreIndependentCopy: verify the snapshotted
points collection is a separate instance from the original
1 parent 2c2cc92 commit 7b85a5d
File tree
4 files changed
+1266
-2
lines changed- broker/src
- main/java/org/apache/rocketmq/broker/metrics
- test/java/org/apache/rocketmq/broker/metrics
- common/src/main/java/org/apache/rocketmq/common
4 files changed
+1266
-2
lines changed
0 commit comments