JMX Metric unit labeling is confusing at best

FROM TS017727981

We are using JMX metrics for datasources

# Java & Custom JMX

com.instana.plugin.java:

jmx:

- object_name: 'oracle.ucp.admin.UniversalConnectionPoolMBean:*'

metrics:

- attributes: 'borrowedConnectionsCount,availableConnectionsCount,maxPoolSize,connectionsClosedCount,connectionsCreatedCount,cumulativeConnectionBorrowedCount,cumulativeConnectionReturnedCount,cumulativeFailedConnectionWaitCount,peakConnectionsCount,pendingRequestsCount,remainingPoolCapacityCount'

type: 'absolute'

The issue we are seeing that values - even live ones from the last hour - are showing decimal values in the unit of "m". The values are much too large for what we expect as they exceed the maximum value and in other cases they are decimal values indicating part of a connection was borrowed or pending - not possible.

A specific example is oracle.ucp.admin.UniversalConnectionPoolMBean/UniversalConnectionPoolManager(1215391846)-a1a7a5d99-2-bizinttom020/jdbc/adjusterDashboard/pendingRequestsCount showing a value of 800.000m . In the 1 hour window we see values of 200, 600, and 800 pending requsts (not in that order). We know from our web logs that we are not seeing the kind of traffic that would be needed to create even 200 pending db connection pool requests. Additionally, we track the exact same data in Splunk and the values are nowhere need 800, 600, or 200. They are more like 1. Our splunk metrics pull directly from the same JMX mbeans that Instana is.

By our configuration, I would expect to see absolute values with no unit as they are counts.

What are these units "m." What can explain the large discrepancy?

Per Support
The notation used for these metrics is the SI notation, which is a standard for metrics and the m that you see stands for millis (not to be confused with milliseconds).

Me
I am not following you. Yes, per https://www.nist.gov/pml/owm/metric-si-prefixes, m means millis. So if I was measuring length, I might write 10mm - meaning 10 milli-meters (dash on purpose for illustration). And yes, that would mean 10/1000 of a m(eter).

In Instana's case there is no measurement specified - it is just m. m what? If I read 800.000m of pendingRequestCount and there is 1000m's in a single pendingRequestCount, then that would mean there was 0.80 pending requests. This is not possible. Pending requests have to be integer values.

Support
We can see how this can be counter intuitive for a metric that you would consider as a round number. This is due to the bucketing that takes place. By default, built-in dashboards show the means over a bucket. In the case of a 1h window, the bucket size is 1 minute. So if there was 1 pending request for a duration of 10s, then over 1 minute, there would have been 0.1 pending requests.