Natvie Prometheus support

Instana supports Prometheus in a customised way to transfer and work with Prometheus metrics. Basically, Prometheus metrics can be collected and transferred to Instana.

But core features like Metrics Labels do not work with the Instana backend in events or dashboards.

We would like to get a better implementation of Prometheus Metrics. From our point of view this is necessary for the following reasons:

• More and more services (operators) especially in the Kubernetes world are implementing prometheus endpoints for monitoring.

• It is very easy and widespread to implement the endpoint and therefore also a great desire for our developers.

• Broad community with many pre-built dashboards/alarms

• Our developers/engineers demand it as standard

We would like to have a Prometheus implementation that

• Collects and imports metrics in native Prometheus format (i.e. with labels)

• The use of metrics labels is possible in events and dashboards e.g. to create one event that targets multiple metrics based on labels

• It is possible to perform simple arithmetic calculations with the metrics (in dashboards and events)

Since custom metrics are hard to use in Instana and Prometheus is not very well supported, developers are getting creative. As a result, there are now scripts that query Prometheus endpoints and trigger alerts directly (without instana). Another common pattern is writing log messages and generating metrics from them in Datadog. This is obviously not the way we (the platform team) want to handle metrics as it is unreliable and hard to maintain.

Datadog could serve as a template of how it could work:
https://docs.datadoghq.com/containers/kubernetes/prometheus/?tab=kubernetesadv2