Add Metric-Specific Labels to googlecloudmonitoringreceiver Metrics enrichment

[navaneeth-c]

Component(s)

receiver/googlecloudmonitoring

Is your feature request related to a problem? Please describe.

The Google Cloud Monitoring Receiver currently enriches metrics with resource-level tags but lacks metric-specific labels (e.g., nat_ip for NAT gateway metrics). Without these labels, users miss crucial context directly related to the metric, limiting filtering and grouping capabilities when monitoring specific resources. Including these metric labels would enhance usability by allowing more precise filter, grouping and tracking based on metric-specific attributes.

Describe the solution you'd like

I propose updating the convertGCPTimeSeriesToMetrics function (here) within the googlecloudmonitoringreceiver/receiver.go file to capture and append metric-specific labels alongside resource tags. Specifically, this enhancement would add metric labels from timeSeries.Metric.Labels to the metric attributes, enriching each metric with both resource and metric-specific context.

Suggested Code:

// Add metric-specific labels if they are present 
if len(timeSeries.Metric.Labels) > 0 {
	for k, v := range timeSeries.Metric.Labels {
		resource.Attributes().PutStr(k, fmt.Sprintf("%v", v))
	}
}

This code snippet ensures that metric-specific labels are added only if they are present.

Describe alternatives you've considered

Custom Processor: We considered building a custom processor to add metric labels post-collection. Howeever, this approach requires additional processing steps and does not fully integrate with the Google Cloud Monitoring Receiver.

Additional context

This feature request aims to improve the observability and filtering options for users relying on GoogleCloudMonitoring metrics. By incorporating metric labels, we offer a more complete metric context without adding significant overhead or complexity. If this aligns with the project's goal, I am ready to contribute by implementing this feature in a PR.

[github-actions]

Pinging code owners:

• receiver/googlecloudmonitoring: @dashpole @TylerHelmuth @abhishek-at-cloudwerx

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dashpole]

Oh, dang. We are completely ignoring/dropping metric labels:

opentelemetry-collector-contrib/receiver/googlecloudmonitoringreceiver/receiver.go

Lines 301 to 323 in 27aa54b

	// Create a new Metric
	m := sm.Metrics().AppendEmpty()
	// Set metric name, description, and unit
	m.SetName(metricDesc.GetName())
	m.SetDescription(metricDesc.GetDescription())
	m.SetUnit(metricDesc.Unit)
	// Convert the TimeSeries to the appropriate metric type
	switch timeSeries.GetMetricKind() {
	case metric.MetricDescriptor_GAUGE:
	mr.metricsBuilder.ConvertGaugeToMetrics(timeSeries, m)
	case metric.MetricDescriptor_CUMULATIVE:
	mr.metricsBuilder.ConvertSumToMetrics(timeSeries, m)
	case metric.MetricDescriptor_DELTA:
	mr.metricsBuilder.ConvertDeltaToMetrics(timeSeries, m)
	// TODO: Add support for HISTOGRAM
	// TODO: Add support for EXPONENTIAL_HISTOGRAM
	default:
	metricError := fmt.Sprintf("\n Unsupported metric kind: %v\n", timeSeries.GetMetricKind())
	mr.logger.Info(metricError)
	}
	}

[dashpole]

@abhishek-at-cloudwerx is this something you were planning to do?

[andrewegel]

Started using this receiver at my organization - Not adding the metric label can cause miss-reporting; You can see this in the metric router.googleapis.com/nat/sent_bytes_count where if the ip_protocol label isn't added, we were getting reported values of 0 when Google showed these as having vlaues; We found that the Zeros were the cases where ip_protocol=1 or ip_protocol=17, but when ip_protocol=6 there were actual reported values.

I don't have enough knowledge of Open Telemetry's internals to know if thats an issue or not, but we're using the prometheusremotewrite exporter, and since all three series without a ip_protocol attribute, the values for ip_protocol=6 were "lost" when values for ip_protocol=1 or ip_protocol=17 were added.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• receiver/googlecloudmonitoring: @dashpole @TylerHelmuth @abhishek-at-cloudwerx

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dashpole]

Fixed by ##35828. Unit tests to follow