[7.17](backport #33654) Azure Platform Logs: rename identity as ident…

…ity_name when the value is a string (#33957) * Azure Platform Logs: rename identity as identity_name when the value is a string (#33654) * Rename identity as identity_name when is a string The identity field comes in different flavors, depending on the specific log category. If it comes as a string, the pipeline renames it as identity_name to avoid collisions when the value is an object. (cherry picked from commit ea9b0cb) # Conflicts: # x-pack/filebeat/module/azure/fields.go * Remove extra entries in changelog * Update fields.go * Remove extra entries in changelog Co-authored-by: Maurizio Branca <[email protected]> Co-authored-by: Maurizio Branca <[email protected]>
elastic · Dec 12, 2022 · 8edcc0b · 8edcc0b
1 parent 505cd00
commit 8edcc0b
Show file tree

Hide file tree

Showing 7 changed files with 55 additions and 1 deletion.
diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc
@@ -38,6 +38,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 
 *Filebeat*
 
+- Rename identity as identity_name when the value is a string in Azure Platform Logs. {pull}33654[33654]
 
 *Heartbeat*
 

diff --git a/filebeat/docs/fields.asciidoc b/filebeat/docs/fields.asciidoc
@@ -3216,6 +3216,16 @@ type: keyword
 ActivityId
 
 
+type: keyword
+
+--
+
+*`azure.platformlogs.identity_name`*::
++
+--
+Identity name
+
+
 type: keyword
 
 --

diff --git a/x-pack/filebeat/module/azure/fields.go b/x-pack/filebeat/module/azure/fields.go
diff --git a/x-pack/filebeat/module/azure/platformlogs/_meta/fields.yml b/x-pack/filebeat/module/azure/platformlogs/_meta/fields.yml
@@ -57,6 +57,10 @@
       type: keyword
       description: >
         ActivityId
+    - name: identity_name
+      type: keyword
+      description: |
+        Identity name
     - name: properties
       type: flattened
       description: >

diff --git a/x-pack/filebeat/module/azure/platformlogs/ingest/pipeline.yml b/x-pack/filebeat/module/azure/platformlogs/ingest/pipeline.yml
@@ -15,6 +15,12 @@ processors:
 - json:
     field: message
     target_field: azure.platformlogs
+- rename:
+    field: azure.platformlogs.identity
+    target_field: azure.platformlogs.identity_name
+    ignore_missing: true
+    description: 'Rename the field to `identity_name` to avoid conflicts with the `identity` containing a JSON object.'
+    if: "ctx.azure?.platformlogs?.identity instanceof String"
 - date:
     field: azure.platformlogs.time
     target_field: '@timestamp'

diff --git a/x-pack/filebeat/module/azure/platformlogs/test/platformlogs-identity-raw.log b/x-pack/filebeat/module/azure/platformlogs/test/platformlogs-identity-raw.log
@@ -0,0 +1 @@
+{"Cloud":"AzureCloud","Environment":"prod","category":"kube-audit","ccpNamespace":"5e4bf4baee195b00017cdbfa","identity":"Michael Dell","operationName":"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read","properties":{"log":"{\"kind\":\"Event\"}","pod":"kube-apiserver-666bd4b459-hjgdc"},"resourceId":"/SUBSCRIPTIONS/70BD6E77-4B1E-4835-8896-DB77B8EEF364/RESOURCEGROUPS/OBS-INFRASTRUCTURE/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/OBSKUBE","time":"2020-11-09T10:57:31.0000000Z"}
diff --git a/x-pack/filebeat/module/azure/platformlogs/test/platformlogs-identity-raw.log-expected.json b/x-pack/filebeat/module/azure/platformlogs/test/platformlogs-identity-raw.log-expected.json
@@ -0,0 +1,32 @@
+[
+    {
+        "@timestamp": "2020-11-09T10:57:31.000Z",
+        "azure.platformlogs.Cloud": "AzureCloud",
+        "azure.platformlogs.Environment": "prod",
+        "azure.platformlogs.category": "kube-audit",
+        "azure.platformlogs.ccpNamespace": "5e4bf4baee195b00017cdbfa",
+        "azure.platformlogs.event_category": "Administrative",
+        "azure.platformlogs.identity_name": "Michael Dell",
+        "azure.platformlogs.operation_name": "Microsoft.ContainerService/managedClusters/diagnosticLogs/Read",
+        "azure.platformlogs.properties.log.kind": "Event",
+        "azure.platformlogs.properties.pod": "kube-apiserver-666bd4b459-hjgdc",
+        "azure.resource.group": "OBS-INFRASTRUCTURE",
+        "azure.resource.id": "/SUBSCRIPTIONS/70BD6E77-4B1E-4835-8896-DB77B8EEF364/RESOURCEGROUPS/OBS-INFRASTRUCTURE/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/OBSKUBE",
+        "azure.resource.name": "OBSKUBE",
+        "azure.resource.provider": "MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS",
+        "azure.subscription_id": "70BD6E77-4B1E-4835-8896-DB77B8EEF364",
+        "cloud.provider": "azure",
+        "event.action": "Microsoft.ContainerService/managedClusters/diagnosticLogs/Read",
+        "event.dataset": "azure.platformlogs",
+        "event.kind": "event",
+        "event.module": "azure",
+        "event.original": "{\"Cloud\":\"AzureCloud\",\"Environment\":\"prod\",\"category\":\"kube-audit\",\"ccpNamespace\":\"5e4bf4baee195b00017cdbfa\",\"identity\":\"Michael Dell\",\"operationName\":\"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read\",\"properties\":{\"log\":\"{\\\"kind\\\":\\\"Event\\\"}\",\"pod\":\"kube-apiserver-666bd4b459-hjgdc\"},\"resourceId\":\"/SUBSCRIPTIONS/70BD6E77-4B1E-4835-8896-DB77B8EEF364/RESOURCEGROUPS/OBS-INFRASTRUCTURE/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/OBSKUBE\",\"time\":\"2020-11-09T10:57:31.0000000Z\"}",
+        "fileset.name": "platformlogs",
+        "input.type": "log",
+        "log.offset": 0,
+        "service.type": "azure",
+        "tags": [
+            "forwarded"
+        ]
+    }
+]
Original file line number	Diff line number	Diff line change
Expand Up		@@ -38,6 +38,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d

		Filebeat

		- Rename identity as identity_name when the value is a string in Azure Platform Logs. {pull}33654[33654]

		Heartbeat

Expand Down
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		{"Cloud":"AzureCloud","Environment":"prod","category":"kube-audit","ccpNamespace":"5e4bf4baee195b00017cdbfa","identity":"Michael Dell","operationName":"Microsoft.ContainerService/managedClusters/diagnosticLogs/Read","properties":{"log":"{\"kind\":\"Event\"}","pod":"kube-apiserver-666bd4b459-hjgdc"},"resourceId":"/SUBSCRIPTIONS/70BD6E77-4B1E-4835-8896-DB77B8EEF364/RESOURCEGROUPS/OBS-INFRASTRUCTURE/PROVIDERS/MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/OBSKUBE","time":"2020-11-09T10:57:31.0000000Z"}