Soft delete feature implementation for Google Cloud Storage

Backport hashicorp/terraform-provider-google-beta#7119

Adds support for Soft Delete feature, which allows setting soft delete policy on 'google_storage_bucket' resource.

This fixes hashicorp/terraform-provider-google#17127

storage: added 'soft_delete_policy' to 'google_storage_bucket' resource
Derived from GoogleCloudPlatform/magic-modules#10171

config/crds/resources/apiextensions.k8s.io_v1_customresourcedefinition_storagebuckets.storage.cnrm.cloud.google.com.yaml

@@ -287,6 +287,23 @@ spec:
                 required:
                 - retentionPeriod
                 type: object
+              softDeletePolicy:
+                description: The bucket's soft delete policy, which defines the period
+                  of time that soft-deleted objects will be retained, and cannot be
+                  permanently deleted. If it is not provided, by default Google Cloud
+                  Storage sets this to default soft delete policy.
+                properties:
+                  effectiveTime:
+                    description: Server-determined value that indicates the time from
+                      which the policy, or one with a greater retention, was effective.
+                      This value is in RFC 3339 format.
+                    type: string
+                  retentionDurationSeconds:
+                    description: The duration in seconds that soft-deleted objects
+                      in the bucket will be retained and cannot be permanently deleted.
+                      Default value is 604800.
+                    type: integer
+                type: object
               storageClass:
                 description: 'The Storage Class of the new bucket. Supported values
                   include: STANDARD, MULTI_REGIONAL, REGIONAL, NEARLINE, COLDLINE,

scripts/generate-google3-docs/resource-reference/generated/resource-docs/storage/storagebucket.md

@@ -156,6 +156,9 @@ resourceID: string
 retentionPolicy:
   isLocked: boolean
   retentionPeriod: integer
+softDeletePolicy:
+  effectiveTime: string
+  retentionDurationSeconds: integer
 storageClass: string
 uniformBucketLevelAccess: boolean
 versioning:
@@ -684,6 +687,36 @@ Enables Bucket PolicyOnly access to a bucket.{% endverbatim %}</p>
             <p>{% verbatim %}The period of time, in seconds, that objects in the bucket must be retained and cannot be deleted, overwritten, or archived. The value must be less than 3,155,760,000 seconds.{% endverbatim %}</p>
         </td>
     </tr>
+    <tr>
+        <td>
+            <p><code>softDeletePolicy</code></p>
+            <p><i>Optional</i></p>
+        </td>
+        <td>
+            <p><code class="apitype">object</code></p>
+            <p>{% verbatim %}The bucket's soft delete policy, which defines the period of time that soft-deleted objects will be retained, and cannot be permanently deleted. If it is not provided, by default Google Cloud Storage sets this to default soft delete policy.{% endverbatim %}</p>
+        </td>
+    </tr>
+    <tr>
+        <td>
+            <p><code>softDeletePolicy.effectiveTime</code></p>
+            <p><i>Optional</i></p>
+        </td>
+        <td>
+            <p><code class="apitype">string</code></p>
+            <p>{% verbatim %}Server-determined value that indicates the time from which the policy, or one with a greater retention, was effective. This value is in RFC 3339 format.{% endverbatim %}</p>
+        </td>
+    </tr>
+    <tr>
+        <td>
+            <p><code>softDeletePolicy.retentionDurationSeconds</code></p>
+            <p><i>Optional</i></p>
+        </td>
+        <td>
+            <p><code class="apitype">integer</code></p>
+            <p>{% verbatim %}The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Default value is 604800.{% endverbatim %}</p>
+        </td>
+    </tr>
     <tr>
         <td>
             <p><code>storageClass</code></p>

third_party/github.com/hashicorp/terraform-provider-google-beta/google-beta/services/storage/resource_storage_bucket.go

@@ -438,6 +438,28 @@ func ResourceStorageBucket() *schema.Resource {
 				Computed:    true,
 				Description: `Prevents public access to a bucket.`,
 			},
+			"soft_delete_policy": {
+				Type:        schema.TypeList,
+				MaxItems:    1,
+				Optional:    true,
+				Computed:    true,
+				Description: `The bucket's soft delete policy, which defines the period of time that soft-deleted objects will be retained, and cannot be permanently deleted. If it is not provided, by default Google Cloud Storage sets this to default soft delete policy`,
+				Elem: &schema.Resource{
+					Schema: map[string]*schema.Schema{
+						"retention_duration_seconds": {
+							Type:        schema.TypeInt,
+							Default:     604800,
+							Optional:    true,
+							Description: `The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Default value is 604800.`,
+						},
+						"effective_time": {
+							Type:        schema.TypeString,
+							Computed:    true,
+							Description: `Server-determined value that indicates the time from which the policy, or one with a greater retention, was effective. This value is in RFC 3339 format.`,
+						},
+					},
+				},
+			},
 		},
 		UseJSONNumber: true,
 	}
@@ -563,6 +585,10 @@ func resourceStorageBucketCreate(d *schema.ResourceData, meta interface{}) error
 		sb.CustomPlacementConfig = expandBucketCustomPlacementConfig(v.([]interface{}))
 	}
 
+	if v, ok := d.GetOk("soft_delete_policy"); ok {
+		sb.SoftDeletePolicy = expandBucketSoftDeletePolicy(v.([]interface{}))
+	}
+
 	var res *storage.Bucket
 
 	err = transport_tpg.Retry(transport_tpg.RetryOptions{
@@ -722,6 +748,12 @@ func resourceStorageBucketUpdate(d *schema.ResourceData, meta interface{}) error
 		sb.IamConfiguration = expandIamConfiguration(d)
 	}
 
+	if d.HasChange("soft_delete_policy") {
+		if v, ok := d.GetOk("soft_delete_policy"); ok {
+			sb.SoftDeletePolicy = expandBucketSoftDeletePolicy(v.([]interface{}))
+		}
+	}
+
 	res, err := config.NewStorageClient(userAgent).Buckets.Patch(d.Get("name").(string), sb).Do()
 	if err != nil {
 		return err
@@ -1083,6 +1115,32 @@ func flattenBucketRetentionPolicy(bucketRetentionPolicy *storage.BucketRetention
 	return bucketRetentionPolicies
 }
 
+func expandBucketSoftDeletePolicy(configured interface{}) *storage.BucketSoftDeletePolicy {
+	configuredSoftDeletePolicies := configured.([]interface{})
+	if len(configuredSoftDeletePolicies) == 0 {
+		return nil
+	}
+	configuredSoftDeletePolicy := configuredSoftDeletePolicies[0].(map[string]interface{})
+	softDeletePolicy := &storage.BucketSoftDeletePolicy{
+		RetentionDurationSeconds: int64(configuredSoftDeletePolicy["retention_duration_seconds"].(int)),
+	}
+	softDeletePolicy.ForceSendFields = append(softDeletePolicy.ForceSendFields, "RetentionDurationSeconds")
+	return softDeletePolicy
+}
+
+func flattenBucketSoftDeletePolicy(softDeletePolicy *storage.BucketSoftDeletePolicy) []map[string]interface{} {
+	policies := make([]map[string]interface{}, 0, 1)
+	if softDeletePolicy == nil {
+		return policies
+	}
+	policy := map[string]interface{}{
+		"retention_duration_seconds": softDeletePolicy.RetentionDurationSeconds,
+		"effective_time":             softDeletePolicy.EffectiveTime,
+	}
+	policies = append(policies, policy)
+	return policies
+}
+
 func expandBucketVersioning(configured interface{}) *storage.BucketVersioning {
 	versionings := configured.([]interface{})
 	if len(versionings) == 0 {
@@ -1599,6 +1657,9 @@ func setStorageBucket(d *schema.ResourceData, config *transport_tpg.Config, res
 	if err := d.Set("custom_placement_config", flattenBucketCustomPlacementConfig(res.CustomPlacementConfig)); err != nil {
 		return fmt.Errorf("Error setting custom_placement_config: %s", err)
 	}
+	if err := d.Set("soft_delete_policy", flattenBucketSoftDeletePolicy(res.SoftDeletePolicy)); err != nil {
+		return fmt.Errorf("Error setting soft_delete_policy: %s", err)
+	}
 
 	if res.IamConfiguration != nil && res.IamConfiguration.UniformBucketLevelAccess != nil {
 		if err := d.Set("uniform_bucket_level_access", res.IamConfiguration.UniformBucketLevelAccess.Enabled); err != nil {

third_party/github.com/hashicorp/terraform-provider-google-beta/google-beta/services/storage/resource_storage_bucket_test.go

@@ -1127,6 +1127,66 @@ func TestAccStorageBucket_retentionPolicyLocked(t *testing.T) {
 	})
 }
 
+func TestAccStorageBucket_SoftDeletePolicy(t *testing.T) {
+	t.Parallel()
+
+	var bucket storage.Bucket
+	bucketName := fmt.Sprintf("tf-test-acc-bucket-%d", acctest.RandInt(t))
+
+	acctest.VcrTest(t, resource.TestCase{
+		PreCheck:                 func() { acctest.AccTestPreCheck(t) },
+		ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories(t),
+		CheckDestroy:             testAccStorageBucketDestroyProducer(t),
+		Steps: []resource.TestStep{
+			{
+				Config: testAccStorageBucket_basic(bucketName),
+				Check: resource.ComposeTestCheckFunc(
+					testAccCheckStorageBucketExists(
+						t, "google_storage_bucket.bucket", bucketName, &bucket),
+					resource.TestCheckResourceAttr(
+						"google_storage_bucket.bucket", "soft_delete_policy.0.retention_duration_seconds", "604800"),
+				),
+			},
+			{
+				ResourceName:            "google_storage_bucket.bucket",
+				ImportState:             true,
+				ImportStateVerify:       true,
+				ImportStateVerifyIgnore: []string{"force_destroy"},
+			},
+			{
+				Config: testAccStorageBucket_SoftDeletePolicy(bucketName, 7776000),
+				Check: resource.ComposeTestCheckFunc(
+					testAccCheckStorageBucketExists(
+						t, "google_storage_bucket.bucket", bucketName, &bucket),
+					resource.TestCheckResourceAttr(
+						"google_storage_bucket.bucket", "soft_delete_policy.0.retention_duration_seconds", "7776000"),
+				),
+			},
+			{
+				ResourceName:            "google_storage_bucket.bucket",
+				ImportState:             true,
+				ImportStateVerify:       true,
+				ImportStateVerifyIgnore: []string{"force_destroy"},
+			},
+			{
+				Config: testAccStorageBucket_SoftDeletePolicy(bucketName, 0),
+				Check: resource.ComposeTestCheckFunc(
+					testAccCheckStorageBucketExists(
+						t, "google_storage_bucket.bucket", bucketName, &bucket),
+					resource.TestCheckResourceAttr(
+						"google_storage_bucket.bucket", "soft_delete_policy.0.retention_duration_seconds", "0"),
+				),
+			},
+			{
+				ResourceName:            "google_storage_bucket.bucket",
+				ImportState:             true,
+				ImportStateVerify:       true,
+				ImportStateVerifyIgnore: []string{"force_destroy"},
+			},
+		},
+	})
+}
+
 func testAccCheckStorageBucketExists(t *testing.T, n string, bucketName string, bucket *storage.Bucket) resource.TestCheckFunc {
 	return func(s *terraform.State) error {
 		rs, ok := s.RootModule().Resources[n]
@@ -1926,6 +1986,19 @@ resource "google_storage_bucket" "bucket" {
 `, bucketName)
 }
 
+func testAccStorageBucket_SoftDeletePolicy(bucketName string, duration int) string {
+	return fmt.Sprintf(`
+resource "google_storage_bucket" "bucket" {
+  name          = "%s"
+  location      = "US"
+  force_destroy = true
+  soft_delete_policy {
+    retention_duration_seconds = %d
+  }
+}
+`, bucketName, duration)
+}
+
 func testAccStorageBucket_websiteNoAttributes(bucketName string) string {
 	return fmt.Sprintf(`
 resource "google_storage_bucket" "website" {

third_party/github.com/hashicorp/terraform-provider-google-beta/website/docs/r/storage_bucket.html.markdown

@@ -128,6 +128,8 @@ The following arguments are supported:
 
 * `custom_placement_config` - (Optional) The bucket's custom location configuration, which specifies the individual regions that comprise a dual-region bucket. If the bucket is designated a single or multi-region, the parameters are empty. Structure is [documented below](#nested_custom_placement_config).
 
+* `soft_delete_policy` - (Optional, Computed) The bucket's soft delete policy, which defines the period of time that soft-deleted objects will be retained, and cannot be permanently deleted. If the block is not provided, Server side value will be kept which means removal of block won't generate any terraform change. Structure is [documented below](#nested_soft_delete_policy).
+
 <a name="nested_lifecycle_rule"></a>The `lifecycle_rule` block supports:
 
 * `action` - (Required) The Lifecycle Rule's action configuration. A single block of this type is supported. Structure is [documented below](#nested_action).
@@ -224,6 +226,12 @@ The following arguments are supported:
 
 * `data_locations` - (Required) The list of individual regions that comprise a dual-region bucket. See [Cloud Storage bucket locations](https://cloud.google.com/storage/docs/dual-regions#availability) for a list of acceptable regions. **Note**: If any of the data_locations changes, it will [recreate the bucket](https://cloud.google.com/storage/docs/locations#key-concepts).
 
+<a name="nested_soft_delete_policy"></a>The `soft_delete_policy` block supports:
+
+* `retention_duration_seconds` - (Optional, Default: 604800) The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Default value is 604800. The value must be in between 604800(7 days) and 7776000(90 days). **Note**: To disable the soft delete policy on a bucket, This field must be set to 0. 
+
+* `effective_time` - (Computed) Server-determined value that indicates the time from which the policy, or one with a greater retention, was effective. This value is in RFC 3339 format.
+
 ## Attributes Reference
 
 In addition to the arguments listed above, the following computed attributes are