Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate AWS::CUR::ReportDefinition in template in all regions #1825

Open
iainelder opened this issue Oct 23, 2023 · 1 comment
Open

Validate AWS::CUR::ReportDefinition in template in all regions #1825

iainelder opened this issue Oct 23, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@iainelder
Copy link

Name of the resource

Other

Resource name

AWS::CUR::ReportDefinition

Description

#1565 explains that AWS::CUR::ReportDefinition is available only in us-east-1.

I accept that, but I think that you could improve the validation behavior for the resource in other regions.

The CUDOS deployment guide provides a template that conditionally creates the resource. If the stack is in us-east-1 the template creates the resource directly. Otherwise the template creates a Lambda that makes a cross-region request to create the CUR.

In practice the template works outside us-east-1. I have created a stack from the template in eu-central-1.

When I call ValidateTemplate on the same file, I get an error.

An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: Unrecognized resource types: [AWS::CUR::ReportDefinition]

Instead I expect the template to pass validation.

Other Details

Extract from CloudFormation template:

####
# Local CUR 
####

  ## Deploy CUR nativly via CFN resource if we are in us-east-1
  LocalCurInSource:
    Type: AWS::CUR::ReportDefinition
    Condition: DeployCURViaCFNInSource
    DependsOn:
      - SourceS3BucketPolicy
    Properties:
      AdditionalArtifacts:
        - ATHENA
      AdditionalSchemaElements:
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: True
      ReportName: !Ref ResourcePrefix
      ReportVersioning: OVERWRITE_REPORT
      S3Bucket: !If [ IsDestinationAccount, !Ref DestinationS3, !Ref SourceS3 ]
      S3Prefix: !Sub "cur/${AWS::AccountId}"
      S3Region: !Ref AWS::Region
      TimeUnit: HOURLY

  LocalCurInDestination:
    Type: AWS::CUR::ReportDefinition
    Condition: DeployCURViaCFNInDestination
    DependsOn:
      - DestinationS3BucketPolicy # Conditional DependsOn is not supported, so we need 2 resources
    Properties:
      AdditionalArtifacts:
        - ATHENA
      AdditionalSchemaElements:
        - RESOURCES
      Compression: Parquet
      Format: Parquet
      RefreshClosedReports: True
      ReportName: !Ref ResourcePrefix
      ReportVersioning: OVERWRITE_REPORT
      S3Bucket: !If [ IsDestinationAccount, !Ref DestinationS3, !Ref SourceS3 ]
      S3Prefix: !Sub "cur/${AWS::AccountId}"
      S3Region: !Ref AWS::Region
      TimeUnit: HOURLY


  # Deploy CUR via lambda due to missing cfn resource definition
  # AWS::CUR::ReportDefinition outside us-east-1
  CURinUSEAST1:
    Type: Custom::CURCreator
    Condition: DeployCURViaLambda
    Properties:
      ServiceToken: !GetAtt CIDLambdaCURCreator.Arn
      BucketPolicyWait: !If [ IsDestinationAccount, !Ref DestinationS3BucketPolicy, !Ref SourceS3BucketPolicy ]
      ReportDefinition:
        AdditionalArtifacts:
          - ATHENA
        AdditionalSchemaElements:
          - RESOURCES
        Compression: Parquet
        Format: Parquet
        RefreshClosedReports: True
        ReportName: !Ref ResourcePrefix
        ReportVersioning: OVERWRITE_REPORT
        S3Bucket: !If [ IsDestinationAccount, !Ref DestinationS3, !Ref SourceS3 ]
        S3Prefix: !Sub "cur/${AWS::AccountId}"
        S3Region: !Ref AWS::Region
        TimeUnit: HOURLY

###########################################################################
# Lambda CUR Creator: used to create cur from outside us-east-1
###########################################################################

  CIDLambdaCURCreatorRole: #Execution role for the custom resource CIDLambdaAnalyticsExecutor
    Type: AWS::IAM::Role
    Condition: DeployCURViaLambda
    Properties:
      Path:
        Fn::Sub: /${ResourcePrefix}/
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: "ExecutionDefault"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - logs:CreateLogStream
              - logs:PutLogEvents
              - logs:CreateLogGroup
              Resource:
              - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${ResourcePrefix}-CID-CURCreator"
              - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${ResourcePrefix}-CID-CURCreator:*"
              - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/${ResourcePrefix}-CID-CURCreator:*:*"
        - PolicyName: "ExecutionSpecific"
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Effect: Allow
              Action:
              - cur:PutReportDefinition
              - cur:ModifyReportDefinition
              - cur:DeleteReportDefinition
              Resource:
                - Fn::Sub: arn:${AWS::Partition}:cur:us-east-1:${AWS::AccountId}:definition/*

  CIDLambdaCURCreator:
    Type: AWS::Lambda::Function
    Condition: DeployCURViaLambda
    Properties:
      Runtime: python3.9
      FunctionName:
        Fn::Sub: ${ResourcePrefix}-CID-CURCreator
      Handler: index.lambda_handler
      MemorySize: 128
      Role:
        Fn::GetAtt: CIDLambdaCURCreatorRole.Arn
      Timeout: 15
      Code:
        ZipFile: |
          import boto3
          import cfnresponse
          import uuid
          import json

          # Create a cur client in us-east-1 region
          client = boto3.client('cur', region_name='us-east-1')

          def lambda_handler(event, context):

            print(json.dumps(event))
            reason = ""

            try:
              report = event['ResourceProperties']['ReportDefinition']
              report_name = event['ResourceProperties']['ReportDefinition']['ReportName']

              refresh_closed_report = event['ResourceProperties']['ReportDefinition']["RefreshClosedReports"]
              if refresh_closed_report in ["True", "true"]:
                  report["RefreshClosedReports"] = True
              elif refresh_closed_report in ["False", "false"]:
                  report["RefreshClosedReports"] = False
              else:
                  raise Exception("RefreshClosedReports is not a boolean")

              if event['RequestType'] == 'Create':
                  res = client.put_report_definition(
                      ReportDefinition=report
                  )
                  print(json.dumps(res))
              elif event['RequestType'] == 'Update':
                  old_report_name = event['OldResourceProperties']['ReportDefinition']['ReportName']
                  if report["ReportName"] != old_report_name:
                      res = client.put_report_definition(
                          ReportDefinition=report
                      )
                      print(json.dumps(res))
                  else:
                      res = client.modify_report_definition(
                          ReportName=old_report_name,
                          ReportDefinition=report
                      )
                      print(json.dumps(res))
              elif event['RequestType'] == 'Delete':
                  try:
                      res = client.delete_report_definition(
                          ReportName=report_name
                      )
                      print(json.dumps(res))
                  except:
                      pass # Do not block deletion
              else:
                  raise Exception("Unknown operation: " + event['RequestType'])

            except Exception as e:
                reason = str(e)
                print(e)
            finally:
                physicalResourceId = event.get('ResourceProperties',{}).get('ReportDefinition').get('ReportName', None) or str(uuid.uuid1())
                if reason:
                    print("FAILURE")
                    cfnresponse.send(event, context, cfnresponse.FAILED, {"Data": reason }, physicalResourceId)
                else:
                    print("SUCCESS")
                    cfnresponse.send(event, context, cfnresponse.SUCCESS, {}, physicalResourceId)
@prerna-p prerna-p added enhancement New feature or request and removed Coverage labels Dec 7, 2023
@tlinkin
Copy link

tlinkin commented Mar 13, 2024

Please merge !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Coming Soon
Development

No branches or pull requests

3 participants