• 實踐 Infrastructure as Code，不需要手動建立資源，而且可以對 Source Code 做 Version Control。
  • CloudFormation 本身是免費的，只需要付建立的資源費用即可。
  • 可以透過 CloudFormation Designer 產生基本的架構圖
  • Template 必須要上傳到 S3 ，Template 格式可以是 Yaml or Json
  • Stack 是由一個 Template 產生的包含多個資源，Stack 名字當作識別。

CloudFormation Template Section

  • Parameters : 當要建立 Stack ，可以由外面設定動態參數
    • 可設定資訊
      • Type: String, Number, CommaDelimitedList, List, AWS Parameter
      • Description
      • Constraints
      • ConstraintDescription
      • Min/MaxLength
      • Defaults
      • AllowedValues (array)
      • AllowedPattern (regular expression)
      • NoEcho(Boolean)
    • 可以從 SSM 取得資料
    1VpcId: !Ref MyVPC
2
3Parameters:
4  InstanceType:
5    Type: 'AWS::SSM::Paramter::Value<String>'
6    Default: /EC2/InstanceType
7#there is public SSM parameters by AWS
  • Conditions : 建立資源的條件
    • intrinsic function : Fn::And, Fn::Equals, Fn::If, Fn::Not, Fn::Or
1Conditions:
2	CreateProdResources: !Equals [!Ref EnvType, prod] 
3
4Resource:
5  MountPoint:
6    Type: "AWS::EC2::VolumeAttachment"
7    Condition: CreateProdResources
  • Resource: 要建立的 AWS 資源，此為必要欄位
    • 格式 : AWS::aws-product-name::data-type-name
  • Mapping: 建立 Key Value Mapping
 1Mapping:
 2  RegionMap:
 3    user-east-1:
 4      "32" : "ami-aaa"
 5      "64" : "ami-bbb"
 6    user-east-2:
 7      "32" : "ami-ccc"
 8      "64" : "ami-ddd"
 9
10ImagesId: !FindInMap [RegionMap, !Ref "AWS:Region", 32]
  • Transforms: 主要用於 Serve-less Application Model (SAM) ，宣告這是 SAM 的格式
  • Output: 可以將 Stack 建立的 Resource Output 出來，所以其他 Stack 可以建立 reference
    • 如果被其他 Stack reference ，那就不能 delete
 1Outputs:
 2  StackSSHSecurityGroup:
 3    Description: The SSH Security Group for our Company
 4    Vaule: !Ref MyCompanyWideSSHSecurityGroup
 5  Export:
 6    Name: SSHSecurityGroup
 7
 8Resource:
 9  MySecureInstance:
10  Type: AWS::EC2::Instance
11  Properties:
12    AvailabilityZone: us-east-1a
13    ImageId: ami-a5c7edb2
14    InstanceType: t2.micro
15    SecurityGroup:
16      - !ImportValue SSHSecurityGroup
  • Metadata: 提供額外有關 Template 資訊

Intrinsic Functions

  • Fn::Ref
    • Parameters ⇒ 返回 parameter 對應的值
    • Resources ⇒ 返回資源的 physical ID
  • Fn:GetAtt
 1Resource:
 2  EC2Instance: 
 3    Type: "AWS::EC2::Instance"
 4    Properties:
 5      ImageId: ami-1234567
 6      InstanceType: t2.micro
 7
 8NewVolume:
 9  Type: "AWS::EC2::Volume"
10  Confition: CreateProdResources
11  Properties:
12    Size: 100
13    AvailabilityZone:
14    !GetAtt EC2Instance.AvailabilityZone
  • Fn::Join
1"Fn::Join" : [ ",", [ "a", "b", "c" ] ] => "a,b,c"
  • Fn::Sub
1Fn::Sub:
2  - String
3  - Var1Name: Var1Value
4    Var2Name: Var2Value
5
6Name: !Sub
7  - www.${Domain}
8  - { Domain: !Ref RootDomainName }
  • Fn::Split
1!Split [ "|" , "a|b|c" ] => ["a", "b", "c"]
  • Fn::Select
1!Select [ "1", [ "apples", "grapes", "oranges", "mangoes" ] ]
  • Fn::Base64
    • 可以用在 EC2 pass 整個 script 在 user data
    • user data script log ⇒ /var/log/cloud-init-output.log
 1Resource:
 2  EC2Instance: 
 3    Type: "AWS::EC2::Instance"
 4    Properties:
 5      ImageId: ami-1234567
 6      InstanceType: t2.micro
 7    UserData:
 8      Fn::Base64: |
 9        yum update -y
10        yum install -y httpd
11        systemctl start httpd
12        systemctl enable httpd
13        echo "Hello World from user data" > /var/www/html/index.html

cfn-init & cfn-signal & cfn-hup

  • AWS::CloudFormation::Init 目的是讓 EC2 的設定可讀性更高， 需要放在 metadata section
  • init 的 log 會寫在 /var/log/cfn-init.log
  • cfn-init & cfn-signal 流程
    • CloudFormation 先建立 EC2 Instance ，如果有設定 WaitCondition 這時 CloudFormation 會等待 EC2 這邊的 signal
    • EC2 上面執行 User Data 時執行 cfn-init，這時 cfn-init 會去 CloudFormation 拉 metadata 的設定回來並執行
    • cfn-signal 用於執行完 cfn-init 告訴 CloudFormation 成功與否
  • 沒有收到 Signal 怎麼辦
    • 確認 Script 是否有安裝
    • 可以disable rollback，進機器看 /var/log/cfn-init.log 和 /var/log/cloud-init-output.log
    • 由於 Signal 必須透過 Internet 通知 CloudFormation ，確認 EC2 可以 Access Internet.
  • cfn-hup : 是一個 daemon 去檢查是否有 metadata 要更新， Default Interval 是 15 分鐘
 1Resource:
 2  EC2Instance: 
 3    Type: "AWS::EC2::Instance"
 4    Properties:
 5      ImageId: ami-1234567
 6      InstanceType: t2.micro
 7    UserData:
 8      Fn::Base64: |
 9        yum update -y aws-cfn-bootstrap
10        # Start cfn-init
11        /opt/aws/bin/cfn-init -s ${AWS::StackId} -r MyInstance --region ${AWS::Region} || error_exit 'Failed to run cfn-init' 
12        
13        # start the cfn-hup daemon
14        /opt/aws/bin/cfn-hup || error_exit "Failed to start cfn_hup"
15        # Start cfn-signal to the wait condition
16        /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource SampleWaitCondition --region ${AWS::Region}        
17
18Metadata:
19  Comment: Install a simple Apache HTTP page
20  AWS:CloudFormation::Init:
21    config:
22      packages:
23        yum:
24          httpd: []
25      files:
26        "/var/www/html/index.html";
27        content: |
28          <h1>Hello World from Ec2 Instance</h1>          
29        mode: '000644'
30        "/etc/cfn/cfn-hup.conf":
31        content: !Sub |
32            [main]
33            stack=${AWS::StackId}
34            region=${AWS::Region}
35            interval=2
36        mode: "000400"
37        owner: "root"
38        group: "root"
39
40        "/etc/cfn/hooks.d/cfn-auto-reloader.conf":
41        content: !Sub |
42            [cfn-auto-reloader-hook]
43            triggers=post.update
44            path=Resources.WebServerHost.Metadata.AWS::CloudFormation::Init
45            action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource WebServerHost --region ${AWS::Region}
46        mode: "000400"
47        owner: "root"
48        group: "root"
49      commands:
50        hello:
51          command: "echo 'hello world'"
52      services:
53        sysvinit:
54          httpd:
55            enabled: 'true'
56            ensureRunning: 'true'
57
58SampleWaitCondition:
59  CreationPolicy:
60    ResourceSignal:
61      Timeout: PT1M #等一分鐘
62      Count:2 #要兩個成功的 Signal
63    Type: AWS::CoudFormation::WaitCondition

CloudFormation Rollback

  • Stack 建立失敗時，Stack 正常狀態會是 ROLLBACK_COMPLETE ，並且只能刪除不能更新
    • 預設 : 所有的資源會 rollback ，但 Stack 還在，可以看原因
      • OnFailure=Rollback
    • 可以 Disable rollback，所以可以做 troubleshooting
      • OnFailure=DO_NOTHING
    • 可以刪除整個 Stack
      • OnFailure=Delete
  • Stack 更新失敗
    • Stack 如果 rollback 成功 Stack 狀態會是 UPDATE_ROLLBACK_COMPLETE ，可以嘗試更新或刪除
    • Stack 如果 rollback 失敗 Stack 狀態會是 UPDATE_ROLLBACK_FAIL，可以嘗試修復問題再繼續 rollback

Nested Stacks

  • 可以重複使用其他 Stack 當作整個 Stack 一部分
1Resources:
2  myStack:
3    Type: AWS::CloudFormation::Stack
4    Properties:
5      TemplateURL:
6        xxx.com/...
7      Parameters:
8        key: value

ChangeSet

  • 可以在 CloudFormation 更新新的Template 之前先評估有什麼資源會變動，並確認是否要繼續執行，但執行了並不代表一定會成功。

Drift

  • CloudFormation 建立的資源是沒有預防人為手動更新的
  • Drift 是用來檢查哪些資源跟當初 CloudFormation 建立的不一樣，並且告訴你哪些不同

Deletion Policy

  • 用來控制當 Stack 被刪除時特定資源要做的事情
  • 三種 Policy
    • DeletionPolicy=Retain: 預防資源被刪除
    • DeletionPolicy=Snapshot: 先將資料備份再刪除
      • Support: EBS Volume, ElasticCache, Cluster, ElasticCache, ReplicationGroup, RDS DBInstance, RDS DBCluster, Redshift Cluster
    • DeletionPolicy=Delete 刪除資源，但並非都會刪除成功，例如 S3 Bucket 裡面有 Object 就會刪除失敗
    • RDS 預設 Snapshot , 其他預設 Delete

Termination Protection

  • 用於保護 Stack 不會被不小心刪除

Creation Policy & Update Policy

  • 像 Auto Scaling Group 可以需要啟多個 Ec2 Instance，這時 Creation Policy 就可以設定在 Auto Scaling Group，要收時間內收多少個 Signal 才算成功
  • UpdatePolicy
    • 如果直接更新 template 中的 user data，Auto Scaling Group 裡面的 EC2 還會是舊的 user data。
    • UpdatePolicy Attribute 可以使用在三種 AWS Resource

      • auto scaling group 有三種 Policy

        • AutoScalingReplacingUpdate: WillReplace可以用來決定建立全新的 Auto Scaling Group 取代舊，還是單純建立新的 Instance 來取代現有的 Instance 但 Auto Scaling Group 還是舊的。
        • AutoScalingRollingUpdate: 在 Auto Scaling Group，將現有的 EC2 Instance 停掉，並建立新的取代
        • AutoScalingScheduledAction
         1Resources:
 2  AutoScalingGroup:
 3  Type: AWS::AutoScaling::AutoScalingGroup
 4  Properties:
 5    AvailabilityZones:
 6      Fn::GetsAZs:
 7        Ref: "AWS::Region"
 8    LaunchConfigurationName:
 9      Ref: LaunchConfig
10    Desiredcapacity: '3''
11    MinSize: '1'
12    MaxSize: '4'
13  CreationPolicy:
14    ResourceSignal:
15    Count: '3'
16    Timeout: PT15M
17  UpdatePolicy:
18    ## Example1 start
19    AutoScalingRollingUpdate:
20      MinInstancesInService: '1'
21      MaxBatchSize: '2'
22      #how much time to wait for the signal
23      PauseTime: PT1M
24      WaitOnResourceSignals: 'true'
25    ## 預防CloudFormation 有排程去 update min,max or desired size 
26    AutoScalingScheduledAction:
27      IgnoreUnmodifiedGroupSizeProperties: 'true
28    ## Example1 end
29    ## Example1 and Example2 不會同時存在
30
31    ## Example 2 start
32    ## 會建立全新的 Auto Scaling Group 取代舊得
33    AutoScalingReplacingUpdate:
34      WillReplace: 'true'
35    ## Example 2 end
36
37LaunchConfig:
38  Type: AWS::AutoScaling::LaunchConfiguration
39  Properties:
40    ImageId: ami-aaa
41    InstanceType: t2.micro
42    UserData:
43      Fn::Base64: |
44        yum update -y
45        yum install -y httpd
46        systemctl start httpd
47        systemctl enable httpd
48        echo "Hello World from user data" > /var/www/html/index.html

      • lambda alias

        • CodeDeployLambdaAliasUpdate
        1UpdatePolicy:
2  CodeDeployLambdaAliasUpdate:
3    AfterAllowTrafficHook: String
4    ApplicationName: String
5    BeforeAllowTrafficHook: String
6    DeploymentGroupName: String

      • elastic cache replication group

        • UseOnlineResharding
        1UpdatePolicy:
2EnableVersionUpgrade: Boolean

Depends ON

  • 控制資源建立的順序，以下面為例，EC2 Instance 需要在 RDS 建立好以後才建立
 1Resources:
 2  Ec2Instance:
 3    Type: AWS::EC2::Instance
 4    Properties:
 5      ImageId:
 6        Fn::FindInMap:
 7        - RegionMap
 8        - Ref: AWS::Region
 9        - AMI
10    DependsOn: myDB
11  myDB:
12    Type: AWS::RDS::DBInstance

Stack Policy

  • 像是 IM Policy ，可以用來控制哪些 Resource 可以做什麼，哪些不能做什麼
 1{
 2  "Statement" : [
 3    {
 4      "Effect" : "Allow",
 5      "Action" : "Update:*",
 6      "Principal": "*",
 7      "Resource" : "*"
 8    },
 9    {
10      "Effect" : "Deny",
11      "Action" : "Update:*",
12      "Principal": "*",
13      "Resource" : "LogicalResourceId/ProductionDatabase"
14    }
15  ]
16}

StackSets

  • 可以一次更動多個 Region 或多個 Account 的 Stack
  • 只有 Administrator Account 可以建立 StackSets
  • Trust Account 可以更新 StackSets
  • 每次更動 StackSets 設定，全部的 Stack 都會被更新
  • 可以刪除整個 StackSets 或單獨刪除 StackSets 裡面的一個 Stack

Custom Resource

  • 由於不是所有的 AWS Resource, CloudFormation 都有支援，所以有時需要透過 Lambda Function 去 call api
  • 當 CloudFormation 在 Create, Update or delete 時都會觸發 Lambda Function

Delete S3 Custom Resource

  • Lambda Function
 1Resources: 
 2  LambdaExecutionRole:
 3    Type: AWS::IAM::Role
 4    Properties:
 5      AssumeRolePolicyDocument:
 6        Version: '2012-10-17'
 7        Statement:
 8        - Effect: Allow
 9          Principal:
10            Service:
11            - lambda.amazonaws.com
12          Action:
13          - sts:AssumeRole
14      Path: "/"
15      Policies:
16      - PolicyName: root
17        PolicyDocument:
18          Version: '2012-10-17'
19          Statement:
20          - Effect: Allow
21            Action:
22            - "s3:*"
23            Resource: "*"
24          - Effect: Allow
25            Action:
26            - "logs:CreateLogGroup"
27            - "logs:CreateLogStream"
28            - "logs:PutLogEvents"
29            Resource: "*"
30
31  EmptyS3BucketLambda: 
32    Type: "AWS::Lambda::Function"
33    Properties: 
34      Handler: "index.handler"
35      Role: 
36        Fn::GetAtt: 
37          - "LambdaExecutionRole"
38          - "Arn"
39      Runtime: "python3.7"
40      # we give the function a large timeout 
41      # so we can wait for the bucket to be empty
42      Timeout: 600
43      Code: 
44        ZipFile: |
45          #!/usr/bin/env python
46          # -*- coding: utf-8 -*-
47          import json
48          import boto3
49          from botocore.vendored import requests
50
51          def handler(event, context):
52              try:
53                  bucket = event['ResourceProperties']['BucketName']
54
55                  if event['RequestType'] == 'Delete':
56                      s3 = boto3.resource('s3')
57                      bucket = s3.Bucket(bucket)
58                      for obj in bucket.objects.filter():
59                          s3.Object(bucket.name, obj.key).delete()
60
61                  sendResponseCfn(event, context, "SUCCESS")
62              except Exception as e:
63                  print(e)
64                  sendResponseCfn(event, context, "FAILED")
65
66
67          def sendResponseCfn(event, context, responseStatus):
68              response_body = {'Status': responseStatus,
69                              'Reason': 'Log stream name: ' + context.log_stream_name,
70                              'PhysicalResourceId': context.log_stream_name,
71                              'StackId': event['StackId'],
72                              'RequestId': event['RequestId'],
73                              'LogicalResourceId': event['LogicalResourceId'],
74                              'Data': json.loads("{}")}
75
76              requests.put(event['ResponseURL'], data=json.dumps(response_body))          
77
78Outputs:
79  StackSSHSecurityGroup:
80    Description: The ARN of the Lambda function that empties an S3 bucket
81    Value: !GetAtt EmptyS3BucketLambda.Arn
82    Export:
83      Name: EmptyS3BucketLambda
  • Custom Resource with lambda function
 1---
 2AWSTemplateFormatVersion: '2010-09-09'
 3
 4Resources:
 5  myBucketResource:
 6    Type: AWS::S3::Bucket
 7
 8  LambdaUsedToCleanUp:
 9    Type: Custom::cleanupbucket
10    Properties:
11      ServiceToken: !ImportValue EmptyS3BucketLambda
12      BucketName: !Ref myBucketResource

CloudFormation Request Example

 1{
 2   "RequestType" : "Create",
 3   "ResponseURL" : "http://pre-signed-S3-url-for-response",
 4   "StackId" : "arn:aws:cloudformation:us-west-2:123456789012:stack/stack-name/guid",
 5   "RequestId" : "unique id for this create request",
 6   "ResourceType" : "Custom::TestResource",
 7   "LogicalResourceId" : "MyTestResource",
 8   "ResourceProperties" : {
 9      "Name" : "Value",
10      "List" : [ "1", "2", "3" ]
11   }
12}