This article is a guest post from Olaf Conijn, the creator of org-formation.
In the first part of this series on managing AWS Organizations using org-formation, we covered the basics of how to use org-formation, how to initialize an organization.yml file on disk, and how to make changes using the
update command. In Part 2, we will take managing Organizations to the next level by describing how the update process can be automated using AWS CodePipeline.
Creating a CodePipeline using org-formation
Because Infrastructure as Code (IaC) is particularly useful when stored in source control, and applied automatically upon commit (or merge), org-formation has a command to set up such a pipeline in AWS. Running the
init-pipeline command is a lot like the
init command, but instead of creating a file on disk, it creates AWS CodeCommit, AWS CodeBuild, and CodePipeline resources. It also creates an initial check-in that contains the organization.yml file for the organization and all other files needed to deploy changes to this file automatically.
> org-formation init-pipeline organization.yml --region eu-central-1
This command will create an initial commit to the CodeCommit repository containing the following files:
- organization.yml is the file we created on our local disk using the
- buildspec.yml contains instructions to run org-formation upon every check-in to main. Running org-formation using the
perform-taskscommand allows us to run any number of tasks from within a tasks file.
- organization-tasks.yml is an org-formation tasks file that contains two tasks: a task of type
update-organization, which is used to apply all changes made to the organization.yml file (if any); and an
update-stackstask that can be used to change the pipeline itself.
- templates/org-formation-build.yml contains the AWS CloudFormation template used to create the AWS resources and can be used to modify these.
Contents of the organization-tasks.yml file, as generated by the
Automating deployments using task files
New AWS accounts within an organization typically also come with a basic set of resources created within these accounts. Therefore, updating our organization is likely a process with multiple steps. To do this, org-formation has a command called
perform-tasks. We can run
perform-tasks to execute tasks we would like to be part of the organization build pipeline.
The task file needs to contain at least one
update-organization task that will be executed before all other tasks. If other tasks reference an organization.yml file, this file must always be the same file specified in the
A task file can contain the following task types:
- update-organization, a task used to update the organization resources in the main account
- update-stacks, a task used to create/update CloudFormation templates in the accounts that are part of the organization
- include, a task used to include another tasks file
- update-cdk, a task used to execute an AWS CDK project in the accounts that are part of the organization
- update-serverless.com, a task used to execute a Serverless.com project in the accounts that are part of the organization
- copy-to-s3, a task used to copy a local file to Amazon S3
An example of a task file may look like the following:
We can specify all tasks with the following attributes:
- Use DependsOn to have a task run only after the task(s) specified here have executed successfully.
- Use Skip to skip the execution of a task (when set to
true). Tasks that depend on this using
DependsOnwill be skipped (unless explicitly unskipped).
- Use TaskRoleName to specify the name of the AWS IAM role that will be assumed in the target account when performing the task.
- Use LogVerbose to print debug-level logging when executing a particular task.
Note that the
perform-tasks command has options to run multiple tasks concurrently. It also has options to specify a tolerance for failures on both tasks and stacks. If we are into speeding up our deployment, try adding the option
--max-concurrent-stacks 10 when executing
perform-tasks. If we want the
perform-tasks to continue even after a number of tasks have failed, we can add the option
--failed-tasks-tolerance 5. Tasks that depend on tasks that have failed will not be executed and considered failed. Both options can also be specified on a task with type
A concept at the core of org-formation is the Organization Binding. The Organization Binding allows us to specify a number of target accounts (and regions) and update them all at once. Annotated CloudFormation templates can use multiple Organization Bindings and specify exactly where to deploy needed resources.
An Organization Binding always specifies both the target accounts and target regions. The targets that are used are all the possible combinations of regions and accounts. For example, an Organization Binding with two regions and three accounts will have six targets, but an Organization Binding with zero regions and six accounts will not have any targets.
Because Annotated CloudFormation templates can have multiple bindings, there is the option to specify a default set of regions using
DefaultOrganizationBindingRegion. This prevents us from forgetting to specify a region and not having the resources deployed anywhere.
An Organization Binding can have the following attributes:
- Region used to specify the region(s) for which this binding needs to create targets.
- Account used to include a specific account, or list of accounts, that this binding needs to create targets for. We can also use * to specify all accounts, except for the main account.
- IncludeMasterAccount used to include the MasterAccount in the targets (when value is
- OrganizationalUnit used to include accounts from an Organizational Unit (or list of Organizational Units).
- AccountsWithTag used to include all accounts that declare a specific tag in the organization file.
- ExcludeAccount used to exclude a specific account (or list of accounts) from the targets.
All references use the logical names as declared in the organizational.yml file, and accounts that are not part of the organizational model are not used to create a target for.
Examples of Organization Bindings
Simple list of accounts in
All accounts in an organization (including the main account) in both
All accounts part of the development OU, except for the
All accounts that declare a
Variables and Parameters in the task file
From within the task file, it is possible to reference attributes from the organization.yml using
!Sub (or any combination). This can be useful if we want to parameterize the tasks (or Parameters of a task) using information in our organization.
We can refer to any account in the organization.yml by its logical name. Refer to the account that is part of the current task and target (when executing the task) by
CurrentAccount. We can declare custom parameters in a top-level
Parameters attribute in the task file. Parameters can have default values specified in the template and be overwritten by adding a
--parameters option to the
Declaring and specifying parameter values when running the
In the previous example, we demonstrated how the parameters are:
- Used in a
StackNameattribute to avoid colliding stack names when re-using or testing the tasks file.
- Used in an
OrganizationBindingto conditionally include the
- Passed down to an include task. If nothing is specified in the
Parametersattribute of the include task, parameter values from the parent task file are passed down to included task files. In the example, we assigned the parameter
stackPrefixa specific value in the included task file. However, the value from
includeMasterAccountwill remain the same.
In addition to organization attributes and parameters, CloudFormation exports can be queried using the
!CopyValue function. As opposed to CloudFormations native
!ImportValue function, the stack (and the resources within the stack) that declares the output can also be deleted after the value was copied from the export. We can use
!CopyValue cross account and cross region, however
!ImportValue only works within the same account and region.
The following four examples demonstrate how a task (called
PolicyTemplate) uses a value exported by another task (called
BucketTemplate) and assigns it to a parameter.
!CopyValue function can declare up to three arguments:
- ExportName, the first argument, must contain the name of the export of which the value needs to be resolved.
- AccountId, the second argument will, if specified, contain the account ID of the account that declares the export. This can be either a hard coded AccountId (12 digits), or
!Refto a logical account name in the organization file that will resolve to the account ID when processing the task file.
- Region, the third argument will, if specified, contain the region that declares the export.
If we do not specify
Region, the account and region of the target are used. If we have an Organization Binding with six targets and do not specify
Region, we will find the export in all six targets (Account/Region combinations).
Protecting critical resources
There are several ways we can protect critical resources deployed by org-formation. The
update-stacks tasks allows us to set the
TerminationProtection attribute to
true to prevent a template from being deleted; and sets the
UpdateProtection attribute to
true, thus preventing any of the resources within the template from updating using CloudFormation.
The following is an example of using
TerminationProtection will cause any call to delete the stack to fail (through the CloudFormation console or org-formation). The
UpdateProtection will cause updates of any resource using CloudFormation to fail. This feature uses a CloudFormation StackPolicy that can also be specified explicitly using the
StackPolicy only apply to changes made using CloudFormation. The resources can still be modified directly in the console or using an API. If we want to ensure resources in the accounts remain unchanged, we can specify this as a service control policy in the organization.yml file.
An example service control policy that prevents modifying an IAM role called
The previous example policy will prevent anyone in the organization (including root) from changing the ProtectedRole resource in any of the applicable accounts. If we only want to allow the Organization Build to change these resources, we can add a
Condition to the service control policy:
In Part 2 of this series, we learned how to create a continuous deployment pipeline for changes to our AWS Organizations. We also learned what task files are, how they can be parameterized, and what we can do to prevent resource modification from outside the pipeline.
In the final installment of this series, we will learn about annotation we can add to the CloudFormation language, parameterizing templates with attributes defined in our AWS Organizations, and creating references between resources across different AWS accounts and regions.
The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.