Cloud Journey: Starting with Enterprise Scale — Part 4

Photo by Tim Mossholder on Unsplash

(PART 1: Isolate cloud infrastructure for IaC Automation)
(PART 2: Organisation structure and hierarchy in the cloud)
(PART 3: Single control and management plane)

PART 4: Unified policy and governance model (Policy as code)

Goal : To catch non-compliant resources before they get deployed

• Enforce policies on your resources to set guardrails and make sure future configurations will be compliant with organizational or external standards and regulations.
• Manage your policies in a centralized location where you can track their compliance status and dig into the specific changes that made resources non-compliant

We can divide the whole process of unified policy and governance into three parts

1. Authoring policies across cloud providers
2. Pre-deployment policy validations
3. Post-deployment policy validations

Simplified IaC CI/CD workflow (drawn using draw.io)

1. Authoring policies across cloud providers

The cloud platform comes with a native way to define, monitor compliance for example :

AWS: Service control policies, AWS GuardDuty, AWS Config

Azure: Azure Policy + Azure Security Center

Google Cloud: GCO Organization policy, Cloud Security Command Center, Forseti Config Validator for GCP

Managing policies in centralized location across platforms and services become challenging as most of these policies are written in JSON or Native formats which are not easy to understand for non-technical team members.

Typical workflow for policy authoring (drawn using draw.io)

Behaviour-Driven Development (BDD)

BDD is a way for software teams to work that closes the gap between business people and technical people by:

• Encouraging collaboration across roles to build a shared understanding of the problem to be solved
• Working in rapid, small iterations to increase feedback and the flow of value
• Producing system documentation that is automatically checked against the system’s behavior

*Using Terraform as base IaC tools, The basic approach of policy validation should remain same if you are using pulumi, ansible, or native IaC tools like ARM,Bicep,Cloudformation,CDK etc.

Example — Terraform-compliance (https://terraform-compliance.com/)

Scenario Outline: Ensure that specific tags are defined
    Given I have resource that supports tags defined
    When it contains tags
    Then it must contain <tags>
    And its value must match the "<value>" regex

Example authoring of Azure Policy in Azure DevOps(drawn using draw.io)

2. Pre-deployment validation

There are various approaches and it very easy to get lost in the world of various CICD tools and support for policy compliance checks and extensions available. I will discuss 3 choices.

• Using cloud provider-specific scan approaches

Azure — Policy compliance scan tool

AWS — Compliance by default through Open Policy Agent (OPA) policy check integrated with a CI/CD pipeline — Reference Click here

Google Cloud — Protecting your GCP infrastructure with Forseti Config Validator — Reference Click here

• Cloud agnostic open-source tools like terraform-compliance or Checkov, Chef InSpec
• Terraform Sentinel —

Terraform Sentinel workflow (reference)

3. Post-deployment validation

Each cloud provider exposes the whole infrastructure as plain REST/JSON API (or gRPC) as well as SDKs for common programming languages.

• AWS API Reference and AWS SDKs
• Azure API Reference and SDKs and resource explorer
• Google Cloud API Reference and API SDKs

It is very easy to use a language of choice and a favorite testing framework to easily create tests with the use of language SDK or even pure REST API and tools like REST Assured or JSON Assert.

References :

• https://adinermie.com/the-good-the-bad-and-the-ugly-of-using-terraform-compliance/
• https://www.nearform.com/blog/getting-devsecops-right-in-azure/
• https://blog.gft.com/pl/2020/03/12/secure-terraform-delivery-pipeline-best-practices-part-2-2/