Azure Policy - Using the Append Effect as a More Gentle Deny

Recently I've been having a lot of fun working with a customer to operationalize Azure in their enterprise. As these type of engagements go, there's a ton of up-front knowledge transfer followed by rubber meeting the road.

We're firmly in the execution phase and have been leveraging both built-in and custom Azure Policy to help enforce cloud controls. If you're not familiar with Azure Policy, here's a brief description from the documentation:

Azure Policy is a service in Azure that you use to create, assign, and manage policies. These policies enforce different rules and effects over your resources, so those resources stay compliant with your corporate standards and service level agreements.

At the heart of Azure Policy is the policy rule which is essentially an if..then statement:

{
    "if": {
        <condition> | <logical operator>
    },
    "then": {
        "effect": "deny | audit | append | auditIfNotExists | deployIfNotExists | disabled"
    }
}

The Scenario

Data security is of the utmost importance to all enterprises moving to the cloud -- one misconfiguration and an enterprise could end up having a really, really bad day.

Preventing these types of misconfigurations is where Azure Policy shines as it provides enterprises a means of implementing controls at scale; for example, enforcing encryption at rest and in transit on all Storage Accounts within an enterprise's Azure environment.

In the case of this customer, we wanted to do exactly that -- prevent the possibility of Storage Accounts being created (or updated) without requiring encryption at rest as well as HTTPS for encryption in transit.

With that in mind, we created the following Azure Policy definitions in their Azure environment (both are available in the Azure Policy samples on GitHub):

Ensure https traffic only for storage account

{
	"if": {
		"allOf": [
			{
				"field": "type",
				"equals": "Microsoft.Storage/storageAccounts"
			},
			{
				"not": {
					"field": "Microsoft.Storage/storageAccounts/supportsHttpsTrafficOnly",
					"equals": "true"
				}
			}
		]
	},
	"then": {
		"effect": "deny"
	}
}

Ensure storage file encryption

{
	"if": {
		"allOf": [
			{
				"field": "type",
				"equals": "Microsoft.Storage/storageAccounts"
			},
			{
				"field": "Microsoft.Storage/storageAccounts/enableFileEncryption",
				"equals": "false"
			}
		]
	},
	"then": {
		"effect": "deny"
	}
}

Dissecting the policies above, the if conditions are looking for Azure resource types that are Storage Accounts and have supportsHttpsTrafficOnly or enableFileEncryption property values that are not true or false, respectively (the manner in which these rules are authored matter, more on that below). When allOf these conditions are met the then part of the rule is executed and the policy effect will apply, which in this case is to deny the operation.

To test, we assigned the policies, waited a few minutes for them to take effect, and then attempted to create or update Storage Accounts in a manner that would trigger the deny effect and voila...

Result of the policy denying creation of Storage Account without HTTPS:

Result of the policy denying the update of an existing Storage Account to HTTP:

Done and done! Well, almost...

Houston, We Have a Problem

The great thing about Azure Policy is that like any other code you write, it does EXACTLY what you tell it to do. The bad thing about Azure Policy is that it does EXACTLY what you tell it to do, which can have unintended consequences.

The issue we experienced with the policy rules as written, is the deny effect will block the creation of ANY Storage Account where supportsHttpsTrafficOnly and enableFileEncryption properties are not explicitly set to true. For example, we found this to be problematic with end-users new to Azure creating VMs via the portal and not understanding why their deployment was failing validation. It's not immediately obvious, but when you walk through the Create a VM wizard, it prompts for the diagnostic Storage Account to use. If you select create new, a Storage Account will be added to the deployment which doesn't explicitly set supportsHttpsTrafficOnly and enableFileEncryptionto true and bam, deployment fails validation.

After taking a longer look at Azure Policy Effects, we settled on reworking the policy rules and changing the deny effects to append. What specifically led us to this decision was the description of the append effect evaluation in the documentation:

Append evaluates before the request gets processed by a Resource Provider during the creation or updating of a resource. Append adds fields to the resource when the if condition of the policy rule is met. If the append effect would override a value in the original request with a different value, then it acts as a deny effect and rejects the request.

If the property is missing altogether, the policy will set to the value we have defined. If the property is set to a value that is contrary to the value defined in the policy, then the deployment will be denied. Bingo!

Our re-worked policy rules look like the following:

Updated Ensure https traffic only for storage account

{
    "if": {
        "allOf": [
            {
                "field": "type",
                "equals": "Microsoft.Storage/storageAccounts"
            },
            {
                "not": {
                    "field": "Microsoft.Storage/storageAccounts/supportsHttpsTrafficOnly",
                    "equals": "true"
                }
            }
        ]
    },
    "then": {
        "effect": "append",
        "details": [
            {
                "field": "Microsoft.Storage/storageAccounts/supportsHttpsTrafficOnly",
                "value": "true"
            }
        ]
    }
}

Updated Ensure storage file encryption

{
    "if": {
        "allOf": [
            {
                "field": "type",
                "equals": "Microsoft.Storage/storageAccounts"
            },
            {
                "not": {
                    "field": "Microsoft.Storage/storageAccounts/enableFileEncryption",
                    "equals": "true"
                }
            }
        ]
    },
    "then": {
        "effect": "append",
        "details": [
            {
                "field": "Microsoft.Storage/storageAccounts/enableFileEncryption",
                "value": "true"
            },
            {
                "field": "Microsoft.Storage/storageAccounts/encryption.keySource",
                "value": "Microsoft.Storage"
            }
        ]
    }
}

After updating the policy rules to the definitions above, our deployment failures caused by the Storage Account properties not being explicitly set were resolved.

A couple things to note:

1. The if part of the enableFileEncryption policy rule needed to be rewritten to look for the condition of not true vs an explicit false - this is critical for these policies to work correctly!
2. The then effect was updated to be append with the details containing the properties and associated values we want.

So there you have it!. When writing your Azure Policy rules, consider using the append effect as a more gentle version of deny.

Hope this helps!

Ryan

---