Aggregating output values from linked ARM templates in copy operation

Azure Resource Manager templates are a powerful way to describe and automate the creation of Azure services and they're getting more powerful with each release of the service.

I was recently confronted with the challenge of aggregating output values of linked templates deployed in a copy operation so the values could be used in the creation of services later in the template. This post presents a recipe for how you can do the same should you run across this situation - I hope you'll join me.

The Scenario

As part of a spike I'm working on for a customer, I wanted to create an ARM template that would make it a breeze for them to duplicate what I had done in my subscription and follow along. Without getting too far in the weeds, essentially what I needed to do was deploy a few static Public IPs in various regions and then then grant those IP addresses access to a set of Storage Accounts created by same template by adding the IP addresses to the Storage Accounts' Firewall IP rules.

Whenever possible, the proper way of deploying more than one instance is to use the copy element which is exactly what I wanted to do this for the Pubic IPs and Storage Accounts in this template. After using a copy for both the Public IPs and Storage Accounts I established that the Storage Accounts had a dependency on the Public IPs. Great, now I'll just aggregate the Public IP values and pass them into the Storage Account resource definition and call it a day, right? Not quite...

As it turns out, ARM templates don't currently have the native ability to aggregate output from copy operations to be used in dependent resource creation. But surely there must be a way...and it turns out there is!

The Recipe

After racking my brain for awhile (and an exhaustive search or two later), I stumbled across this Stack Overflow post which suggested the clever approach of passing the output from a previous copyIndex iteration to the input of the next iteration. Then in the current iteration, you concatenate the input with the current iteration output and return the resulting concatenated array. Rinse, repeat.

However, you'll notice the accepted answer deploys the first instance outside the copy to get an instance of the array and then deploys the remainder inside the copy. I cleaned this up a bit so all instances could be deployed within the copy.

Here's a sample root template:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "quantity": {
            "type": "int",
            "defaultValue": 3
        }
    },
    "variables": {
        "emptyArray": []
    },
    "resources": [
        {
            "apiVersion": "2017-05-10",
            "name": "[concat('linked-', copyIndex('exampleCopy', 1))]",
            "type": "Microsoft.Resources/deployments",
            "copy": {
                "name": "exampleCopy",
                "count": "[parameters('quantity')]",
                "mode": "Serial"
            },
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[uri(deployment().properties.templateLink.uri, 'linked.json')]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "state": {
                        "value": "[if(equals(copyIndex('exampleCopy'), 0), variables('emptyArray'), reference(concat('linked-', copyIndex('exampleCopy'))).outputs.state.value)]"
                    },
                    "ping": {
                        "value": "[copyIndex('exampleCopy', 1)]"
                    }
                }
            }
        }
    ],
    "outputs": {
        "aggregate": {
            "type": "array",
            "value": "[reference(concat('linked-', parameters('quantity'))).outputs.state.value]"
        }
    }
}

And the linked template:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "state": {
            "type": "array"
        },
        "ping": {
            "type": "int"
        }
    },
    "variables": {
        "pong": "[concat('pong-', parameters('ping'))]"
    },
    "resources": [],
    "outputs": {
        "state": {
            "type": "array",
            "value": "[concat(parameters('state'), array(variables('pong')))]"
        }
    }
}

You'll note that in the root template, I've updated the "state" parameter of the linked template deployment with an if function. This function checks to see if the current copyIndex value is 0, and if so, will pass an empty array defined in the variables section otherwise will reference the output from the previous iteration.

I've pushed this sample to GitHub here: https://github.com/rjygraham/arm-templates/tree/master/src/arm-copyindex-aggregate-outputs which you can run via the following CLI commands or by the Deploy to Azure button (don't worry, no resources are actually created).

az group create -g arm-copyindex-aggregate-outputs -l eastus
az group deployment create -g arm-copyindex-aggregate-outputs --template-uri https://raw.githubusercontent.com/rjygraham/arm-templates/master/src/arm-copyindex-aggregate-outputs/azuredeploy.json

One Caveat

Please note that for this to work, you'll need to ensure the copy mode is set to Serial for the resources from which you want to aggregate values. This means that in order to minimize deployment times, you'll want to factor your ARM templates such that only the bare minimum is deployed in the linked templates using this method.

---