Azure Automation: Solve the 'A job schedule for the specified runbook and schedule already exists' issue

This week I was busy with writing ARM templates for deploying a runbook within an Azure Automation account. One of the parts you need for this is the jobSchedule object. The first time I created the object the deployment went succesful but when I deployed the same template a second time the deployment fails with the error message "A job schedule for the specified runbook and schedule already exists". 

After doing some research I found out that I was not the only one. In Azure Feedback I found a bug item from Yulun Zeng dated January 24, 2018 who encountered the same issue. The bug is under review since January 7, 2020 but is still not solved. Some of the Azure Automation resources are conflicting with a fundamental rule of ARM templates documented by Microsoft.

Repeatable results: Repeatedly deploy your infrastructure throughout the development lifecycle and have confidence your resources are deployed in a consistent manner. Templates are idempotent, which means you can deploy the same template many times and get the same resource types in the same state. You can develop one template that represents the desired state, rather than developing lots of separate templates to represent updates.

The jobSchedule object is not obeying the idempotency rule because the first time you deploy it creates the object and the second time it fails with an error message. The only solution is to delete the existing jobSchedule object and create another one. I found out that if you try to recreate the jobSchedule object, after manually removing the existing schedule from the runbook, with the same GUID the deploy will also fail. Every deploy should have an unique GUID.

After doing some research I created an ARM template solution which can solve the idempotency issue. The solution is based on the following components:

1. A PowerShell script to delete an existing jobSchedule object for the runbook.
2. An User Assigned Identity to query the Azure subscription for jobSchedule objects. 
3. An ARM template which uses the Microsoft.Resources/deploymentScripts to execute the PowerShell script and deploy the Microsoft.Automation/automationAccounts/jobSchedules object everytime with an unique GUID.

The PowerShell script needs the name of the Automation Account, the name of the resource group and the name of the runbook from which we want to remove the job schedule. The script will deploy the Automation module if needed and will search for a job schedule. If the job schedule is found it will be unregistered. This is necessary before we can do the deployment of a job schedule.

[CmdletBinding()]
param (
    [Parameter(Mandatory = $true)]
    [String]
    $AutomationAccountName,

    [Parameter(Mandatory = $true)]
    [String]
    $ResourceGroupName,

    [Parameter(Mandatory = $true)]
    [String]
    $RunbookName
)

if (-not (Get-Module -Name Az.Accounts -ListAvailable -ErrorAction SilentlyContinue | Where-Object { $_.Version -eq "2.2.4" })) {
    Install-Module -Name Az.Accounts -RequiredVersion 2.2.4 -Scope CurrentUser -Force
}

if (-not (Get-Module -Name Az.Automation -ListAvailable -ErrorAction SilentlyContinue | Where-Object { $_.Version -eq "1.4.2" })) {
    Install-Module -Name Az.Automation -RequiredVersion 1.4.2 -Scope CurrentUser -Force
}

Import-Module -Name Az.Accounts -RequiredVersion 2.2.4
Import-Module -Name Az.Automation -RequiredVersion 1.4.2

$context = (Get-AzContext).Name
Write-Output "Context: $context"

$jobSchedules = Get-AzAutomationScheduledRunbook -ResourceGroupName $ResourceGroupName -AutomationAccountName $AutomationAccountName -ErrorAction SilentlyContinue
$jobSchedule = $jobSchedules | Where-Object -Property "RunbookName" -eq $RunbookName

if ($null -ne $jobSchedule) {
    Write-Host "Unregister job schedule $($jobSchedule.JobScheduleId) from runbook $($jobSchedule.RunbookName)."
    Unregister-AzAutomationScheduledRunbook -JobScheduleId $jobSchedule.jobScheduleId -ResourceGroupName $ResourceGroupName -AutomationAccountName $AutomationAccountName -Force
}
else {
    Write-Host "No job schedule found for runbook $RunbookName to unregister."
}

To execute this PowerShell script in the ARM template we use the ARM deployment script resource option. Deployment scripts give us the option to execute Azure PowerShell and Azure CLI deployment scripts on a Linux environment runtime which will be created in a container instance during the execution of the ARM template deployment. For more information read the Microsoft documentation.

The deployment script resource needs an user assigned identity to work correctly. I gave the identity contributor rights on the resource group of the Automation account. The deployment script will reference the PowerShell script which I deployed to a storage account. In the argument section the parameters of the script are specified.

{
	"name": "unregisterExistingJobSchedule",
	"type": "Microsoft.Resources/deploymentScripts",
	"apiVersion": "2020-10-01",
	"identity": {
	    "type": "userAssigned",
        "userAssignedIdentities": {
            "/subscriptions/01234567-89AB-CDEF-0123-456789ABCDEF/resourceGroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myID": {}
        }
    },
	"location": "[resourceGroup().location]",
	"kind": "AzurePowerShell",
	"properties": {
		"azPowerShellVersion": "5.0",
		"cleanupPreference": "Always",
		"primaryScriptUri": "[concat(parameters('StorageAccount'), '/', 'Unregister-JobSchedule.ps1', parameters('StorageAccountSASToken'))]",
		"arguments": "[concat('-AutomationAccountName ', parameters('automationAccountName'), ' -ResourceGroupName ', resourceGroup().name, ' -RunbookName ', parameters('RunbookName'))]",
		"retentionInterval": "PT1H",
		"timeout": "PT15M"
	}
}

To make the GUID of the job schedule name everytime unique I append a date/time stamp to the functional name of the job schedule name and than generate the GUID with the guid() function based on that value. The parameter now is defined in the parameter section with a default value based on the utcNow() function.

{
	"name": "deployJobSchedule",
	"type": "Microsoft.Resources/deployments",
	"apiVersion": "2020-06-01",
	"dependsOn": [
		"unregisterExistingJobSchedule"
	],
	"properties": {
		"mode": "Incremental",
		"expressionEvaluationOptions": {
			"scope": "inner"
		},
		"parameters": {
			"automationAccountName": {
				"value": "[parameters('automationAccountName')]"
			},
			"jobScheduleName": {
				"value": "[guid(concat(parameters('jobScheduleName'), parameters('now')))]"
			}
		},
		"template": {
			"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
			"contentVersion": "1.0.0.0",
			"parameters": {
				"automationAccountName": {
					"type": "string"
				},
				"jobScheduleName": {
					"type": "string",
					"metadata": {
						"description": "The name of the job schedule should be unique every time we deploy."
					}
				}
			},
			"resources": [
				{
					"name": "[concat(parameters('automationAccountName'), '/', parameters('jobScheduleName'))]",
					"type": "Microsoft.Automation/automationAccounts/jobSchedules",
					"apiVersion": "2020-01-13-preview",
					"properties": {
					   {
                            "name": "[parameters('RunbookName')]",
                            "properties": {
                                "schedule": {
                                    "name": "[parameters('ScheduleName')]"
                                },
                                "runbook": {
                                    "name": "[parameters('RunbookName')]"
                                }
                            }
                        }
					}
				}
			]
		}
	}
}

This ARM template implementation will workaround the idempotency issue of the current jobSchedule implementation. I hope that Microsoft will fix the Azure Automation ARM template implementation in such a way that it is idempotent.