Skip to main content

Azure Automation: Solve the 'A job schedule for the specified runbook and schedule already exists' issue

This week I was busy with writing ARM templates for deploying a runbook within an Azure Automation account. One of the parts you need for this is the jobSchedule object. The first time I created the object the deployment went succesful but when I deployed the same template a second time the deployment fails with the error message "A job schedule for the specified runbook and schedule already exists". 

After doing some research I found out that I was not the only one. In Azure Feedback I found a bug item from Yulun Zeng dated January 24, 2018 who encountered the same issue. The bug is under review since January 7, 2020 but is still not solved. Some of the Azure Automation resources are conflicting with a fundamental rule of ARM templates documented by Microsoft.

Repeatable results: Repeatedly deploy your infrastructure throughout the development lifecycle and have confidence your resources are deployed in a consistent manner. Templates are idempotent, which means you can deploy the same template many times and get the same resource types in the same state. You can develop one template that represents the desired state, rather than developing lots of separate templates to represent updates.

The jobSchedule object is not obeying the idempotency rule because the first time you deploy it creates the object and the second time it fails with an error message. The only solution is to delete the existing jobSchedule object and create another one. I found out that if you try to recreate the jobSchedule object, after manually removing the existing schedule from the runbook, with the same GUID the deploy will also fail. Every deploy should have an unique GUID.

After doing some research I created an ARM template solution which can solve the idempotency issue. The solution is based on the following components:

  1. A PowerShell script to delete an existing jobSchedule object for the runbook.
  2. An User Assigned Identity to query the Azure subscription for jobSchedule objects. 
  3. An ARM template which uses the Microsoft.Resources/deploymentScripts to execute the PowerShell script and deploy the Microsoft.Automation/automationAccounts/jobSchedules object everytime with an unique GUID.

The PowerShell script needs the name of the Automation Account, the name of the resource group and the name of the runbook from which we want to remove the job schedule. The script will deploy the Automation module if needed and will search for a job schedule. If the job schedule is found it will be unregistered. This is necessary before we can do the deployment of a job schedule.

[CmdletBinding()]
param (
    [Parameter(Mandatory = $true)]
    [String]
    $AutomationAccountName,

    [Parameter(Mandatory = $true)]
    [String]
    $ResourceGroupName,

    [Parameter(Mandatory = $true)]
    [String]
    $RunbookName
)

if (-not (Get-Module -Name Az.Accounts -ListAvailable -ErrorAction SilentlyContinue | Where-Object { $_.Version -eq "2.2.4" })) {
    Install-Module -Name Az.Accounts -RequiredVersion 2.2.4 -Scope CurrentUser -Force
}

if (-not (Get-Module -Name Az.Automation -ListAvailable -ErrorAction SilentlyContinue | Where-Object { $_.Version -eq "1.4.2" })) {
    Install-Module -Name Az.Automation -RequiredVersion 1.4.2 -Scope CurrentUser -Force
}

Import-Module -Name Az.Accounts -RequiredVersion 2.2.4
Import-Module -Name Az.Automation -RequiredVersion 1.4.2

$context = (Get-AzContext).Name
Write-Output "Context: $context"

$jobSchedules = Get-AzAutomationScheduledRunbook -ResourceGroupName $ResourceGroupName -AutomationAccountName $AutomationAccountName -ErrorAction SilentlyContinue
$jobSchedule = $jobSchedules | Where-Object -Property "RunbookName" -eq $RunbookName

if ($null -ne $jobSchedule) {
    Write-Host "Unregister job schedule $($jobSchedule.JobScheduleId) from runbook $($jobSchedule.RunbookName)."
    Unregister-AzAutomationScheduledRunbook -JobScheduleId $jobSchedule.jobScheduleId -ResourceGroupName $ResourceGroupName -AutomationAccountName $AutomationAccountName -Force
}
else {
    Write-Host "No job schedule found for runbook $RunbookName to unregister."
}

To execute this PowerShell script in the ARM template we use the ARM deployment script resource option. Deployment scripts give us the option to execute Azure PowerShell and Azure CLI deployment scripts on a Linux environment runtime which will be created in a container instance during the execution of the ARM template deployment. For more information read the Microsoft documentation.

The deployment script resource needs an user assigned identity to work correctly. I gave the identity contributor rights on the resource group of the Automation account. The deployment script will reference the PowerShell script which I deployed to a storage account. In the argument section the parameters of the script are specified.

{
	"name": "unregisterExistingJobSchedule",
	"type": "Microsoft.Resources/deploymentScripts",
	"apiVersion": "2020-10-01",
	"identity": {
	    "type": "userAssigned",
        "userAssignedIdentities": {
            "/subscriptions/01234567-89AB-CDEF-0123-456789ABCDEF/resourceGroups/myResourceGroup/providers/Microsoft.ManagedIdentity/userAssignedIdentities/myID": {}
        }
    },
	"location": "[resourceGroup().location]",
	"kind": "AzurePowerShell",
	"properties": {
		"azPowerShellVersion": "5.0",
		"cleanupPreference": "Always",
		"primaryScriptUri": "[concat(parameters('StorageAccount'), '/', 'Unregister-JobSchedule.ps1', parameters('StorageAccountSASToken'))]",
		"arguments": "[concat('-AutomationAccountName ', parameters('automationAccountName'), ' -ResourceGroupName ', resourceGroup().name, ' -RunbookName ', parameters('RunbookName'))]",
		"retentionInterval": "PT1H",
		"timeout": "PT15M"
	}
}

To make the GUID of the job schedule name everytime unique I append a date/time stamp to the functional name of the job schedule name and than generate the GUID with the guid() function based on that value. The parameter now is defined in the parameter section with a default value based on the utcNow() function.

{
	"name": "deployJobSchedule",
	"type": "Microsoft.Resources/deployments",
	"apiVersion": "2020-06-01",
	"dependsOn": [
		"unregisterExistingJobSchedule"
	],
	"properties": {
		"mode": "Incremental",
		"expressionEvaluationOptions": {
			"scope": "inner"
		},
		"parameters": {
			"automationAccountName": {
				"value": "[parameters('automationAccountName')]"
			},
			"jobScheduleName": {
				"value": "[guid(concat(parameters('jobScheduleName'), parameters('now')))]"
			}
		},
		"template": {
			"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
			"contentVersion": "1.0.0.0",
			"parameters": {
				"automationAccountName": {
					"type": "string"
				},
				"jobScheduleName": {
					"type": "string",
					"metadata": {
						"description": "The name of the job schedule should be unique every time we deploy."
					}
				}
			},
			"resources": [
				{
					"name": "[concat(parameters('automationAccountName'), '/', parameters('jobScheduleName'))]",
					"type": "Microsoft.Automation/automationAccounts/jobSchedules",
					"apiVersion": "2020-01-13-preview",
					"properties": {
					   {
                            "name": "[parameters('RunbookName')]",
                            "properties": {
                                "schedule": {
                                    "name": "[parameters('ScheduleName')]"
                                },
                                "runbook": {
                                    "name": "[parameters('RunbookName')]"
                                }
                            }
                        }
					}
				}
			]
		}
	}
}

This ARM template implementation will workaround the idempotency issue of the current jobSchedule implementation. I hope that Microsoft will fix the Azure Automation ARM template implementation in such a way that it is idempotent.

Comments

Popular posts from this blog

CS8357: The specified version string contains wildcards, which are not compatible with determinism.

Today I was busy with creating a WCF service solution in Visual Studio Enterprise 2017 (15.9.2). In this solution I use a few C# class libraries based on .NET 4.7.2. When I compiled the solution I got this error message: Error CS8357: The specified version string contains wildcards, which are not compatible with determinism. Either remove wildcards from the version string, or disable determinism for this compilation The error message is linking to my AssemblyInfo.cs file of the Class library projects. In all the projects of this solution I use the wildcard notation for generating build and revision numbers. // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.

Fixing HTTP Error 401.2 unauthorized on local IIS

Sometimes the Windows Authentication got broken on IIS servers so you cannot log in locally on the server. In that case you get the dreadfully error message HTTP Error 401.2 - Unauthorized You are not authorized to view this page due to invalid authentication headers. To fix this issue you can repair the Windows Authentication feature with the following PowerShell commands: Remove-WindowsFeature Web-Windows-Auth Add-WindowsFeature Web-Windows-Auth

Azure DevOps YAML based CD pipeline with multiple artifacts

In my post called " Switch from Azure DevOps release pipeline to YAML based CI/CD pipeline " I explained how to implement a combined CI/CD pipeline based on YAML templates. Because most of the deployments in my daily work are consuming multiple artifacts I had to do some additional R&D to find a solution for those releases in Azure DevOps. Base structure In the first post about the transition to YAML CI/CD pipelines I introduced the base structure. The templates I want to reuse are added to the yaml-template folders in the root of my GIT repo. For the CD pipelines which lack a CI phase I introduce a new folder called yaml-releases which will contain the YAML files for releases which are based on multiple artifacts. The folder structure within my GIT repo is thus: yaml-releases releaseArea myMultipleArtifactsRelease azure-pipelines.yml cd-job.yml yaml-templates jobs ... templates ...