Skip to main content

API Management CI/CD using ARM Templates - API Management service and custom hostnames

Eldert Grootenboer (Motion10) posted a series of blog posts around setting up CI/CD for Azure API Management using Azure Resource Manager templates. In his series he describes how you can using Visual Studio Team Services to host GIT repositories and set up a build and release pipeline for deploying your API Management service and it's APIs.

This blog post is an extension on his first article wich describes the setup for deploying the API Management service and is done as a nested ARM template (as described in the linked template article).

Scenario

The API Management service is default hosted under Microsoft's azure-api.net DNS domain. To provide a professional experience to our developers we want to host our API Management service under our own DNS domain. The SSL certificate will be stored in Azure Key Vault and the configuration will be part of the CI/CD pipeline as described by Eldert in his blog posts.

To make the scenario more production like we will store the Key Vault and the API Management service in different resource groups and also use multiple subscriptions. The SSL certificate wil be a self-signed certificate and is created with the certificates functionality of Key Vault. Normally you will buy one from an official SSL Certificate provider like Verisign or Symantec.

Certificates

A SSL certificate (X.509) always has to have a subject. This subject field consists about the common name (the DNS name from the website) and information about the organization for which the certificate is issued. Nowedays Subject Alternative Names (SAN) are almost manditory. Browsers like Chrome check the existence of this field and declare your website insecure if it's absent.

An API Management service uses in total four DNS names. The four addresses we will register as SANs on the self-generated certificate. If you will buy a certificate you should do this also or buy a wildcard certificate instead.

DNS changes

For the custom domain name to work you will have to make some changes in your DNS. You will have to register four CNAME records. These records will couple your DNS names to the Microsoft generated ones.

Register the following CNAME records:
  1. Proxy endpoint: api.[your-domain].[domain-extension] to [api-instance].azure-api.net
  2. Management endpoint: management-api.[your-domain].[domain-extension] to [api-instance].management.azure-api.net
  3. Portal endpoint: portal-api.[your-domain].[domain-extension] to [api-instance].portal.azure-api.net
  4. SCM endpoint: scm-api.[your-domain].[domain-extension] to [api-instance].scm.azure-api.net

Key Vault

In my setup I use a Key Vault instance I created earlier. The Key Vault is hosted in a different resource group than the API Management service. To create the self-signed certificate we go to the certificates section of our Key Vault and start the procedure with the 'Generate/import' button.

Fill out the form as following:
  1. Method of Certificate Creation: Generate
  2. Certificate Name: [name-of-the-certificate]
  3. Type of Certificate Authority (CA): Self-signed certificate
  4. Subject: CN=[dns-name], OU=[organizational-unit], O=[organization], L=[locality], ST=[state-province], C=[two-letter-country-code]. For example the subject will be: CN=api.domain.com, OU=ICT, O=Antiohne, L=Gouda, ST=Zuid-Holland, C=NL.
  5. DNS Names: [The URIs of the four API Management endpoints]
  6. Validity Period (in months): 12
  7. Content Type: PKCS #12
  8. Lifetime Action Type: Automatically renew at given percentage lifetime
  9. Percentage Lifetime: 80
Configure under the advanced policy configuration part the following settings:
  1. Extended Key Usages (EKUs): 1.3.6.1.5.5.7.3.1, 1.3.6.1.5.5.7.3.2
  2. X.509 Key Usage Flages: Digital Signature and Key Encipherment
  3. Reuse Key on Renewal: No
  4. Exportable Private Key: Yes
  5. Key Type: RSA
  6. Key Size: 4096
  7. Certificate Type: [empty]
After filling start the creation of the self-signed certificate by pressing the Create button. After a few seconds the certificate will be generated and the status will be completed.

Open the current version of the generated certificate. The properties will be shown. Copy the secret identifier URI into a text document. You will need it later in your CI/CD pipeline to populate the ARM template with the correct value.

NOTE:
Pay attention to the ACL of (the resource group of) the Key Vault when you do a deployment which spans across multiple subscriptions. Each VSTS endpoint which deploys an API Management instance should have the correct rights to write mutations to the Key Vault instance. Without this right the deployment will fail.

ARM template

To access the Key Vault we need to run the service under a Managed service identity. Modify the service.json (instance.json in Eldert's blog) so it reflects the following ARM template.
{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "publisherEmail": {
            "type": "string",
            "defaultValue": "ci-cd@contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The email address of the owner of this API Management service"
            }
        },
        "publisherName": {
            "type": "string",
            "defaultValue": "CI/CD APIM",
            "minLength": 1,
            "metadata": {
                "description": "The name of the owner of this API Management service"
            }
        },
        "serviceName": {
            "type": "string",
            "defaultValue": "ci-cd-api",
            "minLength": 1,
            "metadata": {
                "description": "The name of this API Management service"
            }
        },
        "sku": {
            "type": "string",
            "defaultValue": "Developer",
            "allowedValues": [
                "Basic",
                "Developer",
                "Premium",
                "Standard"
            ],
            "metadata": {
                "description": "The pricing tier of this API Management service"
            }
        },
        "skuCount": {
            "type": "string",
            "defaultValue": "1",
            "allowedValues": [
                "1",
                "2"
            ],
            "metadata": {
                "description": "The instance size of this API Management service."
            }
        }
    },
    "variables": {},
    "resources": [
        {
            "apiVersion": "2017-03-01",
            "type": "Microsoft.ApiManagement/service",
            "name": "[parameters('serviceName')]",
            "location": "[resourceGroup().location]",
            "tags": {},
            "identity": {
                "type": "SystemAssigned"
            },
            "properties": {
                "publisherEmail": "[parameters('publisherEmail')]",
                "publisherName": "[parameters('publisherName')]",
                "customProperties": {
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls10": "False",
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls11": "False",
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Ssl30": "False",
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TripleDes168": "False",
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls10": "False",
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls11": "False",
                    "Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Ssl30": "False"
                }
            },
            "sku": {
                "name": "[parameters('sku')]",
                "capacity": "[parameters('skuCount')]"
            }
        }
    ]
}
The next step is to give the Managed service identity the rights to read the certificate in the Key Vault. Create in the local repository in the templates folder a new file called key-vault-permissions.json. Add the following ARM template to the file.
{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "serviceName": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "ci-cd-api",
            "metadata": {
                "description": "The name of this API Management service"
            }
        },
        "keyVaultSubscriptionId": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "[subscription().subscriptionId]",
            "metadata": {
                "description": "The subscription in which the Key Vault is located"
            }
        },
        "keyVaultResourceGroup": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "[resourceGroup().name]",
            "metadata": {
                "description": "The name of the resource group in which the Key Vault is located"
            }
        },
        "keyVaultName": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "ci-cd-api",
            "metadata": {
                "description": "The name of the Key Vault where the certifcates are stored"
            }
        }
    },
    "variables": {
        "serviceIdentityResourceId": "[concat(resourceId('Microsoft.ApiManagement/service', parameters('serviceName')), '/providers/Microsoft.ManagedIdentity/Identities/default')]"
    },
    "resources": [
        {
            "apiVersion": "2017-05-10",
            "name": "accessPoliciesTemplate",
            "type": "Microsoft.Resources/deployments",
            "subscriptionId": "[parameters('keyVaultSubscriptionId')]",
            "resourceGroup": "[parameters('keyVaultResourceGroup')]",
            "properties": {
                "mode": "Incremental",
                "template": {
                    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                    "contentVersion": "1.0.0.0",
                    "resources": [
                        {
                            "type": "Microsoft.KeyVault/vaults/accessPolicies",
                            "name": "[concat(parameters('keyVaultName'), '/add')]",
                            "apiVersion": "2015-06-01",
                            "properties": {
                                "accessPolicies": [
                                    {
                                        "tenantId": "[reference(variables('serviceIdentityResourceId'), '2015-08-31-PREVIEW').tenantId]",
                                        "objectId": "[reference(variables('serviceIdentityResourceId'), '2015-08-31-PREVIEW').principalId]",
                                        "permissions": {
                                            "secrets": [ "get" ]
                                        }
                                    }
                                ]
                            }
                        }
                    ]
                }
            }
        }
    ]
}
Because we first have to give the Managed service identity it's rights in the Key Vault we cannot configure the custom hostnames in the same service.json file. Create in the local repository in the templates folder a new file called service-hostnames.json. Add the following ARM template to the file.

Most of it is duplicate code from our service.json. The reason for that is because the duplicate fields are mandatory.
{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "publisherEmail": {
            "type": "string",
            "defaultValue": "ci-cd@contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The email address of the owner of this API Management service"
            }
        },
        "publisherName": {
            "type": "string",
            "defaultValue": "CI/CD APIM",
            "minLength": 1,
            "metadata": {
                "description": "The name of the owner of this API Management service"
            }
        },
        "serviceName": {
            "type": "string",
            "defaultValue": "ci-cd-api",
            "minLength": 1,
            "metadata": {
                "description": "The name of this API Management service"
            }
        },
        "sku": {
            "type": "string",
            "defaultValue": "Developer",
            "allowedValues": [
                "Basic",
                "Developer",
                "Premium",
                "Standard"
            ],
            "metadata": {
                "description": "The pricing tier of this API Management service"
            }
        },
        "skuCount": {
            "type": "string",
            "defaultValue": "1",
            "allowedValues": [
                "1",
                "2"
            ],
            "metadata": {
                "description": "The instance size of this API Management service."
            }
        },
        "managementCustomHostname": {
            "type": "string",
            "defaultValue": "management-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the management endpoint of this API Management service"
            }
        },
        "portalCustomHostname": {
            "type": "string",
            "defaultValue": "portal-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the portal endpoint of this API Management service"
            }
        },
        "proxyCustomHostname": {
            "type": "string",
            "defaultValue": "proxy-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the proxy endpoint of this API Management service"
            }
        },
        "scmCustomHostname": {
            "type": "string",
            "defaultValue": "scm-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the SCM endpoint of this API Management service"
            }
        },
        "keyVaultSslSecret": {
            "type": "string",
            "defaultValue": "https://ci-cd-api.vault.azure.net/secrets/ci-cd-api/46095e3d7bf74ff7a02d202451ce836c",
            "minLength": 1,
            "metadata": {
                "description": "Url to the Key Vault Secret containing the Ssl Certificate (application/x-pkcs12)"
            }
        }
    },
    "variables": {},
    "resources": [
        {
            "apiVersion": "2017-03-01",
            "type": "Microsoft.ApiManagement/service",
            "name": "[parameters('serviceName')]",
            "location": "[resourceGroup().location]",
            "identity": {
                "type": "SystemAssigned"
            },
            "properties": {
                "publisherEmail": "[parameters('publisherEmail')]",
                "publisherName": "[parameters('publisherName')]",
                "hostnameConfigurations": [
                    {
                        "type": "Portal",
                        "hostName": "[parameters('portalCustomHostname')]",
                        "keyVaultId": "[parameters('keyVaultSslSecret')]",
                        "negotiateClientCertificate": false,
                        "defaultSslBinding": false
                    },
                    {
                        "type": "Proxy",
                        "hostName": "[parameters('proxyCustomHostname')]",
                        "keyVaultId": "[parameters('keyVaultSslSecret')]",
                        "negotiateClientCertificate": false,
                        "defaultSslBinding": true
                    },
                    {
                        "type": "Management",
                        "hostName": "[parameters('managementCustomHostname')]",
                        "keyVaultId": "[parameters('keyVaultSslSecret')]",
                        "negotiateClientCertificate": false,
                        "defaultSslBinding": false
                    },
                    {
                        "type": "Scm",
                        "hostName": "[parameters('scmCustomHostname')]",
                        "keyVaultId": "[parameters('keyVaultSslSecret')]",
                        "negotiateClientCertificate": false,
                        "defaultSslBinding": false
                    }
                ]
            },
            "sku": {
                "name": "[parameters('sku')]",
                "capacity": "[parameters('skuCount')]"
            }
        }
    ]
}
To be able to configure the custom hostnames from our CI/CD pipeline we have to adjust our nested template. Add the following ARM template to the file.
{
    "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "storageAccount": {
            "type": "string",
            "defaultValue": "https://apimcicd.blob.core.windows.net/templates/",
            "metadata": {
                "description": "The storage account which will be used for storing the ARM templates"
            }
        },
        "storageAccountSASToken": {
            "type": "string",
            "defaultValue": "",
            "metadata": {
                "description": "The SAS token to access the storage account"
            }
        },
        "publisherEmail": {
            "type": "string",
            "defaultValue": "ci-cd@contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The email address of the owner of this API Management service"
            }
        },
        "publisherName": {
            "type": "string",
            "defaultValue": "CI/CD APIM",
            "minLength": 1,
            "metadata": {
                "description": "The name of the owner of this API Management service"
            }
        },
        "serviceName": {
            "type": "string",
            "defaultValue": "ci-cd-api",
            "minLength": 1,
            "metadata": {
                "description": "The name of this API Management service"
            }
        },
        "sku": {
            "type": "string",
            "defaultValue": "Developer",
            "allowedValues": [
                "Basic",
                "Developer",
                "Premium",
                "Standard"
            ],
            "metadata": {
                "description": "The pricing tier of this API Management service"
            }
        },
        "skuCount": {
            "type": "string",
            "defaultValue": "1",
            "allowedValues": [
                "1",
                "2"
            ],
            "metadata": {
                "description": "The instance size of this API Management service."
            }
        },
        "managementCustomHostname": {
            "type": "string",
            "defaultValue": "management-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the management endpoint of this API Management service"
            }
        },
        "portalCustomHostname": {
            "type": "string",
            "defaultValue": "portal-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the portal endpoint of this API Management service"
            }
        },
        "proxyCustomHostname": {
            "type": "string",
            "defaultValue": "proxy-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the proxy endpoint of this API Management service"
            }
        },
        "scmCustomHostname": {
            "type": "string",
            "defaultValue": "scm-api.contoso.com",
            "minLength": 1,
            "metadata": {
                "description": "The custom hostname for the SCM endpoint of this API Management service"
            }
        },
        "keyVaultSubscriptionId": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "[subscription().subscriptionId]",
            "metadata": {
                "description": "The subscription in which the Key Vault is located"
            }
        },
        "keyVaultResourceGroup": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "[resourceGroup().name]",
            "metadata": {
                "description": "The name of the resource group in which the Key Vault is located"
            }
        },
        "keyVaultName": {
            "type": "string",
            "minLength": 1,
            "defaultValue": "ci-cd-api",
            "metadata": {
                "description": "The name of the Key Vault where the certifcates are stored"
            }
        },
        "keyVaultSslSecret": {
            "type": "string",
            "defaultValue": "https://ci-cd-api.vault.azure.net/secrets/ci-cd-api/46095e3d7bf74ff7a02d202451ce836c",
            "minLength": 1,
            "metadata": {
                "description": "Url to the Key Vault Secret containing the Ssl Certificate (application/x-pkcs12)"
            }
        }
    },
    "variables": {},
    "resources": [
        {
            "apiVersion": "2017-05-10",
            "name": "serviceTemplate",
            "type": "Microsoft.Resources/deployments",
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[concat(parameters('storageAccount'), '/', 'service.json', parameters('storageAccountSASToken'))]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "publisherEmail": { "value": "[parameters('publisherEmail')]" },
                    "publisherName": { "value": "[parameters('publisherName')]" },
                    "serviceName": { "value": "[parameters('serviceName')]" },
                    "sku": { "value": "[parameters('sku')]" },
                    "skuCount": { "value": "[parameters('skuCount')]" }
                }
            }
        },
        {
            "apiVersion": "2017-05-10",
            "name": "keyVaultPermissionsTemplate",
            "type": "Microsoft.Resources/deployments",
            "dependsOn": [
                "[resourceId('Microsoft.Resources/deployments', 'serviceTemplate')]"
            ],
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[concat(parameters('storageAccount'), '/', 'key-vault-permissions.json', parameters('storageAccountSASToken'))]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "serviceName": { "value": "[parameters('serviceName')]" },
                    "keyVaultSubscriptionId": { "value": "[parameters('keyVaultSubscriptionId')]" },
                    "keyVaultResourceGroup": { "value": "[parameters('keyVaultResourceGroup')]" },
                    "keyVaultName": { "value": "[parameters('keyVaultName')]" }
                }
            }
        },
        {
            "apiVersion": "2017-05-10",
            "name": "serviceHostnamesTemplate",
            "type": "Microsoft.Resources/deployments",
            "dependsOn": [
                "[resourceId('Microsoft.Resources/deployments', 'keyVaultPermissionsTemplate')]"
            ],
            "properties": {
                "mode": "Incremental",
                "templateLink": {
                    "uri": "[concat(parameters('storageAccount'), '/', 'service-hostnames.json', parameters('storageAccountSASToken'))]",
                    "contentVersion": "1.0.0.0"
                },
                "parameters": {
                    "publisherEmail": { "value": "[parameters('publisherEmail')]" },
                    "publisherName": { "value": "[parameters('publisherName')]" },
                    "serviceName": { "value": "[parameters('serviceName')]" },
                    "sku": { "value": "[parameters('sku')]" },
                    "skuCount": { "value": "[parameters('skuCount')]" },
                    "managementCustomHostname": { "value": "[parameters('managementCustomHostname')]" },
                    "portalCustomHostname": { "value": "[parameters('portalCustomHostname')]" },
                    "proxyCustomHostname": { "value": "[parameters('proxyCustomHostname')]" },
                    "scmCustomHostname": { "value": "[parameters('scmCustomHostname')]" },
                    "keyVaultSslSecret": { "value": "[parameters('keyVaultSslSecret')]" }
                }
            }
        }
    ]
}
That’s it for our ARM templates, so commit and push it to the VSTS repository.

Deployment pipeline

Make the necessary adjustments to the release definition. Add the new Key Vault and custom hostname parameters to the 'Override template parameters'. Use for the keyVaultSslSecret parameter the value which you earlies copied to a text document when you was generating the self-signed certificate.

Testing

To test our process we can now simply make a change to our local ARM template and check it in to the repository. This will trigger our build and release pipeline, and finally deploy the API into our API Management instance. After a succesfull deployment the API Management service can be reached under our own domain name.
Labels:

Comments

Post a Comment

Popular posts from this blog

CS8357: The specified version string contains wildcards, which are not compatible with determinism.

Today I was busy with creating a WCF service solution in Visual Studio Enterprise 2017 (15.9.2). In this solution I use a few C# class libraries based on .NET 4.7.2. When I compiled the solution I got this error message: Error CS8357: The specified version string contains wildcards, which are not compatible with determinism. Either remove wildcards from the version string, or disable determinism for this compilation The error message is linking to my AssemblyInfo.cs file of the Class library projects. In all the projects of this solution I use the wildcard notation for generating build and revision numbers. // Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.
Post a Comment
Read more

Fixing HTTP Error 401.2 unauthorized on local IIS

Sometimes the Windows Authentication got broken on IIS servers so you cannot log in locally on the server. In that case you get the dreadfully error message HTTP Error 401.2 - Unauthorized You are not authorized to view this page due to invalid authentication headers. To fix this issue you can repair the Windows Authentication feature with the following PowerShell commands: Remove-WindowsFeature Web-Windows-Auth Add-WindowsFeature Web-Windows-Auth
Post a Comment
Read more

Make steps conditional in multi-stage YAML pipelines

To make the switch from the graphical release pipelines in Azure DevOps I am missing two features. The first one is to be able to defer a deploy and the second one is to exclude certain deployment steps without the need for editing the YAML file.  The defer option is something Microsoft has to solve in their Azure DevOps proposition. It's a feature which you have in the graphical release pipeline but what they have not implemented yet in their YAML pipeline replacement. Approvals and certain gate conditions are implemented on the environment but the defer option is still missing .  Pipeline The conditional deployment option can be implemented with the help of runtime parameters and expressions . In the parameter section you define boolean parameters which will control the deploy behavior. With the expressions you can control which stage/job/task should be executed when the pipeline runs. In the below YAML sample I experimented with conditions in the azure-pipelines.yml  file
Post a Comment
Read more