Skip to main content

Low-code/serverless (integration) environments and testability

This month I attended the Integrate 2018 integration congress which was held from June 4-6 in London. These three days I and all other attendees where informed about the latest low-code and serverless integration solutions in the Azure cloud. Think about solutions like Logic Apps, Azure Functions, Event Grid and API Management.

One general theme I was missing during all these sessions is the testability of the solution. For BizTalk my team invested a lot of time in writing unit tests for our solutions. One of my colleagues wrote an unit test framework which let us test our XML schemas, XSLTs and pipeline components for example based on the MSTest framework. Our CI/CD pipeline will execute these test every time the code is checked in and the BizTalk solution has not to be deployed on an actual BizTalk environment. With these low-code/serverless tools it feels I am back in the work on my machine (actual environment) and the labour intensive manually testing era. None of the speakers mentioned the concepts of unit testing and automatic integation/acceptance testing during the conference for these products.

Within our company we have a team of developers which use the low-code Mendix platform. This is also a platform which seems to lack the possibility to write unit tests for it. When observing that team I see that they do a lot of manually testing. They have a few testers in their Scrum team which do al the manually testing to verify that the apps are doing the right things. On the possitive side they are utilizing the test plan functionality of VSTS Test Manager for their manually testing but on the negative side they still not invested the time to write automatic web tests for their (web) apps.

Possible solutions for testing serverless applications like Logic Apps

Back to my own serverless/low-code integration stack. I don't like the idea that my integration solutions need a lot of manual testing every time I make a change to it. What are possible solutions to mitigate this?

For verifying that your ARM templates, which are Logic Apps in essence also, are not containing mistakes you can utilize for example JSON Lint to verify that the JSON code is valid. You can add this as a step in your VSTS CI builds by installing it via NPM. Add the Node Tool Installer step and the NPM step. In this NPM step you install JSON Lint with the command install jsonlint -g. To verify the actual JSON code you have to add a command line step. With the command FOR /r %%A IN (*.json) DO jsonlint %%A I can verify if every JSON file contains valid JSON. The next step to verify that the ARM template is deployable is doing a trial Azure Resource Group deployment with the 'Validation only' switch. It's not fool proof but will detect most errors. You can extend the validation of the ARM template by using Pester. I found the article "Testing ARM Templates with Pester" written by Justin Yoo which describes that scenario.

Based on my search findings unit testing Logic Apps looks not to be possible. We have to deploy our Logic Apps and utilize the concepts of automated integration/acceptance tests with BDD frameworks like SpecFlow. I found the following articles on the web about this solution path:
I will have to spent some time to investigate if SpecFlow is a solution for my team. It looks better than doing only manual testing thats for sure.


  1. The term "low-code app development" didn't exist until a few years ago but the concept isn't a new one. business users who see an opportunity to optimize a process and take it upon themselves to create their own apps. Rather than spend the time and manual effort to code an app from scratch that is made up of common features and components, low code development platforms let the developers work from existing templates and drag prebuilt elements, forms, and objects together to get a particular department or team the simple working app they need with a lot less hassle.


Post a Comment

Popular posts from this blog

CS8357: The specified version string contains wildcards, which are not compatible with determinism.

Today I was busy with creating a WCF service solution in Visual Studio Enterprise 2017 (15.9.2). In this solution I use a few C# class libraries based on .NET 4.7.2. When I compiled the solution I got this error message:
Error CS8357: The specified version string contains wildcards, which are not compatible with determinism. Either remove wildcards from the version string, or disable determinism for this compilation The error message is linking to my AssemblyInfo.cs file of the Class library projects. In all the projects of this solution I use the wildcard notation for generating build and revision numbers.
// Version information for an assembly consists of the following four values: // // Major Version // Minor Version // Build Number // Revision // // You can specify all the values or you can default the Build and Revision Numbers // by using the '*' as shown below: // [assembly: AssemblyVersion("1.0.*")] [assembly: AssemblyVersion("1.0.*&qu…

Deploy with ARM templates an Azure DevTest Lab environment

Within the company I work for we use Azure DevTest Labs already for a year to host our personal development computer and the BizTalk CI machines we need. We are planning to host also our development and test environments in Lab environments. At the moment we configure the lab environments manually.

I invested some time to write ARM templates to deploy Azure DevTest Labs within 5 minutes based on Azure DevOps CI/CD pipelines. In the process of writing these scripts I learned a lot. For example that the Azure documentation is not always up to date and that includes also tooling like Azure Resource Explorer. Based on querying the REST API with Postman I found the missing pieces for deploying a DevTest Lab environment based on nested ARM templates.

In this post I will document the nested ARM template I created the last few days. Lab With the first template you can create the lab environment itself and configure the following parts of the environment
Lab settings; like the storage which th…

Assign an existing certificate to your IIS website with WiX

Recently I had to change the bindings of existing IIS hosted websites and APIs from HTTP to HTTPS. They are installed with a MSI file created with the WiX Toolset.

Because I have to use an already on the server installed certificate I cannot use the Certificate element from the IIS Extension because this element only supports installing and uninstalling certificates based on PFX files. After doing some research I found the blog article Assign Certificate (Set HTTPS Binding certificate) to IIS website from Wix Installer which described the usage of Custom Actions for this purpose. I adopted this approach and rewrote the code for my scenario.

With WiX I still create the website.
<iis:WebSite Id="WebSite" ConfigureIfExists="yes" AutoStart="yes" Description="MyWebsite" Directory="IISROOT" StartOnInstall="yes"> <iis:WebAddress Id="WebSite" …