This month I attended the Integrate 2018 integration congress which was held from June 4-6 in London. These three days I and all other attendees where informed about the latest low-code and serverless integration solutions in the Azure cloud. Think about solutions like Logic Apps, Azure Functions, Event Grid and API Management.
One general theme I was missing during all these sessions is the testability of the solution. For BizTalk my team invested a lot of time in writing unit tests for our solutions. One of my colleagues wrote an unit test framework which let us test our XML schemas, XSLTs and pipeline components for example based on the MSTest framework. Our CI/CD pipeline will execute these test every time the code is checked in and the BizTalk solution has not to be deployed on an actual BizTalk environment. With these low-code/serverless tools it feels I am back in the work on my machine (actual environment) and the labour intensive manually testing era. None of the speakers mentioned the concepts of unit testing and automatic integation/acceptance testing during the conference for these products.
Within our company we have a team of developers which use the low-code Mendix platform. This is also a platform which seems to lack the possibility to write unit tests for it. When observing that team I see that they do a lot of manually testing. They have a few testers in their Scrum team which do al the manually testing to verify that the apps are doing the right things. On the possitive side they are utilizing the test plan functionality of VSTS Test Manager for their manually testing but on the negative side they still not invested the time to write automatic web tests for their (web) apps.
For verifying that your ARM templates, which are Logic Apps in essence also, are not containing mistakes you can utilize for example JSON Lint to verify that the JSON code is valid. You can add this as a step in your VSTS CI builds by installing it via NPM. Add the Node Tool Installer step and the NPM step. In this NPM step you install JSON Lint with the command
Based on my search findings unit testing Logic Apps looks not to be possible. We have to deploy our Logic Apps and utilize the concepts of automated integration/acceptance tests with BDD frameworks like SpecFlow. I found the following articles on the web about this solution path:
One general theme I was missing during all these sessions is the testability of the solution. For BizTalk my team invested a lot of time in writing unit tests for our solutions. One of my colleagues wrote an unit test framework which let us test our XML schemas, XSLTs and pipeline components for example based on the MSTest framework. Our CI/CD pipeline will execute these test every time the code is checked in and the BizTalk solution has not to be deployed on an actual BizTalk environment. With these low-code/serverless tools it feels I am back in the work on my machine (actual environment) and the labour intensive manually testing era. None of the speakers mentioned the concepts of unit testing and automatic integation/acceptance testing during the conference for these products.
Within our company we have a team of developers which use the low-code Mendix platform. This is also a platform which seems to lack the possibility to write unit tests for it. When observing that team I see that they do a lot of manually testing. They have a few testers in their Scrum team which do al the manually testing to verify that the apps are doing the right things. On the possitive side they are utilizing the test plan functionality of VSTS Test Manager for their manually testing but on the negative side they still not invested the time to write automatic web tests for their (web) apps.
Possible solutions for testing serverless applications like Logic Apps
Back to my own serverless/low-code integration stack. I don't like the idea that my integration solutions need a lot of manual testing every time I make a change to it. What are possible solutions to mitigate this?For verifying that your ARM templates, which are Logic Apps in essence also, are not containing mistakes you can utilize for example JSON Lint to verify that the JSON code is valid. You can add this as a step in your VSTS CI builds by installing it via NPM. Add the Node Tool Installer step and the NPM step. In this NPM step you install JSON Lint with the command
install jsonlint -g. To verify the actual JSON code you have to add a command line step. With the command FOR /r %%A IN (*.json) DO jsonlint %%A I can verify if every JSON file contains valid JSON. The next step to verify that the ARM template is deployable is doing a trial Azure Resource Group deployment with the 'Validation only' switch. It's not fool proof but will detect most errors. You can extend the validation of the ARM template by using Pester. I found the article "Testing ARM Templates with Pester" written by Justin Yoo which describes that scenario.Based on my search findings unit testing Logic Apps looks not to be possible. We have to deploy our Logic Apps and utilize the concepts of automated integration/acceptance tests with BDD frameworks like SpecFlow. I found the following articles on the web about this solution path:
- "Automated Testing with Logic Apps and Specflow" by Michael Stephenson.
- "A simple unit testing framework for Azure Logic Apps" by Shankars
- "Is Your Serverless Application Testable? - Azure Logic Apps" by Justin Yoo and his article "Is Your Serverless Applicatin Testable? - Azure Functions"
- "Automated Testing Logic Apps with Specflow" by Félix Mondelo
Comments
Post a Comment