Skip to content

Home

Jump to bottom
jbjakobs edited this page Oct 14, 2020 · 28 revisions

Overview

This repository contains a solution for enabling postman tests as first class citizens in Azure DevOps pipelines, meaning that each postman test case is represented by an Azure DevOps (ADO) work item of type "test case", and can thus participate in Test Plans.

ADO pipeline overview

There are in general two ways of running test cases in ADO:

  1. In a build pipeline. Test source code is checked into the repository. It might need to be compiled, depending upon the test framework. A build task in the pipeline can compile the test source code. Another build task in the pipeline can execute the tests, and the test results can be viewed as part of the build pipeline result. This is convenient for e.g. unit tests, for which you typically are not interested in setting up test plans etc.

  2. In a release pipeline. In order to use a release pipeline for testing, you setup a Test Plan (TP) in ADO, and add test cases to the TP. Then you can run automated tests in a TP by specifying which release pipeline to use. In the release pipeline setup, you specify which build artefacts to use (like test code/binaries and related dependencies, test execution prerequisites, etc). An automated test must contain metadata about the location and type of the test.

Test case automation in ADO

How does test cases based on MSTest fit into ADO?

MSTest is a Microsoft based unit test type framework, and is equivalent to many other unit test framework, like nunit, junit, Google test, etc. The source code for a test class typically looks like

[TestClass] public class MyTestSuite { [TestMethod] public void MyTestCase1() { //Test code }

[TestMethod] public void MyTestCase2() { //Test code } }

When compiling a MSTest project in Visual Studio (VS), you can execute test cases in the Test Explorer window. If you run use these automated test cases in a Test Plan, you would need to do the following:

  1. Manually create a workitem of type Test Case in ADO.
  2. From Test Explorer in VS, map the coded test case to the work item Test Case in ADO. Doing this will fill out the "Associated Automation" tab in the work item Test Case you mapped to, enabling ADO to know where to find that automated case, when you want to run it.

How does test cases based on Postman fit into ADO?

First of all, it is relevant to understand how Postman test cases are structured, and how you save Postman test cases into ADO. In Postman, a collection has a layout like:

MyCollection1 MyFolder1 Request1 Request2 MyFolder2 Request3 Request4 MyCollection2 MyFolder1 Request1 Request2 Request3

So, a collection can contain folders, which in turn can contain requests. This is very similar to the layout of MSTest described above. Since this implementation use MSTest as a wrapper to execute Postman tests, the following, natural mapping has been made:

MSTest TestClass = Postman Collection MSTest TestMethod = Postman Folder

Clone this wiki locally