Assertion errors

[fsaltiu]

Hello,

I have an issue that I'm not sure how to fix, it's probably me doing something wrong but I didn't find any other way of getting in contact. I use your Wrapper to convert my Postman Collections into Test Cases and run them with the release pipeline as specified in the wiki. The thing is that these Test Cases always seem to be passing when actually in Postman the Tests scripts do not pass.
For example:
I add pm.response.to.have.status(201) to my 'Tests' tab in Postman request and it returns 404 but the Test Case is still marked as passed. I have tried with different assertions like tests["Status code is 200"] = responseCode.code === 200 but I never seem to be able to make the Test Case fail (in Azure, in Postman it works as expected).
The question would be if I need to add some updates to the Wrapper to also check for the Tests scripts or if these assertions should be done in a different way.

Thank you very much.

[jbjakobs]

Hi,
Sorry for the slow response.
I dont have access to a setup right now where I can test it, but I expect you would need to update the parsing of the output from PostMan, which is in PostmanWrapper.ParseOutputFile. Right now it relies on Failures collection from the deserialized object from PostMan. Likely that error is not captured in that collection, but exists elsewhere in the deserialized object, meaning you have to Assert for that type of failure as well.

[fsaltiu]

Sorry for the late reply, I was on holidays last week. Thanks for the answer I will try it this week.

[fsaltiu]

After some time I was able to come back to this.
The only way I have found to make a request fail in the "Tests" tab of Postman after a request is to use pm.response.to.have.status(201). The bad thing of this solution is that it stops the execution of any other request that comes after it, but it's the patch I have been using.
The solution that you pointed, to check the ouput.json file, does not work because in that file I see the content inside of the json and the assertions do appear in the tests section, but the result of their execution is not anywhere. The assertions I have tested and do not make any output are:

• tests["Status code is 201"] = responseCode.code === 201;
• pm.expect(pm.response.code).to.eql(200);
• pm.test("Some tests", () => {...};)
  I attach what the output.json looks like after the execution of different requests with assertions like the stated before in the after tests.
  

Just to clarify, because it may be a bit confusing, this is where I put this assertions:

Thank you!