Hi everyone!
Since Katalon Studio 7.8.2, Katalon TestOps will give more details about assertions in execution reports. Assertions ensure that the application is working correctly by checking whether a condition is true, i.e., whether labels, data, API responses, etc., are rendered correctly.
This new feature will give you a better grasp of the severity of a failure. For example, a test result with 1% of its assertions have failed usually indicates a better product quality than that with 99% of the assertions have failed.
Moreover, the number of assertions of a test case also tells something about this test case’s quality. A test case without any assertions is usually less useful than a test case with a dozen assertions.
As usual, this new feature required the integration between Katalon Studio projects and Katalon TestOps.
Whenever an execution is submitted to Katalon TestOps, you can see the number of assertions for each test result.
In a specific test result, you can see a list of all failed assertions without having to go through the log file.
Katalon TestOps also shows the number of assertions a test case normally has so that you can identify test cases that need more assertions.
Built-in assertion keywords have names prefixed by verify
, e.g. landingPage = WebUI.verifyElementPresent(findTestObject('Page_CuraAppointment/div_Appointment'), GlobalVariable.G_Timeout)
.
Custom assertion keywords are Groovy/Java methods that make sure of KeywordUtil.markPassed(message)
, KeywordUtil.markFailed(message)
, KeywordUtil.markFailedAndStop(message)
.
You can also exclude a custom keyword from being identified as an assertion in the Project Setting page of Katalon TestOps.