When I looking generated reports of my tests i can detect some necessary ways to catalog the test status (some of them already exists in katalon):
- PASSED: i need to know if my test case finished correctly.
- FAILED: opposed to “passed”, tell when test case didn’t finish as we expected.
- ERROR: thits state is very important. With this state i can know if my test case didn’t work as i expected AND this is because i didn´t build my test case correctly (my fault).
- IMCOMPLETE: i never get this state, buy i know it exists. (or not?)
i really consider one more status necessary (which doesn’t exist yet) → “WARNING”: This state should tell me when my test case PASSED correctly “but” i need to pay attention to some test steps (abaout his behavior or some optional validation), that is, a test case valid with some test steps that give them attention.
for example:
step 1 → login
step 2 → fill some textboxs
step 3 → select some objects (depending on the result of textboxs)
step 4 → go to next page
if i dont find this “objects” in step 3, this doesn’t mean that failed. But it’s important to me to know if this object doesn’t exist and see them in my report by status.
the issue is: it will be awesome a new status like “Warning” in reports, like Visual Studio IDE have in his error status.
(Sorry for my bad English, Im an enthusiastic Mexican Tester (: Great job guys! I love Katalon)