As I posted here more than a year ago, I’d like to now how Katalon TestOps sees the concept of “Test Environments”.
Use case: I have several projects where we run the Katalon tests on multiple environments, namely TST and ACC. Since we do agile iterative development, the tests on TST could break and often need rework. On ACC we have a production-like application that normally should always be green. It might be, due to integration with external parties, that on ACC a test suddenly starts failing (in other words: the external parties did some changes and these might impact our application in a negative way!).
So it is important for us to be able to quickly report on a state of TST & ACC.
We have the Katalon “profile” concept, where we do make a distinction on the environment (among other things) via the profile name. So we can filter out via ExecutionTestResult.profile:“~acc” all the test results from ACC, but that’s not ideal. Furthermore, we also upload jUnit test reports from ReadyAPI for both environments onto the Katalon TestOps via the Katalon Report Uploader (see Upload JUnit and report files to using command line | Katalon Docs). There we do not have the concept of “profile”, so we cannot filter those out.
I tried to use the labels feature as described in my post above:
So, I created a custom field “Environment” with labels “TST” & “ACC”. I manually linked it to a test Suite Collection result and hoped to be able to have a quick filter. It works… but it’s not ideal. Some shortcomings:
- Obviously, goal should be to have this label automatically set to my test runs. Perfect place for me would be as a parameter in both the Katalon Runtime Engine command line (see Command-Line Syntax in Katalon Runtime Engine | Katalon Docs) AND in the Katalon Report Uploader command line.
- There is no url adaptation (see screenshot) for this filter, so I cannot store it as a favorite “application X - ACC results” in my browser. Each time, I will have to go for the custom filter and filter out the “TST” or “ACC”. Developers might for instance be only interested in the view for TST, while management wants to quickly see the state of ACC before an approval to go to PRODUCTION. I cannot go and ask them to play with the filter themselves…