Hi. With great interest I tried out the labeling system. I hoped it would be a way to suit my need in Katalon Testops: Test Environment reporting. I would think this would be a common concept in the testing world, but I don’t see this reflected in Katalon TestOps.
Use case: I have several projects where we run the Katalon tests on multiple environments, namely TST and ACC. Since we do agile iterative development, the tests on TST could break and often need rework. On ACC we have a production-like application that normally should always be green. It might be, due to integration with external parties, that on ACC a test suddenly starts failing (in other words: the external parties did some changes and these might impact our application in a negative way!).
So it is important for us to be able to quickly report on a state of TST & ACC.
We have the Katalon “profile” concept, where we do make a distinction on the environment (among other things) via the profile name. So we can filter out via ExecutionTestResult.profile:“~acc” all the test results from ACC, but that’s not ideal. Furthermore, we also upload jUnit test reports from ReadyAPI (sorry, their API testing tool still is more advanced than what is offered within Katalon) for both environments onto the Katalon TestOps via the Katalon Report Uploader (see https://docs.katalon.com/docs/analyze/reports/upload-test-reports/upload-junit-and-report-files-to-using-command-line). There we do not have the concept of “profile”, so we cannot filter those out.
So, I created a custom field “Environment” with labels “TST” & “ACC”. I manually linked it to a test Suite Collection result and hoped to be able to have a quick filter. It works… but it’s not ideal. Some shortcomings:
-
Obviously, goal should be to have this label automatically set to my test runs. Perfect place for me would be as a parameter in both the Katalon Runtime Engine command line (see https://docs.katalon.com/docs/execute/katalon-runtime-engine/command-line-syntax-in-katalon-runtime-engine) AND in the Katalon Report Uploader command line.
-
There is no url adaptation (see screenshot) for this filter, so I cannot store it as a favorite “application X - ACC results” in my browser. Each time, I will have to go for the custom filter and filter out the “TST” or “ACC”. Developers might for instance be only interested in the view for TST, while management wants to quickly see the state of ACC before an approval to go to PRODUCTION. I cannot go and ask them to play with the filter themselves…
Just my 2 cents.
Kind regards,
Joost
