[New Release] Katalon Platform Updates - December 7, 2022

Hi Community members :wave:,

Happy holiday seasons! :christmas_tree: :gift:

As you may aware, we launched the new modern and comprehensive Katalon Platform on September [Breaking News] The new Katalon Platform :tada: . Going forward, we would like to announce all releases with Module format (Plan, Author, Organize, Execute, Analyze and optional Administer). In this release note, we would like to share:

Author

  • To improve the product compatibility and security compliance, we are happy to release those items:
    • We added Chrome 107 and Edge 107 compatibility for Katalon Studio 8.5.5 and later releases
    • Katalon Studio now has an updated look, courtesy of our new brand logo
    • Security compliance enhancements that come with Katalon 8.5.5
    • Added the option to automatically generate HTML reports for test suite collections
    • Email addresses that contain the special character ' can now activate their accounts in Katalon Studio
    • Some bug fixes

More details on Author release can be found here

Organize

  • We are happy to announce our partnership with XRay:
    • The Jira/Xray integration offers the ability to link and manage their manual & automation test from TestOps & Xray
    • Now, users can define their Jira project key in the Jira/Xray integration. This feature will help users define the Jira projects that are allowed to be shared, minimizing risks while using Jira/Xray integration.

More details here

  • In order to facilitate user experience on test suite organization, we also deployed a labeling system:
  • Users who use the Robot framework can orchestrate testing activities effectively and efficiently by integrating with TestOps. In addition, the status of visual test cases developed by Robot can be promptly driven by Visual Testing results.

Analyze

  • To improve our reports, we implemented some enhancements for Test Run report and user can now:
    • Synchronize test run list and test result by status chart
    • Customizable filter
    • Interactive search bar
    • Advance Time range filter

More details on Organize and Analyze release can be found here

1 Like

Hi. With great interest I tried out the labeling system. I hoped it would be a way to suit my need in Katalon Testops: Test Environment reporting. I would think this would be a common concept in the testing world, but I donā€™t see this reflected in Katalon TestOps.

Use case: I have several projects where we run the Katalon tests on multiple environments, namely TST and ACC. Since we do agile iterative development, the tests on TST could break and often need rework. On ACC we have a production-like application that normally should always be green. It might be, due to integration with external parties, that on ACC a test suddenly starts failing (in other words: the external parties did some changes and these might impact our application in a negative way!).
So it is important for us to be able to quickly report on a state of TST & ACC.

We have the Katalon ā€œprofileā€ concept, where we do make a distinction on the environment (among other things) via the profile name. So we can filter out via ExecutionTestResult.profile:ā€œ~accā€ all the test results from ACC, but thatā€™s not ideal. Furthermore, we also upload jUnit test reports from ReadyAPI (sorry, their API testing tool still is more advanced than what is offered within Katalon) for both environments onto the Katalon TestOps via the Katalon Report Uploader (see Upload JUnit and report files to using command line | Katalon Docs). There we do not have the concept of ā€œprofileā€, so we cannot filter those out.

So, I created a custom field ā€œEnvironmentā€ with labels ā€œTSTā€ & ā€œACCā€. I manually linked it to a test Suite Collection result and hoped to be able to have a quick filter. It worksā€¦ but itā€™s not ideal. Some shortcomings:

  1. Obviously, goal should be to have this label automatically set to my test runs. Perfect place for me would be as a parameter in both the Katalon Runtime Engine command line (see Command-Line Syntax in Katalon Runtime Engine | Katalon Docs) AND in the Katalon Report Uploader command line.

  2. There is no url adaptation (see screenshot) for this filter, so I cannot store it as a favorite ā€œapplication X - ACC resultsā€ in my browser. Each time, I will have to go for the custom filter and filter out the ā€œTSTā€ or ā€œACCā€. Developers might for instance be only interested in the view for TST, while management wants to quickly see the state of ACC before an approval to go to PRODUCTION. I cannot go and ask them to play with the filter themselvesā€¦

Just my 2 cents.

Kind regards,
Joost

1 Like

Thank you @joost.degeyndt for your insightful sharing, tag @anh.chu @duy.nguyen for your information.

Youā€™re welcome. Just noticed that there was a new release, with some new changes.
1Ā° The results are now no longer visible, only an image of a bar which upon mouse-over shows the actual passed, failed, total etc.
2Ā° The filter moves on top of the graph ā€œTest run by statusā€.

For me those are unwelcome changes :frowning:
1Ā° I want to see the results immediately in as less clicks as possible. I often take a screenshot of this page to report on the status of the application and now this will be no longer possible. Note: Thatā€™s why I also mentioned once here that Iā€™d prefer to see the absolute date-time of when a run started, even when it was a result of today, instead of the ambiguous ā€œ4 hours agoā€, which loses meaning.
2Ā° I never had any use of this graph due to the aforementioned impossibility to distinguish between TST & ACC. Also, on this page, my number 1 priority is to (quickly) see the results, so I am not interested in that graph. Previously, I had the possibility to minimize the graph, but this minimize button is now gone.

Again, just my 2 cents. Interested to hear from other Katalon end usersā€¦
(PS: I could understand that other users might have different needs, so perhaps a feature to allow customizing the reporting view would be a possibility one day?)