[Recap] Ask Katalon Anything Sep 2023

[albert.vu]

Hi Community members,

Thank you very much for participating in our recent Ask Katalon Anything session! We hope our Product team members were able to help you with any questions that you had regarding the Katalon Platform, Katalon’s product roadmap, and more.

Below you will find the list of questions which has been categorized into respective modules of the Katalon Platform.

1. Katalon TestOps

1.1. Is Katalon considering revamping TestOps' UI/UX?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #5 by Monty_Bagati

@Monty_Bagati

Hello to TestOps and Cloud Studio Team : @tri.tle @vy.huynh

Not a query, but a proposal or advice that I have previously made. In contrast to the vibrant katalon.com website, the design/UI layout of TestOps is not very appealing. The TestOps platform would seem more appealing and visible to the end user, especially to CTOs who are seeking to confirm KS/KSE/TestOps as their Automation solution, if it could be completely revamped like your main website, www.katalon.com.

Answer

@tri.tle

Hello @Monty_Bagati, it’s a pleasure to reconnect with you.

The Katalon team is currently engaged in enhancing the interface to provide a more appealing and user-friendly experience. Naturally, such transformations demand a significant amount of time and it’s quite challenging to implement them throughout the entire application.

As we progress, we would greatly value insights from our users. We are considering conducting usability testing and may extend an invitation soon. It would be greatly appreciated if you could participate.

1.2. How to get Katalon to generate the required properties for each validation step in the JUnit .XML Report?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #19 by Testinator-X

@Testinator-X

We have integrated our Katalon tests with TestRail. For this we use the TestRail CLI tool instead of the Katalon TestRail Integration plugin, as the plugin does not meet our requirements. Now we would like to use the new feature of the TestRail CLI tool to automatically add step information in the results. For this purpose the JUnit XML report generated by Katalon should be enriched with additional properties as described here or here .

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="0.05" name="tests.LoginTests">
    <properties><property name="setting1" value="True"/></properties>
    <testcase classname="tests.LoginTests" name="test_case_2" time="650">
     <properties>
         <property name="testrail_attachment" value="path_to/logs.log"/>
         <property name="testrail_attachment" value="path_to/screenshot.jpg"/>
         <property name="testrail_result_field" value="version:1.2"/>
         <property name="testrail_result_field" value="custom_environment:qa02"/>
         <property name="testrail_result_comment" value="Finding 1"/>
         <property name="testrail_result_comment" value="Finding 2"/>
         <property name="testrail_result_step" value="passed: Insert login credentials"/>
         <property name="testrail_result_step" value="failed: Click submit"/>
         <property name="testrail_result_step" value="untested: User should be logged in"/>
     </properties>
    </testcase>
  </testsuite>
</testsuites>

How to get Katalon to generate the required properties for each validation step in the JUnit .XML Report?

Answer

@tri.tle

Hi @Testinator-X, apologies for the delayed response. Your question required some contemplation. Regrettably, we currently do not have the capability to generate properties as you described in Katalon reports. However, your suggestion is valuable, my team will explore this further and consider incorporating it into our future roadmap.

@Testinator-X

We have integrated our Katalon tests with TestRail. For this we use the TestRail CLI tool instead of the Katalon TestRail Integration plugin, as the plugin does not meet our requirements.

In regards to your requirements for the plugin, could you please provide us with more details? This would greatly assist us in developing a plugin that is tailored to your needs.

1.3. Is TestOps equipped to allow me to seamlessly manage both my test cases and requirements?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #22 by karthik.srivatsan

@karthik.srivatsan

I have a TestOps related question for @tri.tle as follow: Is TestOps equipped to allow me to seamlessly manage both my test cases and requirements?

Looking forward to your answer. Thanks!

Answer

@tri.tle

Hi @karthik.srivatsan, indeed, Katalon Platform provides the capability to manage your test cases along with requirements. If you are utilizing Jira to manage your requirements, you can seamlessly integrate your Jira projects with Katalon projects. Subsequently, you can link your Jira requirements with the corresponding test cases. For more information about Jira integration, you can follow it here.

1.4. Whenever I run my Test Suite/Test Suite Collection on Windows 11, the report always shows Windows 10 which is rather misleading.

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #37 by atul.rai

@atul.rai

This is a great initiative, thanks Katalon Team! I have some question as below-
Test run on safari browser is not much compatible compare to other browser like chrome, Firefox, Edge chromium etc. So we expect improvement for this.

@tri.tle - when we run our test suite/testsuiteCollection on Windows 11 OS and view report on TestOps,then always showing Windows 10.
Sometimes are showing N/A as attacheched in the screenshot.

Answer

@tri.tle

Hi @atul.rai,
Apologies for the delayed response and thank you for your questions. I will address them in the following text.

@atul.rai

Test run on safari browser is not much compatible compare to other browser like chrome, Firefox, Edge chromium etc. So we expect improvement for this.

Could you provide more specifics about the issues you’re experiencing with the Safari browser? Understanding your needs better will help us make the necessary improvements.

@atul.rai

When we run our test suite/testsuiteCollection on Windows 11 OS and view report on TestOps,then always showing Windows 10.
Sometimes are showing N/A as attacheched in the screenshot.

In relation to the issues with Windows 11, following a period of investigation, we have determined that the problem originates from our system. We are currently addressing the issue and I will get back to you once it has been resolved.

1.5. Are there plans to develop plugins/integration between Katalon and other test management tools such as TestMo or Qase?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #43 by Testinator-X

@Testinator-X

Many TestRail customers are switching to alternative tools such as testmo or Qase, because the product quality there is apparently much better and the tools are also much more modern. In particular, the migration from TestRail to Testmo is said to be relatively easy, because Testmo’s roots are in TestRail. Are further plugins planned to support integrating Katalon to these or other test management tools?

Answer

@tri.tle

Hi @Testinator-X,

@Testinator-X

In particular, the migration from TestRail to Testmo is said to be relatively easy, because Testmo’s roots are in TestRail. Are further plugins planned to support integrating Katalon to these or other test management tools?

Thank you for your valuable insights. Katalon has plans to support integration with other test management tools such as Xray, which was released by Katalon in 2023. However, the implementation timeline will depend on the priority of integrations, so Katalon will have an official announcement if an integration will be released in the near future.

---

2. Katalon Studio

2.1. Is there any documentation available about developing Katalon tests without the use of Katalon Studio?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #6 by coen.schalkwijk

@coen.schalkwijk

Hey there Katalon peeps/ @xuan.tran

As a developer/API Test Automation engineer I find the tie-in with Eclipse bothersome, there’s some issues with the IDE I can really do without. There’s also a lot of overhead mainly supporting non-technical users that I could do without.

I was wondering if there is any documentation available about developing Katalon tests without the use of Katalon Studio?

For higher level FE test development, Katalon Studio still has the same issues, but the pro’s outweigh them (less need of technical knowledge/experience).

Kind Regards,

Coen

Answer

@xuan.tran

Hi @coen.schalkwijk,
As Russ Thomas and Kazurayam already replied, you can see a few threads on the forum on how to write Katalon tests with other IDEs, and I think it’s really worth trying. However, we do not officially support this approach yet, so currently there’s no official document on docs.katalon.com for this tutorial. We’re still developing Katalon Studio so that we can bring a better experience to users, so we acknowledge your need and will consider it in the future.
Thank you so much and happy testing
Xuan Tran.

2.2. Is Katalon considering JavaScript as a potential development language for Katalon Studio?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #9 by Russ_Thomas

@Russ_Thomas

My question:

Is anyone brave enough to set a date for the elevation of JavaScript to become a first class development language for Test Cases in Katalon Studio?

@duyluong and I had several discussions about the possibility. In late 2021 he told me it is being worked on.

My suspicion is, in the rush to move into the cloud and latterly the heavy focus on “AI”, JS promotion and adoption has fallen off the development radar.

Anyone crazy enough to poke their head above of the trench?

Answer

@xuan.tran

Hi @Russ_Thomas,
Thank you so much for bringing this up. JavaScript sounds interesting, let’s see what we could do about it.

In the meantime, Groovy is still the main focus for us, but we would love to hear your thoughts on this topic. Thumb-up if you want JavaScript in some next couple of releases, and don’t hesitate to comment if you want Studio to support other languages as well

2.3. What is Katalon's plan regarding supporting Selenium 4 in Katalon Studio?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #29 by f_poccard

@f_poccard

Hi @xuan.tran, I just want to know what is Katalon’s plans are regarding supporting Selenium 4 in Katalon Studio as this question has been asked before on the forum…

Hi guys, I found, that Katalon Studio uses selenium version 3.141.59, which is actually latest stable version, but there is already Selenium 4 release candidate, which has some new features I would like to use. So my question is, will you upgrade Selenium to version 4 as soon as it’s released? If I wanted to use it right now, is it possible to upgrade Selenium manually?

Thanks!

Answer

@xuan.tran

Hi @f_poccard,
Thank you for sharing this question!

As you might know, we had a PoC for Selenium 4 a while ago, and we’re keeping this item as a candidate for some upcoming releases.

We would love to hear your feedback on the PoC, so here’s the link for you to download and try it out: 8.2.1.alpha. Let us know your thoughts on this PoC , and happy testing!

2.4. How to set up Sharing Test Artifacts and Object Repository within Katalon Studio?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #30 by chihung.le

@chihung.le

Hi Albert

We are new to Katalon.

We like to setup the sharing Test sets and Object Repo b/w the development. How to set it up?

Also, like to scheduling the automation testing.

It would be great if you can send us some info.

Best regards

Chi Hung Le

Answer

@albert.vu

Hi @chihung.le,

I am sending you a few articles from our documentation base which I believe can help you as below:

For sharing Test sets/test artifacts:

https://docs.katalon.com/docs/create-tests/manage-test-artifacts/test-artifacts-sharing-in-katalon-studio

For setting up Object Repository in Katalon Studio:

https://docs.katalon.com/docs/create-tests/test-objects/web-test-objects/manage-web-test-objects-in-katalon-studio#create-web-objects-manually

For Scheduling Test Run in Katalon TestOps:

https://docs.katalon.com/docs/execute/schedule-test-execution/schedule-test-runs-in-testops

For Scheduling Test Suite Collection in Katalon Studio:

https://docs.katalon.com/docs/execute/schedule-test-execution/test-suite-collection-scheduler-for-katalon-studio

I hope this helps. Should you have further questions, don’t hesitate to tag one of our PMs above by simply adding a @ and then type in their usernames. Thanks!

2.5. Why has Katalon not address the issue with Drag and Drop in Build Web Service keywod yet?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #40 by Monty_Bagati

@Monty_Bagati

Hello @xuan.tran : Why it is taking too long to get the below issue resolved? The ageing of this issue is more than 180 days now.

When I tried to use Katalon in-build drag and drop Web Service Keyword, the drag and drop feature was not working for the web testing Even though that function did not work, the Katalon IDE caught up with that drag-and-drop feature included the test step test passed, and gave a successful result.

Answer

@xuan.tran

Hi @Monty_Bagati,
Sorry for the late reply. I’m a bit confused, related to Drag and Drop keywords, we have these webUI keywords:

• [WebUI] Drag And Drop By Offset
• [WebUI] Drag And Drop To Object

I’ve got our dev to check and these two are still working fine. Can you provide more details on the scenario you are testing? If possible, can you also share the web page that you’re testing?
I’m looking forward to hearing more to see if I can help.

2.6. I need a way to manually terminate the KatalonC.exe process in case I need to observe some Test Cases fails.

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #41 by digvijak

@digvijak

Hi @xuan.tran,
I am using KRE 7.2.1 for executing some test cases in one of my Test suites in Katalon studio project on the same machine. There since each of the Test case needs some data set up before execution we need to work on few mins on data set up for each of the TCs. And if for some reason any test case fails, the data set up has to be done again from scratch for it.
So in case of failure of a TC in a suite, all subsequent TCs should not be executed or skipped so as to investigate the failure and also to avoid corrupting the test data by NOT executing subsequent TCs unnecessarily. I tried a feature of MaxFailedTCs argument in KRE v8.5 as well, but does not help in my case.

In event of failure, i was trying to kill the Katalonc.exe, but even after doing it, it keeps launching subsequent TCs in suite as per the sequence. I want to manually terminate the KatalonC.exe in case i observe some TCs failed.

How can i do it or is there any way, please help me.

Answer

@xuan.tran

Hi @digvijak,

We don’t have any option to manually terminate KRE in the middle of the execution. I guess the most suitable way for your scenario is to use the command -maxFailedTests, which requires Studio version 8.1.0 onwards. For more details on how to use this command, you can see this doc: Terminate execution conditionally | Katalon Docs.

For example, if you want to stop the test suite after 4 test failures → Set T = 4. The test suite is terminated once the number of failures becomes 4. → The execution ends, and the rest test cases do not run. So I was wondering why this command does not help in your case. Can you elaborate a bit more?

2.7. User Unexpected behavior noticed in Katalon Studio when using the method WebUI.verifyElementNotPresent or WebUI.verifyElementPresent.

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #42 by mike.wardrop

@mike.wardrop

Question: User Unexpected behavior I noticed in Katalon Studio. When using the methods WebUI.verifyElementNotPresent or WebUI.verifyElementPresent with the following syntax’
TestObject export_button = findTestObject(‘imging/action_export_button’)
button_state = WebUI.verifyElementVisible(export_button) and expect a returned boolean.
If the element is not present, it fails the test at this step. Even though Katalon indicates that it will return a boolean value based on element’s presents or not.

The only way I was able to get this to work is by using the Failure Handling and it has to use the OPTIONAL option not CONTINUE. Then even when using OPTIONAL I still see a warning level event when the element is not visible. WebUI.verifyElementVisible(export_button, FailureHandling.OPTIONAL)

What I expected with these methods is a cleaner exit of returning a boolean if the element is present or not and the step to continue. I didn’t expect the verifyElementPresents to be a hard assertion failure and stop the test. Is this the correct usage of these methods? I’ve seen other Selenium implementations where there were methods that can check the element presents and return a boolean and allow the test steps to continue -versus- an assertion method that will hard fail the test step.
Best,
Mike W.

Answer

Hi @mike.wardrop,

The reason here is that when you use the WebUI.verifyElementVisible keyword, Studio needs to make sure that the element is present on the DOM before checking if it’s visible in the UI or not. If the element is not present in the DOM, then this step status will be marked as false. The keyword returns false if the element is present and NOT visible. I guess we have to update our docs

Hope this is clear to you

---

3. Katalon TestCloud

3.1. Which environment does TestCloud support at the moment, and how do I run tests from TestCloud from my CI/CD pipeline?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #20 by Surendrakumar

@Surendrakumar

hi everyone, I’ve just started using Katalon, and I have some questions about Katalon Testcloud for @ha.tpham:

• Which environments does Testcloud support at the moment?
• How can I run test from Testcloud from my ci/cd pipeline?

Thanks

Answer

@ha.tpham

Hi @Surendrakumar, thank you very much for your questions. Please find my answers below.

@Surendrakumar

Which environments does Testcloud support at the moment?

TestCloud allows you to run tests of a Katalon project on a wide range of browsers, operating systems and mobile devices. You can learn more by going to our documentation below:

Supported environments for Katalon TestCloud | Katalon Docs

@Surendrakumar

How can I run test from Testcloud from my ci/cd pipeline?

TestCloud integrates seamlessly with your CI/CD pipeline to enable smooth collaboration. Explore TestCloud’s Jenkins integration - one of the most popular CI/CD tools.

---

Don’t hesitate to ask me any other questions about TestCloud

---

4. Cloud Studio

4.1. What is Cloud Studio, and what are its advantages compare to Katalon Studio?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #23 by imeelmax

@imeelmax

Hi all, I have a question for @vy.huynh about Cloud Studio as I have been seeing a few topics here and there about it popping up on the forum: What is it, and what are Cloud Studio’s advantage compare to Katalon Studio?

Thanks!

Answer

@vy.huynh

Hi @imeelmax
Katalon Cloud Studio is our new web-based test creation tool that offers a more intuitive UI experience. It is specifically designed for those new to automation testing or anyone looking for a quick and hassle-free test case creation process without complex installations.

Here’s what sets it apart:

• Intuitive interface: With its user-friendly interface directly on TestOps, the product enables easy creation of test automation, providing a lightweight web experience.
• Enhanced readability: You can translate your recorded test steps into plain English, improving the readability and understanding of your tests.
• Cloud-based test execution: You can effortlessly execute your automated tests in a cloud-based environment named TestCloud with up to 4 parallel sessions

Katalon Cloud Studio is conveniently located on Katalon TestOps, offering a unified platform for managing and scheduling test cases, as well as accessing comprehensive reports.

I hope this provides a clear understanding of the advantages of Katalon Cloud Studio compared to Katalon Studio.

The product is in closed beta phrase, so to register your interest, simply fill out this form .

How will Katalon Recorder within Cloud Studio (KR-CS) compare to the existing Katalon Recorder within Katalon Studio (KR-KS4)?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #33 by guy.mason

@guy.mason

I am a long term Katalon Recorder user (having used it since 2016), and have no interest in using applications such as Katalon Studio, or other similar products.

I exclusively want a browser-based / browser-integrated solution, and a standalone IDE, even if it offers its own browser extensions, is still far more bloated a solution than what I would wish to use, when the greater majority of my work is performed and kept solely within the browser (with the exception being Slack, for comms). I have not developed a knowledge of Java, Playwright, C++, Perl, etc either to use something like Katalon Studio in the same way that I could use the scripting language present within Katalon Recorder.

You have recently made the decision to EoL Katalon Recorder development (and only offering to do one more maintenance release), which feels to me like a massive misstep. What you have already developed there over the years is great, and has the potential to really transform scripting based tools like these to another level if some of the suggestions already present in your forum were adopted, so it is deeply disappointing that you have chosen not to go down this route.

Whilst there are those who mistakenly view Katalon Recorder as a toy, those views come from a shallow understanding of what is possible with this tool. Yes, it is built upon a JS framework, and this has limitations, and at the end of the day being an open source application, you have no obligation as a company to develop it further, but you have built up a community around it over the years, and it is abandoning that community by moving away from it.

I personally have really valued the work Katalon has put into this over the years, too. Due to this, I have over the years attempted to show my appreciation for your efforts, by helping others in this community, via your forums.

I am aware that you are now developing (and looking to replace Katalon Recorder with) KR-CS from KR-KS4, but from what I can tell this is a very different application, even if it is aimed at a similar target market. It also appears to be vastly more limited in its capability and flexibility to what KR-KS4 offers, too (…and I do use the majority of what KR-KS4 currently offers, as well).

So, speaking from the PoV of a (long term and dedicated) KR-KS4 user, what I am hoping for clarity on is:

• Understanding whether KR-CS will offer feature parity with everything KR-KS4 already offers (and if so, within what timelines)
• Understanding whether existing test scripts / suites, etc will work OOTB with CS
• …and subsequently whether existing knowledge around Selenese scripting will continue to be useful there (otherwise this means that all KR-KS4, Selenium IDE, etc users knowledge will become invalidated)
• Whether there will now be a price barrier (or not) that will lock out a segment of your existing market

I have spent well over a decade developing my knowledge and understanding of Selenese, and used Selenium IDE (and a rival browser-based extension) for many years prior to my discovery of Katalon Recorder back in 2016.

Because of this, I have real concerns that all of this expertise will also no longer be usable by KR-CS. Likewise, all the years i’ve put into developing scripts that I use in KR-KS4 may be trashed, if I am unable to directly re-use those via KR-CS.

So, if you could please come back to me on the points and questions above, this would be appreciated! Thanks!

Answer

@vy.huynh

Hi @guy.mason

Thank you for reaching out. We appreciate your inquiry and value your interest in Katalon Cloud Studio. Please find below the information you requested:

1. Feature parity between Katalon Cloud Studio and Katalon Recorder:

• Katalon Cloud Studio offers a user-friendly test authoring capability with test steps presented in plain English, comprehensive keyword suggestions, and descriptions. This makes test creation and review easier for users.
• Additionally, as a web-based tool located on TestOps, Cloud Studio offers convenient access from any web browser, allowing you to work on your tests anytime and anywhere.
• You can freely execute Cloud Studio test cases with Katalon TestCloud and conveniently view the test reports on TestOps, providing an all-in-one platform experience.
• The product is still in the development stage and requires further improvements to match the functionality of KR-KS4. We highly value the sincere feedback from our valuable users, as it helps us make Cloud Studio significantly more useful and efficient for you.
• While we have planned release timelines, they are subject to changes based on new findings and user feedback. We assure you that we will keep our users informed of every new update.

2. Compatibility of existing KR test scripts/suites with Cloud Studio: This is not available for now, we have it in our backlog to consider. We will inform KR users when we have any specific plan for it.
3. Scripting support:
   At the moment, our Cloud Studio doesn’t support scripting, which means you won’t be able to apply your scripting knowledge.
   While we understand your interest in scripting, we cannot provide a definite confirmation regarding its introduction in the future. If scripting is considered at a later stage, it is important to be aware that there may be slight differences in the way you write tests due to the use of different frameworks.
   We appreciate your understanding in this matter, and we will make sure to keep you updated on any further developments regarding scripting capabilities in Cloud Studio.
4. Pricing: The product is currently in the Beta phase, and during this period, it is completely free to use. We will provide further information about pricing as we near the general availability launch.

We appreciate your loyalty as a KR-KS4 user. Your feedback is important to us as we strive to make Cloud Studio even better. If you have any further questions, please feel free to ask. Thank you for choosing Katalon.

---

5. Miscellaneous questions

5.1. What are the GPT-powered functions that we can expect in the upcoming release of the Katalon Platform?

[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬 - #32 by Shakkir

@Shakkir

what are the GPT powered functions we can expect in the upcoming releases/

Answer

@xuan.tran

Hi @Shakkir,

We had StudioAssist out for beta with 2 AI features, which are generating code and explaining code in Katalon Studio. Here’s the link for you to try out: 8.6.7.beta.

These features will be in an official release soon. And after that, we will also have some enhancements on mobile testing, so stay tuned for it. Are there any features in mind that you want us to support? Let us know by commenting below. Thank you so much.

---

Feel free to let us know below what you would like to see in our next AKA session, of if there is any suggestions for us to improve how AKA is organized and carried out in the future

Have a better one,
Albert from Katalon Community team