[Extended] Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬

[vy.huynh]

Hi @imeelmax
Katalon Cloud Studio is our new web-based test creation tool that offers a more intuitive UI experience. It is specifically designed for those new to automation testing or anyone looking for a quick and hassle-free test case creation process without complex installations.

Here’s what sets it apart:

• Intuitive interface: With its user-friendly interface directly on TestOps, the product enables easy creation of test automation, providing a lightweight web experience.
• Enhanced readability: You can translate your recorded test steps into plain English, improving the readability and understanding of your tests.
• Cloud-based test execution: You can effortlessly execute your automated tests in a cloud-based environment named TestCloud with up to 4 parallel sessions

Katalon Cloud Studio is conveniently located on Katalon TestOps, offering a unified platform for managing and scheduling test cases, as well as accessing comprehensive reports.

I hope this provides a clear understanding of the advantages of Katalon Cloud Studio compared to Katalon Studio.

The product is in closed beta phrase, so to register your interest, simply fill out this form.

[tri.tle]

Hi @Testinator-X, apologies for the delayed response. Your question required some contemplation. Regrettably, we currently do not have the capability to generate properties as you described in Katalon reports. However, your suggestion is valuable, my team will explore this further and consider incorporating it into our future roadmap.

@Testinator-X

We have integrated our Katalon tests with TestRail. For this we use the TestRail CLI tool instead of the Katalon TestRail Integration plugin, as the plugin does not meet our requirements.

In regards to your requirements for the plugin, could you please provide us with more details? This would greatly assist us in developing a plugin that is tailored to your needs.

[tri.tle]

Hi @karthik.srivatsan, indeed, Katalon Platform provides the capability to manage your test cases along with requirements. If you are utilizing Jira to manage your requirements, you can seamlessly integrate your Jira projects with Katalon projects. Subsequently, you can link your Jira requirements with the corresponding test cases. For more information about Jira integration, you can follow it here.

[Testinator-X]

Hi @tri.tle thanks for your feedback, here are a few examples why we don’t use the Katalon TestRail integration.

1. We are TestRail Server Customers and according to your TestRail integration documentation

The TestRail Integration plugin only supports integrating Katalon Studio with TestRail Cloud

2. It is only possible to link Katalon TestCases with TestRail TestCases and Katalon TestSuites with TestRail Runs at a high level. However, unfortunately not all parameters of the TestRail API (https://www.gurock.com/testrail/docs/api/reference/results/) can be passed via the Katalon Integration to TestRail, which is why the result, if it can be pushed at all, is only very superficial in terms of the level of detail. e.g.:

a) In our TestRail configuration the Result field “Version” is a mandatory field. Katalon can run the test successfully, but cannot push the result because you cannot submit the version. This should be possible by, for example, reading the currentVersion at runtime from the frontend of the application under test and passing it to the plugin for push.

b) Currently only the overall status of the test result can be passed. This is determined automatically. If a Katalon test is successful, the TestRail test is also successful or if a TestStep in Katalon fails, the overall status in TestRail is also set to “Failed”. You can’t see in TestRail which Step failed.

c) It should be possible to define in Katalon which Katalon test step/test step is to be assigned to which TestRail test step. On the one hand it should be possible to set the test result automatically, on the other hand it should be possible to overwrite it with an own status if desired. It should be possible to set the default states (Passed/Blocked/Failed/Retest) as well as the custom states (e.g. “Passed with Errors”) both on step level and on overall level. As a workaround, it would help if you could at least push a comment or screenshots on the overall level.

d) The highlight would be if you could also interact with a Defect plugin. That means that e.g. in case of a “Failed” status a Jira-Issue is created automatically and the ID of the Jira-Issue is pushed to TestRail together with the TestRail-Result. However, this would only be “nice to have”.

3. Even though the CLI tool is far from perfect, its feature set could be a good starting point for what the Katalon TestRail plugin should support out of the box. For example, a code first approach, with which TestRail cases could be automatically created and linked, would also be helpful.

4. It should be possible to jump from Katalon to a TestRail Case or vice versa.

5. If there are several Katalon tests for a TestCase, Results are alienated. If the first katalon test fails but a subsequent casekatalon test for the same test is passed, the overall result is “passed”. For the overall result it should be possible to consider the results of all Katalon tests.

[albert.vu]

Hi folks,

We have decided to extend Ask Katalon Anything (AKA) till next Friday 2023-09-22T16:59:00Z so that both you and our PMs can come up with more questions and answers.

@trust_level_0 Thank you for all of your questions thus far, and please keep 'em coming!

And don’t forget, we are also giving out some small gifts when you submit your questions in this thread → Check out the first post for more details!

[f_poccard]

Hi @xuan.tran, I just want to know what is Katalon’s plans are regarding supporting Selenium 4 in Katalon Studio as this question has been asked before on the forum…

Is it possible to upgrade Selenium to Selenium 4?

Hi guys, I found, that Katalon Studio uses selenium version 3.141.59, which is actually latest stable version, but there is already Selenium 4 release candidate, which has some new features I would like to use. So my question is, will you upgrade Selenium to version 4 as soon as it’s released? If I wanted to use it right now, is it possible to upgrade Selenium manually?

Thanks!

[chihung.le]

Hi Albert

We are new to Katalon.

We like to setup the sharing Test sets and Object Repo b/w the development. How to set it up?

Also, like to scheduling the automation testing.

It would be great if you can send us some info.

Best regards

Chi Hung Le

[mike.wardrop]

Coming from Ruby, I’ve been liking the stability and functionality of Groovy. If there were other language flavors for consideration I’d love to see Ruby as one of them. @xuan.tran!! Thank you for all you help and support!!

[Shakkir]

what are the GPT powered functions we can expect in the upcoming releases/

[guy.mason]

I am a long term Katalon Recorder user (having used it since 2016), and have no interest in using applications such as Katalon Studio, or other similar products.

I exclusively want a browser-based / browser-integrated solution, and a standalone IDE, even if it offers its own browser extensions, is still far more bloated a solution than what I would wish to use, when the greater majority of my work is performed and kept solely within the browser (with the exception being Slack, for comms). I have not developed a knowledge of Java, Playwright, C++, Perl, etc either to use something like Katalon Studio in the same way that I could use the scripting language present within Katalon Recorder.

You have recently made the decision to EoL Katalon Recorder development (and only offering to do one more maintenance release), which feels to me like a massive misstep. What you have already developed there over the years is great, and has the potential to really transform scripting based tools like these to another level if some of the suggestions already present in your forum were adopted, so it is deeply disappointing that you have chosen not to go down this route.

Whilst there are those who mistakenly view Katalon Recorder as a toy, those views come from a shallow understanding of what is possible with this tool. Yes, it is built upon a JS framework, and this has limitations, and at the end of the day being an open source application, you have no obligation as a company to develop it further, but you have built up a community around it over the years, and it is abandoning that community by moving away from it.

I personally have really valued the work Katalon has put into this over the years, too. Due to this, I have over the years attempted to show my appreciation for your efforts, by helping others in this community, via your forums.

I am aware that you are now developing (and looking to replace Katalon Recorder with) KR-CS from KR-KS4, but from what I can tell this is a very different application, even if it is aimed at a similar target market. It also appears to be vastly more limited in its capability and flexibility to what KR-KS4 offers, too (…and I do use the majority of what KR-KS4 currently offers, as well).

So, speaking from the PoV of a (long term and dedicated) KR-KS4 user, what I am hoping for clarity on is:

• Understanding whether KR-CS will offer feature parity with everything KR-KS4 already offers (and if so, within what timelines)
• Understanding whether existing test scripts / suites, etc will work OOTB with CS
• …and subsequently whether existing knowledge around Selenese scripting will continue to be useful there (otherwise this means that all KR-KS4, Selenium IDE, etc users knowledge will become invalidated)
• Whether there will now be a price barrier (or not) that will lock out a segment of your existing market

I have spent well over a decade developing my knowledge and understanding of Selenese, and used Selenium IDE (and a rival browser-based extension) for many years prior to my discovery of Katalon Recorder back in 2016.

Because of this, I have real concerns that all of this expertise will also no longer be usable by KR-CS. Likewise, all the years i’ve put into developing scripts that I use in KR-KS4 may be trashed, if I am unable to directly re-use those via KR-CS.

So, if you could please come back to me on the points and questions above, this would be appreciated! Thanks!

[dineshh]

Any plans to integrate with copilot or code whisperers?

[albert.vu]

Hi Dinesh,

Thank you for your question. By Copilot, did you mean Microsoft 365 Copilot, or GitHub Copilot, or both? Please be more specific in your questions so that our PMs can provide you with the answers you need. Thanks!

[albert.vu]

Hi @chihung.le,

I am sending you a few articles from our documentation base which I believe can help you as below:

For sharing Test sets/test artifacts:

Test Artifacts Sharing in Katalon Studio | Katalon Docs

For setting up Object Repository in Katalon Studio:

Manage Web test objects in Katalon Studio | Katalon Docs

For Scheduling Test Run in Katalon TestOps:

Schedule test runs in TestOps | Katalon Docs

For Scheduling Test Suite Collection in Katalon Studio:

Test suite collection scheduler for Katalon Studio | Katalon Docs

I hope this helps. Should you have further questions, don’t hesitate to tag one of our PMs above by simply adding a @ and then type in their usernames. Thanks!

[atul.rai]

This is a great initiative, thanks Katalon Team! I have some question as below-
Test run on safari browser is not much compatible compare to other browser like chrome, Firefox, Edge chromium etc. So we expect improvement for this.

@tri.tle - when we run our test suite/testsuiteCollection on Windows 11 OS and view report on TestOps,then always showing Windows 10.
Sometimes are showing N/A as attacheched in the screenshot.

[xuan.tran]

Hi @f_poccard,
Thank you for sharing this question!

As you might know, we had a PoC for Selenium 4 a while ago, and we’re keeping this item as a candidate for some upcoming releases.

We would love to hear your feedback on the PoC, so here’s the link for you to download and try it out: 8.2.1.alpha. Let us know your thoughts on this PoC , and happy testing!

[xuan.tran]

Hi @Shakkir,

We had StudioAssist out for beta with 2 AI features, which are generating code and explaining code in Katalon Studio. Here’s the link for you to try out: 8.6.7.beta.

These features will be in an official release soon. And after that, we will also have some enhancements on mobile testing, so stay tuned for it. Are there any features in mind that you want us to support? Let us know by commenting below. Thank you so much.

[Monty_Bagati]

Hello @xuan.tran : Why it is taking too long to get the below issue resolved? The ageing of this issue is more than 180 days now.

Drag and Drop In Build Web Service Keyword is Not Working

When I tried to use Katalon in-build drag and drop Web Service Keyword, the drag and drop feature was not working for the web testing Even though that function did not work, the Katalon IDE caught up with that drag-and-drop feature included the test step test passed, and gave a successful result.

[digvijak]

Hi @xuan.tran,
I am using KRE 7.2.1 for executing some test cases in one of my Test suites in Katalon studio project on the same machine. There since each of the Test case needs some data set up before execution we need to work on few mins on data set up for each of the TCs. And if for some reason any test case fails, the data set up has to be done again from scratch for it.
So in case of failure of a TC in a suite, all subsequent TCs should not be executed or skipped so as to investigate the failure and also to avoid corrupting the test data by NOT executing subsequent TCs unnecessarily. I tried a feature of MaxFailedTCs argument in KRE v8.5 as well, but does not help in my case.

In event of failure, i was trying to kill the Katalonc.exe, but even after doing it, it keeps launching subsequent TCs in suite as per the sequence. I want to manually terminate the KatalonC.exe in case i observe some TCs failed.

How can i do it or is there any way, please help me.

[mike.wardrop]

Question: User Unexpected behavior I noticed in Katalon Studio. When using the methods WebUI.verifyElementNotPresent or WebUI.verifyElementPresent with the following syntax’
TestObject export_button = findTestObject(‘imging/action_export_button’)
button_state = WebUI.verifyElementVisible(export_button) and expect a returned boolean.
If the element is not present, it fails the test at this step. Even though Katalon indicates that it will return a boolean value based on element’s presents or not.

The only way I was able to get this to work is by using the Failure Handling and it has to use the OPTIONAL option not CONTINUE. Then even when using OPTIONAL I still see a warning level event when the element is not visible. WebUI.verifyElementVisible(export_button, FailureHandling.OPTIONAL)

What I expected with these methods is a cleaner exit of returning a boolean if the element is present or not and the step to continue. I didn’t expect the verifyElementPresents to be a hard assertion failure and stop the test. Is this the correct usage of these methods? I’ve seen other Selenium implementations where there were methods that can check the element presents and return a boolean and allow the test steps to continue -versus- an assertion method that will hard fail the test step.
Best,
Mike W.

[Testinator-X]

Many TestRail customers are switching to alternative tools such as testmo or Qase, because the product quality there is apparently much better and the tools are also much more modern. In particular, the migration from TestRail to Testmo is said to be relatively easy, because Testmo’s roots are in TestRail. Are further plugins planned to support integrating Katalon to these or other test management tools?