Ask Katalon Anything - Sep 6 to Sep 22, 2023 💬

My question:

Is anyone brave enough to set a date for the elevation of JavaScript to become a first class development language for Test Cases in Katalon Studio?

@duyluong and I had several discussions about the possibility. In late 2021 he told me it is being worked on.

My suspicion is, in the rush to move into the cloud and latterly the heavy focus on “AI”, JS promotion and adoption has fallen off the development radar.

Anyone crazy enough to poke their head above of the trench?

1 Like

See this link:

I think this link isn’t interesting for @coen.schalkwijk

2 Likes

No responses in over 24hrs? Hmm.

:thinking:

Hi all,

Thank you for your questions so far. Our APMs will be responding to your questions as soon as possible. Keep them coming :+1:

1 Like

Hi @coen.schalkwijk,
As Russ Thomas and Kazurayam already replied, you can see a few threads on the forum on how to write Katalon tests with other IDEs, and I think it’s really worth trying. However, we do not officially support this approach yet, so currently there’s no official document on docs.katalon.com for this tutorial. We’re still developing Katalon Studio so that we can bring a better experience to users, so we acknowledge your need and will consider it in the future.
Thank you so much and happy testing :two_hearts:
Xuan Tran.

2 Likes

Hi @Russ_Thomas,
Thank you so much for bringing this up. JavaScript sounds interesting, let’s see what we could do about it. :sparkles:

In the meantime, Groovy is still the main focus for us, but we would love to hear your thoughts on this topic. Thumb-up :+1:t2: if you want JavaScript in some next couple of releases, and don’t hesitate to comment if you want Studio to support other languages as well :two_hearts:

Hello, every one

Hello @Monty_Bagati, it’s a pleasure to reconnect with you.

The Katalon team is currently engaged in enhancing the interface to provide a more appealing and user-friendly experience. Naturally, such transformations demand a significant amount of time and it’s quite challenging to implement them throughout the entire application.

As we progress, we would greatly value insights from our users. We are considering conducting usability testing and may extend an invitation soon. It would be greatly appreciated if you could participate.

2 Likes

Hello Win @tri.tle : thanks for your reply. I completely understand your point.

Looking forward to help @Katalon_team in my best capacity

2 Likes

Hi Noor,

Good to have you here. Is there anything about the Katalon Platform that you would like to ask our APMs? Maybe a new GPT-powered feature? :eyes:

We have integrated our Katalon tests with TestRail. For this we use the TestRail CLI tool instead of the Katalon TestRail Integration plugin, as the plugin does not meet our requirements. Now we would like to use the new feature of the TestRail CLI tool to automatically add step information in the results. For this purpose the JUnit XML report generated by Katalon should be enriched with additional properties as described here or here.

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="0.05" name="tests.LoginTests">
    <properties><property name="setting1" value="True"/></properties>
    <testcase classname="tests.LoginTests" name="test_case_2" time="650">
     <properties>
         <property name="testrail_attachment" value="path_to/logs.log"/>
         <property name="testrail_attachment" value="path_to/screenshot.jpg"/>
         <property name="testrail_result_field" value="version:1.2"/>
         <property name="testrail_result_field" value="custom_environment:qa02"/>
         <property name="testrail_result_comment" value="Finding 1"/>
         <property name="testrail_result_comment" value="Finding 2"/>
         <property name="testrail_result_step" value="passed: Insert login credentials"/>
         <property name="testrail_result_step" value="failed: Click submit"/>
         <property name="testrail_result_step" value="untested: User should be logged in"/>
     </properties>
    </testcase>
  </testsuite>
</testsuites>

How to get Katalon to generate the required properties for each validation step in the JUnit .XML Report?

hi everyone, I’ve just started using Katalon, and I have some questions about Katalon Testcloud for @ha.tpham:

  • Which environments does Testcloud support at the moment?
  • How can I run test from Testcloud from my ci/cd pipeline?

Thanks

Hi @Surendrakumar, thank you very much for your questions. Please find my answers below.

TestCloud allows you to run tests of a Katalon project on a wide range of browsers, operating systems and mobile devices. You can learn more by going to our documentation below:

TestCloud integrates seamlessly with your CI/CD pipeline to enable smooth collaboration. Explore TestCloud’s Jenkins integration - one of the most popular CI/CD tools.


Don’t hesitate to ask me any other questions about TestCloud :smile:

2 Likes

I have a TestOps related question for @tri.tle as follow: Is TestOps equipped to allow me to seamlessly manage both my test cases and requirements?

Looking forward to your answer. Thanks!

Hi all, I have a question for @vy.huynh about Cloud Studio as I have been seeing a few topics here and there about it popping up on the forum: What is it, and what are Cloud Studio’s advantage compare to Katalon Studio?

Thanks!

Hi @imeelmax
Katalon Cloud Studio is our new web-based test creation tool that offers a more intuitive UI experience. It is specifically designed for those new to automation testing or anyone looking for a quick and hassle-free test case creation process without complex installations.

Here’s what sets it apart:

  • Intuitive interface: With its user-friendly interface directly on TestOps, the product enables easy creation of test automation, providing a lightweight web experience.
  • Enhanced readability: You can translate your recorded test steps into plain English, improving the readability and understanding of your tests.
  • Cloud-based test execution: You can effortlessly execute your automated tests in a cloud-based environment named TestCloud with up to 4 parallel sessions

Katalon Cloud Studio is conveniently located on Katalon TestOps, offering a unified platform for managing and scheduling test cases, as well as accessing comprehensive reports.

I hope this provides a clear understanding of the advantages of Katalon Cloud Studio compared to Katalon Studio.

The product is in closed beta phrase, so to register your interest, simply fill out this form.

2 Likes

Hi @Testinator-X, apologies for the delayed response. Your question required some contemplation. Regrettably, we currently do not have the capability to generate properties as you described in Katalon reports. However, your suggestion is valuable, my team will explore this further and consider incorporating it into our future roadmap.

In regards to your requirements for the plugin, could you please provide us with more details? This would greatly assist us in developing a plugin that is tailored to your needs.

Hi @karthik.srivatsan, indeed, Katalon Platform provides the capability to manage your test cases along with requirements. If you are utilizing Jira to manage your requirements, you can seamlessly integrate your Jira projects with Katalon projects. Subsequently, you can link your Jira requirements with the corresponding test cases. For more information about Jira integration, you can follow it here.

Hi @tri.tle thanks for your feedback, here are a few examples why we don’t use the Katalon TestRail integration.

  1. We are TestRail Server Customers and according to your TestRail integration documentation

The TestRail Integration plugin only supports integrating Katalon Studio with TestRail Cloud

  1. It is only possible to link Katalon TestCases with TestRail TestCases and Katalon TestSuites with TestRail Runs at a high level. However, unfortunately not all parameters of the TestRail API (https://www.gurock.com/testrail/docs/api/reference/results/) can be passed via the Katalon Integration to TestRail, which is why the result, if it can be pushed at all, is only very superficial in terms of the level of detail. e.g.:

a) In our TestRail configuration the Result field “Version” is a mandatory field. Katalon can run the test successfully, but cannot push the result because you cannot submit the version. This should be possible by, for example, reading the currentVersion at runtime from the frontend of the application under test and passing it to the plugin for push.

b) Currently only the overall status of the test result can be passed. This is determined automatically. If a Katalon test is successful, the TestRail test is also successful or if a TestStep in Katalon fails, the overall status in TestRail is also set to “Failed”. You can’t see in TestRail which Step failed.

c) It should be possible to define in Katalon which Katalon test step/test step is to be assigned to which TestRail test step. On the one hand it should be possible to set the test result automatically, on the other hand it should be possible to overwrite it with an own status if desired. It should be possible to set the default states (Passed/Blocked/Failed/Retest) as well as the custom states (e.g. “Passed with Errors”) both on step level and on overall level. As a workaround, it would help if you could at least push a comment or screenshots on the overall level.

d) The highlight would be if you could also interact with a Defect plugin. That means that e.g. in case of a “Failed” status a Jira-Issue is created automatically and the ID of the Jira-Issue is pushed to TestRail together with the TestRail-Result. However, this would only be “nice to have”.

  1. Even though the CLI tool is far from perfect, its feature set could be a good starting point for what the Katalon TestRail plugin should support out of the box. For example, a code first approach, with which TestRail cases could be automatically created and linked, would also be helpful.

  2. It should be possible to jump from Katalon to a TestRail Case or vice versa.

  3. If there are several Katalon tests for a TestCase, Results are alienated. If the first katalon test fails but a subsequent casekatalon test for the same test is passed, the overall result is “passed”. For the overall result it should be possible to consider the results of all Katalon tests.

1 Like

Hi folks, :wave:

We have decided to extend Ask Katalon Anything (AKA) till next Friday 2023-09-22T16:59:00Z so that both you and our PMs can come up with more questions and answers. :sunglasses:

@trust_level_0 Thank you for all of your questions thus far, and please keep 'em coming! :+1:

And don’t forget, we are also giving out some small gifts when you submit your questions in this thread → Check out the first post for more details!

7 Likes