Hi Community members,
In this thread, we are going to summarize all of the questions & answer that have been asked during Ask Katalon Anything (AKA) thus far. Each comment in this thread will correspond to a separate module within the Katalon Platform to easier navigation:
List of Katalon Studio questions & answers
List of Katalon TestOps questions & answers
Our team has also decided to extend AKA for another week. Hence, from now until 2024-03-12T16:59:00Z , feel free to share any questions that you may have about the Katalon Platform i.e. product roadmap, upcoming features, best practices, or troubleshooting.
Keep your questions coming!
1. Desktop application automation testing within Katalon Studio Enterprise (KSE) is still lacking. Does Katalon have plans to improve this area of KSE
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/4?u=albert.vu
Hello, we intensively use WebUI and WS features of KSE yet noticed that regarding desktop app automation KSE seems behind the competition: missing WebUI features (no video record, no screenshot, no “rerun from here”) and not handling all desktop app development languages meaning for some desktop app interactions is difficult and object recognition not working well.
Question: is it intended to remain like this, because KSE is actually mostly used for WebUI / WS test automation, or is it intended to catchup competitors in this domain?
Answer
Hi @yann.sautreuil ,
Thank you for trusting Studio for your day-to-day work, I really appreciate your feedback on WebUI and WS features. We’re still working on enhancements and developing new features every releases.
For the object recognition, we’ve just released 9.3.1 yesterday, introducing Smart Locator , where it can be used in advanced cases that previous locators might not work well. We would love to hear if this new feature can help your testing work easier.
Some other docs I believe could help you:
If you have other issues or feature suggestions for video record, screenshot or debugging, please let me know. We would love to hear more from you so that we can improve Studio better.
2. Would it be possible to make a time boundary on test execution (stopping Katalon Runtime Engine test suites run) and then generate the report as "finally" part of execution (by sending it to TestOps)?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/5?u=albert.vu
Answer
Thanks @wbalien for your suggestion.
Currently, reports are only generated when execution is completed, and there will be no reports if the execution is stopped in the middle. We value your suggestion, and we might consider it for future releases.
3. Are there plans for Katalon StudioAssist to use alternative Large Langugage Models (LLM) such as Google Gemini or other (free/no-cost) models instead of ChatGPT?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/6?u=albert.vu
With this question, however, I would also like to emphasize the matter of data protection. The issue is not only important for large companies such as Apple, which is said to have issued a general ban on ChatGPT as early as the beginning of 2023, but also for small companies such as ours, which want to ensure that no sensitive information leaves the company.
Here, fore example you can find some LLMs to host yourself: library
Answer
Hi @Testinator-X ,
A 30-day data retention on OpenAI
No data is used to train the OpenAIs model
For more information on OpenAI’s data policies, refer to: Enterprise privacy . On an account level, we can set the data retention to 0 days if desired.
We do have plans to support Azure OpenAI soon. Their policies on data are more stringent as the data will completely be controlled by Microsoft. Refer to Data, privacy, and security for Azure OpenAI Service - Azure AI services | Microsoft Learn for the data policy for Microsoft.
Other than the above-mentioned, our long-term plan is to support other LLMs (both propriety & open source models).
I hope that this addresses your concern.
4. Is there any plans to incorporate Performance/Load-testing capabilities into Katalon Platform such that someone does not need to use third-party tool to get performance insights about their application whose test cases are primarily housed in Katalon Studio?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/7?u=albert.vu
A question from @katamba :
Answer
5. Which security-related information is Katalon Studio getting from our usage of the product itself?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/9?u=albert.vu
I received an email from Katalon that they “knew that I used Keywords” in my test scripts and to talk to Katalon about the new KS product, version 9.0, in which, if you use Keywords, you need to pay for subscription. Note: This is on my home machine in which I don’t do much except respond to forum questions.
Answer
Hi @grylion54 ,
We do collect some information about your application use for the purpose of analyzing and improving the product better, eg what types of keywords you use, but not more than that, we cannot read your script. You can refer to this document: Privacy Policy for further details of our tracking.
In case you do not want to be tracked, you have the option to disable usage tracking, but pls note that it will require a KSE license. I hope this info can help you.
6. Is it possible to run Katalon Runtime Engine on Apple Silicon (the M1 chip specifically)?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/15?u=albert.vu
Answer
Hi @trungnghia.ctn94 ,
I want to use KRE in my M1 mac. Is there any way to do that?
Yes, you can use KRE on Apple Silicon, KRE for macOS can run well on both Intel and Apple Silicon chips. I hope that this answer could help, if you need further support, please let me know.
Happy testing,
7. I am trying to iterate through the list of web elements but kept getting error (expand to see more
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/21?u=albert.vu
Answer
The error MultipleCompilationErrorsException means you have defined the same variable more than once. Somewhere in your code, you likely copied and pasted the variables, Table , rows_table and Columns_row . What you need to do is remove the definition from in front of the variables. You can leave the definition in front of the very first one in your code, then remove all others. As an example, keep your definition in the first instance:
WebElement Table = driver.findElement(...)
but any other use of the same variable, you just use:
Table = driver.findElement(...)
Same goes for the others listed in your error message.
8. How do I customize the reports generated by Katalon?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/22
Answer
9. Does Katalon Studio support multiple languages for testing?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/23
Answer
10. I am facing issues when trying to set up a mobile automation project using Perfecto Cloud
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/25?u=albert.vu
Hi Team,
I’m trying to setup a mobile automation project using perfecto cloud.
I used the plugins provided in Katalon Store but unable to achieve the result.
Answer
Hi @syed.shoaib ,Perfecto Integration v2022 . If you have any specific error message or screenshot of where you’re stuck, please share it with us so that we can help you further.
11. Can you walk us through on how each module of the Katalon Platform can be integrated with JIRA (no Xray), and more specifically, the activities that can be synced between the two platforms?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/26?u=albert.vu
Can the team walk through how each piece of the Katalon suite of products can be integrated with JIRA(no Xray), specifically the activities that can be synchronized between the two platforms? A high-level outline/diagram or flowchart?
We have Studio and TestOps attempting to bring TrueTest online as well.
Answer
Hey Nathaniel,
Here’s a video I recorded against Jira in the pas it also has GitLab integration but you can skip that part.
Note, some of the screens have changed but I believe it’s still applicable.
[Tutorial] Integrating Katalon TestOps With Jira and Gitlab
Let me know if you have any questions.
Philip B
12. Can we directly upload test cases/suit/folder from Katalon Studio to TestOps?
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874/27?u=albert.vu
Answer
Let us know if we missed anything.
If you have a question about Katalon Studio, then don’t hesitate to let our Product team know in the thread below
http://forum.katalon.com/t/aka-katalon-studio-feb-26-to-mar-04-2024/120874?u=albert.vu
1. Katalon TestOps has received a new feature recently where you can record and playback test cases in TestOps via the Katalon Recorder plugin. However, whilst recording some test cases, I kept getting a crash (expand to see more)
http://forum.katalon.com/t/aka-katalon-testops-feb-26-to-mar-04-2024/120864/5?u=albert.vu
Katalon test ops has recently launched a new feature where you can record and playback test cases in the test ops using the Katalon recorder plugin.
However, while recording some test cases, I keep on getting a crash browser when I reach a particular step that I am recording, the step is expected to open a modal and continue my next step there, the page freezes and eventually crashes after a few seconds.
**Answer **
Hi @rlinatoc ,
We took a closely look at the inspect that you shared in the video. So far, there is no error nor issue related to the Recorder engine spotted in the inspection. We need to collect more information from the Dev Tools of Katalon Platform Recorder Extension to better troubleshoot the issue. If you don’t mind, please help to reproduce it, and at the frozen step, record what you can find in the Dev Tools for us. It would help a lot in figuring out what is going on there.
To access the Dev Tools, please open the “Manage Extensions” dialog on your browser. Then, under “Katalon Platform Recorder” extension, click on “service worker”, and capture all you see under the Console log.
Looking forward to your reply. Thank you very much for your time.
[details="2. A question regarding the pricing model of Katalon TestOps (expand to see more)]http://forum.katalon.com/t/aka-katalon-testops-feb-26-to-mar-04-2024/120864/7?u=albert.vu
Hi.
A question with regard to the pricing on TestOps. Not so long ago, the pricing changed from “number of tests executed through testOps” to “number of results within testOps”. As you can see from the screenshot on your pricing page, the Monthly test results limit for e.g. 10.000 test results is $1.300/year. These test results include Katalon tests but also imported tests.
A couple of points I’d like to make on this:The Evolution of the Testing Model and Where It Will Go Next
2° To me, the practice of making the customer pay for the number of tests they execute is a bad practice. The goal of test automation is just to have as much automated as possible. If customers need to take into account the number of test results they are allowed to have, Quality Assurance will be compromised. Then things will be pushed which should be avoided, like
Let’s disable those tests,
Let’s not automate this part,
Let’s only run on day X
Let’s just do manual testing…
I do understand the hosting and server costs for Katalon increase the more data is stored on testOps. I also understand that Katalon wants to pass on these costs to the customer…
But isn’t there a better way to do this?
For instance:
Allow customers to, on a project basis determine which and how long test results are stored. There are always projects which may be less “active”. Perhaps once delivered there might be only sporadic new releases needed. For those projects, we want to keep on running the tests (as external changes could always cause regression), but perhaps we don’t need to have those test results stored for longer than a week. Of those, perhaps we only want to keep on storing the tests runs from the highest (most production-like) environment (see my other question I’ll raise
Make not the number of test results be the cost-driver, but the actual size of storage space. There will be much less storage use for jUnit imported tests (without a screenshot). The Katalon tests that fail do have screenshots and those might then also be a motivator to get those fixed (either the application or the test case itself).
As I posted here more than a year ago, I’d like to now how Katalon TestOps sees the concept of “Test Environments”.
Use case: I have several projects where we run the Katalon tests on multiple environments, namely TST and ACC. Since we do agile iterative development, the tests on TST could break and often need rework. On ACC we have a production-like application that normally should always be green. It might be, due to integration with external parties, that on ACC a test suddenly starts failing (in other words: the external parties did some changes and these might impact our application in a negative way!).
We have the Katalon “profile” concept, where we do make a distinction on the environment (among other things) via the profile name. So we can filter out via ExecutionTestResult.profile:“~acc” all the test results from ACC, but that’s not ideal. Furthermore, we also upload jUnit test reports from ReadyAPI for both environments onto the Katalon TestOps via the Katalon Report Uploader (see Upload JUnit and report files to using command line | Katalon Docs ). There we do not have the concept of “profile”, so we cannot filter those out.
I tried to use the labels feature as described in my post above:
Obviously, goal should be to have this label automatically set to my test runs. Perfect place for me would be as a parameter in both the Katalon Runtime Engine command line (see Command-Line Syntax in Katalon Runtime Engine | Katalon Docs ) AND in the Katalon Report Uploader command line.
There is no url adaptation (see screenshot) for this filter, so I cannot store it as a favorite “application X - ACC results” in my browser. Each time, I will have to go for the custom filter and filter out the “TST” or “ACC”. Developers might for instance be only interested in the view for TST, while management wants to quickly see the state of ACC before an approval to go to PRODUCTION. I cannot go and ask them to play with the filter themselves…
Answer
A question with regard to the pricing on TestOps. Not so long ago, the pricing changed from “number of tests executed through testOps” to “number of results within testOps”. As you can see from the screenshot on your pricing page, the Monthly test results limit for e.g. 10.000 test results is $1.300/year. These test results include Katalon tests but also imported tests.
Hi @joost.degeyndt , thanks for your dedicated feedback regarding the pricing model. I would like to apologize for any inconvenience and fully understand your concerns here. To be honest, the business is crucial to Katalon’s existence, and the changes in the pricing model have aided us in scaling and developing more capabilities that benefit Katalon users.
Although business is important, the Katalon team always aims for Product-Model Fit, which reflects the balance between pricing models and product growth. Your feedback is very valuable at this point. It’s my pleasure to incorporate your feedback into the further discussion of the Katalon team regarding the pricing model.
[/details]
If you have a question about Katalon TestOps, then don’t hesitate to share with our Product team at:
http://forum.katalon.com/t/aka-katalon-testops-feb-26-to-mar-04-2024/120864?u=albert.vu
