1. Katalon TestOps has received a new feature recently where you can record and playback test cases in TestOps via the Katalon Recorder plugin. However, whilst recording some test cases, I kept getting a crash (expand to see more)
[AKA] Katalon TestOps - Feb 26 to Mar 04, 2024 🪄 - #5 by rlinatoc
Katalon test ops has recently launched a new feature where you can record and playback test cases in the test ops using the Katalon recorder plugin.
However, while recording some test cases, I keep on getting a crash browser when I reach a particular step that I am recording, the step is expected to open a modal and continue my next step there, the page freezes and eventually crashes after a few seconds.
**Answer **
Hi @rlinatoc ,
We took a closely look at the inspect that you shared in the video. So far, there is no error nor issue related to the Recorder engine spotted in the inspection. We need to collect more information from the Dev Tools of Katalon Platform Recorder Extension to better troubleshoot the issue. If you don’t mind, please help to reproduce it, and at the frozen step, record what you can find in the Dev Tools for us. It would help a lot in figuring out what is going on there.
To access the Dev Tools, please open the “Manage Extensions” dialog on your browser. Then, under “Katalon Platform Recorder” extension, click on “service worker”, and capture all you see under the Console log.
Looking forward to your reply. Thank you very much for your time.
"2. A question regarding the pricing model of Katalon TestOps (expand to see more)
[AKA] Katalon TestOps - Feb 26 to Mar 04, 2024 🪄 - #7 by joost.degeyndt
Hi.
A question with regard to the pricing on TestOps. Not so long ago, the pricing changed from “number of tests executed through testOps” to “number of results within testOps”. As you can see from the screenshot on your pricing page, the Monthly test results limit for e.g. 10.000 test results is $1.300/year. These test results include Katalon tests but also imported tests.
A couple of points I’d like to make on this:The Evolution of the Testing Model and Where It Will Go Next
2° To me, the practice of making the customer pay for the number of tests they execute is a bad practice. The goal of test automation is just to have as much automated as possible. If customers need to take into account the number of test results they are allowed to have, Quality Assurance will be compromised. Then things will be pushed which should be avoided, like
Let’s disable those tests,
Let’s not automate this part,
Let’s only run on day X
Let’s just do manual testing…
I do understand the hosting and server costs for Katalon increase the more data is stored on testOps. I also understand that Katalon wants to pass on these costs to the customer…
But isn’t there a better way to do this?
For instance:
Allow customers to, on a project basis determine which and how long test results are stored. There are always projects which may be less “active”. Perhaps once delivered there might be only sporadic new releases needed. For those projects, we want to keep on running the tests (as external changes could always cause regression), but perhaps we don’t need to have those test results stored for longer than a week. Of those, perhaps we only want to keep on storing the tests runs from the highest (most production-like) environment (see my other question I’ll raise
Make not the number of test results be the cost-driver, but the actual size of storage space. There will be much less storage use for jUnit imported tests (without a screenshot). The Katalon tests that fail do have screenshots and those might then also be a motivator to get those fixed (either the application or the test case itself).
As I posted here more than a year ago, I’d like to now how Katalon TestOps sees the concept of “Test Environments”.
Use case: I have several projects where we run the Katalon tests on multiple environments, namely TST and ACC. Since we do agile iterative development, the tests on TST could break and often need rework. On ACC we have a production-like application that normally should always be green. It might be, due to integration with external parties, that on ACC a test suddenly starts failing (in other words: the external parties did some changes and these might impact our application in a negative way!).
We have the Katalon “profile” concept, where we do make a distinction on the environment (among other things) via the profile name. So we can filter out via ExecutionTestResult.profile:“~acc” all the test results from ACC, but that’s not ideal. Furthermore, we also upload jUnit test reports from ReadyAPI for both environments onto the Katalon TestOps via the Katalon Report Uploader (see Upload JUnit and report files to using command line | Katalon Docs ). There we do not have the concept of “profile”, so we cannot filter those out.
I tried to use the labels feature as described in my post above:
Obviously, goal should be to have this label automatically set to my test runs. Perfect place for me would be as a parameter in both the Katalon Runtime Engine command line (see Command-Line Syntax in Katalon Runtime Engine | Katalon Docs ) AND in the Katalon Report Uploader command line.
There is no url adaptation (see screenshot) for this filter, so I cannot store it as a favorite “application X - ACC results” in my browser. Each time, I will have to go for the custom filter and filter out the “TST” or “ACC”. Developers might for instance be only interested in the view for TST, while management wants to quickly see the state of ACC before an approval to go to PRODUCTION. I cannot go and ask them to play with the filter themselves…
Answer
A question with regard to the pricing on TestOps. Not so long ago, the pricing changed from “number of tests executed through testOps” to “number of results within testOps”. As you can see from the screenshot on your pricing page, the Monthly test results limit for e.g. 10.000 test results is $1.300/year. These test results include Katalon tests but also imported tests.
Hi @joost.degeyndt , thanks for your dedicated feedback regarding the pricing model. I would like to apologize for any inconvenience and fully understand your concerns here. To be honest, the business is crucial to Katalon’s existence, and the changes in the pricing model have aided us in scaling and developing more capabilities that benefit Katalon users.
Although business is important, the Katalon team always aims for Product-Model Fit, which reflects the balance between pricing models and product growth. Your feedback is very valuable at this point. It’s my pleasure to incorporate your feedback into the further discussion of the Katalon team regarding the pricing model.
If you have a question about Katalon TestOps, then don’t hesitate to share with our Product team at:
[Extended]
Hey TestOps Crew!