KS 8.1.0 stucked in running (after more than 5 days of running long haul test) - need help to generate partial report

And this is were we are starting to converge.
This option it is available only for enterprise users, as you correctly pointed.

And, even if this will be made available for free version, the user may still like to have an option to watch the reports ‘on the fly’, particularly for such cases with large datasets / long running suites.

So, no matter if i am an enterprise user or free user, I should have the option to turn off the log viewer, but refreshing the browser some time to time (I won’t mind doing this manually)

This is why both requests should be considered.

Of-course, running the tests with KRE may be a solution (since the console runner shouldn’t bother with the log viewer) but KRE … guess what, it is yet another licensed only feature …

to the product owner of Katalon Studio, @Jass

Let me tell you, I see Katalon Studio is not good for large scale testing.

I am already on the ‘dark-side’ (pyhton + robot framework).
And actually … funny, the main reason for this was not the development inconsistencies and licensing schema … but something more serious for me … the Linux support.
Yet again, starting from 7.9.0, the linux version is broken, at least on Fedora, due to the eclipse upgrade most probably.
But I won’t bother to submit another bug report, looks like the Linux usage it is simply ignored … some people are thinking that Linux is running only in the cloud …

Have you ever reported a bug report of Katalon GUI on Fedora? Which URL? Let me bookmark it!

Is the Eclipse project on a decline nowadays?

here is one of them (fixed by accident by a previous eclipse upgrade):

Is Eclipse project on a decline nowadays?

In theory, upgrades to newest eclipse engines should fix issues.
But in practice, for katalon just breaks more stuff.

The main reason is, I think, Linux development seems to be targeted for some ancient ubuntu only distribution, but does not consider some other big players on the linux desktop scene.
Therefore, lack of testing …

“Linux development” of what?

Do you mean “Katalon Studio GUI for Linux”? or “Eclipse for Linux desktop”?

yes, I am refering to Katalon Studio GUI for Linux.
of-course, could be that the root causes are due to the changes in eclipse engine himself (or at least in the gtk runner) but still, the final product should be tested on different environments, at least the important ones.

you know ‘It is working on my machine …’
well, we don’t ship your machine …

@binel

Thank you for sharing

I know some people want to go for other ‘dark-side’ (TypeScript + Playwright).

… well, i should stop this off-topics.

I have also somewhere a POC project using groovy + spock which is mimicking the ‘global variable profile’ behavior.
It can run both with Maven or Gradle.
I created that for API testing just to have it at hand in case everything will go wrong with katalon but i have to quickly migrate to a java/groovy opensource solution.
In the mean time I moved to another company which is more python oriented.

That can be easily extended with Geb to have a ui own framework at hand too.

:stuck_out_tongue:

GEB + Spock + WebDriver + JUnit + Gradle … that is my favourite tools combination as well. This tool set requires solid skill for Java programming.

A few years ago I used to develop UI tests with this golden tools combination while I worked for my one-man project. Later I was assigned to work in a team where most of the members are non-programmers. I found it is impossible to ask them to use this tool set. So I switched to Katalon Studio. KS was easy to introduce to them. KS got them feel started with UI test automation.

I will send what i did in a private message, just not to offend somebody.
It is kind of old, i did that 2 years ago but feel free to use it for your inspiration.

@bionel

Let’s suspend our off-topics.

We should to go back to the original issue of @david.casiano.

Updating this thread. So far, disabling the “step execution logs” works like a charm. It seems to be faster and I was able to finish 6K overnight (compared to several days before) and KS is still running!

Though, I’m observing that doing Project->“Close and cleanup” does not seem to decrease the memory usage of KS as reported by Task Manager. I have to close KS and start it again to see the memory to decrease.

Thanks again for all the tips!

You once wrote,

Your test runs too slow. I can guess why your test is so slow.

Your test is not slow due to WinSocket APi calls.

Your test is slow because your test is too busy emitting bulky useless messages into the Log Viewer.


The “Log Viewer” GUI buffers and keep all the messages displayed in memory.

“Project -> Cleanup” will have no effect to sweep out the “Log Viewer” buffer, as this operation only removes some temporary files in the project directory.

It is quite likely. The buffers inside Log Viewer can be garbage-collected only after your Test Suite finished and you closed the Log Viewer of the Test Suite.

You should try to minimise the volume of messages that are directed to the “Log Viewer”.

But how?

  1. One approach was disabling the step execution logs (You are lucky, you have an Enterprise license!). Still you have 2 more options.

  2. “logs configuration”.
    See https://docs.katalon.com/katalon-studio/docs/working-with-execution-log.html#logs-configuration
    For example, edit your Include/config/log.properties to have a line:

logging.level.testcase=INFO

By this, many of DEBUG messages logged by test cases will be suspended. I would expect a decrease of -10% of messages (not very much effective).

  1. Modularize your Test Case (Groovy script). Identify a code section that are repetitively executed, reform it to be a Groovy class as a Custom Keyword. If you have a Test Case script which is called by other Test Cases using WebUI.callTestCase(), that would be a good candidate to refactor.

Katalon Studio tends to log how it executed Test Case scrips verbosely, but it NEVER logs how the Custom Keywords (Groovy class) worked internally. Therefore, moving your code sections from Test Cases into Keyword will significantly reduce the amount of messages directed into the Log Viewer.

@david.casiano

Your code is repeating 16000 times of processing something (WinSocket APi call). How is the processing implemented? — As a plain Test Case Script? or as a Custom Keyword? If you are not using Custom Keyword, definitely you should do. When your repetitive processing is transformed to be a Custom Keyword, the Log Viewer will become much silent. I would expect a decrease of -90% message in the Log Viewer. This change will make your test run far faster.

I would guess, if you could get rid of verbose messages in the Log Viewer, one verification processing for telnet call can run in 5 seconds or quicker. Then a sequential repetition of 16000 times will finish in 22 hours. Much quicker than 5 days!

The winsocket is imitating telnet connection so there is some delay there of about a second between connecting and wait for the prompt. It is open, action, close so it will be a tad slower. Not 30 seconds though but expecting it to be within 10 seconds with all the delay in between calls.

Also, the tests are implemented using keywords so the main test cases are just calling keywords and just a verification to pass or fail(using the KeywordUtil). Maybe 10 lines or so.

Thanks for all the input.

Also, open and close for both sockets, read/write buffer are using try/finally so exceptions will clean up the handles and stuff. I also don’t have any tight loops that will not enable gc to happen. KS will be looping to the data in the data files(executing the TC one by one sequentially).

Thanks again for all the tips!

Or you may prefer reusing the existing Test Case script, rather than creating a new Custom Keyword. Then the following code would suffice.
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import com.kms.katalon.core.testdata.TestData

// read the Data from the CSV file
TestData data = findTestData("sampleData")

// iterate over the rows in the table body
for (int i: 1..data.getRowNumbers()) {
	// find value of the 'ID' column
	String id = data.getValue('ID', i)
	if (id != null & id.length() > 0) {
		// do whatever you want
        WebUI.callTestCase(findTestCase("your existing test case name"), ["ID": id])
	}
}
This Test Case will be execute with a single TestCaseExecutor object. Therefore it is lighter-weighted than asking a Test Suite to execute a Test Case 16K times. This Test Case will not instantiate/dispose multiple child TestCaseExecutor objects. The `WebUI.callTestCase()` is executed by the single caller TestCaseExecutor object. No more TestCaseExecutor objects will be created.

20 Nov 2021

I removed a few post of mine, because I wrote something unexamined, and I found they were wrong.