Running test suite/ test suites collection is super slow

I notice since yesterday when im running a test suite or test suites collection it is extremly slow,
there are like 3 scenarios:
-i dont always see a web browser opening but tests are passing
-i see the browser opening tests are running and result are showed aftere a big delay
-in test suite collection when one test suite is running browser is gone after a few test cases and it keeps running somehow for the rest of the test case but im unable to see the browser

It is very slow when switching to next test suite, i can upload my report folder if someone wants to have a look, thanks for the help

Try setting “Default Smart Wait” to “Disable”.

Did not really help,
it more looks like katalon does not keep up with the browser if you know what i mean, if i launch a single suite i can see all the steps already finished on the browser and its closing but results are still slowly appearing in the log viewer and Job progress.
I guess its happening the same for test suite collections that’s why its so slow, cause Katalon is waiting for a results of one suite to finish and then its launching next one.
So the issue is that it doesn’t keep up with the browser which is super weird, here are screenshots from the console of example suite that im seeing this, maybe it can help somehow cause i have no more ideas, you can see it finished in about a minute but in fact job progress was spinning for about 8-9 minutes which is stoping next suite to launch


the Katalon runtime log can prove very helpful. You can access the runtime log via the menu:

image

I see lots of this warnings there, not sure what it means :

!ENTRY org.eclipse.jdt.core 2 0 2021-05-19 13:36:53.570
!MESSAGE Invalid package binding for default import: java.io

!ENTRY org.eclipse.jdt.core 2 0 2021-05-19 13:36:53.571
!MESSAGE Invalid package binding for default import: java.net

!ENTRY org.eclipse.jdt.core 2 0 2021-05-19 13:36:53.571
!MESSAGE Invalid package binding for default import: java.util

Here is the file:
log.txt (707.1 KB)

In the log.txt I found quite a lot of fatal Exceptions occurred. No doubt, your KS was installed badly.

The screenshot revealed that you installed Katalon Studio in the “C:\Program Files(x86)” folder. I suppose it is not a good location. Your Windows 10 user usually does NOT have the write access privilege to that folder. On the other hand, KS tries to write temp files into the installed folder. So KS fails.

Please remove KS currently you have, and reinstall Katalon Studio under the “C:\Users\yourname” folder where you have full access privileges, and try again.

That did not help sadly…
I just checked on other projects and it all looks the same, tests are running but results are not showing in the progress and its not launching next suite,
Even better i checked on a windows virtual machine and problem is the same, can it be something with the way i saved projects maybe? no idea, i will gladly hear any other ideas cause not gonna lie, starting to lose hope here

Here is an issue quite similar to yours:

Again, did not help

I tried re-installing, using diffrent projects, older version of the project (from january-march), using Virtual machine, changing directory of the projects and Katalon, updating web browser, im running out of ideas and starting to think it can’t be just me with this issue.
Its usually the same pattern, one or two test cases are showing as ‘passed’ and then you have to wait long time for the rest of the results

You can see same collection took nearly 4 hours to finish today when a few days ago it was about an 1hour!


You reported about the slowness of Test Suites/Test Suite Collections. But let me ask you, do all of individual Test Cases run OK quickly?

If any of Test cases run slowly, you should look at the slow test case and make it fast enough.

I have never seen a case where all of Test Cases run fine but a Test Suite, which is just a bunch of the test cases, runs slow.

In the original post you wrote:

How long did your test take before you noticed the extreme slowness?
10 seconds? 3 minutes? 20 minutes? or over 1 hour?

Do you regard 1hour as normal? To me, 1hour looks to be an extreme case.

I have ever experienced a lot of strangeness when I ran “hours-running as normal” tests. For example, when I left KS to run for hours, my Windows OS went down to Power Save mode, then every thing including Katalon Studio halted without saving miscellaneous resources; … after resume, KS got crazy.

Long running tests are hard to care, deserve special treatments. I suppose that you should try finding ways to make your tests to run quicker.

example :
1 test suite contatins 9 diffrent test cases,2 for login and logout and 7 for going to diffrent url and check element on the page. This took about 50 seconds before i notice the issue. Multiplaying that by 68 similar test suites we can get about ~60-70 minutes of running tests. Im not saying its perfect and there is room to improve but for now in my case it does the trick.
Now suddenly it takes 3-4 hours to finish which is extreme

I checked individual test cases as you suggested and they work normal, I dont see its the issue with tests beacuse they run normal if i look at the browser and what is happenning there. Im not sure if i explain it correctly, to me it looks like only the results of the tests are delaying the whole process, beacuse after browser is closed and tests did the actuall checks Katalon is showing you result of the first test case and ‘Job progress’ tab is not being updated in real time, that is stopping the next test suite from launching

Here you can see that it still “running” but in the console we can see the info about that the ended test suite

Project > Settings… > Katalon TestOps opens the Katalon TestOps dialog.

Do you have Enable Katalon TestOps Integration option ON?

I guess, you happened to turn the option from OFF to ON. Or you installed a new version of KS which has the option ON as default. Or you started using VisualTesting by TestOps recently.

Why not you to try disabling it.


If the TestOps integration option is ON, KS uploads the Test Suite’s results to the remote TestOps server. Your test cases seem to be large., and a test suite with large test cases emits bulky reports. Let me guess, uploading the bulky reports take 3 minutes after each Test Suite execution. You wrote that you repeat Test Suites 68 times. So, upload will possibly take over 3 hours= (3 minutes of upload * 68 times).

If you are using VisualTesting feature of TestOps, it would take even longer time to upload screenshot images from your PC to the remote TestOps server.

You are right! Uploading test suite results & images to TestOps — these tasks are done by KS behind the scene after a test suite finished and before the next test suite in a Test Suite Collection starts.

I am afraid your TestSuiteCollection is too heavy to utilise the TestOps features comfortably. You need to make your test cases flyweight.

I would suggest you to try disabling TestOps integration. Your test suite collection will run briskly as before, I think.

@ThanhTo

The “Job Progress” popup window does not show the progress of the interaction of Katalon Studio with TestOps server. It makes users think that nothing is going on while in fact KS is busy talking to TestOps. The popup makes user confused.

I suppose KS should change “Job Progress” popup to show the progress of the TestOps interaction, or to equip “TestOps Integration Progress bar”. Or writing a progress animation in character in the LogView could be an idea:
#
##
###
####
#####
######

2 Likes

I wrote:

@wolczoskar

There is a few things you can try to make your test emit less volume of logs:

Disable logging executed test steps

If you are KS Enterprise user, you can disable verbose logs of test step execution:
https://docs.katalon.com/katalon-studio/docs/working-with-execution-log.html#log-executed-test-steps

Configure Log level

edit Include/Config/log.properties file:

https://docs.katalon.com/katalon-studio/docs/working-with-execution-log.html#logs-configuration

for example:

logging.level.testcase=INFO

This will reduce the volume of logs.

Hi all, coming back to this after holiday,
as its often happening, issue solved itself, i didnt try new suggestion with TestOps or disabling logs, running test suites is back to normal and its not taking half a working day, i really wish i could help and tell you what was the reason but im not sure now, starting to think it was my working environment but i was trying on 3 diffrent ones so that can’t be true

I close this as his problem ceased.

PLS have a look at this topc: