UI test cases are not executing on headless mode

Hi,

I am trying to execute UI test cases on chrome headless mode from Jenkins and also local.

From Jenkins, If I execute test suite with 20 test cases, all are executing fine. But when I kept suite with 200+ test cases, its getting stuck for more than hour(aborted the jib manually), not giving any error as well.

Tried to execute the same test case which got stuck from local, facing the same issue there as well.

And when I tried to execute 200+ test cases in Test suite collection file by keeping all 15+ test suite file, all got failed with error of webelement not found. But when execute individual test suites its working fine.

what could be reason for failing in Test Suite collection file and test cases are getting stuck ?

Hi,

This problem is really hard to reproduce. With a large number of test cases like that, it will be easy to be crashed, please help to double-check your CPU utilization of the Katalon.exe process. Which katalon version have you been using?

Do you run multiple test suites in a test suite collection in parallel mode? i know parallel execution is very unstable. Why not change the collection to run sequentially?

tried with sequential execution as well, still test cases are getting stuck.

You should read execution log catefully, identify which statement in your test case script caused the failure, and think why. Only your careful investigation could solve your problem.

At which stage your test suite with 200+ test cases got stuck?

Did it stuck at the 1st test case? 2nd test case? … at 100th test case? I guess, all your test cases ran ok but after the test case finished, your test suite stuck when it started compiling an execution report.

Please check the CPU usage and Memory usage using the Performance Monitor tool.


I guess your test suite gets stuck due to insufficient memory is allocated.

Compiling a execution report of 200+ test case would require very large memory of JVM. Possibly you want to allocate more memory to Katalon Studio.

The default size of recent versions of Katalon Studio is set

-Xmx2048m

You may want to enlarge it, for example,

-Xmx8192m

or

-Xmx2g

I think that you are too optimistic to execute 200+ test cases as one batch. I think that Katalon Studio is not designed with such large scale projects in mind, therefore it requires much memory for compiling execution reports. You should divide and conquer your targets for less troubles.