Have you ever successfully run the test suite collection on any other environment but on the CI/CD pipeline? Are you sure, on other environment, the test suite collection runs fine? Any evidence?
150 test cases in a test suite — this looks very big. I think it is too big to debug when any little trouble occurs.
A run of Test Suite will try to compile an execution report (HTML,PDF,CSV etc) that contains all the test cases’ results combined. The report will become big as the number of test cases grows. The report compilation process is pretty memory hungry, requires larger memory accordingly. I guess, the process got an OutOfMemory error, which will cause the JVM process to stop suddenly and uncontrollably.
I would recommend:
- Make sure that the Test Suite Collection ran in the sequential mode, not in the parallel mode.
- try setting to emit no execution report. Then, the test suite collection may run fine and faster.
- extend the JVM Heap memory for Katalon runtime.
- split the large test suites into smaller pieces; prefer smaller number of test cases per test suite.
How can you extend the JVM Heap memory? Where is the explanation how to in the official Katalon documentation? ------- I was surprised to find there is none! => @nghi.phan
See OutOfMemoryError in console mode for my old and poor description. You want to change -Xmx2048m in katalon.ini file to, for example, -Xmx4096m or -Xmx8g