Katalon always exits with code 0 in error cases

Hi friends,

I have running the Katalon Docker Image in Gitlab-CI.
But there is a Problem that I could not solve.

If everything is going well and my tests are starting then it works like expected:

  • If a testsuite finishes with success the container exits with code 0
  • If a testsuite failed the container exits with code 1

But there are some special cases, like the browser could not start because of an incompatible webdriver:

2023-08-10 16:22:43.012 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Unable to open browser with url: 'http://my-url/' (Root cause: com.kms.katalon.core.exception.StepFailedException: Unable to open browser with url: 'http://my-url/'
Caused by: org.openqa.selenium.SessionNotCreatedException: session not created: This version of ChromeDriver only supports Chrome version 110
Current browser version is 115.0.5790.170 with binary path /usr/bin/google-chrome

This is an example and I experienced other cases where the tests also couldn’t start because of an error.
Katalon always delivers exit code 0 in these cases.

And the gitlab-pipeline is marked as successful because of this.
This is a big problem and right now I can’t see if I can do anything to solve it. I think this comes out of the Katalon Engine and I am not a Katalon developer.

Am I right or is there a solution for this?
Would be just awesome!


1 Like


Can you please give us the device, Katalon and browser version you are using? Thank you

Device: docker with os = ubuntu:23.04
Katalon: 8.6.0
Browser: Chrome 115.0.5790.170


You might need to refer this: As of Chrome 115, Tools > Update WebDrivers > Chrome no longer works - #31 by Elly_Tran

@Elly_Tran you don’t got the actual problem.

It is not about Chrome and Webdrivers. I fixed this specific problem with the incompatible webdriver already. It was just an example to show you that there can be error cases where Katalon gives an exit code 0 (that stands for success)
And that was totally not a successful test run.

When we run hundreds of tests later on, we cannot check each individual test log to make sure that such a problem is not occuring again spontaneously. In such a case, Katalon must return an ‘exit code 1’, so that we can immediately see: There are issues.

Perhaps not 1, but definetly no 0.

According to the doc:

  • 1: The execution has failed test cases.

But for this case, looks like there is an uncaught situation.
No test was started, so KRE wrongly reports 0 because no failure was detected (or other known situation)
Which is wrong.
Should report at least 4:

4: The execution cannot start because …

Kindly pass this info to the development team to investigate

@simon.klotz since you are using KRE, most likely you have a licence.
So you can rise also a support ticket using the HelpDesk

1 Like

@bionel thanks for your further information.
This documentation is helpful, and yes this should be probably exit code 4.

I meanwhile remembered by myself that we’re actually paying for this and I reached out to our account manager and described the issue. It is up to them now to forward the issue to a developer team or whatever.


I had a similar situation where no tests ran but but KRE reporting exit code 3 which is for

  • 3: The execution has failed test cases and error test cases.

Details are here

Katalon team should consider it on priority because the job statuses are misleading for CI/CD workflow.