I have made a demo project and pushed in GitHub:
I made a simple Katalon Studio project, which has a test suite named
TS0. It invokes 2 test cases named
TC2. The TC1 always gets success while the TC2 always fails. Consequently the TS0 always finishes with 1 failure.
I wanted to execute the test suite in console. I read the documentation of Console Mode Execution. I made a Windows batch file named
run_console_mode.bat. Its initial version was like this:
@echo off echo KATALONSTUDIO_HOME="%KATALONSTUDIO_HOME%" set DIR=%~dp0set SCRIPT_DIR=%DIR:~0,-1% echo SCRIPT_DIR=%SCRIPT_DIR% cd "%KATALONSTUDIO_HOME%" .\katalon.exe -noSplash -runMode=console -summaryReport -projectPath="%SCRIPT_DIR%" -testSuitePath="Test Suites\TS0" -browserType="Firefox" -headless --config -proxyOption=MANUAL_CONFIG -proxy.server.type=HTTP -proxy.server.address="172.24.2.10" -proxy.server.port="8080"cd /d "%SCRIPT_DIR%"
(Please note that here is no code to deal with the exitCode from
When I ran this bat file in the Windows command prompt
TS0 was executed. As expected
TC1 passed and
Then I put the project into GitHub and configured my Continuous Integration server (Jenkins) so that it continuously build the project. When I tried for the first time, in the Jenkins dashboard I found a blue ball icon for the project. The blue ball icon means that Jenkins sees the project ran passed.
Obviously Jenkins was not informed of the
TC2 failure. I realized that I need to rewrite the
Here is my revised
The point is that it receives the exitCode from
.\katalon.exe and pass it to the caller process.
... .\katalon.exe -noSplash -runMode=console ... set exitCode=%ERRORLEVEL% ... exit /B %exitCode%
exit /B %exitCode% solved my problem. Jenkins is now informed of the
TC2 failure and can deal with the job correctly.