Getting test result when running in console mode

Integrations
,
1

I would launch a Katalon test suite using console mode execution on a Windows box within cmd shell. I would like to run a test synchronously and get 0 if tests were successful, 1 if there are some failed tests.
I read Jenkins integration example suggesting to use command mode execution omitting -noExit and -consoleLog and running as a windows batch command.
So I opened a cmd and try to run

katalon -noSplash -runMode=console -projectPath=“D:\Workspace\Katalon\CreateStream\CreateStream.prj” -retry=0 -testSuitePath=“Test Suites/CreateProjectStream” -executionProfile=“test2” -browserType=“Chrome (headless)”

When I run this command it returns immediately with return code 0 while the json report file get written after some 20 seconds. So, return code 0 means that the test launched successfully and call to test suite is asynchronous.
How can I get a synchronous call where exit code is 0 upon success and 1 upon test failure ?
Alternatively, how can I get the same outcome from the json report ? I tried to read it but I really don’t know where to look for the test suite and/or test results.

1 Like
2

I did a few tests and eventually set up a Jenkins build to observe the way Jenkins call the katalon command. Jenkins actually generates a bat file containing the provided commands and then launches it with a "cmd /c call . So I try to mimic it in my windows box launched cmd /c katalon -noSplash -runMode=console… and got the desired behaviour, that is command return at test completion and exit code is 0 for success and 1 for failure.

This solves my issue. As a matter of curiosity, is there anybody that is able to explain the different observed behaviour ? Is it a matter of windows inner working or is it originated by the katalon.exe implementation ?

2 Likes
3

Upvoted.

Evelino,

Unfortunately, I don’t have an answer for you but I wanted to congratulate you on the excellent work to figure out the exact workings of the interface between Jenkins, the shell and Katalon.

It would be really nice if you rewrote this as a tip for others and posted it in the Tips and Tricks forum – @Katalon Team should then pick it up and repost it to the documentation website.