Katalon Jenkins Plug in Issue


#1

Setup: I have suites of very simple tests that generate a GET request to a url and verify that the response code is between 200-399.

I’m attempting to use the Katalon plugin in Jenkins to run them, and I’ve come across an unusual problem.

I created a Katalon project with two test cases - one positive (url returns between 200-399), one negative (url 404’s). Add those to a test suite, committed to GIT. I use the GIT extension with a sparse checkout to pull the project i need (multiple projects in the repo). I use the Jenkins workspace.

Created two projects in Jenkins, one using the Katalon Plugin, one using a windows batch command to launch the tests. What I’m seeing is the when launched from the command line the test performs as I expect with the positive and negative being flagged appropriately.

  2019-05-09 15:37:00.255 DEBUG testcase.HomePage                        - 1: url = HomePage
2019-05-09 15:37:00.255 DEBUG testcase.HomePage                        - 2: println(url)
https://xxxxxxxxxxxxxxx/asdfad
2019-05-09 15:37:00.256 DEBUG testcase.HomePage                        - 3: requestObject = requestBuilder.RequestBuilder.buildRequest(url)
2019-05-09 15:37:00.257 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - requestBuilder.RequestBuilder.buildRequest is PASSED
2019-05-09 15:37:00.258 DEBUG testcase.HomePage                        - 4: response = sendRequest(requestObject)
2019-05-09 15:37:13.435 INFO  c.k.k.core.util.BrowserMobProxyManager   - HAR: c:\Katalon GIT repo\SQA\Jenkins\workspace\Test_GSW_CMD\Katalon\GSW_v2\Reports\MVP\20190509_153624\requests\main\null.har
2019-05-09 15:37:13.445 DEBUG testcase.HomePage                        - 5: verifyResponseStatusCodeInRange(response, 200, 399, STOP_ON_FAILURE)
2019-05-09 15:37:13.448 ERROR c.k.k.core.keyword.internal.KeywordMain  - ? Expected status code is from '200' to '399' but actual status code is '404'
2019-05-09 15:37:13.454 ERROR c.k.k.core.keyword.internal.KeywordMain  - ? Unable to verify response status code (Root cause: com.kms.katalon.core.exception.StepFailedException: Expected status code is from '200' to '399' but actual status code is '404'

Running thru the Katalon Jenkins Plugin, it’s not logging the failure, and the thread hangs, requiring a restart of jenkins to kill.

_> 2019-05-09 15:56:36.716 DEBUG testcase.HomePage                        - 1: url = HomePage_
_> 2019-05-09 15:56:36.716 DEBUG testcase.HomePage                        - 2: println(url)_
_> https://xxxxxxxxxxxxxxxxxxxxx/asdfad_
_> 2019-05-09 15:56:36.717 DEBUG testcase.HomePage                        - 3: requestObject = requestBuilder.RequestBuilder.buildRequest(url)_
_> 2019-05-09 15:56:36.718 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - requestBuilder.RequestBuilder.buildRequest is PASSED_
_> 2019-05-09 15:56:36.824 DEBUG testcase.HomePage                        - 4: response = sendRequest(requestObject)_
_> 2019-05-09 15:56:37.850 INFO  c.k.k.core.util.BrowserMobProxyManager   - HAR: c:\Katalon GIT repo\SQA\Jenkins\workspace\Test_GSW_KatalonPlugin\Katalon\GSW_v2\Reports\MVP\20190509_155624\requests\main\null.har_
_> 2019-05-09 15:56:37.860 DEBUG testcase.HomePage                        - 5: verifyResponseStatusCodeInRange(response, 200, 399, STOP_ON_FAILURE)_
_> 2019-05-09 15:56:37.871 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/HomePage_
_> 2019-05-09 15:56:37.908 INFO  c.k.katalon.core.main.TestSuiteExecutor  - --------------------_
_> 2019-05-09 15:56:37.908 INFO  c.k.katalon.core.main.TestSuiteExecutor  - END Test Suites/MVP_
_> 2019-05-09 15:56:37.908 INFO  c.k.katalon.core.main.TestSuiteExecutor  - ====================_
_> Launcher status after execution process completed: Running_
_> Start writing reports to folder: c:\Katalon GIT repo\SQA\Jenkins\workspace\Test_GSW_KatalonPlugin\Katalon\GSW_v2\Reports\MVP\20190509_155624_
_> Reports were generated at folder: c:\Katalon GIT repo\SQA\Jenkins\workspace\Test_GSW_KatalonPlugin\Katalon\GSW_v2\Reports\MVP\20190509_155624_
_> _
_> --------------------------------------------------------------------------------_
_> Test Suites/MVP - Chrome - 20190509_155624.............................2/2(100%)_
_> --------------------------------------------------------------------------------_
_> _
_> Build timed out (after 10 minutes). Marking the build as failed._

Thoughts? I’d prefer to use the Katalon Plugin to be able to download Katalon Studio as needed.

Also noticed that it wasn’t saving out the HAR’s. Note the null.har filename.