How to integrate Katalon Studio to TestRail to report test case results

staff-20

#21

@ leszek.urbanik
@thebenluker

I’m getting the same NullPointerException as “leszek.urbanik”. My default profile has a value set for global variable G_testrail_run_id. The test script has id in single quotes. Is there a solution?


#22

No, not really. We just bought the plugin…


#23

How do you get all of this to work in the plugin? I installed the plugin, setup the connection, and tested it, but it is not clear how I am supposed to actually use the plugin. There were no instructions on the plugin page.


#24

hi bro @Jayesh_Mohane for me it’s not working. Now i am trying that your tutorial. because i am trying that plugin it;s not working.
May you help?


#25

What is the issue that you are having? I no longer use Katalon :frowning:


#26

On TestRail not updated the status

but i get this

TestRail: Start sending summary message to channel:

Summary execution result of test suite: Test Suites/As a Brand/Advanced Regression Testing/Sign in Module

Total test cases: 1

Total passes: 1

Total failures: 0

Total errors: 0

Total skipped: 0

TestRail: Summary message has been successfully sent

Slack: Start sending summary message to channel: ui_automation

Slack: Summary message has been successfully sent


#27

2019-10-16 14:33:28.260 INFO c.k.katalon.core.main.TestSuiteExecutor - START Test Suites/As a Brand/Advanced Regression Testing/Sign in Module
2019-10-16 14:33:28.339 INFO c.k.katalon.core.main.TestSuiteExecutor - hostName =
2019-10-16 14:33:28.340 INFO c.k.katalon.core.main.TestSuiteExecutor - os = Mac OS X 64bit
2019-10-16 14:33:28.341 INFO c.k.katalon.core.main.TestSuiteExecutor - hostAddress =
2019-10-16 14:33:28.368 INFO c.k.katalon.core.main.TestSuiteExecutor - katalonVersion = 6.3.3.11
2019-10-16 14:33:28.697 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-10-16 14:33:28.698 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/As a Brand/01. Sign in Module/1.2 Check sign in/TC01_Sign in with valid account
2019-10-16 14:33:28.912 INFO c.k.katalon.core.main.TestCaseExecutor - (Default) testrail_tc_id = 3
2019-10-16 14:33:29.157 DEBUG t.TC01_Sign in with valid account - 1: openBrowser("")
2019-10-16 14:33:29.367 INFO c.k.k.core.webui.driver.DriverFactory - Starting ‘Chrome’ driver
Oct 16, 2019 2:33:29 PM org.openqa.selenium.remote.DesiredCapabilities chrome
INFO: Using new ChromeOptions() is preferred to DesiredCapabilities.chrome()
2019-10-16 14:33:29.404 INFO c.k.k.core.webui.driver.DriverFactory - Action delay is set to 0 seconds
Starting ChromeDriver 76.0.3809.68 (420c9498db8ce8fcd190a954d51297672c1515d5-refs/branch-heads/3809@{#864}) on port 29471
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
[1571211224.736][WARNING]: This version of ChromeDriver has not been tested with Chrome version 77.
Oct 16, 2019 2:33:45 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
2019-10-16 14:33:45.127 INFO c.k.k.core.webui.driver.DriverFactory - sessionId = 40dacde7e034f6f34504d92c2dfe05a7
2019-10-16 14:33:45.160 INFO c.k.k.core.webui.driver.DriverFactory - browser = Chrome 77.0.3865.120
2019-10-16 14:33:45.161 INFO c.k.k.core.webui.driver.DriverFactory - platform = Mac OS X
2019-10-16 14:33:45.161 INFO c.k.k.core.webui.driver.DriverFactory - seleniumVersion = 3.141.59
2019-10-16 14:33:45.162 INFO c.k.k.core.webui.driver.DriverFactory - proxyInformation = ProxyInformation{proxyOption=USE_SYSTEM, proxyServerType=HTTP, password=, proxyServerAddress=, proxyServerPort=0}
2019-10-16 14:33:45.164 DEBUG t.TC01_Sign in with valid account - 2: navigateToUrl(G_SiteURL, STOP_ON_FAILURE)
2019-10-16 14:33:49.203 DEBUG t.TC01_Sign in with valid account - 3: setText(findTestObject(“Object Repository/Login Page/ts-hq-textfield-email”), emailbrand, STOP_ON_FAILURE)
2019-10-16 14:33:49.561 DEBUG t.TC01_Sign in with valid account - 4: setText(findTestObject(“Object Repository/Login Page/ts-hq-textfield-pwd”), password, STOP_ON_FAILURE)
2019-10-16 14:33:49.731 DEBUG t.TC01_Sign in with valid account - 5: click(findTestObject(“Object Repository/Login Page/ts-hq-button-signin”), STOP_ON_FAILURE)
2019-10-16 14:33:49.895 DEBUG t.TC01_Sign in with valid account - 6: delay(wait_to_medium, STOP_ON_FAILURE)
2019-10-16 14:33:54.964 DEBUG t.TC01_Sign in with valid account - 7: verifyTextPresent(“ITEMS”, false, STOP_ON_FAILURE)
2019-10-16 14:33:57.924 DEBUG t.TC01_Sign in with valid account - 8: verifyTextPresent(“Lite description about Item”, false, STOP_ON_FAILURE)
2019-10-16 14:33:58.159 ERROR c.k.k.core.context.internal.TestHooker - :x: org.codehaus.groovy.runtime.InvokerInvocationException: groovy.lang.MissingMethodException: No signature of method: java.lang.String.add() is applicable for argument types: (java.lang.String) values: [3]
Possible solutions: any(), any(groovy.lang.Closure), wait(), find(), wait(long), find(java.lang.CharSequence)
2019-10-16 14:33:58.162 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/As a Brand/01. Sign in Module/1.2 Check sign in/TC01_Sign in with valid account
2019-10-16 14:33:58.286 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-10-16 14:33:58.286 INFO c.k.katalon.core.main.TestCaseExecutor - CALL Test Cases/TestRail_Update_Run_With_Results
2019-10-16 14:33:58.819 DEBUG t.TestRail_Update_Run_With_Results - 1: response = com.testrail.syncTestRail.get_tests(G_testrail_run_id)
2019-10-16 14:33:58.906 ERROR k.k.c.m.CustomKeywordDelegatingMetaClass - :x: No such property: WSBuiltInKeywords for class: com.testrail.syncTestRail
2019-10-16 14:33:58.909 ERROR c.k.katalon.core.main.TestCaseExecutor - :x: Test Cases/TestRail_Update_Run_With_Results FAILED.
Reason:
groovy.lang.MissingPropertyException: No such property: WSBuiltInKeywords for class: com.testrail.syncTestRail
at com.testrail.syncTestRail.get_tests(syncTestRail.groovy:49)
at com.testrail.syncTestRail.invokeMethod(syncTestRail.groovy)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:17)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
at TestSuiteTestRailID.afterTestSuite(TestSuiteTestRailID.groovy:32)
at TestSuiteTestRailID.invokeMethod(TestSuiteTestRailID.groovy)
at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:111)
at com.kms.katalon.core.context.internal.TestHooker.lambda$3(TestHooker.java:95)
at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:94)
at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:88)
at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:85)
at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:99)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1571211205127.run(TempTestSuite1571211205127.groovy:35)

2019-10-16 14:33:58.915 INFO c.k.katalon.core.main.TestCaseExecutor - END CALL Test Cases/TestRail_Update_Run_With_Results
2019-10-16 14:33:58.915 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-10-16 14:33:58.942 ERROR c.k.k.core.keyword.internal.KeywordMain - :x: Unable to call Test Case ‘Test Cases/TestRail_Update_Run_With_Results’ (Root cause: com.kms.katalon.core.exception.StepErrorException: Call Test Case ‘Test Cases/TestRail_Update_Run_With_Results’ failed because of error(s)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:66)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
at TestSuiteTestRailID.afterTestSuite(TestSuiteTestRailID.groovy:32)
at TestSuiteTestRailID.invokeMethod(TestSuiteTestRailID.groovy)
at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:111)
at com.kms.katalon.core.context.internal.TestHooker.lambda$3(TestHooker.java:95)
at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:94)
at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:88)
at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:85)
at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:99)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1571211205127.run(TempTestSuite1571211205127.groovy:35)
Caused by: groovy.lang.MissingPropertyException: No such property: WSBuiltInKeywords for class: com.testrail.syncTestRail
at com.testrail.syncTestRail.get_tests(syncTestRail.groovy:49)
at com.testrail.syncTestRail.invokeMethod(syncTestRail.groovy)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:17)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
… 18 more
)
2019-10-16 14:33:58.956 ERROR c.k.k.core.context.internal.TestHooker - :x: com.kms.katalon.core.exception.StepErrorException: Call Test Case ‘Test Cases/TestRail_Update_Run_With_Results’ failed because of error(s)
2019-10-16 14:33:59.210 INFO c.k.katalon.core.main.TestSuiteExecutor - --------------------
2019-10-16 14:33:59.211 INFO c.k.katalon.core.main.TestSuiteExecutor - END Test Suites/As a Brand/Advanced Regression Testing/Sign in Module
2019-10-16 14:33:59.212 INFO c.k.katalon.core.main.TestSuiteExecutor - ====================


#28

10-18-2019 11:10:45 AM G_run_testrail_tc_id.add(tc_ids[n])

Elapsed time: 0.062s

org.codehaus.groovy.runtime.InvokerInvocationException: groovy.lang.MissingMethodException: No signature of method: java.lang.String.add() is applicable for argument types: (java.lang.String) values: []

Possible solutions: any(), any(groovy.lang.Closure), wait(), find(), wait(long), take(int)

Anyone can help?
@devalex88


#29

Hi @januari

Can you express clearly what you’re trying to do, the expected behavior and what seems to be wrong. And please try to separate between plain text and code (you can wrap the code with triple single quote so that they display like

code

Instead of

code


#30

Hi @ThanhTo, I hope you can directing me, Thank you

The point is I already following step by step like documentation, and I got errors like this image. I did not reduce or add the step expect URL my TestRail and Authorization.

defaultFile