How to integrate Katalon Studio to TestRail to report test case results

This post will help those understand how to integrate Katalon Studio with TestRail properly. The current guides on Katalon is not very clear in terms of explanation so I will post the proper guide I have written for my company here.

Link test cases between Katalon Studio and TestRail

To submit test results from Katalon Studio to TestRail, you need to link a test case from Katalon Studio to the one in TestRail. In order to do that, you need to save the information of the test case in TestRail to Katalon Studio’s global variable.

Setting up global variables

In the default profile of Katalon Studio, create these three variables:

G_run_testrail_tc_id : a list of executed TestRail test case IDs

G_run_testrail_tc_status : a list of executed TestRail test case status

G_testrail_run_id : a string value type which will include current TestRail test run ID.

To get the Value for the Test Run ID, go to your project on TestRail > click on Test Runs > Open the Test Run created > and see the ID next to the name i.e.: image The value will be the numbers after the letter.

To add the above, go to Profiles > default and click Add at the top. See example below of how it should look:image

Set up Katalon Studio test cases

In your Test Cases folder on Katalon, where you will have the test cases for automation, each test case will need to store a Variable related to the TestRail test case IDs. This variable should store numbers only. For instance, if you have a test case with ID C2510, the variable should be filled with “2510.” If any of Katalon Studio’s test cases relates to some TestRail test cases (to demonstrate an end-to-end test flow), the variable should be entirely filled with related TestRail test case IDs, separated by commas such as “2510, 2511, 2512.”

Although this step is manual for each test case—meaning that it takes time—it is an essential part of the process. To do this, see instructions below:

  • Open the test case on Katalon Studio
  • Click Variables at the bottom: image
  • Click Add at the top: image
  • Double Click Name field and add this vairable- testrail_tc_id
  • Leave Value Type as String
  • In Default Value, enter the number of the test case from TestRail.

Example:

Set up TestListener to collect tests and results after executing Katalon Studio test cases

Test Listeners are test steps that created based on your own criterias and will be executed when the condition is matched.

Create an AfterTestCase listener to get the testrail_tc_id and the status of the executed TestRail test case. These information are then updated to global variables for later use. To do this:

  1. Right Click Test Listeners folder on Katalons Test Explore > New > New Test Listener: image
  2. Set it to a suitable name i.e.: Get_TC_STATUS
  3. Check the AfterTestCase and AfterTestSuite method boxes
  4. Click OK
  5. Copy and paste the following code inside @AfterTestCase
/**
 * Executes after every test case ends.
 */
@AfterTestCase
def afterTestCase(TestCaseContext testCaseContext) {
    try {
        def tc_ids = testCaseContext.getTestCaseVariables()['testrail_tc_id'].split(",")
        for (def n : (0 .. tc_ids.length - 1)) {
            GlobalVariable.G_run_testrail_tc_id.add(tc_ids[n])
            GlobalVariable.G_run_testrail_tc_status.add(testCaseContext.getTestCaseStatus())
        }
    } catch (NullPointerException ex) {
    }
}

We will come back to the other method in there at the end so continue…

Create custom keywords to interact with TestRail from Katalon Studio via API

In addition to the built-in keywords, users can define custom keywords to extend the capability of Katalon Studio. Once created, custom keywords can be used in test cases just like other built-in keywords. This feature allows users to expand keywords and reuse them across projects.

The Custom Keywords will contain the API requests to TestRail to get test case from the run, update the test run and add results to the run.

  1. Create a New Package inside Keywords: image
  2. Enter a suitable name i.e.: sync_testrail
  3. Right Click the package and select New > Keyword: image
  4. Enter a class name i.e.: Testrail
  5. Click OK

Once the keyword file is created, please copy and paste the below into the file so that the code works and it has the right packages:

import groovy.json.JsonSlurper as JsonSlurper
import com.kms.katalon.core.testobject.TestObjectProperty as TestObjectProperty
import com.kms.katalon.core.testobject.ResponseObject as ResponseObject
import com.kms.katalon.core.testobject.ConditionType
import com.kms.katalon.core.testobject.RequestObject as RequestObject
import com.kms.katalon.core.testobject.impl.HttpTextBodyContent

Methods inside the Custom Keywords

Once the above is done, we will create methods inside the customer keyword we created. All the methods will need to be part of this one Keyword to make it easy. I’ve added comments in the code to make it easy to understand.

Methods below will require your credentials encoded to Base64%. To get this:

  1. Go to https://www.base64encode.org/

  2. Enter email address for TestRail on Line 1

  3. Enter your password for TestRail on Line 2

  4. Click >Encode<

  5. This will generate your base64 encoded credentials to paste after ‘Basic’ in the codes below. Replace everything from %xxxx% with your base64 credentials

This first method makes a GET request to TestRail to receive all test case IDs from your selected TestRail run. These IDs will be used later to update the execution results from Katalon Studio to TestRail. Copy and paste the code inside the public class:

//Makes a GET request to TestRail to get all the test case IDs from the selected TestRail run.
@Keyword
def get_tests(String id) {
    def slurper = new JsonSlurper()
    RequestObject ro = new RequestObject('a')
    ro.setRestRequestMethod('GET')
    ro.setRestUrl('https://%YOUR PROJECT%/index.php?/api/v2/get_tests/'+id)
    def httpheader = new ArrayList<TestObjectProperty>()
    httpheader.add(new TestObjectProperty(
            'Content-Type', ConditionType.EQUALS, 'application/json'))
    httpheader.add(new TestObjectProperty(
            'Authorization', ConditionType.EQUALS,
            'Basic %PASTE YOUR ENCODED CREDENTIALS HERE%'))
    ro.setHttpHeaderProperties(httpheader)
    ro.setBodyContent(new HttpTextBodyContent('', 'UTF-8', 'application/json'))
    def response = WSBuiltInKeywords.sendRequest(ro)
    return slurper.parseText(response.getResponseText())
}

This method makes a POST request to TestRail to update the selected test run with a new list of test case IDs. This is a common practice advised by TestRail itself. Please note that you can change the request body ‘include_all’ parameter to ‘true’ if you want to include all available test cases to the selected test run.

@Keyword
def update_run(String id, String array) {
    def slurper = new JsonSlurper()
    RequestObject ro = new RequestObject('Update TestRail test run')
    ro.setRestRequestMethod('POST')
    ro.setRestUrl('https://%YOUR PROJECT%/index.php?/api/v2/update_run/'+id)
    def httpheader = new ArrayList<TestObjectProperty>()
    httpheader.add(new TestObjectProperty(
            'Content-Type', ConditionType.EQUALS, 'application/json'))
    httpheader.add(new TestObjectProperty(
            'Authorization', ConditionType.EQUALS,
            'Basic %PASTE YOUR ENCODED CREDENTIALS HERE%'))
    ro.setHttpHeaderProperties(httpheader)
    //StringArray array = new String(tc_id.size())
    def body ='{"include_all": false,"case_ids": '+ array +'}'
    WebUI.comment('body='+body)
    ro.setBodyContent(new HttpTextBodyContent(body, 'UTF-8', 'application/json'))
    def response = WSBuiltInKeywords.sendRequest(ro)
    return slurper.parseText(response.getResponseText())
}

This final method also makes a POST request to TestRail to to add test results to TestRail.

//Makes a POST to add the test results to the TestRail run.
@Keyword
def add_results(String id, String request) {
    def slurper = new JsonSlurper()
    RequestObject ro = new RequestObject('a')
    ro.setRestRequestMethod('POST')
    ro.setRestUrl('https://%YOUR PROJECT%/index.php?/api/v2/add_results_for_cases/'+id)
    def httpheader = new ArrayList<TestObjectProperty>()
    httpheader.add(new TestObjectProperty(
            'Content-Type', ConditionType.EQUALS, 'application/json'))
    httpheader.add(new TestObjectProperty(
            'Authorization', ConditionType.EQUALS,
            'Basic %PASTE YOUR ENCODED CREDENTIALS HERE%'))
    ro.setHttpHeaderProperties(httpheader)
    WebUI.comment('body='+request)
    ro.setBodyContent(new HttpTextBodyContent(
            request, 'UTF-8', 'application/json'))
    def response = WSBuiltInKeywords.sendRequest(ro)
    def response_array = slurper.parseText(response.getResponseText())
    return slurper.parseText(response.getResponseText())
}

To make sure you’ve made everything correct you can play with same requests via Postman app after reading TestRail API documentation.

Create Katalon Studio test cases to handle communication between Katalon Studio and TestRail

Once all API call methods are ready, you will finally be able to start the integration. This will be done using an automated test case that we need to create inside the Test Cases Folder.

So create a test case and call it anything you want i.e.: TestRail_Update_Run_With_Results

Once you have created the test case, open it and click on Script at the bottom. Clear everything and copy and paste the code below. Also read the things to remember which are important after the below code:

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WSBuiltInKeywords
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable
import java.util.ArrayList as ArrayList
def response = CustomKeywords.'Testrail.get_tests'(GlobalVariable.G_testrail_run_id)
def tcs_to_add = []
for (def n : (0..response['id'].size)) {
    if (((response['case_id'])[n]) != null) {
        tcs_to_add.add((response['case_id'])[n])
    }
}
response = CustomKeywords.'Testrail.update_run'(GlobalVariable.G_testrail_run_id, tcs_to_add.toString())
def total_tcs_to_update = []
for (def n : (0..GlobalVariable.G_run_testrail_tc_id.size)) {
    if ((GlobalVariable.G_run_testrail_tc_id[n]) != null) {
        total_tcs_to_update.add(GlobalVariable.G_run_testrail_tc_id[n])
    }
}
for (def n : (0..tcs_to_add.size)) {
    if ((tcs_to_add[n]) != null) {
        total_tcs_to_update.add(tcs_to_add[n])
    }
}
def PASSED = '1';
def FAILED = '5';
def status_id
String request = '{"results": ['
// Generate a request for update test results
for (def n : (0..GlobalVariable.G_run_testrail_tc_id.size)) {
    if ((GlobalVariable.G_run_testrail_tc_id[n]) != null) {
        if ((GlobalVariable.G_run_testrail_tc_status[n]) == 'PASSED') {
            status_id = PASSED
        } else {
            status_id = FAILED
        }
         
        request = request.concat(((('{"case_id":' + (GlobalVariable.G_run_testrail_tc_id[n])) + ',"status_id":') + status_id) +
            ',"comment":"Automation test via Katalon"},')
    }
}
request = request.substring(0, request.length() - 1) //removing last excessive comma from request
request = request.concat(']}')
WebUI.comment('request=' + request.toString())
response = CustomKeywords.'Testrail.add_results'(GlobalVariable.G_testrail_run_id, request)
WebUI.comment('response=' + response.toString())

Things to remember with the above script:

  • With any of the code files above, you can press CTRL + A to select ALL code and then press CTRL+SHIFT+F to automagically indent it properly the groovy java way.
  • image - After CustomKeywords, you must tell it where the keyword is. In our case, we called the file Testrail hence it’s TestRail.get_tests where the API methods live
  • You can change the Comment in the request to whatever you want Katalon to report to TestRail

AfterTestSuite Listener

The final step is adding another listener at TestSuite to call the Katalon Studio test case you created in the previous step. To do this, go back to the Test Listener created earlier and paste this after the @AfterTestCase code:

/**
 * Executes after every test suite ends.
 */
@AfterTestSuite
def afterTestSuite() {
    'push results to testrail'
    WebUI.callTestCase(findTestCase('%replace this with path to your test case, for example: Test Cases/TestRail_Update_Run_With_Results%'), [:], FailureHandling.CONTINUE_ON_FAILURE)
}

To find the path of the test case you created with the long script, right click the test case and click on Properties. Copy the whole ID and pate it above. Below is an example of how it should actually look like if you followed the name in the guide:
example only:

How to report to TestRail once set up

After the above has all been set up, you will need to ensure that whatever test cases you have on your TestRail runs, the same automation tests are added to a Test Suite before running it. Each test case must have the testrail_tc_id variable with the correct test case ID as explained above.
To add tests to the Test Suite:

  1. Create a New Test Suite: image
  2. Enter a Name and click OK
  3. Click Add
  4. Select the Test Case that you want to add which are part of the Test Run on TestRail
  5. DO NOT SELECT the test case which communicates with TestRail (the one with the long script. There is no harm but it will mean your results will be reported twice to TestRail.):
    Once the test cases are added to the Test Suite, Hit the Run button at the top, relax and watch the magic happen as your test case are ran and the results are reported to TestRail.

Conclusion

Integrating between Katalon Studio and TestRail can help users not only reduce the complexity and cost during the testing process, but also improve the productivity and efficiency in automation testing and test case management.

Groovy is a messy language however there is so much on the web to learn about it. If you have any issues or questions, please let me know. Also, would be great to get feedback on this below to see how people got along and whether the guide was easy to follow or not.

4 Likes

Thank you @Jayesh_Mohane for this tutorial :slight_smile:

I have to ask a question though. What would I need to fix if I encounter the following error?

response = Testrail.get_tests(G_testrail_run_id) FAILED.

Reason:
java.lang.ClassNotFoundException: Testrail
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.getCustomKeywordClassAndSetMetaClass(CustomKeywordDelegatingMetaClass.java:98)

Hi @jason.ogayon,

Apologies for the delayed response.

‘Testrail’ should be the name of the keyword inside the package you created. So in the long piece of script inside the test case, the very first line should be CustomKeywords.xxxxxxxxxx - where xxxx should be the location of your Keyword and package. If you press CTRL+ Space after the full stop, you should get a list of options to choose from.

Let me know.

Thanks

Thank you so much @Jayesh_Mohane. Your tutorial helped us understand more about TestRail and we have built a TestRail integration plugin based on these ideas.

2 Likes

HI @devalex88, that is great to hear and thank you for developing the plugin to make integration with TestRail easier.

1 Like

Thank you @Jayesh_Mohane for such a great tutorial.
But I occured to face a problem while accomplishing it:

response = get_tests(G_testrail_run_id) FAILED.
Reason:
java.lang.StringIndexOutOfBoundsException: String index out of range: -1

how do you think what could be wrong?

Hi @lubomir.pe,

No worries. Can you please send the whole error and the part of the code. I believe this will be in the long script test case and it needs to be the following:
image

1 Like

Additionally, I would strongly recommend that you download the TestRail Plugin that has been developed which is a lot easier than this script. IT can be downloaded from here:
https://store.katalon.com/product/13/TestRail-Integration

I have been using it and it is simple and fast to use.

Thanks,
Jayesh

2 Likes

@Jayesh_Mohane thank you, it totally worked for me
Did you manage to run testsuits with TestRail integration in Jenkins? I am asking because I also have issues with it, but I believe this is not the right topic, just curious.
If someone has this flow (Katalon-Testrail-Jenkins), I would highly appreciate any help
lubomir.peretyatko@gmail.com

Glad that has worked for you. Unfortunately, this is not something I have come across and have done :frowning: might be worth checking on Google to see if someone has come across this. We are now moving to AzureDevOps where I hope to continue using Katalon Studio to run test cases

1 Like

Hello all,

did anyone encountered such issue:

04-16-2019 01:10:46 PM for (def n : (0…response[id].size))

Elapsed time: 0.023s

Get_TC_STATUS.afterTestSuite:49

Get_TC_STATUS.invokeMethod:0

for (def n : (0…response[id].size)) FAILED.
Reason:
java.lang.NullPointerException: Cannot get property ‘size’ on null object
at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:22)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:331)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:322)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:301)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:293)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:227)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:66)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
at Get_TC_STATUS.afterTestSuite(Get_TC_STATUS.groovy:49)
at Get_TC_STATUS.invokeMethod(Get_TC_STATUS.groovy)
at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:132)
at com.kms.katalon.core.context.internal.TestHooker.lambda$5(TestHooker.java:116)
at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:115)
at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:60)
at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:57)
at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:89)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1555413021757.run(TempTestSuite1555413021757.groovy:35)

?

I’m struggling with it for some time and I’m afraid I’m out of ideas what possibly I can be doing wrong in here…

I tried TR Integration Plugin and it works well so it must be something with those scripts.

Thanks,
Leszek

Hi there,

I am getting something similar. The tests execute fine but fail to update TR run with results:

2019-04-16 08:47:58.711 INFO c.k.katalon.core.main.TestCaseExecutor - CALL Test Cases/TestRail_Update_Run_With_Results
2019-04-16 08:47:58.811 DEBUG t.TestRail_Update_Run_With_Results - 1: response = Testrail.get_tests(G_testrail_run_id)
2019-04-16 08:47:58.862 ERROR k.k.c.m.CustomKeywordDelegatingMetaClass - :x: Testrail
2019-04-16 08:47:58.873 ERROR c.k.katalon.core.main.TestCaseExecutor - :x: response = Testrail.get_tests(G_testrail_run_id) FAILED.
Reason:
java.lang.ClassNotFoundException: Testrail
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.getCustomKeywordClassAndSetMetaClass(CustomKeywordDelegatingMetaClass.java:98)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:44)
at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:18)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:342)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:333)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:312)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:304)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:238)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:66)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
at GET_TC_STATUS.afterTestSuite(GET_TC_STATUS.groovy:45)
at GET_TC_STATUS.invokeMethod(GET_TC_STATUS.groovy)
at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:109)
at com.kms.katalon.core.context.internal.TestHooker.lambda$3(TestHooker.java:93)
at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:92)
at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:88)
at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:85)
at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:89)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1555426068992.run(TempTestSuite1555426068992.groovy:35)

2019-04-16 08:47:58.876 ERROR c.k.katalon.core.main.TestCaseExecutor - :x: Test Cases/TestRail_Update_Run_With_Results FAILED.
Reason:
java.lang.ClassNotFoundException: Testrail
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.getCustomKeywordClassAndSetMetaClass(CustomKeywordDelegatingMetaClass.java:98)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:44)
at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:18)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:342)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:333)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:312)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:304)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:238)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:66)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
at GET_TC_STATUS.afterTestSuite(GET_TC_STATUS.groovy:45)
at GET_TC_STATUS.invokeMethod(GET_TC_STATUS.groovy)
at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:109)
at com.kms.katalon.core.context.internal.TestHooker.lambda$3(TestHooker.java:93)
at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:92)
at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:88)
at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:85)
at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:89)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1555426068992.run(TempTestSuite1555426068992.groovy:35)

2019-04-16 08:47:58.877 INFO c.k.katalon.core.main.TestCaseExecutor - END CALL Test Cases/TestRail_Update_Run_With_Results
2019-04-16 08:47:58.877 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-04-16 08:47:58.881 ERROR c.k.k.core.keyword.internal.KeywordMain - :x: Unable to call Test Case ‘Test Cases/TestRail_Update_Run_With_Results’ (Root cause: com.kms.katalon.core.exception.StepErrorException: Call Test Case ‘Test Cases/TestRail_Update_Run_With_Results’ failed because of error(s)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:66)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)
at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:66)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)
at GET_TC_STATUS.afterTestSuite(GET_TC_STATUS.groovy:45)
at GET_TC_STATUS.invokeMethod(GET_TC_STATUS.groovy)
at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:109)
at com.kms.katalon.core.context.internal.TestHooker.lambda$3(TestHooker.java:93)
at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:92)
at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:88)
at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:85)
at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:89)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1555426068992.run(TempTestSuite1555426068992.groovy:35)
Caused by: java.lang.ClassNotFoundException: Testrail
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.getCustomKeywordClassAndSetMetaClass(CustomKeywordDelegatingMetaClass.java:98)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:44)
at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:18)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:342)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:333)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:312)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:304)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:238)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)
… 18 more
)

I was following this guide:

I urgently need some help to resolve this.

thank you

1 Like

Hi @leszek.urbanik,

Sorry for the late reply - I think I know where the problem is. Can you please check your long script Test Case where the long code is…
Looking at where it is failing, I can see that you are missing ’ ’ in [id]. This is where it is failing:

image

It should be the following in your script:
_for (def n : (0…response[id].size)) {_

Let me know if that works for you.

Thank you

Please see this response.

Hello @Jayesh_Mohane

no problem. Thanks for the answer.

Unfortunately it’s the response which is missing ’ ’ signs. They are present in the script:

Any other ideas? :slight_smile:

Hi @leszek.urbanik,

Seems to be working fine for me. Can you please confirm if you have G_testrail_run_id set in the default profile and given it the value of whatever the Test Plan ID is from TestRail?

Yes, it’s there.

Hi @Jayesh_Mohane, Thanks for the tutorial. It has helped a lot.

But I had followed exactly the same steps mentioned by you, but I am facing an issue which is like -

Test Cases/TestRail_Update_Run_With_Results FAILED.

Reason:

java.lang.StringIndexOutOfBoundsException: String index out of range: -1

at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:43)

at TestRail_Update_Run_With_Results.run(TestRail_Update_Run_With_Results:18)

at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)

at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)

at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:336)

at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:327)

at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:306)

at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:298)

at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:232)

at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)

at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.doCall(CallTestCaseKeyword.groovy:59)

at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword$_callTestCase_closure1.call(CallTestCaseKeyword.groovy)

at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:66)

at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.callTestCase(CallTestCaseKeyword.groovy:81)

at com.kms.katalon.core.keyword.builtin.CallTestCaseKeyword.execute(CallTestCaseKeyword.groovy:44)

at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)

at com.kms.katalon.core.keyword.BuiltinKeywords.callTestCase(BuiltinKeywords.groovy:334)

at Get_TC_STATUS.afterTestSuite(Get_TC_STATUS.groovy:44)

at Get_TC_STATUS.invokeMethod(Get_TC_STATUS.groovy)

at com.kms.katalon.core.context.internal.TestHooker.invokeMethod(TestHooker.java:109)

at com.kms.katalon.core.context.internal.TestHooker.lambda$3(TestHooker.java:93)

at com.kms.katalon.core.context.internal.TestHooker.invokeContextMethods(TestHooker.java:92)

at com.kms.katalon.core.context.internal.TestListenerCollector.lambda$3(TestListenerCollector.java:88)

at com.kms.katalon.core.context.internal.TestListenerCollector.handleListenerEvent(TestListenerCollector.java:85)

at com.kms.katalon.core.context.internal.ExecutionEventManager.publicEvent(ExecutionEventManager.java:36)

at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:89)

at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)

at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)

at TempTestSuite1558947967342.run(TempTestSuite1558947967342.groovy:35)

I am not able to understand exactly what is the issues here… can you kindly help me what is the issue here?

Thanks in advance!

Running integration with katalon 6.2 and got below issue

Test Cases/TestRail_Update_Run_With_Results FAILED.
Reason:
org.codehaus.groovy.runtime.metaclass.MissingMethodExceptionNoStack: No signature of method: Testrail.get_tests() is applicable for argument types: (java.lang.String) values: [1]
Possible solutions: getClass(), getAt(java.lang.String)

@Jayesh_Mohane please help

thank you

Did you ever solve this? Got the exact same problem over here. Thanks