How to integrate katalon studio suite collection results with xray without TestOps integration

Product Forums Katalon Studio
1

Hi, I need to show the results of a suite collection execution in a xray test execution ticket.Jira is on premises and Katalon is activated offline therefore there is no integration with TestOps (there is no internet connection abroad). I need the results to be shown as in screenshot.

image
image794×752 51.1 KB

Any help would be great…

1 Like
2

To integrate Katalon Studio test results with Xray in Jira (on-premises) without TestOps, follow these steps:

1. Generate Xray-Compatible Reports in Katalon

Configure Katalon to produce test results in a format Xray supports (e.g., JUnit XML or Xray JSON).

Option A: Use JUnit XML Reports

  1. Enable JUnit Reports in Katalon:
  • Go to Project Settings > Report > Enable JUnit Report.
  1. Map Katalon Tests to Xray Test Issues:
  • Prefix Katalon test case names with Xray test keys (e.g., TEST-123: Login Test).
  1. Run Tests:
  • Results are saved in Reports/<execution_folder>/junitreports/*.xml.

Option B: Generate Custom Xray JSON Reports

Use a custom listener in Katalon to create Xray-specific JSON:

  1. Add a Test Listener:

groovy

// TestListener.groovy
import com.kms.katalon.core.annotation.AfterTestCase
import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.context.TestCaseContext
import com.kms.katalon.core.context.TestSuiteContext
import groovy.json.JsonOutput

class XrayJSONListener {
    static results = []

    @AfterTestCase
    void afterTestCase(TestCaseContext testCase) {
        def xrayKey = testCase.testCaseId.split(':')[0].trim() // Extract "TEST-123" from test case name
        results << [
            testKey: xrayKey,
            status : testCase.testCaseStatus == 'PASSED' ? 'PASS' : 'FAIL',
            comment : "Execution notes"
        ]
    }

    @AfterTestSuite
    void generateXrayJSON(TestSuiteContext testSuite) {
        def jsonReport = [
            tests: results
        ]
        new File('./Reports/xray_report.json').write(JsonOutput.prettyPrint(JsonOutput.toJson(jsonReport)))
    }
}
  1. Run Tests:
  • A custom xray_report.json file will be generated in the Reports folder.

2. Import Results into Xray

Option 1: Manual Upload via Xray UI

  1. Navigate to Xray Test Execution:
  • Open the Test Execution issue in Jira.
  1. Upload Results:
  • Go to More > Import Execution Results.
  • Select the generated xray_report.json or JUnit XML file.

Option 2: Use Xray REST API (Internal Network)

If your Jira instance allows internal API access:

  1. Authenticate:

bash

# Get authentication token
curl -H "Content-Type: application/json" -X POST --data '{"client_id": "your_id","client_secret": "your_secret"}' https://<jira-url>/rest/oauth2/latest/token
  1. Import Results:

bash

# For JSON
curl -H "Authorization: Bearer $TOKEN" -F "file=@xray_report.json" "https://<jira-url>/rest/raven/2.0/import/execution"

# For JUnit XML
curl -H "Authorization: Bearer $TOKEN" -F "file=@TEST-Katalon.xml" "https://<jira-url>/rest/raven/1.0/import/execution/junit"

3. Map Katalon Tests to Xray Issues

  • Naming Convention: Name Katalon test cases as XRAY_TEST_KEY: Test Name (e.g., TEST-456: Checkout Workflow).
  • Tags: Use Xray test keys as tags in Katalon:

groovy

@TestCaseTag(name = "TEST-456")
def "Checkout Workflow"() { ... }

Sample Xray JSON Report Format

json

{
  "testExecutionKey": "TEST-EXEC-123",
  "tests": [
    {
      "testKey": "TEST-456",
      "status": "PASS",
      "comment": "No issues found."
    }
  ]
}

Troubleshooting

  • Formatting Issues: Validate JSON/XMl with Xray’s schema.
  • Permissions: Ensure API tokens or users have rights to update Test Executions in Jira.

By generating Xray-compatible reports and using manual uploads or internal API calls, you can integrate Katalon results into on-premises Jira/Xray without TestOps

3

Hi @dineshh , thanks a lot for your reply it was very helpful, it works just fiine.

image
image1423×750 50.3 KB