# How to update azuretestplan status when katalon tests run from pipeline with agent?

**URL:** <https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284>\
**Category:** Miscellaneous\
**Created:** [June 17, 2025, 2:23am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284 "2025-06-17T02:23:25Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![testql](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/testql/32/22831_2.png) [@testql](https://forum.katalon.com/u/testql)\
**Post date:** [June 17, 2025, 2:23am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/1 "2025-06-17T02:23:25Z")

</div>

Hello need to know how to make execution status updated when I use my ADO pipeline. This works when I run from my PC, but today nobody use a PC to run automation script. Can you explain step by step how to make this set up please?

---

<div class="post-metadata">

**Author:** ![dineshh](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/dineshh/32/24488_2.png) [@dineshh](https://forum.katalon.com/u/dineshh)\
**Post date:** [June 17, 2025, 7:29am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/2 "2025-06-17T07:29:28Z")

</div>

To update Azure Test Plan statuses when running Katalon tests via an Azure DevOps (ADO) pipeline using a build/release agent:

* * *

## **Prerequisites**

1. **Azure Test Plan Setup** :

- Test cases are created in Azure Test Plans.
- Each test case has a **work item ID** (e.g., `12345`).

1. **Katalon Project Setup** :

- Use the `TestCase` tag in Katalon to map tests to Azure Test Plan IDs (e.g., `@TestCase:12345`).
- Configure Katalon to generate **JUnit** or **TRX** test reports (needed for Azure integration).

1. **Pipeline Permissions** :

- The ADO pipeline service account has permissions to **Update Test Plans**.
- Use a **PAT token** with `Test Management` scope or an **Azure Service Connection** with sufficient access.

* * *

## **Step 1: Link Katalon Tests to Azure Test Cases**

In your Katalon test script, add the `TestCase` tag with the Azure Test Plan work item ID:

groovy

```auto
@Test
@TestCase("12345") // Replace with your Azure Test Case ID
def "Verify Login Functionality"() {
  // Test steps
}

```

* * *

## **Step 2: Configure Katalon to Generate Test Reports**

In your Katalon project’s `build.gradle`, ensure JUnit/TRX reporting is enabled:

text

```auto
test {
  useJUnit {
  }
  reports {
    junitXml.enabled = true
    html.enabled = true
  }
}

```

* * *

## **Step 3: Set Up the Azure Pipeline**

## **YAML Pipeline Example**

text

```auto
trigger:
- main

pool:
  vmImage: 'windows-latest' # or 'ubuntu-latest'

variables:
  katalonVersion: '8.3.5' # Match your Katalon CLI version
  katalonProjectPath: '$(System.DefaultWorkingDirectory)/YourKatalonProject'

steps:
- task: CmdLine@2
  displayName: 'Run Katalon Tests'
  inputs:
    script: |
      cd ./Katalon_Studio_Engine_Linux_64-${katalonVersion} # For Linux agents
      ./katalonc -noSplash -runMode=console \
        -projectPath="${katalonProjectPath}" \
        -retry=0 \
        -testSuitePath="Test Suites/YourTestSuite" \
        -browserType="Chrome" \
        -apiKey="YOUR_KATALON_API_KEY" \
        -reportFolder="$(Agent.TempDirectory)/Reports" \
        -reportFileName="TestResults"

# Publish test results to Azure Test Plans
- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit' # or 'VSTest' for TRX
    testResultsFiles: '**/TEST-*.xml' # Path to JUnit reports
    mergeTestResults: true
    testRunTitle: 'Katalon Test Run'

# Optional: Update Test Plan status via REST API (if needed)
- powershell: |
    $url = "https://dev.azure.com/{org}/{project}/_apis/testplan/Plans/{planId}/Suites/{suiteId}/TestPoint?api-version=7.1-preview.2"
    $body = @{
      outcome = "Passed" # or "Failed"
    } | ConvertTo-Json
    Invoke-RestMethod -Uri $url -Method Patch -Body $body -Headers @{
      Authorization = "Bearer $(System.AccessToken)"
      "Content-Type" = "application/json"
    }
  displayName: 'Update Test Plan Status (Optional)'

```

* * *

## **Step 4: Map Test Results to Azure Test Cases**

1. In Azure Test Plans, ensure the **test case IDs** match the tags in Katalon.
2. Use the `PublishTestResults@2` task to upload results. Azure will auto-link results to test cases if IDs match.

* * *

## **Step 5: Authentication**

- If using **PAT** :
  - Add a secret variable `AZURE_TOKEN` in the pipeline and use it in REST API calls.

- If using **System.AccessToken** :
  - Enable the **Allow scripts to access the OAuth token** option in the pipeline job.

* * *

## **Troubleshooting**

1. **Test results not linked** :

- Ensure the `@TestCase` tag matches the Azure Test Case ID exactly.
- Verify the **JUnit/TRX report** contains the correct IDs.

1. **Permission errors** :

- Grant the **Project Collection Build Service** account **Contributor** rights in Test Plans.

1. **Report path issues** :

- Use `$(Agent.TempDirectory)` for report paths to avoid permission conflicts.

* * *

## **Alternate Approach: Use the Katalon Azure DevOps Plugin**

1. Install the **Katalon Studio** extension from the [Azure Marketplace](https://marketplace.visualstudio.com/items?itemName=KatalonLLC.katalon-studio).
2. Use the **Katalon Studio** task in your pipeline:

text

```auto
- task: KatalonStudio@2
  inputs:
    version: '8.3.5'
    command: '-runMode=console -projectPath="$(System.DefaultWorkingDirectory)/Project" -testSuitePath="Test Suites/YourSuite" -reportFolder="$(Agent.TempDirectory)"'

```

* * *

By following these steps, your Azure Test Plan statuses will update automatically when Katalon tests run via the pipeline.

---

<div class="post-metadata">

**Author:** ![testql](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/testql/32/22831_2.png) [@testql](https://forum.katalon.com/u/testql)\
**Post date:** [June 17, 2025, 8:14am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/3 "2025-06-17T08:14:41Z")

</div>

Thanks Dineshh,  
I use the extension Katalon for azuredevops and I have already linked azure testcase ID in every katalon testcase using Integration tab.

Katalon is installed in a VM and run by self hosted agent.  
Here how I run Katalon in the actual pipeline :

- task: [katalon-llc.katalon.katalonTask.katalonTask@1](mailto:katalon-llc.katalon.katalonTask.katalonTask@1)

displayName: ‘Execute Katalon Studio project’

inputs:

location: ‘C:\Delivery\Applications\KATALON\Katalon\_Studio\_Engine\_Windows\_64-9.1.0’

executeArgs: 'katalonc -noSplash -runMode=console -retry=0 -testSuitePath=“Test Suites/DSK” -executionProfile=“UAT” -browserType=“Chrome” --config -proxy.auth.option=USE\_SYSTEM -proxy.system.option=USE\_SYSTEM -proxy.system.applyToDesiredCapabilities=true -webui.autoUpdateDrivers=true

Can you tell me how to update this with the task KatalonStudio@2 you gave me ?

---

<div class="post-metadata">

**Author:** ![dineshh](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/dineshh/32/24488_2.png) [@dineshh](https://forum.katalon.com/u/dineshh)\
**Post date:** [June 18, 2025, 5:36am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/4 "2025-06-18T05:36:45Z")

</div>

[quote=“testql, post:3, topic:176284”]  
katalon-llc.katalon.katalonTask.katalonTask@1  
[/quote] couldn’t access the link

---

<div class="post-metadata">

**Author:** ![dineshh](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/dineshh/32/24488_2.png) [@dineshh](https://forum.katalon.com/u/dineshh)\
**Post date:** [June 18, 2025, 5:45am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/5 "2025-06-18T05:45:22Z")

</div>

## **Step-by-Step Pipeline Update**

Replace your current `katalonTask@1` with the following YAML configuration for `KatalonStudio@2`:

```auto
- task: KatalonStudio@2
  displayName: 'Execute Katalon Studio Project'
  inputs:
    katalonRuntime: 'CUSTOM' # Required for self-hosted installations
    customKatalonLocation: 'C:\Delivery\Applications\KATALON\Katalon_Studio_Engine_Windows_64-9.1.0' # Path to your Katalon installation
    projectPath: '$(System.DefaultWorkingDirectory)\YourKatalonProjectFolder' # Path to the Katalon project (e.g., repo root)
    command: '-noSplash -runMode=console -retry=0 -testSuitePath="Test Suites/DSK" -executionProfile="UAT" -browserType="Chrome" -proxy.auth.option=USE_SYSTEM -proxy.system.option=USE_SYSTEM -proxy.system.applyToDesiredCapabilities=true -webui.autoUpdateDrivers=true'

```

* * *

## **Key Changes Explained**

1. **Task Version** :

- Switched from `katalonTask@1` to `KatalonStudio@2` (modern task with better integration).

1. **Custom Katalon Installation** :

- `katalonRuntime: 'CUSTOM'` tells Azure to use your pre-installed Katalon Engine.
- `customKatalonLocation`: Explicit path to your Katalon Engine installation (matches your VM setup).

1. **Project Path** :

- `projectPath`: Ensure this points to the root folder of your Katalon project (e.g., where `prj` and `Test Suites` folders reside). If your repo is cloned to `$(System.DefaultWorkingDirectory)`, adjust the path accordingly (e.g., `$(System.DefaultWorkingDirectory)\YourProjectName`).

1. **Command Arguments** :

- Retained all your existing arguments (test suite, profile, browser, proxy settings).
- **Fix** : Removed `--config` (not needed unless explicitly referencing a config file).
- Use **straight quotes** (`"`) instead of curly quotes (`“`/`”`) to avoid YAML parsing issues.

* * *

## **Additional Configuration**

## **Proxy Settings (If Required)**

If your self-hosted agent needs proxy authentication, add these environment variables to the pipeline agent or task:

```auto
env:
  HTTP_PROXY: 'http://proxy-server:port'
  HTTPS_PROXY: 'http://proxy-server:port'
  NO_PROXY: 'localhost,127.0.0.1'

```

## **Publish Test Results to Azure Test Plans**

Add this step **after** the Katalon task to auto-update Test Plan statuses:

```auto
- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/TEST-*.xml' # Default JUnit report path
    testRunTitle: 'Katalon Test Run'

```

* * *

## **Troubleshooting Tips**

1. **Path Validation** :

- Ensure `customKatalonLocation` points to the **root folder** of Katalon Studio Engine (not the `.exe` file).
- Example valid path:

```auto
C:\Delivery\Applications\KATALON\Katalon_Studio_Engine_Windows_64-9.1.0

```

1. **Katalon CLI Compatibility** :

- Confirm your Katalon version (`9.1.0`) matches the engine installed on the VM.

1. **Self-Hosted Agent Permissions** :

- The agent must have **read/write access** to the Katalon installation directory and project folder.

1. **Logs** :

- Check pipeline logs under the `Execute Katalon Studio Project` step for errors like missing dependencies or invalid paths.

* * *

## **Final Pipeline Example**

```auto
steps:
- checkout: self # Check out your repo

- task: KatalonStudio@2
  displayName: 'Execute Katalon Studio Project'
  inputs:
    katalonRuntime: 'CUSTOM'
    customKatalonLocation: 'C:\Delivery\Applications\KATALON\Katalon_Studio_Engine_Windows_64-9.1.0'
    projectPath: '$(System.DefaultWorkingDirectory)\YourKatalonProject'
    command: '-noSplash -runMode=console -retry=0 -testSuitePath="Test Suites/DSK" -executionProfile="UAT" -browserType="Chrome" -proxy.auth.option=USE_SYSTEM -proxy.system.option=USE_SYSTEM -proxy.system.applyToDesiredCapabilities=true -webui.autoUpdateDrivers=true'

- task: PublishTestResults@2
  displayName: 'Publish Test Results to Azure'
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/TEST-*.xml'
    testRunTitle: 'Katalon Test Run'

```

---

<div class="post-metadata">

**Author:** ![testql](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/testql/32/22831_2.png) [@testql](https://forum.katalon.com/u/testql)\
**Post date:** [June 25, 2025, 8:49am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/6 "2025-06-25T08:49:51Z")

</div>

> [@dineshh](#):
>
> `KatalonStudio@2`

Thanks for your answer. I’ve been trying to find KatalonStudio@2 in tasks but nothing. Do I have to do something special to see this task ready to use in pipeline assistant ? the only one I have for katalon is “Execute Katalon Studio Tests”

---

<div class="post-metadata">

**Author:** ![dineshh](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/dineshh/32/24488_2.png) [@dineshh](https://forum.katalon.com/u/dineshh)\
**Post date:** [June 26, 2025, 6:28am UTC](https://forum.katalon.com/t/how-to-update-azuretestplan-status-when-katalon-tests-run-from-pipeline-with-agent/176284/7 "2025-06-26T06:28:18Z")

</div>

1. **Verify Extension Installation**  
Go to your Azure DevOps organization → Manage extensions → Check if **“Katalon for Azure DevOps”** is installed. If not:

- Visit [Katalon for Azure DevOps extension page](https://marketplace.visualstudio.com/items?itemName=katalon-llc.katalon)
- Click “Get it free” and install it for your organization.

1. **Using the Correct Task Name**  
The task you see (“Execute Katalon Studio Tests”) **is the correct one** for `KatalonStudio@2`. There’s no separate “KatalonStudio@2” in the UI - this is the YAML identifier for the visual task.
2. **Configure Pipeline Task**  
In your YAML pipeline, use this structure:

text

```auto
steps:
- task: KatalonStudio@2
  displayName: 'Execute Katalon Studio Tests'
  inputs:
    katalonRuntime: 'CUSTOM'
    customKatalonLocation: 'C:\Delivery\Applications\KATALON\Katalon_Studio_Engine_Windows_64-9.1.0'
    projectPath: '$(System.DefaultWorkingDirectory)\YourProjectFolder'
    command: '-noSplash -runMode=console -retry=0 -testSuitePath="Test Suites/DSK" -executionProfile="UAT" -browserType="Chrome" -proxy.auth.option=USE_SYSTEM -proxy.system.option=USE_SYSTEM -proxy.system.applyToDesiredCapabilities=true -webui.autoUpdateDrivers=true'

```

Replace `YourProjectFolder` with your Katalon project directory name.  
4. **Publish Test Results** (Critical for Azure Test Plans integration)  
Add this task after Katalon execution:

text

```auto
- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/TEST-*.xml'
    testRunTitle: 'Katalon Test Results'
    mergeTestResults: true

```

## Key Configuration Points

| Parameter | Value | Notes |
| --- | --- | --- |
| `katalonRuntime` | `CUSTOM` | Required for self-hosted installation |
| `customKatalonLocation` | Path to Katalon Engine | Must be exact installation path on VM |
| `projectPath` | `$(System.DefaultWorkingDirectory)\...` | Path to Katalon project root (contains `prj` folder) |
| `command` | Your CLI arguments | Use straight quotes `"` instead of curly quotes |

## Troubleshooting Tips

1. **Agent Permission Issues**  
Ensure the agent service account has:

- Full control of Katalon installation directory (`C:\Delivery\Applications\KATALON`)
- Modify rights to the project directory

1. **Path Validation**  
Add a PowerShell step to verify paths exist:

text

```auto
- powershell: |
    Test-Path "$(env:customKatalonLocation)\katalonc.exe"
    Test-Path "$(projectPath)"
  displayName: 'Validate Paths'

```

1. **Check Katalon Version Compatibility**  
Ensure your Katalon Engine version (9.1.0) matches the CLI syntax. Update Katalon if needed.
2. **Log Collection**  
Add this to your command to get detailed logs:

text

```auto
command: '... -consoleLog -logLevel=INFO -reportFolder="$(System.DefaultWorkingDirectory)\Results"'

```

## Full Pipeline Example

text

```auto
steps:
- checkout: self # Required to get your test scripts

- task: KatalonStudio@2
  displayName: 'Execute Katalon Tests'
  inputs:
    katalonRuntime: 'CUSTOM'
    customKatalonLocation: 'C:\Delivery\Applications\KATALON\Katalon_Studio_Engine_Windows_64-9.1.0'
    projectPath: '$(System.DefaultWorkingDirectory)\KatalonProject'
    command: '-noSplash -runMode=console -retry=0 -testSuitePath="Test Suites/DSK" -executionProfile="UAT" -browserType="Chrome" -proxy.auth.option=USE_SYSTEM -proxy.system.option=USE_SYSTEM -proxy.system.applyToDesiredCapabilities=true -webui.autoUpdateDrivers=true'

- task: PublishTestResults@2
  displayName: 'Publish to Azure Test Plans'
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: '**/TEST-*.xml'
    testRunTitle: 'Katalon Regression Suite'

```

## Post-Execution

After a successful run:

1. Go to Azure Test Plans → Test Runs
2. Find “Katalon Regression Suite” run
3. You should see:

- Test cases linked via ID
- Status updated automatically (Passed/Failed)
- Stack traces for failures

If test cases still don’t update:

1. Verify Azure Test Case IDs are in this format: `AB123` (no extra characters)
2. Check test case IDs are mapped in Katalon: Project → Settings → Integration → Azure DevOps
3. Ensure the Katalon project has API access to Azure DevOps