Why isn't my ADO test plan being updated when running tests from ADO pipeline while it does work when running tests locally from Katalon Studio?

When I run my test suite locally from Katalon Studio it successfully updates my ADO test plan state. However when I run the tests from my ADO pipeline, my tests are executed successfully but my ADO test plan is not updated. I have followed the steps on this page to set things up β†’ here

I am using in my yaml pipeline script the following arguments as specified in the documentation:


Is adoTestRunName mandatory? And does it have to match with the one I have set up in the IDE under Azure DevOps integrations?

Here is my current yaml command:

- main

  vmImage: ubuntu-latest

- task: katalonTask@1
    version: '9.2.0'
    executeArgs: '-retry=0 -testSuitePath="Test Suites/TestSuiteOne" -browserType="TestCloud" -testcloudEnvironmentId="300" -testcloudTunnel="false" -apiKey="xxxx" -licenseRelease="true" -orgID="xxxx" --config -webui.autoUpdateDrivers="true" -adoPlanId="38727" -adoTestRunName="TestCloudTestRun"'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
    JAVA_HOME: $(JAVA_HOME_17_X64)
    PATH: $(JAVA_HOME_17_X64)/bin:$(PATH)
  displayName: 'Katalon test run on TestCloud'

My pipeline succeeds but it does say that it can’t find my test case under my test plan. My test case 38729 is contained in the test plan with ID 38727. Pipeline output:

Uploading report to Azure...

Azure: Parameters received for running: AzureTestPlanId=38727 AzureTestRunName=TestCloudTestRun

Azure: Test Case Id 38,729 is not contained in Test Plan with Id 38,727

Report has been sent to Azure

This is what my test plan looks like:

Add images

1 Like

I solved it. The test plan ID of my actual test plan in ADO was different.

1 Like