What is the most optimal way to run test suites on different environments from a pipeline using TestCloud?

Product Forums Katalon TestCloud
,
1

I am writing my yaml file to run my test suite on different OS and browsers using TestCloud. I am currently configuring my Azure DevOps pipeline with Katalon extension which creates a task to run the tests. As far as I understand I can only specify in the command one environment per task. Meaning that if I’d want to run my tests on 10 different environments I’d need to create 10 tasks. Every time a Katalon task starts it downloads Katalon Studio which takes quite a while. That is why I am asking if there is a better way to concatenate test suites on different environments?

Something better than this:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- task: katalonTask@1
  inputs:
    version: '9.2.0'
    executeArgs: '-retry=0 -testSuitePath="Test Suites/TestSuiteOne" -browserType="TestCloud" -testcloudEnvironmentId=**"300"** -testcloudTunnel="false" -apiKey="xxxx" -licenseRelease="true" -orgID="xxxx" --config -webui.autoUpdateDrivers="true"'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
  env:
    JAVA_HOME: $(JAVA_HOME_17_X64)
    PATH: $(JAVA_HOME_17_X64)/bin:$(PATH)
  displayName: 'Katalon test suite run on **Windows and Edge Chromium**'
- task: katalonTask@1
  inputs:
    version: '9.2.0'
    executeArgs: '-retry=0 -testSuitePath="Test Suites/TestSuiteOne" -browserType="TestCloud" -testcloudEnvironmentId=**"299"** -testcloudTunnel="false" -apiKey="xxxx" -licenseRelease="true" -orgID="xxxx" --config -webui.autoUpdateDrivers="true"'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
  env:
    JAVA_HOME: $(JAVA_HOME_17_X64)
    PATH: $(JAVA_HOME_17_X64)/bin:$(PATH)
  displayName: 'Katalon test suite run on **Windows and Firefox**'
and so on...
2 Likes
2

Hi,

I find this documentation: Override test environments in a test suite collection | Katalon Docs. Is this work for you?

2 Likes
3

Hi @Elly_Tran , that is not really what I am looking for as I would like to trigger my tests from my Azure DevOps pipeline not from TestOps. Maybe @ha.tpham knows?

I still tried running a scheduled test run as indicated on the documentation you provided, and I managed to do so but I am getting that the amount of test cases run is 0 when I actually have one test in my test suite so I expect test cases run counter to be 1. I even get the report under Files tab, but no test result. Why is that? I followed the steps and connected with my repo correctly.

4

If my understanding is correct, please configure test suite collection that has multiple test suites, then you can specify TestCloud environment for each test suite as screenshot below.

image
image720×321 99.6 KB

For the issue that you mentioned in your previous comment, please give us more details to investigate such as debug log, screenshot, video recording, etc. Thank you.

5

Concerning the TestSuites run on different environments, the screenshot you show ia to run this manually isn’t it? I would like to run the tests in different environments from the pipeline. What would the yaml code look like?

6

We recommend configuring the execution in this manner, followed by generating KRE command lines and putting them into the pipeline.