How to use Katalon for Azure DevOps

hello,
I have a question please: should I have Floating license to run the tests on Azure Devops?

to explain I do have Katalon Studio Enterprise (Node-locked) Katalon Runtime Engine (Node-locked) license

but when I run the tests

Search for valid offline licenses in folder: C:\Users\VssAdministrator.katalon\license
2020-12-26T19:05:34.2511542Z
2020-12-26T19:05:34.2611529Z The number of valid offline licenses: 0

when I read about offline license I found out that once I created it I will not be able to move it to other device, so before I do that I need to confirm: can I run the automation tests on Azure Devop having only the node locked license?

please help, I am really in need to your answer
thank you!

Hi Rawan,

Thanks for your question.

I understand that you have KSE (Node-Locked) licenses and KRE (Node-Locked) licenses, and you would like to run tests on Azure DevOps.

Almost all users execute tests on Azure DevOps with Docker, so only KRE Floating licenses can be used to execute tests. As a result, if your environment has Docker, please consider the KRE Floating licenses, supporting you in executing Azure DevOps tests.

Besides, since the offline license files are for fixed machines, and Docker is an environment with dynamic machines, so you cannot create offline license files in that case.

I hope that the information is sufficient.

Please feel free to reply with other questions you might have.

How can I use Katalon for Azure DevOps in a continuous delivery system… we don’t have builds so there is nowhere I can configure the plugin… we have services and we want to run some test suite automatically or triggered directly in Azure DevOps

1 Like

Hello @ThanhTo Please can you help from where I can get the API key, and how to use the offline activation key in my azure-pipelines.yml file?

@delia.ionita Please can you help me with this?

Hi @ravi.arkasali

KRE Offline activation needs a fixed machine ID. Hence, to activate KRE in an offline environment, you have to use Azure Pipelines in a self-hosted agent.

Helpful docs:

Jass

Hi @jerson.martinez, @anhtuan
I am facing the same issue. Could you please help me with what you did to resolve the issue?
I have tried all of the below but still facing the issue

  • The com.kms.katalon.integration.analytics.properties file is the same on both, my PC and the Azure Repo

Is there any other configuration that we need to update?

Hi,

I have a question about test suite execution. If I need to execute the test suite after deploy software in Pipeline and after that when execute test command activated from pipeline. Will the test case execution process running on local machine environment or pipeline server environment?

Thank you!

1 Like

Hi all,

i am trying to evaluate the integration between katalon and azure devops, i start a release pipeline, but i get the error:
2021-10-21T13:26:44.8244070Z Katalon Executable File: undefined
2021-10-21T13:26:44.8276474Z ##[error]Cannot read property ‘indexOf’ of undefined

my Yaml file:
steps:
- task: katalon-llc.katalon.katalonTask.katalonTask@1
displayName: ‘Execute Katalon Studio project’
inputs:
location: ‘D:\Katalon_Studio_Windows’
executeArgs: ‘-browserType=“Chrome” -retry=0 -statusDelay=15 -testSuitePath=“Test Suites/Sample” -projectPath=“C:\Users*****\Katalon Studio\My First Web UI Project”’

any idea what might be the issue?

P.S. i am using a release pipeline in Azure because this is our usecase
thanks

i have an update on this

the paths i was referring to were local paths, but actually they should be the Azure DevOps cloud paths, which are “unknown” to me, so i moved the katalon test to the build pipeline and i uploaded my project in the repo, and hence i could omit the -projectPath option, (although i dont know how this will help my use case, because we are planning to use katalon to test our deployments/releases).

anyway, the build pipeline seems to solve the indexOf problem, but now i face another problem:

Unable to init server: Could not connect: Connection refused

Katalonc: Cannot open display: 

i hope someone could help, and i hope the comment above helps someone

Hi @Jass - do you have any further guidance on using self-hosted Agents please? I need to know what to request with our environments team eg what software should be installed? We would be using Azure DevOp pipelines (with DevOps plugin) and self-hosted agent. I have seen that MS-hosted agent works but unsure about setting up a self-hosted version and what software requirements are. If anyone can tell me how different ADO test project executions can be orchestrated that would be useful - can ADO manage execution to make use of, for example, two KRE licenses between five test projects, so that all projects run nightly, one after the other?

@Dan_Bown,

Please refer to software and hardware specs in Microsoft hosted agents and setup your self-hosted agent with the same specs.

I think the ADO manage the execution via the pipeline.yml but very straight forward. You can setup 2 parallel job via 2 azure-pipelines.yml files that run 2 projects and 3 projects at the same time. Eg:
Job 1:

trigger:
- master

pool:
  vmImage: 'ubuntu-20.04'

steps:
- task: katalonTask@1
  inputs:
    version: '8.1.0'
    executeArgs: '-projectPath=project1 -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key) --config -webui.autoUpdateDrivers=true'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
- task: katalonTask@1
  inputs:
    version: '8.1.0'
    executeArgs: '-projectPath=project2 -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key) --config -webui.autoUpdateDrivers=true'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
- task: PublishPipelineArtifact@0
  inputs:
    artifactName: 'JUnit Report' 
    targetPath: report/

Job 2:

trigger:
- master

pool:
  vmImage: 'ubuntu-20.04'

steps:
- task: katalonTask@1
  inputs:
    version: '8.1.0'
    executeArgs: '-projectPath=project3 -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key) --config -webui.autoUpdateDrivers=true'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
- task: katalonTask@1
  inputs:
    version: '8.1.0'
    executeArgs: '-projectPath=project4 -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key) --config -webui.autoUpdateDrivers=true'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
- task: katalonTask@1
  inputs:
    version: '8.1.0'
    executeArgs: '-projectPath=project5 -testSuitePath="Test Suites/New Test Suite" -browserType="Chrome" -apiKey=$(katalon_api_key) --config -webui.autoUpdateDrivers=true'
    xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'
- task: PublishPipelineArtifact@0
  inputs:
    artifactName: 'JUnit Report' 
    targetPath: report/

How do I upload the test results from the Azure pipeline into the Katalon testOps?

Hello,

Since yesterday my team is unable to run my test scripts in the Azure pipeline (they run with no problems locally) due to the following error:

com.kms.katalon.activation.plugin.models.KStoreClientExceptionWithInfo: Unexpected error occurs during executing get latest plugins
com.kms.katalon.activation.plugin.service.KStoreRestClient.getLatestPlugins(KStoreRestClient.java:83)
com.kms.katalon.activation.plugin.service.PluginService.getOnlinePlugins(PluginService.java:316)
com.kms.katalon.activation.plugin.service.PluginService.reloadPlugins(PluginService.java:96)
com.kms.katalon.activation.plugin.handler.ConsoleModeReloadPluginsHandler.reload(ConsoleModeReloadPluginsHandler.java:24)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.kms.katalon.execution.console.ConsoleMain.reloadPlugins(ConsoleMain.java:560)
com.kms.katalon.execution.console.ConsoleMain.launch(ConsoleMain.java:487)
com.kms.katalon.console.application.ConsoleApplicationStarter.start(ConsoleApplicationStarter.java:37)
com.kms.katalon.application.Application.runConsole(Application.java:125)
com.kms.katalon.application.Application.start(Application.java:88)
org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:137)
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:107)
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:657)
org.eclipse.equinox.launcher.Main.basicRun(Main.java:594)
org.eclipse.equinox.launcher.Main.run(Main.java:1447)
Caused by: com.kms.katalon.activation.plugin.models.KStoreClientException: Invalid Request. Status Code: 422. Message: Unprocessable Entity
com.kms.katalon.activation.plugin.service.KStoreRestClient.executeGetRequest(KStoreRestClient.java:333)
com.kms.katalon.activation.plugin.service.KStoreRestClient.getLatestPlugins(KStoreRestClient.java:59)
com.kms.katalon.activation.plugin.service.KStoreRestClient.executeGetRequest(KStoreRestClient.java:333)
com.kms.katalon.activation.plugin.service.KStoreRestClient.getLatestPlugins(KStoreRestClient.java:59)

The test execution fails because the script are unable to import this plugin: kms.turing.katalon.plugins.assertj.StringAssert

This happened after some configuration changes in the system to upgrade to JDK 11. However:

So I am not sure to what extent this error is happening due to those configuration changes or something else. Any idea how to fix this?

  • Katalon version: Katalon Studio 8.3.5
  • OS: Windows Server 2019 64bit

I have the same issue can any one help me out on this.

I have the same issue.

.

Can the Katalon Azure Devops test result uploaded in test plans?

i have same error did you fix it ?

I have yaml like this.

jobs:
- job: Katalon_API_Test
  displayName: 'Katalon API Test'
  timeoutInMinutes: ${{ parameters.timeoutInMinutes }}
  pool:
    vmImage: 'ubuntu-20.04'
  - task: katalonTask@1
    inputs:
      version: '8.6.6'
      executeArgs: '-projectPath=api-test.prj -testSuitePath="Test Suites/${{ parameters.testSuites }}" -executionProfile="${{ parameters.testProfile }}" -browserType="Web Service" -apiKey="${{ parameters.KatalonRuntimApiKey }}" --config -proxy.auth.option=NO_PROXY -proxy.system.option=NO_PROXY -proxy.system.applyToDesiredCapabilities=true -webui.autoUpdateDrivers=true'
      xvfbConfiguration: '-a -n 0 -s "-screen 0 1024x768x24"'

Got this error:
(katalonc:1870): dbind-WARNING **: 23:38:35.996: AT-SPI: Error retrieving accessibility bus address: org.freedesktop.DBus.Error.ServiceUnknown: The name org.a11y.Bus was not provided by any .service files

Any thought?