Azure DevOps - Execute Katalon Studio Tests

We’re trying to run our Katalon tests through Azure DevOps. We’ve bought 3 Katalon Runtime Engine Licences, and when I log into analytics.katalon.com I can see the 3 licences are taken and have a machine ID in the online licence section. To me this says they’re setup correctly.

We’re using the following CMD against the Katalon Runtime:

-noSplash -runMode=console -projectPath=[ProjectPathToMy.prj file] -retry=1 -testSuitePath=Test Suites/TestSuite1/v1 -executionProfile=Default -browserType=“Chrome” -reportFolder=“Reports/$(Build.BuildId)” -reportFileName=“report” -apiKey= XXXX

It get’s passed all the opening project bit and then fails with the error:

‘InvalidConsoleArgumentException: Test suite ‘Test’ not found.’

This run’s correctly locally, I checked/ changed/ tried different paths to the testsuite


Start check license task

2020-05-01T08:13:51.1584792Z All launchers terminated

2020-05-01T08:13:51.1594449Z 

2020-05-01T08:13:51.1595687Z com.kms.katalon.execution.exception.ExecutionException: com.kms.katalon.execution.exception.InvalidConsoleArgumentException: Test suite 'Test' not found.

2020-05-01T08:13:51.1597431Z    at com.kms.katalon.execution.console.entity.TestSuiteLauncherOptionParser.getConsoleLauncher(TestSuiteLauncherOptionParser.java:240)

2020-05-01T08:13:51.1598968Z    at com.kms.katalon.execution.console.ConsoleExecutor.execute(ConsoleExecutor.java:74)

2020-05-01T08:13:51.1600153Z    at com.kms.katalon.execution.console.ConsoleMain.launch(ConsoleMain.java:320)

2020-05-01T08:13:51.1601242Z    at com.kms.katalon.console.application.Application.runConsole(Application.java:71)

2020-05-01T08:13:51.1602381Z    at com.kms.katalon.core.application.Application.runConsole(Application.java:99)

2020-05-01T08:13:51.1603406Z    at com.kms.katalon.core.application.Application.start(Application.java:74)

2020-05-01T08:13:51.1604523Z    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)

2020-05-01T08:13:51.1605776Z    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)

2020-05-01T08:13:51.1607007Z    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)

2020-05-01T08:13:51.1608219Z    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)

2020-05-01T08:13:51.1609212Z    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)

2020-05-01T08:13:51.1610161Z    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

2020-05-01T08:13:51.1611126Z    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

2020-05-01T08:13:51.1612136Z    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2020-05-01T08:13:51.1613180Z    at java.lang.reflect.Method.invoke(Method.java:498)

2020-05-01T08:13:51.1614065Z    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673)

2020-05-01T08:13:51.1615351Z    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610)

2020-05-01T08:13:51.1616317Z    at org.eclipse.equinox.launcher.Main.run(Main.java:1519)

2020-05-01T08:13:51.1617354Z Caused by: com.kms.katalon.execution.exception.InvalidConsoleArgumentException: Test suite 'Test' not found.

2020-05-01T08:13:51.1618921Z    at com.kms.katalon.execution.console.entity.TestSuiteLauncherOptionParser.throwInvalidTestSuiteIdException(TestSuiteLauncherOptionParser.java:335)

2020-05-01T08:13:51.1620780Z    at com.kms.katalon.execution.console.entity.TestSuiteLauncherOptionParser.getTestSuite(TestSuiteLauncherOptionParser.java:330)

2020-05-01T08:13:51.1622183Z    at com.kms.katalon.execution.console.entity.TestSuiteLauncherOptionParser.buildRunConfig(TestSuiteLauncherOptionParser.java:258)

2020-05-01T08:13:51.1623740Z    at com.kms.katalon.execution.console.entity.TestSuiteLauncherOptionParser.getConsoleLauncher(TestSuiteLauncherOptionParser.java:237)

2020-05-01T08:13:51.1624852Z    ... 17 more

2020-05-01T08:13:51.1625142Z

l

I haven’t used this arg before, but…

$(Build.BuildId) --> ${Build.BuildId}

that’s {} not ()

Sorry - I’ve taken that from AzureDevOps that will be replaced at runtime.

This?

-testSuitePath="Test Suites/TestSuite1/v1"

@Russ_Thomas A* for you :slight_smile:

Anyone else struggling with this both -projectPath and -testSuitePath need to be in double quotes… so this is correct:

-noSplash -runMode=console -projectPath="[ProjectPathToMy.prj file]" -retry=1 -testSuitePath=“Test Suites/TestSuite1/v1” -executionProfile=Default -browserType=“Chrome” -reportFolder=“Reports/$(Build.BuildId)” -reportFileName=“report” -apiKey= XXXX

1 Like

Hi I am seeing this issue couldn’t fix it I am using double quotes