Running Cucumber Test Suite from Command Line

Hello,

I’m currently facing the following problem when running katalon tests with cucumber: When I run my test suite from Katalon Studio GUI, it’s fine. When I run the same test suite from Windows cmd, the suite failes with the following error message:

Full Name: 	ReportTestSuite/Test Cases/FeatureTester
Start / End / Elapsed: 	2019-02-15 12:22:43.122 / 2019-02-15 12:22:46.229 / 00:00:03.107
Status: 	FAILED
Message: 	Test Cases/FeatureTester FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Keyword runWithCucumberRunner was failed (Root cause: groovy.lang.MissingMethodException: No signature of method: static com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed() is applicable for argument types: (java.lang.String) values: [These following reason:
 [Not a file or directory: C:\Users\paulschmidt\Documents\100_Projekte\101_Test-Tool\bdd\Include\features\login.feature]]
Possible solutions: stepFailed(java.lang.String, com.kms.katalon.core.model.FailureHandling), stepFailed(java.lang.String, com.kms.katalon.core.model.FailureHandling, java.lang.String), stepFailed(java.lang.String, com.kms.katalon.core.model.FailureHandling, java.lang.String, java.util.Map))
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:36)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:56)
	at com.kms.katalon.core.keyword.internal.KeywordMain$runKeyword.call(Unknown Source)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords.runWithCucumberRunner(CucumberBuiltinKeywords.groovy:257)
	at com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords$runWithCucumberRunner.call(Unknown Source)
	at FeatureTester.run(FeatureTester:20)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:328)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:319)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:298)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:290)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:224)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:129)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:112)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:81)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:149)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1550229757123.run(TempTestSuite1550229757123.groovy:36)

So basically the expected paths to the feature-file differ when executing from cmd and gui.

On Katalon the expected path is:
C:\J\Include\features\login.feature

But the actual path is
C:\J\cucumber\Include\features\login.feature

The cmd skript is created by the Katalon Command Builder as described here: https://docs.katalon.com/katalon-studio/docs/console-mode-execution.html

katalon -runMode=console -projectPath="C:\Users\paulschmidt\Documents\100_Projekte\101_Test-Tool\bdd\cucumber\katalon-bdd-cucumber-tests.prj" -retry=0 -testSuitePath="Test Suites/ReportTestSuite" -executionProfile="default" -browserType="Chrome"

The structure is the following:
I created a TestSuite named xyTestSuite.

This test suite is calling a test case FeatureTester.
CucumberKW.runWithCucumberRunner(FeatureRunner.class,FailureHandling.STOP_ON_FAILURE)

The FeatureTester is running a FeatureRunner. The FeatureRunner is running some Features.

@RunWith(Cucumber.class)
@CucumberOptions(features=["Include/features/login.feature","Include/features/subscribtions","Include/features/dashboard"],glue="",plugin=["pretty","junit:Reports/Cucumber.xml","html:Reports","json:Reports/Cucumber.json"])
public class FeatureRunner {
}

Having the FeatureRunner as above, i can start the project from GUI without errors in the report, but not from cmd using the following command.

katalon -runMode=console -projectPath=“C:\J\cucumber\katalon-bdd-cucumber-tests.prj” -retry=0 -testSuitePath=“Test Suites/ReportTestSuite” -executionProfile=“default” -browserType=“Chrome”`

When I change the FeatureRunner as below, I can launch it from cmd without errors but not from the GUI.

@RunWith(Cucumber.class)
@CucumberOptions(features=["cucmber/Include/features/login.feature","cucmber/Include/features/subscribtions","cucmber/Include/features/dashboard"],glue="",plugin=["pretty","junit:Reports/Cucumber.xml","html:Reports","json:Reports/Cucumber.json"])
public class FeatureRunner {
}

So currently I can make it either run from cmd or from the katalon gui. It seems to be a bug, or does someone have a suggetion on that?

Added code snippets and moved to bug reports

Hi @Paul_Schmidt

I reproduced and confirmed this is a bug. Added into our back log and we will fix it in the upcoming releases.

Regards !

Thank you @ThanhTo. I have not found the original bug report.

Thank you for the report. This has been fixed in version 6.1. Please follow here.