[Cucumber] Run with Cucumber Runner


This is a companion discussion topic for the original entry at https://docs.katalon.com/katalon-studio/docs/cucumber-kw-run-cucumber-runner.html

I am using 7.2 version of Katalon, I tried to execute BDD with Cucumber Runner. I am facing some difficulty while executing TC with below statement.

CucumberKW.runWithCucumberRunner(CucumberRunner.class)

Kindly let me know if this feature is not working in 7.2 version

Please share us your difficulties. You can also refer to our doc:

This keyword is not working in 7.2.6 version, Please let me know how to resolve it.
Please find the error message

03-12-2020 03:15:59 PM Test Cases/BDDTests/CucumberTC

Elapsed time: 0.439s

Test Cases/BDDTests/CucumberTC FAILED.
Reason:
groovy.lang.MissingPropertyException: No such property: CucumberRunner for class: Script1583939058141
at CucumberTC.run(CucumberTC:23)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1584006356540.run(TempTestCase1584006356540.groovy:23)

You need to declare your own CucumberRunner.class

Thank you for your quickest reply.

I have added my own Cucumber runner, Path is like

Include/scripts/(own package is created)/keyword named as CucumberRunner
please find the code inside it.

package com.cucumber.testrunner
import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.checkpoint.Checkpoint
import com.kms.katalon.core.checkpoint.CheckpointFactory
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testcase.TestCase
import com.kms.katalon.core.testcase.TestCaseFactory
import com.kms.katalon.core.testdata.TestData
import com.kms.katalon.core.testdata.TestDataFactory
import com.kms.katalon.core.testobject.ObjectRepository
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

import internal.GlobalVariable

import org.openqa.selenium.WebElement
import org.openqa.selenium.WebDriver
import org.openqa.selenium.By

import com.kms.katalon.core.mobile.keyword.internal.MobileDriverFactory
import com.kms.katalon.core.webui.driver.DriverFactory

import com.kms.katalon.core.testobject.RequestObject
import com.kms.katalon.core.testobject.ResponseObject
import com.kms.katalon.core.testobject.ConditionType
import com.kms.katalon.core.testobject.TestObjectProperty

import com.kms.katalon.core.mobile.helper.MobileElementCommonHelper
import com.kms.katalon.core.util.KeywordUtil

import com.kms.katalon.core.webui.exception.WebElementNotFoundException

import cucumber.api.java.en.And
import cucumber.api.java.en.Given
import cucumber.api.java.en.Then
import cucumber.api.java.en.When

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(features=“Include/features”, glue=" ", plugin = [“pretty”,
“junit:D:/BDD_Reports/cucumber.xml”,
“html:D:/BDD_Reports”,
“json:D:/BDD_Reports/cucumber.json”])

public class CucumberRunner {
}

Actually I am facing issue when I created a Test case and using the cucumber key word “run with cucumber runner”

CucumberKW.runWithCucumberRunner(CucumberRunner.class)

getting issue with CucumberRunner.class

@m.sivakrishna

You need to import: com.cucumber.testrunner package in the test case script.

import com.cucumber.testrunner

Thank You, Compilation error is gone but unable to run it and getting below error.

Test Cases/BDDTests/CucumberTC FAILED.
Reason:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
file:/C:/Users/msivakrishna/Katalon%20Studio/DemoTest/Scripts/BDDTests/CucumberTC/Script1583939058141.groovy: 17: unable to resolve class com.cucumber.CucumberRunner
@ line 17, column 1.
import com.cucumber.CucumberRunner
^

1 error

at com.kms.katalon.core.main.ScriptEngine.getScript(ScriptEngine.java:199)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1584010286837.run(TempTestCase1584010286837.groovy:23)

Kindly help me on this

You need to import:

import com.cucumber.testrunner.CucumberRunner

or press Ctrl + Shift + O on your test script

Wonderful, Thank you so much for your help.

how can we add tags attribute in Cucumber runner, Could you please suggest. Please find the code I am using but getting error at tags attribute.

package com.cucumber.testrunner
import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.checkpoint.Checkpoint
import com.kms.katalon.core.checkpoint.CheckpointFactory
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testcase.TestCase
import com.kms.katalon.core.testcase.TestCaseFactory
import com.kms.katalon.core.testdata.TestData
import com.kms.katalon.core.testdata.TestDataFactory
import com.kms.katalon.core.testobject.ObjectRepository
import com.kms.katalon.core.testobject.TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

import internal.GlobalVariable

import org.openqa.selenium.WebElement
import org.openqa.selenium.WebDriver
import org.openqa.selenium.By

import com.kms.katalon.core.mobile.keyword.internal.MobileDriverFactory
import com.kms.katalon.core.webui.driver.DriverFactory

import com.kms.katalon.core.testobject.RequestObject
import com.kms.katalon.core.testobject.ResponseObject
import com.kms.katalon.core.testobject.ConditionType
import com.kms.katalon.core.testobject.TestObjectProperty

import com.kms.katalon.core.mobile.helper.MobileElementCommonHelper
import com.kms.katalon.core.util.KeywordUtil

import com.kms.katalon.core.webui.exception.WebElementNotFoundException

import cucumber.api.java.en.And
import cucumber.api.java.en.Given
import cucumber.api.java.en.Then
import cucumber.api.java.en.When

import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(features=“Include/features”, glue="", tags={"@login","@Products"} plugin = [“pretty”,
“junit:Reports/cucumber.xml”,
“html:Reports”,
“json:Reports/cucumber.json”])

public class CucumberRunner {
}

Could you please reply

Hi, is there any way to generate reports using Serenity BDD? By doing @RunWith(CucumberWithSerenity.class)?

I’ve added the depencies using gradle but I’m still getting some errors.

I’m facing issue with running the feature file with cucumber runner. The feature file is executed fine by using test suite. I’m following the docs instructions of simple example #1 to create a cucumber runner, but the execution failed. Thank you for any help:

04-29-2020 02:21:22 PM Test Cases/operations/MyRunner

Elapsed time: 0,676s

Test Cases/operations/MyRunner FAILED.
Reason:
java.lang.IllegalArgumentException: argument type mismatch
at MyRunner.run(MyRunner:18)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:105)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1588162880073.run(TempTestCase1588162880073.groovy:25)

@m.sivakrishna I faced the same issue with exactly the same errors. My problem was that when I created the CucumberRunner class it had groovy extension not java. Make sure that you create the class as a new java class

hello, is there a way to pass other properties than glue such as threads for example ?

I would like to emit this type of cucumber test :

java -cp .;<path to cucumber jar folder>/* io.cucumber.core.cli.Main --threads 4 -g parallel parallel

Hi, I am also facing similar issue. I have created feature file, Runner class, Step definition.

Yet I face the below errors due to the runWithCucumberRunner

import testClass.DummyClassMag
import com.cucumber.testrunner
CucumberKW.runWithCucumberRunner(DummyClassMag.class)

RunnerClass file name: DummyClassMag.java

package testClass;
import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
@CucumberOptions(features="Include/features/Test_Login.feature", glue="", tags="@RegressionTestMag", plugin={"pretty","html:target/cucumber-html-report","json:target/cucumber.json","pretty:target/cucumber-pretty.txt","junit:target/cucumber-results.xml"})

public class DummyClassMag {
}

hello @duyluong,
could you say me which cucumber version is included in katalon ?
I suspect katalon use an old version of cucumber libraries :
image

Is it possible to update io.cucumber.core version to a new one ?

I tried to use --threads option of v4.0.0 but I got error “cucumber.runtime.CucumberException: Unknown option: --threads”

=============== ROOT CAUSE =====================
Caused by: cucumber.runtime.CucumberException: Unknown option: --threads

For trouble shooting, please visit: https://docs.katalon.com/katalon-studio/docs/troubleshooting.html
================================================

01-10-2022 02:10:52 PM Test Cases/Cucumber - FM Parallelised test - Launch Test

Elapsed time: 1,239s

Test Cases/Cucumber - FM Parallelised test - Launch Test FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Keyword runFeatureFile was failed (Root cause: cucumber.runtime.CucumberException: Unknown option: --threads
	at cucumber.runtime.RuntimeOptions.parse(RuntimeOptions.java:180)
	at cucumber.runtime.RuntimeOptions.<init>(RuntimeOptions.java:112)
	at cucumber.runtime.RuntimeOptions.<init>(RuntimeOptions.java:105)
	at cucumber.runtime.RuntimeOptions.<init>(RuntimeOptions.java:97)
	at cucumber.api.cli.Main.run(Main.java:30)
	at cucumber.api.cli.Main$run.call(Unknown Source)
	at test.common.CucumberFmRunner$_runFmParallelFeatureFile_closure1.doCall(CucumberFmRunner.groovy:127)
	at test.common.CucumberFmRunner$_runFmParallelFeatureFile_closure1.doCall(CucumberFmRunner.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:74)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordMain$runKeyword.call(Unknown Source)
	at test.common.CucumberFmRunner.runFmParallelFeatureFile(CucumberFmRunner.groovy:83)
	at test.common.CucumberFmRunner$runFmParallelFeatureFile.callStatic(Unknown Source)
	at test.common.CucumberFmRunner.runFmParallelFeatureFile(CucumberFmRunner.groovy:78)
	at Cucumber - FM Parallelised test - Launch Test.run(Cucumber - FM Parallelised test - Launch Test:9)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:442)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:433)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:412)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:404)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:281)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:138)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:129)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1641820250402.run(TempTestCase1641820250402.groovy:25)
)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:50)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:76)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:68)
	at com.kms.katalon.core.keyword.internal.KeywordMain$runKeyword.call(Unknown Source)
	at test.common.CucumberFmRunner.runFmParallelFeatureFile(CucumberFmRunner.groovy:83)
	at test.common.CucumberFmRunner$runFmParallelFeatureFile.callStatic(Unknown Source)
	at test.common.CucumberFmRunner.runFmParallelFeatureFile(CucumberFmRunner.groovy:78)
	at Cucumber - FM Parallelised test - Launch Test.run(Cucumber - FM Parallelised test - Launch Test:9)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:442)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:433)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:412)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:404)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:281)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:138)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:129)
	at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
	at TempTestCase1641820250402.run(TempTestCase1641820250402.groovy:25)
Caused by: cucumber.runtime.CucumberException: Unknown option: --threads
	at cucumber.runtime.RuntimeOptions.parse(RuntimeOptions.java:180)
	at cucumber.runtime.RuntimeOptions.<init>(RuntimeOptions.java:112)
	at cucumber.runtime.RuntimeOptions.<init>(RuntimeOptions.java:105)
	at cucumber.runtime.RuntimeOptions.<init>(RuntimeOptions.java:97)
	at cucumber.api.cli.Main.run(Main.java:30)
	at cucumber.api.cli.Main$run.call(Unknown Source)
	at test.common.CucumberFmRunner$_runFmParallelFeatureFile_closure1.doCall(CucumberFmRunner.groovy:127)
	at test.common.CucumberFmRunner$_runFmParallelFeatureFile_closure1.doCall(CucumberFmRunner.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:74)
	... 17 more

For information, I tried to rewrite CucumberKW.runFeatureFileWithTag function in order to add this “–threads” parameter but I am stucked at this error :

public class CucumberFmRunner extends BuiltinKeywords {
	
	private static final String CUCUMBER_PLATFORM_NAME = "cucumber";

	private static final KeywordLogger logger = KeywordLogger.getInstance(CucumberFmRunner.class);


	public static List GLUE = [""]

	/**
	 * Runs the given Feature file with <code>featureId</code> by invoking
	 * {@link cucumber.api.cli.Main#run(String[], ClassLoader)}.
	 * If your step definitions are in packages, you can set glue path to help system can detect your step definitions path easier.
	 * </p>
	 * The generated reports will be extracted in the current report folder with the following path: <code>&lt;report_folder&gt;/cucumber_report/&lt;current_time_stamp&gt;<code>
	 * </p>
	 *
	 * Examples:
	 * <ul>
	 * <li>Example #1: Run a single feature file
	 * <pre>
	 * import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
	 * import com.kms.katalon.core.model.FailureHandling
	 *
	 * CucumberKW.runFeatureFile('Include/features/New Feature File.feature', FailureHandling.STOP_ON_FAILURE)
	 * </pre>
	 * </li>
	 * <li>Example #2: Run a single feature file and the step definitions is in pre-defined packages
	 * <pre>
	 * import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
	 * import com.kms.katalon.core.model.FailureHandling
	 *
	 * CucumberKW.GLUE = ['mypackage1', 'mypackage2']
	 * CucumberKW.runFeatureFile('Include/features/New Feature File.feature', FailureHandling.STOP_ON_FAILURE)
	 * </pre>
	 * </li>
	 * </ul>
	 *
	 * @param relativeFilePath
	 * relativeFilePath of Feature file
	 * @param flowControl
	 * an instance {@link FailureHandling} that controls the running flow
	 * @return
	 * an instance of {@link CucumberRunnerResult} that includes status of keyword and report folder location.
	 *
	 * @since 5.7
	 * @see CucumberRunnerResult
	 */
	
	@Keyword
	public static CucumberRunnerResult runFmParallelFeatureFile(int threadNumber, String relativeFilePath,  String[] tags) {
		return runFmParallelFeatureFile(threadNumber, relativeFilePath, tags, RunConfiguration.getDefaultFailureHandling());
	}
	
	@Keyword
	public static CucumberRunnerResult runFmParallelFeatureFile(int threadNumber, String relativeFilePath,  String[] tags, FailureHandling flowControl) {
		return KeywordMain.runKeyword(CUCUMBER_PLATFORM_NAME, {
			if (StringUtils.isEmpty(relativeFilePath)) {
				throw new IllegalArgumentException("relativeFilePath param must not be null or empty")
			}
			String reportDir = RunConfiguration.getReportFolder() + "/cucumber_report/" + System.currentTimeMillis()
			String projectDir = RunConfiguration.getProjectDir()
			RunningMode runningMode = RunConfiguration.getRunningMode()

			logger.logInfo(
					MessageFormat.format("Starting run keyword runFeatureFile: ''{0}'' and extract report to folder: ''{1}''...",
					relativeFilePath, reportDir))
			logger.logDebug('Glue: ' + GLUE)
			
			String[] argv = []
			for (g in GLUE) {
				argv += ['-g', g]
			}
			argv += [
				projectDir + "/" + relativeFilePath,
				"--strict",
				//                "--plugin",
				//                "pretty",
				"--plugin",
				"html:" + reportDir + "/html",
				"--plugin",
				"json:" + reportDir + "/cucumber.json",
				"--plugin",
				"junit:"+ reportDir + "/cucumber.xml",
				"--plugin",
				CucumberReporter.class.getName() + ":" + reportDir + "/k-cucumber.json"
			]
			if (tags != null) {
				for (String tag in tags) {
					argv = argv + ["--tags"]
					argv = argv + tag
				}
			}
			//TODO : https://cucumber.io/docs/guides/parallel-execution/
			argv = argv + ["--threads",threadNumber]
			
			if (runningMode == RunningMode.CONSOLE) {
				argv = argv + ["--monochrome"]
			}
			println "argv : $argv"
			boolean runSuccess = Main.run(argv, CucumberFmRunner.class.getClassLoader()) == 0;
			CucumberRunnerResultImpl cucumberResult = new CucumberRunnerResultImpl(
					runSuccess ? 'passed' : 'failed', reportDir)
			if (runSuccess) {
				logger.logPassed(MessageFormat.format("Feature file: ''{0}'' was passed", relativeFilePath));
			} else {
				KeywordMain.stepFailed(MessageFormat.format("Feature file ''{0}'' was failed", relativeFilePath), flowControl)
			}
			return cucumberResult;
		}, flowControl, "Keyword runFeatureFile was failed");
	}

}