My question is how can i count the scenarios or even make the scenarios be read as test case like on QAF , Because on QAF each scenario is counted as one test
Looking into Katalon’s documentation, there’s a way to use CucumberKW.runFeatureFileWithTags() which allows specifying tags. So instead of running the entire feature file, the test case can run a specific scenario by its tag. By parameterizing the tag, the test case can be reused for each scenario.
Therefore, the solution involves parameterizing the test case with scenario tags and creating a data-driven test suite that runs the test case for each tag. This should result in each scenario being treated as a separate test case in the report.
To achieve scenario-level reporting in Katalon Studio , let’s try to use this workaround:
Solution Architecture
Feature File (Scenarios) → Test Suite (Data-Driven) → Individual Scenario Execution → Separate Reports
Step-by-Step Implementation
A. Modify Your Feature File
Add unique tags to scenarios:
@scenario1
Scenario: First Scenario
When I call "firstRequest"
@scenario2
Scenario: Second Scenario
When I call "secondRequest"
B. Create a Parameterized Test Case
Create a new test case (e.g., BDD_Scenario_Runner):
groovy
// Test Case Script (Groovy)
def tag = (String) CustomKeywords.'com.kms.katalon.core.testobject.ResponseData.getKeywordTestProperty'('tag')
CucumberKW.runFeatureFileWithTags('Include/features/myFeature.feature', tag)
Add a “tag” variable in the test case’s Variables tab.
C. Create a Data-Driven Test Suite
Create a test suite
Add a data file (CSV/Excel) with scenario tags:
tag
@scenario1
@scenario2
Configure the test suite to iterate through rows:
Add BDD_Scenario_Runner test case and bind the “tag” parameter:
Execution & Reporting
Before:
Test Suite → 1 Test Case → 2 Test Steps
After:
Test Suite → 2 Test Cases (1 per scenario)
Alternative Approach: Manual Test Case Creation
If you have few scenarios, manually create test cases for each tag:
groovy
// Test Case for Scenario 1
CucumberKW.runFeatureFileWithTags('Include/features/myFeature.feature', '@scenario1')
// Test Case for Scenario 2
CucumberKW.runFeatureFileWithTags('Include/features/myFeature.feature', '@scenario2')
Key Advantages:
Accurate Scenario Count: Each scenario appears as separate test case in reports
Traceability: Directly map failed tests to specific scenarios
Data-Driven Scaling: Easily add new scenarios via CSV updates
Limitations to Note:
Requires Katalon Studio 8.0+ for full tag-filtering support
Welcome to our community. Thank you for sharing issue.
In Katalon Studio, when you run a feature file using CucumberKW.runFeatureFile(), all scenarios within the feature file are treated as test steps rather than individual test cases. This is why the test report only logs a single test case instead of separate scenarios.
I would like to suggest some of the following workarounds here:
1. Run Each Scenario as a Separate Test Case
Instead of calling CucumberKW.runFeatureFile() at the test case level, use CucumberKW.runFeatureFileWithTags(), which allows running individual scenarios separately.
This way, each scenario will be executed as an individual test case.
2. Use Test Listeners to Modify Reporting
You can create a Test Listener to modify how results are logged, ensuring each scenario appears as a separate test case in reports. However, this requires custom handling.
3. Use a Different BDD-Compatible Framework
Since QAF treats each scenario as a test case, you may consider using QAF with Katalon or an alternative framework like Karate or SpecFlow, which natively report scenarios as test cases.
thank you for your response, some of the images are broken , so the work around is on the test case and test suite level, i was wondering if i can somehow override the reports using test listeners
i was wondering how can i do your second suggestion of using Test Listeners to Modify Reporting, can you point me to some resources or guide me on how i can do this, i have done my research but what i have come up is creation of custom reports but not the modification, it would be nice if i could just modify the default reports generated by katalon, since it would also save execution time
Greetings to all testing and quality assurance specialists!
Today I want to discuss the topic of generating reports similar to QA (Quality Assurance Framework). Have any of you already encountered the creation of such reports or use them in your work?
Please share your experience.:
What tools or platforms do you use to generate reports?
What key metrics and data do you include in your reports?
How can I automate the reporting process to save time?
It is also interesting to know how you visualize data (graphs, charts) and what patterns you use for ease of perception.
He reported that he created a custom report of the productional quality for him in Katalon Studio. As far as I am aware, in this forum, he is the only one who managed it.
reports also include cucumber reports
which is similar to what you made, it contains the test run details as well as the steps, my goal was to override this so that i can use it for my custom reports , any luck with this?
Hi everyone, in the end i ended up making my own reporting,
i found a json file with the scenario details and worked on from there to create my test listener
Solution Architecture
Step-by-Step Implementation
Execution & Reporting
Alternative Approach: Manual Test Case Creation
Accurate Scenario Count: Each scenario appears as separate test case in reports
Requires Katalon Studio 8.0+ for full tag-filtering support
