Hi Guys,
I hope you’re all doing well in this pandemic.
Please take time to read. . . . 
I’ll be sharing one of the tricks that I think will help some or many of you in reading your test execution results. So what is this topic about? As we know the best practice in automation, mainly in Katalon is, when you have a test case that can cover many scenarios, those scenarios can be written and defined in a test data so that you won’t need to create another test case, meaning we are avoiding duplicate test steps because its not a good practice. But the problem is, when we execute a test case in a test suite that has many scenarios, Katalon will log only the test case ID in the test execution reports.
For example, I have a “Login_TestCase” and it has an excel test data with 20 rows, those rows defines the login test scenario. Below are my sample excel data login scenarios:
Assume that our “Login_TestCase” is already bound to our test data and we executed our test suite and Katalon already generated the report.
So now we have the report, as we expect, there are 20 login test case listed in the report since we iterate all the rows in our test data. But the question here is, how can we easily identify the executed test data scenario specially if we have a failure? Or what if you have a hundred rows in your test data and in the execution reports you will also have a hundred of test case id with the same name, how can we easily identify the failure in test data? And I think Katalon still doesn’t support dynamic test case naming. → SOURCE
WORK AROUND
In the image above, the excel and the suite results, as you notice the row numbering is not quite in-sync due to the excel’s first row has been occupied by column and it is not counted as row data. So if we will check its kinda confusing even if the only difference is 1 row. What we are trying to achieve here is the exact identification of the scenario.
The solution we can do for this is to include a scenario ID in your test data, meaning in your test data you will have an extra column for that scenario id, and that scenario id should be unique for every row.
So how this scenario ID works and how we can see it in our results? Scenario ID’s can be called in test case / listener.
First, you need to have a variable in your test case to call that scenario Id
Second, you need to write this code in your test listener. The code is simply just putting logs. So that the scenario id will be captured in every iteration.
You can also put the code in your test case, just remove the testCaseContext and use only the variable.
Remember to add the KeywordUtil dependency.
import com.kms.katalon.core.util.KeywordUtil
In the test suite, Dont forget to bind your test case to your test data
RESULT CHECKING
In my Login_Suite, there are three test scenarios that fails. Let’s check the execution details in order for us to know the exact row in the test data where it fails.
Now refer to the test data with scenario ID image above. As you can see, it points on the exact row where the scenario fails. Same goes to other failures. And even the passed tests has that scenario id logged in the results.
I only put mine in the test listener for it to have a separate view.
That log is also visible in the html results.
So with that work around even if we have a lot of scenarios in our test data, we will not get lost in finding the scenario where it fails.
So there you have it guys. I hope that this tip/trick can help you in your automation.
Cheers 