I am seeking guidance on how Katalon is intended to be used, as it’s not clear how to structure the tests to get the requires result in the reports.
I’ll start with some background for context:
I have an application (AUT) that users log into and navigate through a series of tiles / menus to start a business process (BP), the business process contains several screens (typically 8-15, some are common across multiple business processes)
Based on this context I expect this to be similar to many other applications that others are testing.
Many Test Case Model:
I create separate test cases for login, logout, menu navigation, and individual BP screens I then sequence these test cases in a test suite and execute the test suite, I get a report showing 45 test total, 43 passed, 2 failed etc. So far all good.
Now in my AUT, there are a variety of roles, so BP’s can be run by all roles, some BP’s by selected roles, for this simple example we are using a BP that can be run by all roles, so I set up a data sheet (csv) with a list of logins and roles, I create a new Test Data item in the Data Files section, then using the data binding in the test suite I configure my login test case to use the login names and passwords and execute the test suite again expecting the test cases to be executed sequentially for each login but what actually happens is the login test case is executed once for each login and then the other test cases are executed once using the last login…
… ah, so maybe I need to use one big test case?
One test case model:
I create a new “Parent” test case and call all the smaller individual test cases from the many test case model, this is easy to set up with the call test case test step.
I then create a test suite and add this test case to it, I can easily link the test data I already created and pass the login details, I can see the whole BP will be executed for each login
I configure to run with the first row to start with, I execute the test case and look at the report:
1 test total, 0 passed, 1 failed etc (expected 45 test total, 43 passed, 2 failed etc)
so how many of the test cases that are now test steps failed? I expand the test case but there is no summary of passed/failed test steps for the parent test case
- The One big test case model doesn’t provide the detail in the report that the test team requires.
- The many test case model doesn’t allow the whole BP to be executed for each role.
What is the recommended way to use Katalon test cases / test suites?