Currently I am learning data driven test suite. I have 15 rows of test data and a test suite of three test cases (TC1,TC2,TC3).
When I ran data driven for this test suite, it executed TC1 15 times for the 15 row test data, then TC2 15 times, then TC3 15 times.
However, I wanted to run TC1,TC2,TC3 for the first row, then run them again for the second row, … . Is there a way in Katalon Studio can do that?
The answer is yes, but is there really any functional difference between your two scenarios? In other words, does TC2, iteration 1 depend on the results of TC1, iteration 1? If not, then you’re better off keeping your Test Suite the way it is (I will demonstrate why this is below). If you truly want to run a suite in the way you’ve described, you can accomplish it as follows (I will just use “TC1” as an example, but you will need to perform all of these steps for all TC’s involved):
1.) Pretend we have the following data file and test case:
2.) In TC1, open the variables tab, and enter one variable (and a default value if applicable) for each column in your data file:
3.) Create a new Test Suite and add your test case(s):
4.) At the top right-hand corner of the test suite, click the “Show Data Binding” button:
5.) In the Test Data section, add your test data file by clicking Add, then selecting your data file:
6.) Double click on the Data Iteration cell for your test data. You’ll be prompted with a dialog. In the dialog, select “Run with specific rows” and type “1” in the field and click OK:
7.) In the Variable Binding section, you should already see the variables that we set up for TC1 back in step 2. Click on the first variable binding, then expand “Set Type” and select “Data Column”. Double click on the Test Data cell for a row, and you’ll be prompted to select the appropriate data file from the Test Data section outlined in step 6:
Select your data file and click OK. Then, in the Value cell for that binding, select the appropriate column and click OK:
Do the same thing for the second variable binding, and so on, until you’ve mapped all of the columns from your test data file.
8.) Repeat steps 2-7 for all test cases that you want to share the data file for each iteration.
9.) Re-add your test cases to the suite again, and repeat steps 2-8, and of course be sure to reference the appropriate row in the data file for step 6 each time. Eventually, your test suite would look something like this (again, just using TC1 as an example for time’s sake, you would have TC2 and TC3 in there as well):
Note the Data Iteration column when I highlight each Test Case in the suite.
By now, I hope that you realize that your current solution is better (again, assuming that there are no dependencies between iterations of each of your test cases). This is quite labor intensive. Also, it’s not very friendly to changes! If you wanted to add/remove either test cases or rows from your data file, you would need to adjust this accordingly every time, which isn’t fun. In general, try to think forward when designing your project; in such a way that you limit the amount of maintenance that you will need to do.
Actually, I agree with @Tony3 . It would be very useful for the test suite to run through all test cases in the suite sequentially with row 1 of the test data, then repeat the entire test suite with row 2 of the data.
This allows the creation of modular test cases, e.g. a test case per screen/page of a web site. With a little planning this allows the tests to be sequenced in different orders depending on the end-to-end scenario required. This also greatly reduces test maintenance overheads when there is a UI change because only 1 test case (and perhaps some test data) needs to be modified to correct all test scenarios using that test case.
To keep both @Brandon_Hein and @Tony3 happy, I humbly suggest Katalon offer options with a test suite along the lines of - Read the next row of the data file:
In a scenario with two sequential test cases (e.g. Test 1 and 2) in a test suite using three rows of test data (for customer, A, B and C) this would allow:
Current behaviour - test looping enabled
1A
1B
1C
2A
2B
2C
New behaviour - suite looping enabled
1A
2A
1B
2B
3C
3B
@Tony3 Hi Tony, were you able to figure out a way to do this? I have a similar setup and am trying to figure out how to run a run a test suite using one data row and then moving to the next data row. I dont want to go everytime and readjust all the test cases to the next row.
