I have two sets of testsuites that have the following constraints:
TSType1 (N number tests) - these TS can only run sequential - technically they are just one test suite with a different data iteration to workaround issues with report generation
TSType2 (O number tests)- these TS can only run sequential - same reason as the other TSType
However, you can run TSType1 test with TSType2 tests in parallel.
Normally, you can think of just running all sequentially. However, to optimize for time, you can technically run TSType1 and TSType2 together. I’m looking at the documentation and I can only see that you either run sequential or parallel.
In my configuration, I have 2 test collections. TCTSType1 and TCTSType2 which are both sequential. At first, I thought I can nest it to make the 2 collections parallel (within the inner collection as sequential) but it seems that you can’t do it(you can only select TS and not TC within a TC).
I’m not sure if I did present my case clearly, so please ask for clarification. Do you guys have recommendations for me to optimize for test execution? The tests needs to have report to indicate the data variation that succeded and failed.
I thought about the 2 KS, but as you mentioned, we need 2 licenses. Before the workaround of splitting data across multiple TS(to improve performance and making sure reports will be generated), the solution will just be putting the 2 TS together in a test collection (and you make the execution parallel). However, in the splitted data(you spread the execution to multiple Test Suite), I’m not sure how to make sure that there is only one instance of that type running at the same time(because they need to run sequentially).
I know KS/KSE might not get better performance than KRE does because KS/KSE has Log Viewer which could make performance worse. See Log Viewer slows down your tests. How to prevent it?. But I am not sure if @duylong mean that there could be any other factors (other than Log Viewer) that makes KS/KSE slower than KRE.
That was the idea before, to have both test suite (TSType1 and TSType2) in a test collection as a parallel test. However, I have to split the data of TSType1 to multiple test suites (thus, it can run in parallel in a test collection which will be forbidden-if I enable parallel execution in test collection). Same is true for TSType2. It is a single test suite split into multiple test suites(so you can run data in Data Files in sections so you can get a report after finishing N data items).
I’m not sure if there is a thread affinity type of feature where you can set a type of tests to let say 1, 2 (if you set your max. concurrency to 2). I have the misfortune of splitting the data of a data files to multiple suites so KS can run faster (yes, splitting data of a big data file in a test suite, e.g. 20K entries in CSV-split the test suite so TS1:1-500, TS2: 501-100, etc.)
Perhaps you want to write a small Groovy script (or shell script with grep or awk) that splits the source data file into pieces (e.g. 20K entries in CSV-split the test suite so TS1:1-500, TS2: 501-1000, … etc as you wrote) before running the Test Suite Collection. Is it difficult to implement?