Parallel mode of Test Suite Collection ---- does it make my test to run faster?

I have published a GitHub project:


Problem to solve

Katalon Studio provides a feature Parallel execution of Test Suite Collection. Does it make your test project run faster than the sequential mode?

I did a study about this question. I would present you a data for reference.

Environment

I used Chrome v122, Katalon Studio v9.0.0 with appropriate version of ChromeDriver manually upgraded.

I set the SmartWait ON.

I specified a larger memroy size to Katalon JVM (-Xmx4096m) than the default 2048m.

How to run the exam

Run Test Suites/TSC_parallel and Test Suites/TSC_sequential.

then you will see 3 Chrome windows are opened in parallel:

parallel_TSC

You can see how the Test Suite Collection runs.

A study of speed

Question: How long it takes to run the Test Suite Collection in sequential mode and in parallel mode? Does the parallel mode run faster than the sequential mode?

Speed of Test Cases

I used Chrome browser to run the test cases with.

How long each Test Cases takes to run?

Test Suite seconds Test Case seconds visits which URL?
TS0 24 TC0 23 Kyoto City Official Travel Guide
TS1 16 TC1 14 https://www.esbnyc.com/
TS2 14 TC2 13 Louvre Museum Official Website
TS_012 56 TC0,TC1,TC2

The sum of seconds of TS0 + TS1 + TS2 makes 54 seconds

Speed of Sequential execution

How long the Test Suites/TSC_sequential took to finish executing all 3 Test Cases?

I used my stockwatch device to measure it.

I got the figure: 84 seconds.

Speed of Parallel execution

How long the Test Suites/TSC_parallel took to finish executing the same set?

I set the number of parallel execution to be 3.

I got the figure: 50 seconds.

My Conclusion

I am not very much convinced by the idea that Parallel mode execution of Test Suite Collection will be effective to run my tests faster. So, what’s the benefit of the Parellel mode of TSC? — I don’t see.

1 Like

I noticed that 49 people have cloned this GitHub repository by 3rd March 2024.

4th March, i updated the GitHub repository a lot. The project now measures the duration seconds of Test Cases, Test Suites and Test Suite Collections programatically. It now prints the duration seconds in the Log Viewer. This change would make it easier to see how the project runs overall.

Please fetch the latest updates.

1 Like