I have made a GitHub project:
This project proposes a new concept TestClosure for Katalon Studio. This project provides a preliminary implementation and demonstrations.
Problem to solve
Many of Katalon Studio users want to execute multiple test processings parallelly . Me too. They have their own requirements. In my case, I want to take a lot of web page screenshots as quickly as possible.
As I wrote in the README of ExecutionProfilesLoader project, last year I wanted to take screenshots of 6000 URLs. The job took me 6000 * 10 seconds = over 17 hours. Obviously itâs too long. I wanted to make the job run faster.
It is a simple job to take a screenshot of a web page.
Open browser, navigate to URL, take screenshot, save image to file, close browser .
Every time we navigate to a new URL, we are forced to wait for a few seconds until the page is fully loaded. This wait makes our sequential processing very slow.
If have a machine with 8 core-CPU, I can process these 6000 pages with 8 threads, then the job will be done in 17 hours / 8 threads = 2.2 hours.
But how can I do multi-threading in a Test Case of Katalon Studio?
I know that Katalon Studio offers a feature of executing Test Suites in parallel mode. It isnât satisfactory for me. It is too coarse-grained. I want a way to execute a piece of Groovy scripts in fine-grained / light-weighted Java Threads.
Solution
1. You can call Groovy Closure in Katalon Test Case
It is quite easy to create a Groovy Closure that contains WebUI.*
statements in a Test Case. Executing it is a breeze. See the following example.
- code Test Cases/solution/01_callClosure
- video 01_callClosure
When you execute this, you will see a browser window opens, shows a web page, and closes.
2. You can excute Groovy Closures using ExecutorService utility
java.util.concurrent.ExecutorService
makes concurrent programming in Java/Groovy much easier. The ExecutorService
accepts objects that implements the java.util.concurrent.Callable
interface and executes them. A Groovy Closure implements the java.util.concurrent.Callable
interface. So, you can execute Closures of WebUI.*
statements using ExecutorService
utility in a Katalon Test Case.
The following sample code shows 2 Closures are in 2 threads sequantially in Katalon Test Case.
- code Test Cases/solution/02_executeClosuresByExecutorService
- video 02_executeClosuresByExecutorService
When you execute this code, you will see 2 browser windows opened/closed sequentially.
3. You can parametrize Closures
I want to parametrize a Closure. I want to execute 2 instances of a Groovy Closure with different values of parameters. The following code shows how to do it. I made a Groovy Class URLVisitor
which implements the java.util.concurrent.Callable
interface.
When you execute this code, you will see 2 browser windows opened/closed sequentially.
4. You can only see 1 browser window
Now I want to execute 2 Closures simultaneously, in parallel. The following code does that:
The code difference between this and the previous one is only 1 character. The size of Thread Pool is changed: 1 â 4.
ExecutorService exService = Executors.newFixedThreadPool(4)
Of course I would expect to see 2 Browser windows to open.
- code Test Cases/solutions/04_executeCallablesMultiThreadsLayoutUnmanaged
- video 04_executeCallablesMultiThreadsLayoutUnmanaged
When I executed this example, I got puzzled. I executed 2 threads in parallel, and I saw only 1 window opened. why?
What you see is not what you get, sometimes. The fact is, acutally 2 windows were opened by the script, but the windows were displayed overlayed at the same position (x, y coordinate) with just the same dimension (width, height). So I could see only one. This behavior is very confusing.
5. Introduce TestClosure to manage layout of browser windows
I want to manage the layout of browser windows. I want them displayed at different positions (x, y) so that I can see them identifiable. But how can I do it?
WebDriver API provides tools for us to solve this problem. You can explicitly set Window Position and Size. For example,
driver.manage().window().setPosition(new Point(50,200));
and
driver.manage().window().setSize(new Dimension(300,500));
I can should be able to set position and size to the windows opened by WebUI.openBrowser('')
call inside the Closures in Test Cases. The following code shows how I managed it.
- code Test Cases/solutions/05_executeCallablesMultiThreadsLayoutManaged
- video 05_executeTestClosuresMultiThreadsLayoutManaged
I introduced a custom helper BrowserWindow.layout(metrics, position)
that moves the browser window appropriately.
Problems are resolved.
Miscellaneous demo videos
You can view the videos by clicking the following links:
demo video | Test Suite elasped (seconds) | what it does |
---|---|---|
demo1A | 27 | visits 3 URLs sequentially, windows are not managed |
demo1B | 30 | visits 3 URLs sequetioally, windows are managed in Tile layout |
demo1C | 55 | visits 5 URLs sequentially, windows are managed in Stack layout, takes screenshots |
demo2A | 20 | visits 3 URLs simultaneously, windows are overlayed at the same (x,y) position with the same width/height |
demo2B | 21 | visits 3 URLs simultaneously, windows are managed in Tile layout |
demo2C | 48 | visits 5 URLs simultaneously, windows are managed in Stacklayout, takes screenshots |
demo3D | 156 | takes Full Page screenshots of 8 URLs using 2 browsers simultaneously. A browser is reused to process 4 URLs each. Unfortunately WebUI.takeFullPageScreenshot() takes long time (approx. 30 seconds each). Unable to post video for demo. |
Using the following environment:
- Mac Book Air, Processor: 1.6GHz Dual Core Intel Core i5, Memory 16GB
- macOS Big Sur
- Katalon Studio v7.9.1
Caution: videos are compressed
When I executed a Test Suite in Katalon Studio to take the demo2A, the Test Suite acutally took 20 seconds to finish processing. But you will find the âmovieâ plays in 7 seconds. The movie plays far quicker than the acutal scene. I suppose that, while I uploaded the files to Google Drive, the movies were compressed to reduce the size by chomping the still frames off.
Design Description
This project includes a set of working codes. Please read the source to find the detail. The code will tell you everything I could.
Here I write quick pointers:
Entry point
Have a look at a Test Case that opens 3 browser windows simultaneously:
The code is very short. Is it simple? â not really. This short code triggers a lot.
What is TestClosure? How to write it?
Essentially a TestClosure
object is a pair of a Groovy Closure and a list of parameters for the closure. Plus a TestClosure
carries information where the browser window should be located and how it should be sized. The source of TestClosure
is this:
The following Test Case shows an example of creating a list of TestClosure
objects.
How to execute TestClosures in parallel?
The Test Case Test Cases/demo/demo2B_simultaneous_tiling
calles TestClosureCollectionExcecutor
. This object executes the collection of TestClosures
simultaneoutly.
The source code is here:
TestClosuresCollectionExecutor
creates a thread pool of 4 as default. You can change the maxThread by parameter to the Buiilder. The value 1 - 16 is accepted.
2 strategies of window layout
This project provides 2 strategies of browser window layout:
TestClosuresCollectionExecutor
uses the Tiling strategy as default. You can explicitly specify the strategy as a parameter to the Builder.
Multiple threads â does it run faster?
Yes, it does on a machine with 2 or more core-CPU. If my test case executes mutilple TestClosures with multiple threads, it rus faster than with a single thread.
Of course, the faster processing demands more powerful machine resources. If I can afford Mac Book Air M1 with 8-core CPU, my test case with Thread Pool of 8 will run far faster. I want to see it!
It is pointless to give the maximum number of threads for TestClosureCollectionExecutor
with a value larger (8, 16, 32) than the number of cores you have.
Conclusion
I could implement a Test Case that executes multiple Groovy Closures with multiple Threads. The code ran faster on my dual-core machine than with single Thread.
The codes of this project is yet preliminary. It has a lot more to develop. Especially, it does not consider failure handling seriously.