I have been using Docker to execute a Test Suite Collection of around 30 tests. Predictably, as the number of tests in the collection have grown the execution time of the collection has increased dramatically. This largely seems to be due to:
- Each test suite getting its own instance of chromedriver that occurs at the beginning of the process, resulting in having to wait for 30 instances to open regardless of how many will be executed at a given time. It would be great if only the instances needed to start running the tests were spun up and future ones as needed, as this can take 3 minutes on its own.
- Performance of the test suites seems to also be slow, with execution times taking 2-4x longer that just running the test in Katalon.
I do not have a ton of experience with docker, so does anyone have any general tips that they have found help with expedited the run process, or settings as far as number of tests run simultaneously. In case it is helpful, the docker run command I use is below.
docker run -t --rm --shm-size=4g --env-file envfile -v %cd%:/tmp/project katalonstudio/katalon katalonc.sh -projectPath="/tmp/project" -browserType=“Chrome” -retry=0 -statusDelay=30 -testSuiteCollectionPath=“Test Suites/Automation/Search And Verify 1” -apikey=theKey -executionProfile=“dev” -retryFailedTestCases=true
Thank you!
James