Hii Team, first i like to thank the new forum layout is better now
My team has 300s manual test scenario that will be automated using Katalon studio, for now I already created single Test Suite collection that contain 10 Test suite and run it in parallel. one test suite is for one test scenario. I’m creating component test scenario inside test case so in test suite designer I just arrange the test case block sequentially
I’m using test listener @BeforeTestSuite to open and @AfterTestSuite to close browser, so it will open 10 chrome headless browser when running test suite collection, and run around 10-15 test case inside each test suite.
The problem is when running Katalon in Jenkins server (Ubuntu server) and run it periodically, the result is often not consistent each build, flaky in many test cases (different error across build), often found Selenium stale element exception. but run more stable at my local machine (Mac mini), I wonder it maybe the machine resource is exhausted when running Katalon.
So the my question is how to arrange the test running strategy in proper way?
- Is open 10 browser in parallel is too many?
- What is preferable way to open browser each test suite or each test case running?