A brief information about what we are trying to achieve:
- We have about 2500 scenarios, which have to be covered in about 70 different test suites.
- Each test suite will contain about 5 Test Cases.
- The Object repository would contain about 5000 objects.
- Also, we would have one test suite collection with all these 70 test suites added to it.
This is a two part question:
- Can Katalon have a project with the above configuration?
- Can Katalon execute 70 test suites in parallel?
Regarding parallel execution from a single Test Suite Collection. You can do that, but the overall stability of your execution would suffer, especially if your Test Suites perform complex logic on the same page and are not carefully written in a concurrently safe manner. KS/KSE is intended to be a tool for developing tests.
For true parallel execution, the ideal solution is through multiple instances of Runtime Engine, as they provide you with separate executions. It would also enable you to integrate with other CI/CD systems, since it’s a command-line driven product that is designed for test execution.
Please refer to the FAQ section on our pricing page for more information.
Oh and don’t forget… Katalon is the IDE which helps you do a lot of stuff… executing tests… well, that’s limited to the environment you are running them on. Spin up a server with a single CPU and it will go slowly per test case and I would do it one by one… spin up a cloud with 50 servers each ‘x’ GB of memory and run them in parallel… it’s just about numbers here
I don’t know which company has requirements of running 70 test suites in parallel, that’s outrageous. But you can use agents or virtual machines (I think you need runtime engine license for each VM) to run tests concurrently. Divide up the test suites in each VMs…
Perhaps facebook, google etc …
But i agree with the above opinions. Running tests in parallel at such scale should be a concern of infrastructure, e.g. provide enough agents to be able to handle such task … And if katalon is the choice for the runner, buy enough licences.
Putting such pressure on a single machine will only result in flakiness == irelevant testing == lot of confusions == headaches