Since the launch of True platform as the alternative to TestOps, we have been facing issues with Parallel test collection execution. We are getting curl 56 error.
We typically schedule a test collection consisting of 12 test suites in parallel mode, with the instance count set to 15 and an instance delay of 100 seconds. This setup worked reliably in TestOps. However, in TruePlatform, we are encountering execution issues—only 3–4 suites run successfully, while the remaining 8–9 suites get interrupted.
We consistently receive errors such as “RPC failed; curl 56 Recv failure”, as shown in the image below.
The above issue is very common and usually by executing some shell commands it gets sorted out, but in the NEW UI we have tried many times and tried various permutation and combinations, but result is still the same.
Thanks for the detailed explanation—this helps a lot.
Based on the error logs (RPC failed; curl 56 Recv failure, early EOF), this appears to be related to Git connection resets during repository cloning, rather than the test execution itself.
With TruePlatform, parallel execution is more aggressively distributed across agents, which can result in multiple simultaneous repository clone operations. This may lead to network saturation or throttling from the Git server (Bitbucket), causing intermittent failures like the ones you’re seeing.
To help mitigate this, I’d recommend trying the following:
Reduce the instance count (e.g., from 15 to 6–8) and observe if stability improves
Configure Git on the agent machines with increased buffer:
Use shallow clone (--depth 1) if applicable to reduce payload size
Check if Bitbucket has any rate limits or connection throttling in place
Optionally, consider reusing a pre-cloned repository on agents to avoid repeated cloning under parallel load
This behavior is typically seen under high concurrency when multiple clone operations happen simultaneously, so tuning parallelism and network load usually resolves it.
@athul.kannan I already posted in my query that I have done with all 4 points which you have mentioned and even more things, but yes 5th one I haven’t tried, will try.
I checked with my network team as well, they assured that nothing is getting blocked from them.
But still →
My question is still the same, when me and my team was able to schedule and execute the suites in collection in parallel mode in the OLD UI (i.e. TestOps) with the same configuration, then why we are not able to do the same with the NEW UI (i.e. TRUE Platform) .
I have raised ticket in the Katalon support portal as well, where I have submitted logs and the link of my collection along with some more details. Let’s see what they have to say, I go message from that they are investigating.
Just stop using the Parallel execution of Test Suite Collection. This is my advice to @arvind.choudhary.
The parallel execution is always fragile due to various resource constraints (cpu, memory, network, counter-part’s performance, etc). The parallel execution does not tune itself automatically. It runs as you configured before you start it. It tends to fail easily. When failed, all you can do is to guess the reason, tune the configuration manually, and bet again. You will keep losing bets endlessly.
I suppose that Katalon just wanted to ornament their marketing brochures saying “Yes, you can run your Test Suites in a Collection in parallel”, but I am afraid that Katalon doesn’t mind how long the users would waste their time on struggling with the fragility.
I think you wanted to save your time by introducing the parallelism — if you could, your tests would have run much quicker. But, in fact, you are blocked by this issue, and you are wasting your time and efforts. Unfortunate enough. You had better pass the issue back to Katalon, let them work on it. You should not bother yourself any more.
Sure @kazurayam Thanks for your suggestion, I too feel that parallel execution has some issue in katalon and to get back to the sequential mode only. I wasted enough time on finding solution, now finally if you are also saying, it really makes sense to not waste time anymore.
Yes, to save time me and my team thought to go for parallel, but in Katalon Parallel execution is very fragile.
Or, you may try the smaller number of parallelism = 2. If it works, then the total execution duration may be shorted to the half. It would be a big enough gain.
But, you should measure it if the time duration is really shortened. It is likely that the 2 parallel processes work without error but slowly, resulting no different from a sequential execution.
If the 2 processes are distributed to 2 sets of computing resources (AWS EC2 instances, e.g,) then the duration may be shortened. But if 2 processes are instantiated on a single machine, then I guess the 2 processes will fall into a resource contention. In this case, the total duration would be similar to the sequential execution. No gain by parallelism.
I hope you to try the parallelizm = 2 on the True Platform and examine if you get a benefit of shorter duration. If not, it proves that the parallelizim is not valuable at all. Please report the result back here.
Does the True Platform run on an runtime environment which manages the machine resources dynamically, like AWS Lambda or CloudFlare Workers? I don’t know it. Do you have any idea how the True Platform is achitected on a serveless cloud?
On the other hand, I know Katalon’s products are charged for fixed amount of money per month, NOT charged on a pay-for-use basis. Therefore the parallelism of True Platform would certainly be implemented on a limited hardware resources; so I do not expect a large number of parallelizm (say 8) would ever work stable.
I mean, Parallelism in a commercial service will inevitablly involve the pay-as-you-go scheme, otherwise the provider will suffer from the burden made by free-riders. It’s a fundamental issue. I believe, Katalon has not planned a new payment scheme. So, I expect no updates on the stability of their parallelizm Katalon can make.
Thanks for the clarification—and you’re absolutely right to question this since the same configuration worked in TestOps.
The key difference here is not the configuration itself, but how execution is handled under the hood in TruePlatform vs TestOps.
In TestOps, parallel execution was relatively more staggered and centrally orchestrated, which meant Git operations (like cloning the repository) were spread out over time.
In TruePlatform, execution is more distributed and agent-driven, which can result in multiple instances starting at nearly the same time. This leads to simultaneous Git clone operations, increasing the likelihood of connection resets (curl 56, early EOF)—even if the configuration appears identical.
So while the instance count is the same (15), the actual concurrency at the Git/network level is higher in TruePlatform, which explains why the behavior differs.
Since you’ve already tried most optimizations, your approach of raising this with support is absolutely the right step—they’ll be able to validate this behavior deeper from backend logs.
In the meantime, one thing worth testing (purely for validation) would be slightly reducing concurrency or introducing more stagger between executions to see if stability improves. This would help confirm whether this is indeed concurrency-related.
Even though we are still waiting for a response from the Katalon Support team, we have not received any reply so far. Based on our testing and observations, my conclusions regarding parallel execution are as follows:
Parallel execution works reliably with minimal to negligible issues when tests are triggered directly from Katalon Studio on a local machine.
Parallel execution on the older TestOps UI was generally stable when executed or scheduled on a VM server using the Katalon Agent. While occasional issues occurred, they were manageable, and the old TestOps UI provided reasonable support for parallel execution.
True Platform appears to have significant challenges with parallel execution. Before the launch of the True Platform, we were successfully scheduling parallel executions in TestOps using the Katalon Agent (for example, 4 collections with 8–10 test suites). However, since the introduction of the True Platform, we have been forced to revert to sequential execution mode due to consistent parallel execution failures.
Hope the issue will be resolved or they will provide us the reason what is going wrong, we have provided them log files and collection link
Only the amazing part is that we were able to schedule parallel execution mode in old UI only since the time NEW UI came, not for single time it picked up.
So I feel there is some issue in the NEW UI
