Hello All,
Is there a way to upload only failed test reports to Katalon TestOps from CircleCI after the test execution?
Why do you want to exclude successful tests? I am just curious.
@kazurayam , Katalon recently introduced 2000 test reports per month limit in testOps,
We have regression suite for which we keep on adding test cases which runs in CircleCI. Because of this quota limit I cannot see the test results after it crosses the limit. It gets exceeded by mid-month itself. We usually refer to the detailed report only if the test fails. Hence I need it for only failed ones.
“2,000 monthly Test Result”? Are you using “Free” license ?
If you want to use Katalon products more, I think, you need to pay some.
Do you think they would allow you to?
Alternate, you can use the free Standalone KSE/KRE with (under):
- your custom CI tool (jenkins, azure devops, whatever you may like)
- your custom TestCase Management solution (testrail, xray, whatever you may like)
… and upload as many test reports as you like
You can decrease the frequency of regression test runs so that the number of reports to be under 2000 per month, can’t you?
How do you trigger your regression test to run? How do you set up Circle CI?
Do you trigger it by every pushes of commits into the team’s repository? Then, it would be difficult to control the frequency of triggering the regression test under the limit.
You can trigger it by timer ---- once per day, for example. Then you would be able to control the number of reports under the limit, can’t you?
In this case, @yashaswini.sadananda have to deploy a server where he/she maintains TestRail running. This task requires @yashaswini.sadananda’s continuous efforts and charges for the resource.
Katalon will suggest that @yashaswini.sadananda does not need to make it. TestOps will serve him/her for reasonable fee.
I have no idea how competitive TestOps is to TestRail.
Well … ofcourse.
I was bringing this option into discussion just to point why a licence worth the money for most of the use cases.
@yashaswini.sadananda First you will need to setup your own infrastructure, either physical machines or from a certain cloud provider.
So, you will save some money from a licence but you will ‘benefit’ with additional costs and headache to maintain your own setup 
Well, this has actually some other real benefits, e.g if your company security policy does not allow you to use external resources.
But that is no longer a ‘simple use case’ and to go on that path you have to start with the budget analysis.
OK.
I believe that by this schedule change @yashaswini.sadananda has successfully worked around the “2,000 monthly Test Results” problem.
So I would exit from this discussion.
No @kazurayam , It was not changed now, it was already a scheduled run from the start.
Thanks for your response.
Your test seems to be large enough. You would certainly need to purchase the ultimate license of TestOps.
Otherwise, you can schedule the test even less frequently: e.g, once a week. 
Yes @kazurayam , it is large enough and day by day we are adding more tests. You are right, eventually we will have to purchase it .