I would like to know if there is a test case limit.
1 Like
Yes there are limits to a test case, however, you can have multiple test cases in a Test Suite to overcome this. I will say if you exceed over 300 lines in the Manual tab, then you may encounter a ‘Code Too Large’ error. While I am building my Test Cases, I work in the Script tab, and occasionally I click over to the Manual tab to see where I am at, not to exceed 300 lines . Do NOT change anything on the Manual tab if you work on the Script tab as that will cause major script changes–definitely your code formatting, some things disappear, yikes.
If you work in the Manual tab only, then I would still keep to that 300 line limit and then create another Test Case. If you exceed the 300 lines by a bit, then go ahead and try and run it, but if you get the ‘Code Too Large’ error, then create another Test Case and copy and paste from the Script page to the other Test Case anything in excess of 300 lines. That is just my method and easy to remember.
Also note that you can have multiple lines of code in while loops, for loops, if/else statements and try/catch blocks that can alter the “count” on the Manual tab. In this case, just copy and paste until you can start the Test Case without a ‘Code Too Large’ error.
Edit: I also do not go more than 5 Test Cases within a Test Suite.
1 Like
It is indeed crucial not to overload a test case with too many actions. Reaching or exceeding 300 actions in a test case can lead to issues like the “Code Too Large” error. To avoid this, it is recommended to minimize the number of actions in a test case.
To lighten your main tests and make them more maintainable, it is essential to implement custom keywords, keyword classes, or test case calls. Here are some recommendations to structure your tests more effectively:
- Use custom keywords: Create keywords for repetitive or complex actions. This reduces the number of visible actions in a test and allows for easy code reuse.
- Structure code with classes: Encapsulate common functions within classes. This helps organize the code and makes it more readable and maintainable.
- Call other test cases: Break down your test scenarios into smaller test cases and call them within a main test case. This allows for managing complex scenarios without overloading a single test case.
- Use loops and conditions: When you have repetitive or conditional actions, use loops (while, for) and conditional structures (if/else) to minimize the number of lines in the Manual tab.
Regarding test suites, it is important to structure your test cases logically and efficiently:
- Define coherent test suites: The number of test cases in a suite should reflect the needs of your test campaign. It is not reasonable to arbitrarily limit the number of test cases in a suite. Instead, each test suite should be designed to thoroughly cover a specific aspect of the application.
- Modularize test suites: Create modular test suites that group test cases by functionality or user scenario. This makes it easier to manage and maintain your tests.
- Use test configurations: Configure your test suites to include preconditions and postconditions, which can help maintain order and clarity in test execution.
By applying these best practices, you can avoid issues related to overly large test cases and ensure better maintainability of your tests. The number of test cases in a suite should be tailored to your specific needs and not limited arbitrarily.
In summary, it is preferable to structure your tests to minimize the number of actions in each test case, using keywords, classes, and test case calls to create more modular and maintainable tests. Additionally, test suites should be designed coherently and modularly to reflect the specific needs of your test campaign.
1 Like
Hi there,
Thank you very much for your topic. Please note that it may take a little while before a member of our community or from Katalon team responds to you.
Thanks!
There is no limit hard-coded like “if (testCases.size()) > limit) blow up” in the Katalon software.
Please try creating a lot of test cases in your project yourself. Experience it! Then you will find a practical answer to your question.
As far as you count the number of Test Case’s folders on disk, and as far as you are NOT going to run it,
Dozens … no problem
Some hundreds … could be alright. But you are likely to loose controle over such a large set of codes.
Thousands … nobody has ever tried. I am not going to, of course.
If I am going to run the testcases sequentially,
Dozens … I would be able to wait for them to finish, though it may take long time — 10 minutes? 30 minutes? some hours?
Hundres … No, I can’t wait. Likely to see OutOfMemoryErrors or “Code Too Large” errors.
2 Likes
Are you going to use Katalon’s Web Recording tool to generate tests for a large number of web pages? Do you anticipate that you will end up with a lot of Test Cases auto-generated in your project?
You are quite right. I would warn you; you are likely to get into a problem of maintenance.
I am afraid that those who are new to Katalon Studio may expect the codes generated by Web Recorder are complete so that they would never have to change the codes in future. But, in my humble opinion, that is not the case.
The tool generates Test Cases and Test Objects with a tons of duplications. If you leave the duplications as generated, then sooner or later, you will find it impossible to maintain the dozens or hundreds of duplications in long term (months or years). Every single small change in the target system would result a lot of defects in your test codes. You have to fix all the defects in the duplicating codes. That’s a nightmare.
With this tool, you can start automating your UI tests. In this sense, it is useful. But I assure you, code generation is just the beginning of the job. You have to constantly rethink and rewrite your code. The generated code must be read, rewritten, modularized, renamed, and the duplications must be removed.
You should keep the population of Test Cases and Test Objects to the minimum. Don’t let them explode, otherwise your project will collapse.
1 Like