Maybe you should rewrite your test case script manually to make it fit before complaining about KS.
@tanderson you can use a better aproach by spliting the code in sub-testcases and call them from a control testcase. 700 lines of code in a single testcase is too much for any framework.
personally, i would’nt do a code review on such design, i will simply reject it. but that’s me …
LE: or, use a data driven/parametrized approach …
I won’t even bother replying to kazurayam. He can’t believe that I hand wrote 700 lines of code in 8 hours, then what does that imply? Not sure why he wants to assume these things but I’ve seen him insulting other people’s intelligence on here as well. A form with 70 items is long, but if Katalon cannot handle real world demands that my project manager asked me to do with a short turnaround, then it is not the tool to use for serious web testing. Use it to check your landing page or something.
I have done the same kind of coding with Selenium and thought it would be fun to try out Katalon, but with this kind of pedantry on the forums and the weak sauce that fall over with 70 element forms, good riddance. I love contributing to open source but please.
I did use a data driven approach. I have a form with 70 fields, and I hand created the data for each of the 70 fields. I want to enter the data for the whole form, which cannot be submitted with less than 70 fields. I used xpath selectors. In my experience, the Katalon recorder was terrible to use, because you cannot tune the selectors quickly, so after a short time you should be able to code a groovy script. However, it turns out it’s not just a script, it’s actually rewritten as a single java method.
If Katalon has known of this hard limit for scripts, and have known about it for 1.5 years, why did you waste a whole day of my dev time without making a simple warning or guidance? At this point, it’s clearly easier to just go back to Selenium. I have all the selectors and timeouts I need. How difficult is it to track the size of the method and indicate to the user that they should break up the code before it just refuses to compile? And why is a 2kB groovy script compiling to a method that’s bigger than 64kB anyway?
Moral: Use Katalon if you have some small need for unit tests and haven’t coded 700 lines in your life, or Selenium if you are a power user. Sad, because except for the crashes every hour or so, Katalon does seem to be a fairly good abstraction of Selenium if we don’t worry about the rough edges.
I really don’t understand why you are so upset.
The limitation came from JVM
The value of the code_length item must be less than 65536.
So, even if using Selenium with plain Java … soon or later you may still reach the limit.
That’s why the best solution is to somehow split your code.
Hi.
Its been a few years since I posted this but even after we started splitting up the code the issue comes back.
For example, my colleague has Katalon Studio 5.3.1 and she runs a test case fine. She hands me her project and I run the test case in 7.6.2 and I get “Method code too large!”. But we do not have the time to go through hundreds of test cases and split them up. So I have no choice but to downgrade to her level. If the cause of this error is the nature of the Groovy language, why would the error appear in a later but not earlier version of Katalon.
I assume there is no solution aside from splitting test cases, but is there at least some way to know how close a test case is to hitting the JVM limited code length of 65536, other than running it, that is?
Recently I realised that splitting a large TestCase source into smaller pieces and re-assembling them using WebUI.callTestCase() does NOT help for working around the “Method code too large” constraint. I edited my previous post at Method code too large - #18 by kazurayam
@Russ_Thomas suggested an effective approach to decrease the size of method at Method code too large - #5 by Russ_Thomas
Take your common test code, turn it into Classes/Methods and store them under Keywords
I agree with his suggestion.
I guess, simply because the version of Katalon Studio is different. The newer the version of a software product is, it introduces more features that could benefit users, and at the same time the binaries tend to get fat internally.
For example see:
Sincerely, I do not know. I have never heard of such thing.
I found an interesting article Reproducing “code too large” problem in java. This article shows us how to inspect the size of methods of a class using the javap tool bundled in Oracle JDK.
Let me show you what I did. I have Oracle JDK installed. I have a Katalon Studio project HealthCare. I did in the console.
$ cd ~/katalon-workspace/HealthCare
$ cd bin/lib
$ javap -c TempTestCase1610076062391.class
Compiled from "TempTestCase1610076062391.groovy"
public class TempTestCase1610076062391 extends groovy.lang.Script {
public static transient boolean __$stMC;
public TempTestCase1610076062391();
Code:
0: aload_0
1: invokespecial #13 // Method groovy/lang/Script."<init>":()V
4: invokestatic #17 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
7: astore_1
8: return
public TempTestCase1610076062391(groovy.lang.Binding);
Code:
0: invokestatic #17 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
3: astore_2
4: aload_0
5: aload_1
6: invokespecial #22 // Method groovy/lang/Script."<init>":(Lgroovy/lang/Binding;)V
9: return
public static void main(java.lang.String...);
Code:
0: invokestatic #17 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
3: astore_1
4: aload_1
5: ldc #27 // int 0
7: aaload
8: ldc #29 // class org/codehaus/groovy/runtime/InvokerHelper
10: ldc #2 // class TempTestCase1610076062391
12: aload_0
13: invokeinterface #35, 4 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
18: pop
19: return
public java.lang.Object run();
Code:
0: invokestatic #17 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
3: astore_1
4: aload_1
5: ldc #40 // int 1
7: aaload
8: aload_1
9: ldc #41 // int 2
11: aaload
12: ldc #43 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
14: invokeinterface #46, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
19: aload_1
20: ldc #47 // int 3
22: aaload
23: ldc #49 // class com/kms/katalon/core/webui/contribution/WebUiDriverCleaner
25: invokeinterface #52, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
30: invokeinterface #55, 3 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
35: pop
36: aload_1
37: ldc #56 // int 4
39: aaload
40: aload_1
41: ldc #57 // int 5
43: aaload
44: ldc #43 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
46: invokeinterface #46, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
51: aload_1
52: ldc #58 // int 6
54: aaload
55: ldc #60 // class com/kms/katalon/core/mobile/contribution/MobileDriverCleaner
57: invokeinterface #52, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
62: invokeinterface #55, 3 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
67: pop
68: aload_1
69: ldc #61 // int 7
71: aaload
72: aload_1
73: ldc #62 // int 8
75: aaload
76: ldc #43 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
78: invokeinterface #46, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
83: aload_1
84: ldc #63 // int 9
86: aaload
87: ldc #65 // class com/kms/katalon/core/cucumber/keyword/internal/CucumberDriverCleaner
89: invokeinterface #52, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
94: invokeinterface #55, 3 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
99: pop
100: aload_1
101: ldc #66 // int 10
103: aaload
104: aload_1
105: ldc #67 // int 11
107: aaload
108: ldc #43 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
110: invokeinterface #46, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
115: aload_1
116: ldc #68 // int 12
118: aaload
119: ldc #70 // class com/kms/katalon/core/windows/keyword/contribution/WindowsDriverCleaner
121: invokeinterface #52, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
126: invokeinterface #55, 3 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
131: pop
132: aload_1
133: ldc #71 // int 13
135: aaload
136: aload_1
137: ldc #72 // int 14
139: aaload
140: ldc #43 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
142: invokeinterface #46, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
147: aload_1
148: ldc #73 // int 15
150: aaload
151: ldc #75 // class com/kms/katalon/core/testng/keyword/internal/TestNGDriverCleaner
153: invokeinterface #52, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
158: invokeinterface #55, 3 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
163: pop
164: aload_1
165: ldc #76 // int 16
167: aaload
168: ldc #78 // class com/kms/katalon/core/configuration/RunConfiguration
170: ldc #80 // String /var/folders/7m/lm7d6nx51kj0kbtnsskz6r3m0000gn/T/Katalon/Test Cases/Main Test Cases/TC3_Visual Testing Example/20210108_122102/execution.properties
172: invokeinterface #55, 3 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
177: pop
178: aload_1
179: ldc #81 // int 17
181: aaload
182: ldc #83 // class com/kms/katalon/core/main/TestCaseMain
184: invokeinterface #46, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
189: pop
190: aload_1
191: ldc #84 // int 18
193: aaload
194: ldc #83 // class com/kms/katalon/core/main/TestCaseMain
196: ldc #86 // String Test Cases/Main Test Cases/TC3_Visual Testing Example
198: aload_1
199: ldc #87 // int 19
201: aaload
202: ldc #89 // class com/kms/katalon/core/testcase/TestCaseBinding
204: ldc #86 // String Test Cases/Main Test Cases/TC3_Visual Testing Example
206: iconst_0
207: anewarray #91 // class java/lang/Object
210: invokestatic #97 // Method org/codehaus/groovy/runtime/ScriptBytecodeAdapter.createMap:([Ljava/lang/Object;)Ljava/util/Map;
213: invokeinterface #99, 4 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
218: aload_1
219: ldc #100 // int 20
221: aaload
222: ldc #102 // class com/kms/katalon/core/model/FailureHandling
224: invokeinterface #105, 2 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callGetProperty:(Ljava/lang/Object;)Ljava/lang/Object;
229: iconst_0
230: invokestatic #111 // Method java/lang/Boolean.valueOf:(Z)Ljava/lang/Boolean;
233: invokeinterface #114, 6 // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
238: areturn
239: aconst_null
240: areturn
protected groovy.lang.MetaClass $getStaticMetaClass();
Code:
0: aload_0
1: invokevirtual #120 // Method java/lang/Object.getClass:()Ljava/lang/Class;
4: ldc #2 // class TempTestCase1610076062391
6: if_acmpeq 14
9: aload_0
10: invokestatic #124 // Method org/codehaus/groovy/runtime/ScriptBytecodeAdapter.initMetaClass:(Ljava/lang/Object;)Lgroovy/lang/MetaClass;
13: areturn
14: getstatic #126 // Field $staticClassInfo:Lorg/codehaus/groovy/reflection/ClassInfo;
17: astore_1
18: aload_1
19: ifnonnull 34
22: aload_0
23: invokevirtual #120 // Method java/lang/Object.getClass:()Ljava/lang/Class;
26: invokestatic #132 // Method org/codehaus/groovy/reflection/ClassInfo.getClassInfo:(Ljava/lang/Class;)Lorg/codehaus/groovy/reflection/ClassInfo;
29: dup
30: astore_1
31: putstatic #126 // Field $staticClassInfo:Lorg/codehaus/groovy/reflection/ClassInfo;
34: aload_1
35: invokevirtual #135 // Method org/codehaus/groovy/reflection/ClassInfo.getMetaClass:()Lgroovy/lang/MetaClass;
38: areturn
}
By this output, I found that the TempTestCase1610076062391 class has a method run() of which size is approximately 240 bytes.
My next question was: TempTestCase1610076062391 class must be generated from a Test Case script, but which one? Which one of test case scripts was compiled into the TempTestCase1610076062391 class? The only thing I could try was to decompile the TempTestCase1610076062391 class file into a Java source and read the source code. I used the Java Decompiler (jd-gui-1.6.6.jar). The tool showed the decompiled source of the class as follows:
import com.kms.katalon.core.configuration.RunConfiguration;
import com.kms.katalon.core.cucumber.keyword.internal.CucumberDriverCleaner;
import com.kms.katalon.core.driver.internal.DriverCleanerCollector;
import com.kms.katalon.core.main.TestCaseMain;
import com.kms.katalon.core.mobile.contribution.MobileDriverCleaner;
import com.kms.katalon.core.model.FailureHandling;
import com.kms.katalon.core.testcase.TestCaseBinding;
import com.kms.katalon.core.testng.keyword.internal.TestNGDriverCleaner;
import com.kms.katalon.core.webui.contribution.WebUiDriverCleaner;
import com.kms.katalon.core.windows.keyword.contribution.WindowsDriverCleaner;
import groovy.lang.Binding;
import groovy.lang.MetaClass;
import groovy.lang.Script;
import java.lang.ref.SoftReference;
import org.codehaus.groovy.reflection.ClassInfo;
import org.codehaus.groovy.runtime.InvokerHelper;
import org.codehaus.groovy.runtime.ScriptBytecodeAdapter;
import org.codehaus.groovy.runtime.callsite.CallSite;
import org.codehaus.groovy.runtime.callsite.CallSiteArray;
public class TempTestCase1610076062391 extends Script {
public TempTestCase1610076062391() {
CallSite[] arrayOfCallSite = $getCallSiteArray();
}
public TempTestCase1610076062391(Binding context) {
super(context);
}
public static void main(String... args) {
CallSite[] arrayOfCallSite = $getCallSiteArray();
arrayOfCallSite[0].call(InvokerHelper.class, TempTestCase1610076062391.class, args);
}
public Object run() {
CallSite[] arrayOfCallSite = $getCallSiteArray();
arrayOfCallSite[1].call(arrayOfCallSite[2].call(DriverCleanerCollector.class), arrayOfCallSite[3].callConstructor(WebUiDriverCleaner.class));
arrayOfCallSite[4].call(arrayOfCallSite[5].call(DriverCleanerCollector.class), arrayOfCallSite[6].callConstructor(MobileDriverCleaner.class));
arrayOfCallSite[7].call(arrayOfCallSite[8].call(DriverCleanerCollector.class), arrayOfCallSite[9].callConstructor(CucumberDriverCleaner.class));
arrayOfCallSite[10].call(arrayOfCallSite[11].call(DriverCleanerCollector.class), arrayOfCallSite[12].callConstructor(WindowsDriverCleaner.class));
arrayOfCallSite[13].call(arrayOfCallSite[14].call(DriverCleanerCollector.class), arrayOfCallSite[15].callConstructor(TestNGDriverCleaner.class));
arrayOfCallSite[16].call(RunConfiguration.class, "/var/folders/7m/lm7d6nx51kj0kbtnsskz6r3m0000gn/T/Katalon/Test Cases/Main Test Cases/TC3_Visual Testing Example/20210108_122102/execution.properties");
arrayOfCallSite[17].call(TestCaseMain.class);
return arrayOfCallSite[18].call(TestCaseMain.class, "Test Cases/Main Test Cases/TC3_Visual Testing Example", arrayOfCallSite[19].callConstructor(TestCaseBinding.class, "Test Cases/Main Test Cases/TC3_Visual Testing Example", ScriptBytecodeAdapter.createMap(new Object[0])), arrayOfCallSite[20].callGetProperty(FailureHandling.class), Boolean.valueOf(false));
}
}
In the decompiled source, I found a fragment:
return arrayOfCallSite[18].call(TestCaseMain.class, "Test Cases/Main Test Cases/TC3_Visual Testing Example", ...
This indicates that the TempTestCase1610076062391 class will dynamicall compile & run the test case script Test Cases/Main Test Cases/TC3_Visual Testing Example.
I looked at some amount of “TempTestCase:.class” files with javap -c command, and found none of them have run() method exceeding 240 bytes of size.
I realised, I do not really understand what is the meaning of the figure “240” provided by javap -c command. And I am not sure if it really concerned about the “Method code too large” issue. A lot more of careful studies required about .class file format and JVM behaviour. I am afraid, I am not willing to dive into this issue any more.
Thank you for the help. You are always very helpful and I appreciate it.
Ilya
The article Reproducing “Code too large” problem in java was talking about a class file generated from the source written in Java, which is statically & entirely compiled. The javap -c would tell you precise size of methods of classes generated from Java source.
But Katalon’s Test Case script is written in Groovy, which performs dynamic compilation. I suppose that TempTestCase1610076062391.class plays a role of a launcher from which a massive JVM bytecode is generated dynamically, of which size is unmeasurable.
Therefore I would conclude that it is impossible to measure statically how large the run() method of Katalon TestCases will be.
I am going to revive this old post. Let me add a bit more of analysis.
A Groovy script in general could be interpreted as a Script class. What is Script class? — see Groovy Language Documentation
How a Grovy script text is converted into a method of a Script class? — See Groovy Language Documentation
This is a practically possible scenario how a Groovy script is executed. Katalon Studio internally does a similar interpretation, but not exactly the same. Then what’s going on inside Katalon Studio?
Let me assume that I have a Test Cases/TC1, as follows:
import import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
WebUI.comment("Hello, world!")
WebUI.comment("Hello, world!")
WebUI.comment("Hello, world!")
When I run the TC1, Katalon Studio internally generates a file Libs/TempTestCaseNNNNNNNNNNNNN.groovy, of which contents is something like:
import com.kms.katalon.core.main.TestCaseMain
...
TestCaseMain.runTestCase('Test Cases/NISA/main', new TestCaseBinding('Test Cases/NISA/main',[:]), FailureHandling.STOP_ON_FAILURE , false)
Katalon Studio will execute this generated Groovy code.
Then we have to continue reading the source asking what the TestCaseMain.runTestCase(.....) does. You can find the source of the TestCaseMain class at com.kms.katalon.core.main.TestCaseMain. You can get into the jungle of source codes that comprised the core part of Katalon Studio. You can trace what will happen when you run a Test Case. I will skip the details. … Most notablly, the runScriptMethodAsRowText() method at Line#130 of the com.kms.katalon.core.main.ScriptEngine class is interesting.
return getScript(getGroovyCodeSource(processedScriptText, className),
binding, true).invokeMethod(methodName, null);
This line tells me that Katalon Studio reads a script text as a Test Case, parse it as a Groovy script, and execute it as a method of a class. In other words, a single Test Case is regarded as a method. Therefore a Test Case must not be larger than the size limit of a method (64K bytes as a compiled byte code).