Method code too large

@Ibus @kazurayam I am not using Katalon Web Recorder for creating script. I have a test case which is running on a Jenkins server every 30 mins for each of our website.

The test case perform some important testing along the way till checkout page. There are many important validations it do. Our websites have many 3rd party components and payment processors where we need to keep an eye. The code is also printing a lot of information in logs based on logic and data. This is then feed into Analytics where Management team can see it whenever they want.

@Ibus I don’t see why someone couldn’t have more than > 800 lines test ? It depends on what the requirements we have to test.

well … i do understand that you have many requirements. but in such cases, my approach will be to split the scope of the test in smaller components > more smaller testcases instead of a huge script. it’s just a matter of taste, i hate to debug into thousand lines of code

@Ibus I had similar approach earlier, but because the requirement to read / write into database based on the content and action performed the code started getting lengthier. To overcome the method code too large issue I have separated the code into some small scripts and call those using Call Test Case method in one main script.

yeah, that may work too. and is also easier to debug, since in case of failure you will know exactly what sub-case had issues

Hi all,

This issue @kazurayam mentioned above has been fixed in v6.3.x. Please check again.

https://docs.katalon.com/katalon-studio/new/version-630.html#fixes

Thanks

@duyluong

”Method code too large” and “a test suite with too large test data” — are these 2 problems the same issue?

Could you please show us a bit more info what “a test suite with too large test data” issue is? How Katalon Studio has been changed? Any link to a forum issue?

Hi @kazurayam,

There are 2 problems in this topic:

  1. The issue relates to a test suite using data binding with large Test Data. We already fixed it.
  1. The issue relates to a test case contains 500-1000 lines of code. This is a Groovy limitation. We may try to improve but we also recommend users to separate the main script to multiple methods.

Thanks.

OK, I understand it. Thanks for the clarification.

1 Like

I’m testing a single page form with around 70 elements. Every box in the form needs to be completed. The success or failure of the test is based on whether the page can submit. They have promised the client that we will produce a result with short turnaround. Katalon seemed to streamline the Selenium process. I got to 65 elements and now I got this exception.

70 elements and 700 lines of code is too much, maybe Katalon is better for unit tests and not web page tests.

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during class generation: Method code too large!

Maybe you should rewrite your test case script manually to make it fit before complaining about KS.

@tanderson you can use a better aproach by spliting the code in sub-testcases and call them from a control testcase. 700 lines of code in a single testcase is too much for any framework.
personally, i would’nt do a code review on such design, i will simply reject it. but that’s me …

LE: or, use a data driven/parametrized approach …

I won’t even bother replying to kazurayam. He can’t believe that I hand wrote 700 lines of code in 8 hours, then what does that imply? Not sure why he wants to assume these things but I’ve seen him insulting other people’s intelligence on here as well. A form with 70 items is long, but if Katalon cannot handle real world demands that my project manager asked me to do with a short turnaround, then it is not the tool to use for serious web testing. Use it to check your landing page or something.

I have done the same kind of coding with Selenium and thought it would be fun to try out Katalon, but with this kind of pedantry on the forums and the weak sauce that fall over with 70 element forms, good riddance. I love contributing to open source but please.

I did use a data driven approach. I have a form with 70 fields, and I hand created the data for each of the 70 fields. I want to enter the data for the whole form, which cannot be submitted with less than 70 fields. I used xpath selectors. In my experience, the Katalon recorder was terrible to use, because you cannot tune the selectors quickly, so after a short time you should be able to code a groovy script. However, it turns out it’s not just a script, it’s actually rewritten as a single java method.

If Katalon has known of this hard limit for scripts, and have known about it for 1.5 years, why did you waste a whole day of my dev time without making a simple warning or guidance? At this point, it’s clearly easier to just go back to Selenium. I have all the selectors and timeouts I need. How difficult is it to track the size of the method and indicate to the user that they should break up the code before it just refuses to compile? And why is a 2kB groovy script compiling to a method that’s bigger than 64kB anyway?

Moral: Use Katalon if you have some small need for unit tests and haven’t coded 700 lines in your life, or Selenium if you are a power user. Sad, because except for the crashes every hour or so, Katalon does seem to be a fairly good abstraction of Selenium if we don’t worry about the rough edges.

I really don’t understand why you are so upset.
The limitation came from JVM

The value of the code_length item must be less than 65536. 

So, even if using Selenium with plain Java … soon or later you may still reach the limit.
That’s why the best solution is to somehow split your code.

Hi.

Its been a few years since I posted this but even after we started splitting up the code the issue comes back.

For example, my colleague has Katalon Studio 5.3.1 and she runs a test case fine. She hands me her project and I run the test case in 7.6.2 and I get “Method code too large!”. But we do not have the time to go through hundreds of test cases and split them up. So I have no choice but to downgrade to her level. If the cause of this error is the nature of the Groovy language, why would the error appear in a later but not earlier version of Katalon.

I assume there is no solution aside from splitting test cases, but is there at least some way to know how close a test case is to hitting the JVM limited code length of 65536, other than running it, that is?

@Ilya_Novak

Recently I realised that splitting a large TestCase source into smaller pieces and re-assembling them using WebUI.callTestCase() does NOT help for working around the “Method code too large” constraint. I edited my previous post at Method code too large


@Russ_Thomas suggested an effective approach to decrease the size of method at Method code too large

Take your common test code, turn it into Classes/Methods and store them under Keywords

I agree with his suggestion.


I guess, simply because the version of Katalon Studio is different. The newer the version of a software product is, it introduces more features that could benefit users, and at the same time the binaries tend to get fat internally.

For example see:


Sincerely, I do not know. I have never heard of such thing.

I found an interesting article Reproducing “code too large” problem in java. This article shows us how to inspect the size of methods of a class using the javap tool bundled in Oracle JDK.

Let me show you what I did. I have Oracle JDK installed. I have a Katalon Studio project HealthCare. I did in the console.

$ cd ~/katalon-workspace/HealthCare
$ cd bin/lib
$ javap -c TempTestCase1610076062391.class 
Compiled from "TempTestCase1610076062391.groovy"
public class TempTestCase1610076062391 extends groovy.lang.Script {
  public static transient boolean __$stMC;

  public TempTestCase1610076062391();
    Code:
       0: aload_0
       1: invokespecial #13                 // Method groovy/lang/Script."<init>":()V
       4: invokestatic  #17                 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
       7: astore_1
       8: return

  public TempTestCase1610076062391(groovy.lang.Binding);
    Code:
       0: invokestatic  #17                 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
       3: astore_2
       4: aload_0
       5: aload_1
       6: invokespecial #22                 // Method groovy/lang/Script."<init>":(Lgroovy/lang/Binding;)V
       9: return

  public static void main(java.lang.String...);
    Code:
       0: invokestatic  #17                 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
       3: astore_1
       4: aload_1
       5: ldc           #27                 // int 0
       7: aaload
       8: ldc           #29                 // class org/codehaus/groovy/runtime/InvokerHelper
      10: ldc           #2                  // class TempTestCase1610076062391
      12: aload_0
      13: invokeinterface #35,  4           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
      18: pop
      19: return

  public java.lang.Object run();
    Code:
       0: invokestatic  #17                 // Method $getCallSiteArray:()[Lorg/codehaus/groovy/runtime/callsite/CallSite;
       3: astore_1
       4: aload_1
       5: ldc           #40                 // int 1
       7: aaload
       8: aload_1
       9: ldc           #41                 // int 2
      11: aaload
      12: ldc           #43                 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
      14: invokeinterface #46,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
      19: aload_1
      20: ldc           #47                 // int 3
      22: aaload
      23: ldc           #49                 // class com/kms/katalon/core/webui/contribution/WebUiDriverCleaner
      25: invokeinterface #52,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
      30: invokeinterface #55,  3           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
      35: pop
      36: aload_1
      37: ldc           #56                 // int 4
      39: aaload
      40: aload_1
      41: ldc           #57                 // int 5
      43: aaload
      44: ldc           #43                 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
      46: invokeinterface #46,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
      51: aload_1
      52: ldc           #58                 // int 6
      54: aaload
      55: ldc           #60                 // class com/kms/katalon/core/mobile/contribution/MobileDriverCleaner
      57: invokeinterface #52,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
      62: invokeinterface #55,  3           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
      67: pop
      68: aload_1
      69: ldc           #61                 // int 7
      71: aaload
      72: aload_1
      73: ldc           #62                 // int 8
      75: aaload
      76: ldc           #43                 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
      78: invokeinterface #46,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
      83: aload_1
      84: ldc           #63                 // int 9
      86: aaload
      87: ldc           #65                 // class com/kms/katalon/core/cucumber/keyword/internal/CucumberDriverCleaner
      89: invokeinterface #52,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
      94: invokeinterface #55,  3           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
      99: pop
     100: aload_1
     101: ldc           #66                 // int 10
     103: aaload
     104: aload_1
     105: ldc           #67                 // int 11
     107: aaload
     108: ldc           #43                 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
     110: invokeinterface #46,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
     115: aload_1
     116: ldc           #68                 // int 12
     118: aaload
     119: ldc           #70                 // class com/kms/katalon/core/windows/keyword/contribution/WindowsDriverCleaner
     121: invokeinterface #52,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
     126: invokeinterface #55,  3           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
     131: pop
     132: aload_1
     133: ldc           #71                 // int 13
     135: aaload
     136: aload_1
     137: ldc           #72                 // int 14
     139: aaload
     140: ldc           #43                 // class com/kms/katalon/core/driver/internal/DriverCleanerCollector
     142: invokeinterface #46,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
     147: aload_1
     148: ldc           #73                 // int 15
     150: aaload
     151: ldc           #75                 // class com/kms/katalon/core/testng/keyword/internal/TestNGDriverCleaner
     153: invokeinterface #52,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;)Ljava/lang/Object;
     158: invokeinterface #55,  3           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
     163: pop
     164: aload_1
     165: ldc           #76                 // int 16
     167: aaload
     168: ldc           #78                 // class com/kms/katalon/core/configuration/RunConfiguration
     170: ldc           #80                 // String /var/folders/7m/lm7d6nx51kj0kbtnsskz6r3m0000gn/T/Katalon/Test Cases/Main Test Cases/TC3_Visual Testing Example/20210108_122102/execution.properties
     172: invokeinterface #55,  3           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
     177: pop
     178: aload_1
     179: ldc           #81                 // int 17
     181: aaload
     182: ldc           #83                 // class com/kms/katalon/core/main/TestCaseMain
     184: invokeinterface #46,  2           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;)Ljava/lang/Object;
     189: pop
     190: aload_1
     191: ldc           #84                 // int 18
     193: aaload
     194: ldc           #83                 // class com/kms/katalon/core/main/TestCaseMain
     196: ldc           #86                 // String Test Cases/Main Test Cases/TC3_Visual Testing Example
     198: aload_1
     199: ldc           #87                 // int 19
     201: aaload
     202: ldc           #89                 // class com/kms/katalon/core/testcase/TestCaseBinding
     204: ldc           #86                 // String Test Cases/Main Test Cases/TC3_Visual Testing Example
     206: iconst_0
     207: anewarray     #91                 // class java/lang/Object
     210: invokestatic  #97                 // Method org/codehaus/groovy/runtime/ScriptBytecodeAdapter.createMap:([Ljava/lang/Object;)Ljava/util/Map;
     213: invokeinterface #99,  4           // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callConstructor:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
     218: aload_1
     219: ldc           #100                // int 20
     221: aaload
     222: ldc           #102                // class com/kms/katalon/core/model/FailureHandling
     224: invokeinterface #105,  2          // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.callGetProperty:(Ljava/lang/Object;)Ljava/lang/Object;
     229: iconst_0
     230: invokestatic  #111                // Method java/lang/Boolean.valueOf:(Z)Ljava/lang/Boolean;
     233: invokeinterface #114,  6          // InterfaceMethod org/codehaus/groovy/runtime/callsite/CallSite.call:(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
     238: areturn
     239: aconst_null
     240: areturn

  protected groovy.lang.MetaClass $getStaticMetaClass();
    Code:
       0: aload_0
       1: invokevirtual #120                // Method java/lang/Object.getClass:()Ljava/lang/Class;
       4: ldc           #2                  // class TempTestCase1610076062391
       6: if_acmpeq     14
       9: aload_0
      10: invokestatic  #124                // Method org/codehaus/groovy/runtime/ScriptBytecodeAdapter.initMetaClass:(Ljava/lang/Object;)Lgroovy/lang/MetaClass;
      13: areturn
      14: getstatic     #126                // Field $staticClassInfo:Lorg/codehaus/groovy/reflection/ClassInfo;
      17: astore_1
      18: aload_1
      19: ifnonnull     34
      22: aload_0
      23: invokevirtual #120                // Method java/lang/Object.getClass:()Ljava/lang/Class;
      26: invokestatic  #132                // Method org/codehaus/groovy/reflection/ClassInfo.getClassInfo:(Ljava/lang/Class;)Lorg/codehaus/groovy/reflection/ClassInfo;
      29: dup
      30: astore_1
      31: putstatic     #126                // Field $staticClassInfo:Lorg/codehaus/groovy/reflection/ClassInfo;
      34: aload_1
      35: invokevirtual #135                // Method org/codehaus/groovy/reflection/ClassInfo.getMetaClass:()Lgroovy/lang/MetaClass;
      38: areturn
}

By this output, I found that the TempTestCase1610076062391 class has a method run() of which size is approximately 240 bytes.


My next question was: TempTestCase1610076062391 class must be generated from a Test Case script, but which one? Which one of test case scripts was compiled into the TempTestCase1610076062391 class? The only thing I could try was to decompile the TempTestCase1610076062391 class file into a Java source and read the source code. I used the Java Decompiler (jd-gui-1.6.6.jar). The tool showed the decompiled source of the class as follows:

import com.kms.katalon.core.configuration.RunConfiguration;
import com.kms.katalon.core.cucumber.keyword.internal.CucumberDriverCleaner;
import com.kms.katalon.core.driver.internal.DriverCleanerCollector;
import com.kms.katalon.core.main.TestCaseMain;
import com.kms.katalon.core.mobile.contribution.MobileDriverCleaner;
import com.kms.katalon.core.model.FailureHandling;
import com.kms.katalon.core.testcase.TestCaseBinding;
import com.kms.katalon.core.testng.keyword.internal.TestNGDriverCleaner;
import com.kms.katalon.core.webui.contribution.WebUiDriverCleaner;
import com.kms.katalon.core.windows.keyword.contribution.WindowsDriverCleaner;
import groovy.lang.Binding;
import groovy.lang.MetaClass;
import groovy.lang.Script;
import java.lang.ref.SoftReference;
import org.codehaus.groovy.reflection.ClassInfo;
import org.codehaus.groovy.runtime.InvokerHelper;
import org.codehaus.groovy.runtime.ScriptBytecodeAdapter;
import org.codehaus.groovy.runtime.callsite.CallSite;
import org.codehaus.groovy.runtime.callsite.CallSiteArray;

public class TempTestCase1610076062391 extends Script {
  public TempTestCase1610076062391() {
    CallSite[] arrayOfCallSite = $getCallSiteArray();
  }
  
  public TempTestCase1610076062391(Binding context) {
    super(context);
  }
  
  public static void main(String... args) {
    CallSite[] arrayOfCallSite = $getCallSiteArray();
    arrayOfCallSite[0].call(InvokerHelper.class, TempTestCase1610076062391.class, args);
  }
  
  public Object run() {
    CallSite[] arrayOfCallSite = $getCallSiteArray();
    arrayOfCallSite[1].call(arrayOfCallSite[2].call(DriverCleanerCollector.class), arrayOfCallSite[3].callConstructor(WebUiDriverCleaner.class));
    arrayOfCallSite[4].call(arrayOfCallSite[5].call(DriverCleanerCollector.class), arrayOfCallSite[6].callConstructor(MobileDriverCleaner.class));
    arrayOfCallSite[7].call(arrayOfCallSite[8].call(DriverCleanerCollector.class), arrayOfCallSite[9].callConstructor(CucumberDriverCleaner.class));
    arrayOfCallSite[10].call(arrayOfCallSite[11].call(DriverCleanerCollector.class), arrayOfCallSite[12].callConstructor(WindowsDriverCleaner.class));
    arrayOfCallSite[13].call(arrayOfCallSite[14].call(DriverCleanerCollector.class), arrayOfCallSite[15].callConstructor(TestNGDriverCleaner.class));
    arrayOfCallSite[16].call(RunConfiguration.class, "/var/folders/7m/lm7d6nx51kj0kbtnsskz6r3m0000gn/T/Katalon/Test Cases/Main Test Cases/TC3_Visual Testing Example/20210108_122102/execution.properties");
    arrayOfCallSite[17].call(TestCaseMain.class);
    return arrayOfCallSite[18].call(TestCaseMain.class, "Test Cases/Main Test Cases/TC3_Visual Testing Example", arrayOfCallSite[19].callConstructor(TestCaseBinding.class, "Test Cases/Main Test Cases/TC3_Visual Testing Example", ScriptBytecodeAdapter.createMap(new Object[0])), arrayOfCallSite[20].callGetProperty(FailureHandling.class), Boolean.valueOf(false));
  }
}

In the decompiled source, I found a fragment:

        return arrayOfCallSite[18].call(TestCaseMain.class, "Test Cases/Main Test Cases/TC3_Visual Testing Example", ...

This indicates that the TempTestCase1610076062391 class will dynamicall compile & run the test case script Test Cases/Main Test Cases/TC3_Visual Testing Example.

I looked at some amount of “TempTestCase:.class” files with javap -c command, and found none of them have run() method exceeding 240 bytes of size.

I realised, I do not really understand what is the meaning of the figure “240” provided by javap -c command. And I am not sure if it really concerned about the “Method code too large” issue. A lot more of careful studies required about .class file format and JVM behaviour. I am afraid, I am not willing to dive into this issue any more.

Thank you for the help. You are always very helpful and I appreciate it.
Ilya

The article Reproducing “Code too large” problem in java was talking about a class file generated from the source written in Java, which is statically & entirely compiled. The javap -c would tell you precise size of methods of classes generated from Java source.

But Katalon’s Test Case script is written in Groovy, which performs dynamic compilation. I suppose that TempTestCase1610076062391.class plays a role of a launcher from which a massive JVM bytecode is generated dynamically, of which size is unmeasurable.

Therefore I would conclude that it is impossible to measure statically how large the run() method of Katalon TestCases will be.