Plugin jar created by the com.katalon.gradle-plugin contains a lot of unintended classes

I made a GitHub project GitHub - kazurayam/WebDriverFactory4ks at 0.2.0 which contains Custom Keyword class. I want to make it a Plugin. I refered to the instruction How to develop Custom Keywords Plugins. I could create build/libs/webdriverfactory4ks-all.jar.

I got surprised to find the jar file has size of 7.2 Giga bytes.

I check what class files are included:
jar-content.txt (299.6 KB)

The jar contained a lot of classes whose FQND starts with com/katalon/groovy. Why they are contained in the jar?

I expected that the >gradle katalonPluginPackage command will create a simple and small jar file which contains only the classes of Keywords I developed. Am I right? Or the jar is designed to include more classes which are required for some reasons?

Should I write a Gradle task of type Jar myself to create a simple and small jar file as I wanted?

1 Like

I edited the huge webdriverfactory4ks-all.jar with emacs so that it only contains classes of com.kazurayam.webdriverfactory4ks.*. In other words I removed all of contained class with FQND com.katlaon.groovy.*

I deployed the jar into one of my Katalon project and ran a test which calls com.kazurayam.webdriverfactory4ks.ChromeDriverFactory#openChromeBrowserWithProfile(String).

It failed with the following stack trace:

2019-03-13 16:15:59.906 e[34mINFO e[0;39m e[36mc.k.katalon.core.main.TestCaseExecutor   -e[0;39m e[39m--------------------e[0;39m
2019-03-13 16:15:59.921 e[34mINFO e[0;39m e[36mc.k.katalon.core.main.TestCaseExecutor   -e[0;39m e[39mSTART Test Cases/test/com.kazurayam.ksbackyard/ScreenshotDriverResizingTestRunnere[0;39m
2019-03-13 16:16:01.285 e[34mINFO e[0;39m e[36mc.k.k.c.keyword.builtin.CommentKeyword   -e[0;39m e[39m>>> testSuiteId is '_', testSuiteTimestamp is '_'e[0;39m
2019-03-13 16:16:01.286 e[34mINFO e[0;39m e[36mc.k.k.c.keyword.builtin.CommentKeyword   -e[0;39m e[39m>>> Instance of MaterialRepository(C:\Users\qcq0264\katalon-workspace\ksbackyard\Materials) is set to GlobalVariable.MATERIAL_REPOSITORYe[0;39m
2019-03-13 16:16:01.992 e[1;31mERRORe[0;39m e[36mc.k.junit4ks.JUnitCustomKeywords         -e[0;39m e[31m❌ com.kazurayam.ksbackyard.ScreenshotDriverResizingTest FAILED.e[0;39m
e[31mReason:e[0;39m
e[31mjava.lang.NoClassDefFoundError: com/katalon/groovy/lang/GroovyObjecte[0;39m
e[31m	at com.kazurayam.ksbackyard.ScreenshotDriverResizingTest.beforeClass(ScreenshotDriverResizingTest.groovy:62)e[0;39m
e[31m	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)e[0;39m
e[31m	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
...

Interestingly, my com.kazurayam.webdriverfatory4ks.ChromeDriverFactory in the jar file has become dependent on com.katalon.groogy.lang.GroovyObject class. There seems to be something mysterious.

Anybody has insight into this incident?

1 Like

Try removing these lines:

dependencies {
    compile "org.codehaus.groovy:groovy-all:${groovyVersion}"
}

Yes, removing dependencies solved the problem. The jar has got fit.

1 Like

The reason why I added the dependencies to the groovy-all.jar was that I wanted to generate Groovydoc of my Keywords, as I described at How to generate Groovydoc of Plugin

The groovydoc task seems to require groovy-all.jar in the dependecies declaration, so I added it.

Now I enountered a conflict: Katalon-gradle-plugin does not like groovy-all.jar in the dependencies, on the other hand groovydoc task wants it.

May be I can create another build.gradle file for groovydoc task. Even better, the root build.gradle should be designed to support multi projects: one child project is to build Katlaon plugins, other child project is for user’s custom requirements.

1 Like

Please try compileOnly instead of compile. And thank you for the groovydoc configuration - I like it a lot :smile:.

Dependency as compileOnly to Groovy-all.jar did not save the groovydoc task.

$ gradle groovydoc
> Task :groovydoc FAILED
> Task :publishGroovydoc NO-SOURCE

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':groovydoc'.
> Cannot infer Groovy class path because no Groovy Jar was found on class path: []

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
1 actionable task: 1 executed

Still I need to find out some other way to make Groovy Jar available to the groovydoc task.

I will try defining a Configuration which is meant solely to make Groovy Jar available to the groovydoc task.

I found that Gradle’s (groovydoc task](https://docs.gradle.org/current/dsl/org.gradle.api.tasks.javadoc.Groovydoc.html) has a property named groovyClasspath. I will try this later.

1 Like

I could find a solution. The following build.gradle works for both of building jar file and generating groovy docs.

plugins {
    id 'java'
    id 'groovy'
    id 'com.github.johnrengelman.shadow' version '4.0.4'
    id "com.katalon.gradle-plugin" version "0.0.6"
}

ext {
    groovyVersion   = '2.4.7'
}

configurations {
    generateDocs
}

repositories {
    jcenter()
    mavenCentral()
}
dependencies {
    generateDocs "org.codehaus.groovy:groovy-all:${groovyVersion}"
}
sourceSets {
    main {
        groovy {
            srcDirs = ['Keywords', 'Include/scripts/groovy']
            srcDir 'Libs'
        }
    }
}

katalon {
    dependencyPrefix = "com.katalon"
    minimize = false
}

task groovydoc(type: Groovydoc, overwrite:true) {
    source = sourceSets.main.groovy
    classpath = configurations.compile
    groovyClasspath = project.configurations.generateDocs
    include 'com/kazurayam/junit4ks/*'
    exclude '**/*Test.groovy'
}
task publishGroovydoc(type: Copy) {
    from 'build/docs/groovydoc'
    into 'docs/api'
}
groovydoc.finalizedBy publishGroovydoc

The key point was to define a “configuration” named generateDocs.

2 Likes

How can we install these plugins in other project? simply adding in plugins folder? or anything else??

https://store.katalon.com/ lists these plugins.

1 Like

What i meant was…I created one plugin (.jar file), but Im unable to use that in my other projects. Looks like it is not installing.
I also tried to use your project and created plugin out of it and used it, even that was also not getting installed.
To install plugin in other project, we need to simply go to the plugin option from the header and click install right?? or anything else I need to do?

The orthodox method would be publishing your plugin into the Katalon Plugin Store, and from the Store you bring the jar into your target projects.


Do you hesitate publishing your little plugins to the Store and make them exposed to public? — well, I understand it. Me too. I have many jars suitable for my closed use only and am not interested in monetizing it.

Katalon Studio 6.x.x equipped the orthodox path only. It does not support short-cut path of distributing plugins locally on your machine (for example, via MavenLocal repository) or inside my organization (via organization’s Maven Repository) for free of charge.


The shortest path to bring your jars into another projects is to copy the jar file into the <AnotherProjectDir>/Drivers directory. I swear, it works. Any jar files in Drivers will become available to any components in your AnotherProject. How to copy? — well, just manually.

Why not I copy the jar into the <AnotherProject>/Plugins directory?
— Just because I do not know much about the Plugin system of Katalon Studio yet. I once tested copying my jar into a <project>/Plugin directory. I checked if Katalon Studio recognize the jar or not. Katalon Studio did NOT recognize the jar. I mean, in the KS GUI Tool bar, Project > Settings > Plugins did not show up the jar I put. This experiment indicates that copying jars into the Plguins directory is not enough. It is likely that Katalon Studio generates META data for each indivisual jar files as Plugin when you installed them from the Katalon Store. No technical information about Plugin META data is available. So all we can do is to follow the orthodox path if we stick to using the Plugins directory.

Manually copying the jar from the source project to the other target projects’ Drives directory is dull. I would rather like to copy jars by Gradle script. I would do the following:

  1. In the source project by build.gradle, build the jar and publish it into the MavenLocal repository on my PC
  2. In the target project by build.gradle, download the jar from the MavenLocal repository to the Drivers directory

It is definitely feasible to write Gradle tasks to achieve this export/import processing.

If you are able to implement export/import processing of jar files by Gradle, you can certainly choose any location via which you transfer your artifacts; for example, you can use Github’s Releases page, or your organizational Maven Repository by Bintray.

1 Like

What exactly build.gradle file should contain?

Can I find the detailed implementation of these plugins anywhere?
I have read “https://docs.katalon.com/katalon-store/docs/publisher/how-to-develop-custom-keywords-plugins.html#package-the-custom-keywords-plugin
but is there any other example, in detail??

I think it is the only resource publicly available. It provides enough information for me. I followed it to do my build. But It assumes that you are seasoned Gradle programmer so that details are not explained.

Try it out yourself. If you got any problem/question, why not raise a new issue here.

1 Like

Im getting this message ‘.classpath Not Found’, where do I need to specify this?
image

Usually .classpath file is there in the project directory, as follows:

:webdriverfactory4ks [develop]$ ls -la
total 88
drwxr-xr-x   31 urayamakazuaki  staff    992  4 25 05:22 .
drwxr-xr-x  150 urayamakazuaki  staff   4800  4 22 23:26 ..
-rw-r--r--    1 urayamakazuaki  staff  12660  4 25 05:22 .classpath
-rw-r--r--    1 urayamakazuaki  staff   1519  4 25 05:15 .project
drwxr-xr-x    3 urayamakazuaki  staff     96  3 12 05:48 .settings
drwxr-xr-x    2 urayamakazuaki  staff     64  3 12 05:48 Checkpoints
drwxr-xr-x    2 urayamakazuaki  staff     64  3 12 05:48 Data Files
drwxr-xr-x    3 urayamakazuaki  staff     96  3 14 06:55 Drivers
...

The .classpath file is created by Katalon Studio (to be precise, by Eclipse underneath) when you open the project with Katalon Studio GUI. If you miss the .classpath file, you need to reopen the project with the GUI to let Katalon to generate .classpath file.

Why the .classpath file is missing? — I just guess, you are using Git and you are ignoring .classpath in the .gitignore file. If so, the .classpath file is not included in the repository. You cloned the project out of the repository, and immediately after cloning out without opening it with Katalon Studio GUI, you typed gradle build in the command line. Then you would certainly see

.classpath Not Found. No dependency added!

Ignoring .classpath is a common practice. I do the same. Just memember you need to reopen the project with KS to let .classpath file is refleshed.

2 Likes

Im getting this error now:
image

My build.gradle looks like this:
plugins {
id ‘java’
id ‘groovy’
id ‘com.github.johnrengelman.shadow’ version ‘4.0.4’
id “com.katalon.gradle-plugin” version “0.0.6”
}

repositories {
jcenter()
mavenCentral()
}

sourceSets {
main {
groovy {
srcDirs = [‘Keywords’, ‘Include/scripts/groovy’]
srcDir ‘Libs’
}
}
}

katalon {
dependencyPrefix = “com.katalon”
minimize = false
}

// execute gradle&#32;katalonPluginPackage

ext {
groovyVersion = ‘2.4.9’
logbackVersion = ‘1.2.3’
slf4jVersion = ‘1.7.2’
}

task groovydoc(type: Groovydoc, overwrite: true) {
source = sourceSets.main.groovy
classpath = sourceSets.main.runtimeClasspath
//options.memberLevel = JavadocMemberLevel.PUBLIC
include ‘ert/plugin/.groovy’
exclude '/*Test.groovy’
exclude '
/
.json’
}

Even after this error the plugin is getting created but another issue is that it is creating plugin for all the custom keywords available in my project and not specifically for the ones which I had mentioned in katalon-plugin.json file.

Any idea why?

The Gradle plugin “com.katalon.gradle-plugin” is designed to behave as is.

@devalex88,

any comment?

1 Like