Running test cases with gradle

Product Forums Katalon Studio
, , ,
1

I have created a folder in my Katalon Studio project, for unit testing.

It has become pertinent to do the unit testing on some keyword classes in my code base, as there is necessarily a lot of moving parts involved with some of them (due to business requirements), and it’s too easy to fix some issues while introducing a family of others…

I have the unit test cases written:

  • Unit Tests
    • Rate Increase Model Handlers
      • RateIncreaseExpectationsHandler
        • Write new expectations model - Full
      • RateIncreasePracticeModelHandler
        • Outputted Excel File - Full
        • DiscountChildModelListHandler

// and many others, that I don’t care to enumerate on here…

What did you try?

I ask ChatGPT how we can gradle test the project, via hitting just those test cases, what it tells me, makes only half sense…

Why it doesn’t make the full sense…?

Yes, we create the test task, so that we can…well…gradle test our project (and have a GitHub Action do the same down the line, once we have this working), but…

…the test scripts don’t compile to their own .class files, at least none that I can find in the bin folder…

As a matter of fact, when I hit up my project’s bin/testcase folder, I see that it is empty!

I can’t find any resources on how to do this…

How can we run these test cases?

2

I do unit-tesiting using JUnit4 for my custom Keyword classes in Katalon Studio, as described in the following post:

1 Like
3

What is the test task you mean?
In which folder do you have the task?

4

Per the ChatGPT question, I wrote the following into build.gradle:

test { 
  useJUnitPlatform()

  include '**/RateIncreaseExpectatationsHandler/Write new expectations model - Full.class'
}

However, I face the issue, that it cannot find my test classes:

> Task :katalonPluginAddDependency
Caching disabled for task ':katalonPluginAddDependency' because:
  Build cache is disabled
Task ':katalonPluginAddDependency' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
Resolve mutations for :katalonPluginAddResource (Thread[Execution worker,5,main]) started.
:katalonPluginAddResource (Thread[included builds,5,main]) started.

> Task :katalonPluginAddResource
Caching disabled for task ':katalonPluginAddResource' because:
  Build cache is disabled
Task ':katalonPluginAddResource' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
Resolve mutations for :compileJava (Thread[included builds,5,main]) started.
:compileJava (Thread[included builds,5,main]) started.

> Task :compileJava NO-SOURCE
Skipping task ':compileJava' as it has no source files and no previous output files.
Resolve mutations for :compileGroovy (Thread[Execution worker,5,main]) started.
:compileGroovy (Thread[Execution worker,5,main]) started.

> Task :compileGroovy NO-SOURCE
Skipping task ':compileGroovy' as it has no source files and no previous output files.
Resolve mutations for :processResources (Thread[included builds,5,main]) started.
:processResources (Thread[Execution worker,5,main]) started.

> Task :processResources NO-SOURCE
Skipping task ':processResources' as it has no source files and no previous output files.
Resolve mutations for :classes (Thread[Execution worker,5,main]) started.
:classes (Thread[included builds,5,main]) started.

> Task :classes UP-TO-DATE
Skipping task ':classes' as it has no actions.
Resolve mutations for :compileTestJava (Thread[Execution worker,5,main]) started.
:compileTestJava (Thread[included builds,5,main]) started.

> Task :compileTestJava NO-SOURCE
Skipping task ':compileTestJava' as it has no source files and no previous output files.
Resolve mutations for :compileTestGroovy (Thread[included builds,5,main]) started.
:compileTestGroovy (Thread[included builds,5,main]) started.

> Task :compileTestGroovy NO-SOURCE
Skipping task ':compileTestGroovy' as it has no source files and no previous output files.
Resolve mutations for :processTestResources (Thread[Execution worker,5,main]) started.
:processTestResources (Thread[Execution worker,5,main]) started.

> Task :processTestResources NO-SOURCE
Skipping task ':processTestResources' as it has no source files and no previous output files.
Resolve mutations for :testClasses (Thread[Execution worker,5,main]) started.
:testClasses (Thread[Execution worker,5,main]) started.

> Task :testClasses UP-TO-DATE
Skipping task ':testClasses' as it has no actions.
Resolve mutations for :test (Thread[Execution worker,5,main]) started.
:test (Thread[Execution worker,5,main]) started.

> Task :test NO-SOURCE
Skipping task ':test' as it has no source files and no previous output files.
5

@mwarren04011990

You posted:

This error indicates that the :compileTestJava task does not know where your Groovy source files are located. Have you configured the SourceSets { ... } appropriately? Maybe not. You haven’t informed Gradle where in the Katalon project’s directory structure it should look into for the source; therefore it failed.

Let me elaborate this a bit. I made a sample Gradle project as follows (I used Gradle v8.0.2):

$ cd tmp
:~/tmp
$ cd sampleGradleProject
-bash: cd: sampleGradleProject: No such file or directory
:~/tmp
$ mkdir sampleGradleProject
:~/tmp
$ cd sampleGradleProject/
:~/tmp/sampleGradleProject
$ gradle init
Starting a Gradle Daemon (subsequent builds will be faster)

Select type of project to generate:
  1: basic
  2: application
  3: library
  4: Gradle plugin
Enter selection (default: basic) [1..4] 3

Select implementation language:
  1: C++
  2: Groovy
  3: Java
  4: Kotlin
  5: Scala
  6: Swift
Enter selection (default: Java) [1..6] 2

Select build script DSL:
  1: Groovy
  2: Kotlin
Enter selection (default: Groovy) [1..2] 1

Generate build using new APIs and behavior (some features may change in the next
Project name (default: sampleGradleProject): 
Source package (default: samplegradleproject): 

> Task :init
Get more help with your project: https://docs.gradle.org/8.0.2/samples/sample_building_groovy_libraries.html

BUILD SUCCESSFUL in 33s
2 actionable tasks: 2 executed
:~/tmp/sampleGradleProject

$ tree .
.
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradlew
├── gradlew.bat
├── lib
│   ├── build.gradle
│   └── src
│       ├── main
│       │   ├── groovy
│       │   │   └── samplegradleproject
│       │   │       └── Library.groovy
│       │   └── resources
│       └── test
│           ├── groovy
│           │   └── samplegradleproject
│           │       └── LibraryTest.groovy
│           └── resources
└── settings.gradle

You can see the default directory structure that Gradle v8.0.2 assumes your project to have:

  • your groovy sources should be placed in the lib/src/main/groovy
  • your groovy test sources should be place in the lib/src/test/groovy

All of Gradle tasks (including :compileGroovy, :compileTestGroovy) assumes your project looks as this.

On the other hand, a Katalon Studio project has a directory structure which is completely different from the Gradle’s default.

$ tree -L 3 .
.
├── Checkpoints
├── Data Files
├── Drivers
├── Include
│   ├── config
│   │   └── log.properties
│   ├── features
│   └── scripts
│       └── groovy
├── Keywords
├── Libs
│   ├── CustomKeywords.groovy
│   └── internal
│       └── GlobalVariable.groovy
├── Object Repository
├── Plugins
├── Profiles
│   └── default.glbl
├── Reports
│   └── Self-healing
│       └── broken-test-objects.json
├── Scripts
├── Test Cases
├── Test Listeners
├── Test Suites
├── bin
│   ├── groovy
│   ├── keyword
│   ├── lib
│   │   ├── CustomKeywords.class
│   │   └── internal
│   ├── listener
│   └── testcase
├── build.gradle
├── console.properties
├── settings
│   ├── external
│   │   ├── com.kms.katalon.composer.execution.settings.properties
│   │   └── com.kms.katalon.core.db.DatabaseSettings.properties
│   └── internal
│       ├── com.kms.katalon.composer.testcase.properties
│       ├── com.kms.katalon.execution.properties
│       ├── com.kms.katalon.execution.webui.properties
│       ├── com.kms.katalon.integration.analytics.properties
│       └── com.kms.katalon.integration.qtest.properties
└── skeletonProject.prj

If your project has a unique directory structure which is different from the Gradle’s default, you have to configure your Gradle project to be aware of the directory structure of your project. You have to write a lot of lines in the build.gradle and possibly in the settings.gradle. Theoreticaly it is possible as some articles describe it. See also the official document.

6

You wrote you diid:

This solves nothing.

Why? ---- I would not disscuss this, as it’s just out of focus. I would suggest to you to study the above-mentioned Baeldung article.

7

I have studied for years how to implement it. I have tried several approaches and found what works and what doesn’t.

In my humble opinion, it is a poor idea to try to run “$ gradle test” task in a Katalon Studio project. Why? — It is too messy to configure a Gradle project to recongnize the directory structure of a Katalon Studio project for source codes. Also it is very cumbersome to include the Katalon Studio’s core libraries into the compilation classpath of the Gradle project.

You should know, Katalon Studio is not designed with Gradle in mind at all. These are 2 different beasts. They would not mate well.

8

When I need to develop a set of application classes and use it in my Katalon Studio project, I would create an independent project “MyLib”, outside Katalon Studio.

“MyLib” will be on top of Gradle written in Java/Kotlin/Groovy language. I would develop it using IntelliJ IDEA. The MyLib project will have the directory structure as the Gradle’s default. I would build a jar of the MyLib project’s artifacts utilizing the full features of Gradle, e.g, “$ gradle build”, with the default configuration.

I would export the produced jar into an external Maven repository and import it into a Katalon Studio project’s Libs directory. Let me show you an example of this sort of “MyLibs”:

I intended the “timekeeper” library to be reused in any automated UI testings on JVM; Katalon Studio is one of them. The “timekeeper” project is an independent Gradle project. It produces a jar. I would import the jar into my Katalon projects and use it.

I would design my code to be organized into 2 groups of packeges:

  1. a group of packages that is dependent on com.kms.katalon.** classes → can only be located inside a Katalon Studio project
  2. a group of packages that is not dependent on com.kms.katalon.** classes → can be located outside Katalon Studio project, in any Gradle project

I always try to minimize the size of the group1 codes (com.kms.katalon.**-dependent classes).

9

How can we trigger this from a GitHub action?

10

I think you should find out a way why to make “$ gradle test” successfully pass on your machine without GitHub Action. I think that you haven’t found it yet, right?

11

It seems you want to ask others how to run a gradle build in GitHub Action.

Just read the manual: Executing Gradle builds on GitHub Actions . But I think that it is not an issue to be raised in this Katalon user forum.

If you want to run $ gradle test task in a Katalon Studio project on GitHub Actions to test your com.kms.katalon.**-dependent classes, you need a Katalon Runtime Engine license and somehow you need to enable KRE to run on GitHub Actions. I have never tried that. I do not even know if it is possible or not.

In some projects, I do unit-tests of com.kms.katalon.**-independent calsses on GitHub Actions as a part of Pull-Request processing. I can do it without KRE, of course. On the other hand, I would never want to do unit-tests of com.kms.katalon.**-dependent classes on GitHub Actions, simply because I do not need it. I always make good efforts to minimize the size of com.kms.katalon.**-dependent classes, so the manual testing on my local machine using the junit4ks library inside Katalon Studio is enough. I do not need any CI cycle for my com.kms.katalon.**-dependent classes.

12

Interesting!

Did you ask questions to ChatGPT? I am impressed that ChatGPT replied to you in a way as if it is an expert about the issue: how to make ‘$ gradle test’ task run successful in a Katalon Studio project. ChatGPT is amazingly successful in pretending.

I read your conversation with ChatGPT at the link: https://chat.openai.com/share/72ad9a75-b8c2-4caa-a40b-7b9a33487e7e. Their answer contained some mistakes. I believe that ChatGPT had made no hands-on experiments for you before it returned the advises, which is bad and insufficient as an expert adviser.

Why did ChatGPT make mistakes? — because ChatGPT has not learned a knowledge yet. That is, the directory structure of a Katalon Studio project is completely different from the directory structure that Gradle assumes as default. Therefore the default setup of a Gradle project does not fit to a Katalon project. You have to explicity configure the Gradle project in far more detail so that it work on a Katalo project. Well, possibly up until now, nobody has ever written about this issue. There is no readable stuff has been published on the Internet. Therefore ChatGPT has got no chance to learn the issue deeply.

@mwarren04011990

I think that you, an innocent believer, were confused by the ChatGPT’s fluent but faulty answer.

However, it is likely that in future (days, weeks, months, years, … i don’t know), ChatGPT finds this URL and learn it. Then it may give you a better answer : “you shouldn’t do it”.

1 Like
13

Yeah, I figured so.

Also, I found a simpler way… After rethinking about what I’m asking, a subset of test cases in the unit tests folder to run, as a guardrail against any pull requests being merged to the master branch…

I should just create a test suite just for the unit tests, and then use a GitHub action to run the test suite on pull request or push to master branch:

https://docs.katalon.com/docs/execute/cicd-integrations/katalon-studio-github-action

Way simpler, without having to delve into Gradle. Gradle is a YAGNI in this case it seems…