Visual Testing in Katalon Studio

I have released the version 1.14.0 of VisualTestingInKatalonStudio. Wtih this version, the index.html displays the name of Execution Profiles as the following example shows. Displaying Execution Profile names like “Production” and “Development” makes the report easier to understand.

If you want to enable your own Katalon Studio project to carry out the Visual Testing approach as I propose, please refer to

1 Like

I have updated the VisualTestingInKatalonStudio project to the version 1.17.3.. In this version I fixed a OutOfMemoryError: Java Heap Space problem as described at https://github.com/kazurayam/Materials/issues/17

If you have any project using the VisualTestingInKatalonStudio as framework, you can easily update the framework to the latest version. In your project, you have build.gradle file, right? You change the vt.version as follows:

plugins {
  id "com.github.kazurayam.visualtestinginks" version "0.1.21"
}
vt.version = '1.17.3'

And in the commandline, you want to do:

$ cd $yourVisualTesingProject
$ ./gradlew enableVisualTesting

then, the new version of “VisualTestingInKatalonStudio” artifacts will be automatically downloaded and located in you project. The updates to the 1.17.3 requires no changes to your test case scripts.

In the $KatalonStudioInstalledDirectory/katalon.ini file, you specify the maximum heap space allocated to Java VM of Katalon Studio. I would recommend you to write in the katalon.ini

-Xms512m
-Xmx4096m
1 Like

I have updated the VisualTestingInKatalonStudio project to the version 1.17.4.

This version added information how many screenshots were taken (PNG:x) and how long each Test Suite execution took in seconds (TIME:t). See the following screenshots for example:

When I tried to test over 300 URLs it took approximately 30minutes. I wanted to know exact figures how many URLs, how long it took. So I modified the framework.

If you have any project using the VisualTestingInKatalonStudio as framework, you can easily update the framework to the latest version. In your project, you have build.gradle file, right? You change the vt.version as follows:

plugins {
  id "com.github.kazurayam.visualtestinginks" version "0.1.21"
}
vt.version = '1.17.4'

And in the commandline, you want to do:

$ cd $yourVisualTesingProject
$ ./gradlew enableVisualTesting

then, the new version of “VisualTestingInKatalonStudio” artifacts will be automatically downloaded and located in you project. The updates to the 1.17.4 requires no changes to your test case scripts.

tag 1.23.0 implemented this.

In the Test Cases/CURA/visiteSite script, now write code that calls MaterialRepository#resolve*Path* method with argument of type MaterialDescription . You are supposed to write any description what that screenshot is about. For example, a screenshot of http://demoaut.katalon.com can have a description ホームページ in my native language. The code will look like this.

// create one more screenshot file with name in Japanese
Path png2 = mr.resolveMaterialPath(
					GlobalVariable[MGV.CURRENT_TESTCASE_ID.getName()],
					'screenshots',
					"ホーム.png",
					new MaterialDescription("1", "ホームページ"))

And the Test Suite Collection Test Suite/CURA/Execute_chronos and Test Suite/CURA/Execute_twins will produce Materials/index.html where all “diff images” will be annotated with the description which you added in the Test Cases/CURA/visitSite script.

In short, the Materials/index.html page shows descriptions of each URL’s screenshots (you assign them in your native language, in my case 日本語で). The page looks like this. Much easier to understand, isn’t it?

Hey @kazurayam

How do we enable the remote debugging in katalon.Currently I am using Katalon 7.7.2 version

I do not see what is the remote debugging at all, sorry.

I was talking about below line @kazurayam
Starting Headless (CLI)
The easiest way to get started with headless mode is to open the Chrome binary from the command line. If you’ve got Chrome 59+ installed, start Chrome with the --headless flag:

chrome \ --headless \
**# Runs Chrome in headless mode. --disable-gpu \ **
# Temporarily needed if running on Windows.
–remote-debugging-port=9222 \ https://www.chromestatus.com
# URL to open. Defaults to about:blank.

for this issue

Still I do not see what you are asking, sorry.

If you have a problem concerning the command line option for the chrome.exe, it has little to do with the ‘Visual Testing in Katalon Studio’ issue.

You should create a new post here with appropriate title on its own. Then it would better attract attentions of those who are capable to help you.

is this applicable for desktop app?
if i want to compare email content?

Unfortunately not. You need to develop a way to take PNG screenshots of your target desktop apps. I have not experience about this.
Once you can screenshots, then image-comparison by the VisualTestingInKatalonStudio is applicable, but it would be a tuff job to make a usable software project that implements that idea.

Do you want to compare EMail text messages? Then you don’t need the VisualTestingInKatalonStudio framework. All you need is

If you want to compare 2 HTML content embeded in EMail body, you need to develop a code to read EMail message and extract the HTML content, and load it to new Browser. Possibly you can read Gmail Messages by

According to the change log, version 1.2.0 of this plugin supported reading text/html message.

Though I haven’t tried extracting the embeded HTML content out of GMail messages using this plugin.

Once the embedded HTML content is loaded on browsers, then the image-comparison framework of the VisualTestingInKatalonStudio would be applicable for them. However it would be a tuff job to develop a set of codes that implements this idea.

Katalon Studio v8 is announced; they will support taking screenshots for mobile.

Previously some people asked me if the “Visual Testing in Katalon Studio” framework works on Mobile testing, and I replied “no” because I did not know how to take screenshots of mobile devices in tests. This new feature announced for KS v8 will make it possible: Visual Testing in Katalon Studio for Mobile.

However I am not sure if “Visual Testing for Mobile” is really useful or not. As for Web UI, we can take “full screenshot of a page”(= from start to end of a HTML document), and compare 2 images. But as for Mobile, there is no abstraction of “full size of page”. In many cases we can repeat swiping screen endlessly. Consequently we can not get “full size screenshot” of mobile app at all. All we can do is to take screenshot of just a small area, and our test script must be implemented intelligently enough being aware where to swipe to and to take screenshot of. This is cumbersome.

I want “VisualTestingInKatalonStudio” to be a tool to find out unplanned/silly mistakes in UI. People can not predict where they make mistakes. Therefore, I think, “VisualTestingInKatalonStudio for mobile” may not be very useful.

i do check updates
latest is 7.8 only

I wrote a post that would extend the usability of the VisualTestingInKatalonStudio framework.

The Visual Testing In Katalon Studio project is now superseded by the Visual Testing In Katalon Studio - Revived project backed by the materialstore library.