I have made a GitHub project to share my findings. I made it publicly accessible.
Problem to solve
I want to automate the following tasks by scripts in Katalon Studio:
- to take snapshot of current DOM of Web Page in browser, save it into a local file in the MHTML format.
- to take full screenshots of a web page in multiple viewport sizes (width*height) just as I can do in Chrome DevTools using Device Mode like Galaxy S5, Pixel 2, iPhone 6/7/8, iPad. I want screenshots in the size of PC display of XGA (1024px,768px) and SXGA (1280px,1024px) as well. I want to save the screenshots as *.png files.
- Into your Katalon Studio, you need to install the Katalon Studio Plugin: Chrome DevTools Protocol Integration
- Download the zip file of this project from the zip file from the Releases page, unzip and open it with your local Katalon Studio.
- I tested this project with KS ver 7.2.2. But the older versions should do.
- You should choose
Google Chromeas browser to run the demo scripts
Description of the demos
Save web page as MHTML
- Run Test Cases/Save web page as MHTML
- It will open a web page ‘http://demoaut.katalon.com/’ and save it as a single MHTML file
- It will write a file tmp/snapshot.mht
Capture full page in multiple viewport sizes
- Run TestCases/Capture full page
- It will open a web page ‘http://demoaut-mimic.kazurayam.com/’
- It will take a full page screenshot of the page
- It will write a file tmp/screenshot.png
In October 2020 in a post in the Katalon Forum, Katalon 7.8 beta was announced where included a new Feater called Time Capsule .
Time Capsule captures the state of your application when the test fails. No not a picture of your application when the test fails but the actual HTML, CSS of the application. Yes, that also includes the visual aspect of your application as well.
ThanTo, Katalon Developer, added a description about this feature.
Time Capsule captures the MHTML of the page when a test fails.
MHTML, an initialism of MIME encapsulation of aggregate HTML documents, is a web page archive format used to combine, in a single computer file, the HTML code and its companion resources (such as images, Flash animations, Java applets, and audio and video files) that are represented by external hyperlinks in the web page’s HTML code. The content of an MHTML file is encoded using the same techniques that were first developed for HTML email messages, using the MIME content type multipart/related. MHTML files use a .mhtml or .mht filename extension.
OK. Now I understand what MHTML is.
Inspired by the Time Capsule, I got a new requirement to myself. I want a Custom Keyword which enables me to save snapshot of web page opened in browser into a single file in MHTML format . I want to use the keyword as often as I like during WebUI testing in Katalon Studio. For what? — For my VisualTestingInKatalonStudio project. The current version of this project takes web page screenshots in raster images of PNG format. I thought that MHTML might be better than PNG for recording the web page. Especially when I want to do Visual Testing for multiple viewPortSizes, the MHTML would be useful.
I had another motivation. In another disucusion in the Katalon Forum I was informed of a commercial product Percy, Visual Testing as a service. Percy does visual page comparison, but they wrote “it is not designed to accept screenshots, but instead captures DOM snapshots and page assets (CSS, images, etc.)”. — How they do that? In October 2020 I had no idea. But now in November I have got a guess — Percy possibly utilizes MHTML.
So I have studied how to save pages in MHTML within Katalon Studio. I have learned some points that might be interesting for Katalon users. I have developed a demo project, and here I share it.
Chrome DevTools Protocol supports taking snapshot of current DOM
I happend to find the following thread in the Puppeteer issues.
This suggests that Chrome DevTools Protocol (also known as CDP ) supports
Page.captureSnapshot method, which turns the current DOM of web page into a String as MHTML.
This looked exactly what I want to do! I realised that my question was how to make use of
Page.captureSnapshot method of CDP in Katalon Studio.
Chrome DevTools Protocol (CDP)
But what is “Chrome DevTools Protocol” at all? Quoting from :Chrome DevTools Protocol
The Chrome DevTools Protocol allows for tools to instrument, inspect, debug and profile Chromium, Chrome and other Blink-based browsers. Many existing projects currently use the protocol. The Chrome DevTools uses this protocol and the team maintains its API.
Instrumentation is divided into a number of domains (DOM, Debugger, Network etc.). Each domain defines a number of commands it supports and events it generates. Both commands and events are serialized JSON objects of a fixed structure.
Katalon Plugin: Chrome DevTools Protocol Integration
Integrate Chrome Devtools Protocol with Katalon Studio using https://github.com/kklisura/chrome-devtools-java-client.
This plugin bundles the jar of Chrome DevTools Java Client and make it available to Katalon projects.
And this plugin provides a class
com.katalon.cdp.CdpUtils which implements a factory method that creates an instance of
com.kms.katalon.core.webui.dirver.DriverFactory . Provided with this utility, we can use CDP Java Client API together with Katalon’s WebUI API.
Chrome DevTools Java Client by Kenan Clisura
What is Chrome DevTools Java Client? Quoting from its README:
Chrome DevTools Java Client is a DevTools client - in Java.
Of course this library should work in the Groovy scripts in Katalon Studio. The list of examples would best describe what we can do using Chrome DevTools Protocol in Katalon Studio.
BlockUrlGivenPatternExample.java — opening
http://github.comwhile blocking some links from fetching; specifed by Ant Fileset pattern such as
BlockUrlExample.java — opening
http://github.comwhile blocking some links from fetching; specifed by filename extentions such as
- CssCoverageExample.java — call up CSS Coverage inspection to find unused CSS Code
DumpHtmlFromPageExample.java — print HTML source (
document.documentElement.outerHTML) of the page
FullPageScreenshotExample.java — taks a full page screenshot of
http://github.comand save it into a local file in PNG format
- HighlightElementExample.java — Highlits an element in the same way as the Chrome inspector does
- IncreasedIncomingBufferInTyrusExample.java — quoted from the source: This example demostrates how to increase incoming buffer size in Tyrus client. Use the following example to fix the issues with Tyrus Buffer overflow. This issue occurs when the incoming size is larger than the incoming buffer in Tyrus client at which the client disconnects.
- InterceptAndBlockUrlsExample.java — Intercept and block per URL.
- LogRequestExample.java — Log HTTP requests to linked URLs from a page, print the detail such as Method and URL.
PassingCustomeHeadersToRequests.java — adding HTTP headers to the intercepted requests. E.g, you can insert
Authorizationheader for Basic AUTH.
- PerformacneMetricsExample.java — show Performance metrics measured by Chrome DevTools
- PrintingPageToPdf.java — print web page as a PDF to save into a local file.
SimpleNavigateToUrlExample.java — do the same as
- TakeScreenshotExample.java — take the screenshot of the current viewport
- TracingExample.java — show what Chrome’s Trace Event Profile Tools (about:tracing) records for diagnosing performance problems.
Katalon Studio has been solely dependent on Selenium WebDriver for Web UI testing. What we could do in Katalon Studio stays within the scope of WebDriver. Now I see, Chrome DevTools Protocol enables Web UI testers to do something exceeding the capability of WebDriver. For example, we can save web page into MHTML easily. CDP is interesting. I would look at CDP more deeply looking forward to its better use in Katalon Studio.