I did not took the time to read this thread before. And I feel bad now cause it is very good and add an important value to Katalon. And to my projects.
You did a pretty fine job, @kazurayam
I have updated VisualTestingInKatlaonStudio to version 1.8.0.
This version supported a new feature âignoring web elementsâ. See Step by step instruction, section #10 for detail.
You can specify specific HTML elements to be painted with grey rectangles in the screenshot as follows.
The elements painted grey are effectively ignored when a pair of screen shots are compared.
Hi @kazurayam
Wonderful feature. This is irrelevant to that, but I suggest you create some sort of index (or table of content) since your README file is long. I believe it will prove useful to people who are looking for particular features to identify if your project offers it immediately.
Regards !
I have added Table Of Contents to the Step by step instruction document. TOC is generated by DocToc
good work.
Hi @kazurayam,
Thank you for the great work.
Iâm looking at using this in a project, but I have two questions/problems. I was wondering if you or anyone else has any ready-made solutions for these issues?
(1.) Is there an easy way to compare screenshots that arenât taken at the same time and arenât the most recent screenshots?
In the sample project you provided, screenshots are taken from two different environments and then compared. Instead of taking screen shots from two different environments at the same time, I would like to compare screenshots from different releases of the same environment. For example:
- 2019-01-01: Take screenshot of website after v1.0 release.
- 2019-02-30: Take screenshot of website after v1.1 release and diff with old screenshot.
Basically, I want to remove the âcompare two most recent materialsâ restriction. Is there an easy way to do this currently without making major changes to the Materials library?
(2.) Is there a easy way to mark tests as failed if images are different?
In the sample project, diffing the images is done after the actual tests are run, so itâs not possible to mark the tests as âfailedâ if the images differ. Is there an easy way to diff the images during execution and mark tests as failed if the images are different?
Regards,
eb
I am currently working on enhancement of âVisual Testing in Katalon Studioâ. The typical usecase in my mind is to compare screenshots Before and After any work on a production Web system.
I would take a set of screenshots (may be some hundreds of pages) before any work (application upgrade, applying security patches, DB data migration, network resources reconfiguration, etc) around my Web App. After a few hours once the work is done, I would take another set of screenshots of the same Web App. I will compare the 2 sets of images. If the 2 sets are completely identical, ok, I can go home with relief. But when the comparison revealed any significant image differences, I must have made silly mistakes and caused unplanned/unexpected changes in the system. I will go back to the system and will fix problems BEFORE my customers notice it.
I think I can release the new version in a month by the end of April 2019. This enhancement required quite significant changes of the Materies library, but the job has been done and near completion.
My current idea is to comapre the latest status of a web system against a previous status specifying a timestamp. Your requirement (@eb1, comparing 2019-01-01 and 2019-02-30) is a bit different, but principally similar type of problem. I can imagine a future enhancement to support comparing 2 time slices specifying 2 timestamps.
The coming new version will be capable of marking tests as failed if âone or more significant enough differences foundâ.
I am currently working on a enhancement. The typical usecase in my mind is Compare screenshots Before and After any work on a production system .
The coming new version will be capable of marking tests as failed if âone or more significant enough differences foundâ.
Awesome! Good to know. I will look forward to the next release!
Is there an easy way to compare screenshots that arenât taken at the same time
and arenât the most recent screenshots?
After a bit of thought, I realized that it is possible to fulfill the first part of this requirement by making a few changes to the test suite.
Workaround: how compare images across two different test executions:
Changes:
- Delete the call to âTest Suites/cleanMaterialsâ in the test suite. This prevents our old screenshots from being deleted.
- Delete one of the two calls to âTest Suites/Main/TS1â. Leave the one that uses the profile of the AUT you want to test. This way only one set of screenshots will be taken on each test execution
Execution workflow:
- Run the tests on your pre-deploy âBeforeâ app.
- âBeforeâ images will be saved to the materials folder with timestamp
- âindexâ file will be generated but no image diff will be performed
- Deploy the changes to your app.
- Run the tests on your post-deploy âAfterâ app.
- âAfterâ images will be saved to the materials folder with the most recent timestamp
- The most two recent sets of screen shots will be diffed, and the diff results will be written to the âindexâ file.
- note: If you have run the tests between steps #1 and #3, you will need to move or delete any sets of screenshots that were saved between your âBeforeâ and âAfterâ screenshots
Thank you again for putting this all together, and Iâll look forward to the upcomming release!
-eb
Last weekend, the new version of âVisual Testing in Katalon Studioâ has come up and running on my Mac. Still I need several weekends to write sufficient documentations.
Would it be premature to share it under âdevelopâ or a feature branch on github?
Thanks,
eb
Let me announce the release of new version 1.10.0 of VisualTestingInKatalonStudio project.
Changes include:
- Chronological approach of screenshot-comparison testing of a single URL, namely âChronosâ, was newly supported in addition to the existing approach of comparing 2 URLs (dev & production) of a web application, namely âTwinsâ.
-
Materials/index.html
now presents 3 screenshot images (Back > diff > Forth) in Carousel format. - Gradle Plugin com.github.kazurayam.visualtestinginks has been newly developed. This Gradle Plugin makes resources (Test Cases, Test Suites, Test Objects, Test Listeners, Keywords) of VisualTestingInKatalonStudio portable to other projects.
Please start with the revised README doc
Katalon Studio v6.1.5 was released. This version includes:
- Added the capability to display all user folders/files in Test Explorer.
Now you can view the <projectDir>/Materials/index.html
file generated by VisualTestingInKatalonStudio with browser inside the Katalon Studioâs GUI. You do not have to open a browser window outside KS.
I would recommend to you to upgrade KS to v6.1.5 if you are going to evaluate VisualTestingInKatalonStudio.
You can also edit and save .gitignore
, README.md
file. Any other type of text files including *.js
, *.css
, *.bat
, *.sh
, *.properties
, *.xml
and *.json
. It seems that plain (no syntax-highlighting) editor is only available.
Here is an example output from VisualTestingInKatalonStudio in action
Hello Kazurayam,
The link which is shared for step by step instruction for ignoring web elements - > Says page not found
https://github.com/kazurayam/VisualTestingInKatalonStudio/blob/master/docs/Step_by_step_instruction.md
Can you please share the new link if it is moved from current path.
You want to visit the following url: