Integration to percy.io visual testing

Do Katalon have any plans to allow/facilitate integration with Percy visual testing (now owned by BrowserStack)?

Initially, Percy only works for web, but I would imagine that mobile will follow.

If Katalon could integrate with Percy in the same way it does with Applitools, I believe this would be a fantastic addition to Katalon - not least because Percy is free (for up to 5,000 screenshots per month), whereas Applitools only allows 100 free checkpoints per month (not enough other than to trial the tool, and after that things get expensive).

Or…is there a way to integrate into Percy without a formal integration solution?

https://percy.io/

Yes, I second that, it would be great !

Hi kevin.mcandrew1 and luc.namur,
If you are still looking for a visual testing solution, we have recently released Katalon AI-Driven Visual Testing as part of TestOps. It has seamless integration with Katalon Studio, there is no effort required in setting up. Visual Testing Standard (Free) is readily available to all users with a quota of 1000 images/month. Visual Testing Professional version leverages AI to detect significant changes to the layout and content, greatly reducing false positives detected by the pixel-comparison method, its quota of images is 20,000+/month. Please check it out and let us know your thoughts.
Thank you so much.
All the best,
The Katalon Team

Hi @mai.le, thanks for the update, I’ll take a fresh look at this.

Please could I ask some questions:

  1. Please can you summarise what’s new, as such? Are there new commands in the applications, or is there a new web UI to manage baselines and snapshots, or have you just packaged it all up in a new way?

  2. Is there a full/current user guide or documentation including how to select image comparison methods (e.g. layout / content based), and how to configure ignored zones and so on?

  3. How are multiple mobile devices (and their inevitable different screen resolutions etc.) handled? For example, if I execute ‘Test Case 1’ (that takes a screenshot of a login screen and compares to the baseline) on a Samsung Galaxy S9, AND a Google Pixel 6, how are comparisons between baselines and snapshots handled? Obviously the login screen will look visually different between devices, so I wouldn’t want the test to fail on that basis?

  4. Do you have any plans to introduce this to Windows Desktop testing (this would be a fantastic addition)?

Great to see Katalon have made such good progress in this exciting area :slight_smile:

Many thanks,
Kevin

1 Like

Hi kevin.mcandrew1,
We have done major improvements to our core features. Documentation of the new features can be found at: Visual Testing Overview | Katalon Docs. Here are some highlights:

  • Support multiple baseline collections with versioning (there used to be a single set of baseline images used for all visual test runs). You can choose which baseline collection to use (or generate a new one) for each of the test run via the Schedule Test Run dialog on TO or via Katalon Command
  • Use AI to assist users with image comparison: layout-based and text-content-based methods
  • Ignore dynamic areas during comparison using the Configure ignored zones feature.

Here is a short demo clip: [Tutorial] AI-Driven Visual Testing | Katalon Platform - YouTube.

Checkpoint images and baseline images must be in the same resolutions to be compared. In your case, if the application is rendered on different devices with the same resolutions, then the comparison can be done, it will not work otherwise.

We will consider Desktop application support in the future as it is more complex, technical-wise.

Thank you so much for your interest. Please let us know if you need any help during your trial.
Best,
The Katalon Team