Testing YouTube video in Katalon Studio

I have made a demo project:

-----

Problems to solve

You can make a web page which embeds a YouTube video. YouTube video itself is usually marked up with video element like this:

<video src="https://www.youtube.com/watch?v=Q80JTXYIteU">

And the page is expected to have a start/stop button like this:

<button class="ytp-play-button ytp-button" aria-label=Play>    ...</button>

When a web page with embeded video loaded, the video may autoplay on load or stay still. It depends on how the page is designed and configured. I want to verify if the video element autoplays as expected, or it stays still as designed. I want to do the verification in Katalon Studio.

Solution proposed

aShot, WebDriver Screenshot utility, enables you to take a screenshot of a selected WebElement (e.g. <video">). Also aShot enables you to compare 2 images and know how much different they are. This Katalon Studio project uses aShot. This project provides a set of Custom Keywords in Katalon Studio, which wrap the aShot API. Also the project provides some example Test Cases in Katalon Studio which shows how to make use of the keywords.

You will realize another problem. If you take screen shots, you would inevitably want to save images into files on your local disk. But how you manage the paths of generated files? It is a cumbersome problem. My Materials project provides a smart solution for managing & resolving paths of image files dynamically created by test cases on Katalon Studio.

Demonstration

Input

The demo will open the following URL. Both embeds the same video clip: ‘Kataln Studio - Quick start’.

  1. YouTube: Katalon Quick start
  2. Katalon Docs: Quick start

How to run the demo

  1. download the source ZIP file of this demo project from the releases Page (version 0.4 or higher)
  2. unzip it
  3. start your Katalon Studio, open the TestingYouTubeVideoInKatalonStudio project.
  4. open Test Cases/examples/vefiry-video-autoplay-example and run it. You can use either of Firefox or Chrome.
  5. Or you open a Test Suite Collection named Test Suites/Execute vefiry vidoes, and run it by clicking the Executebutton Execute

Output

Running Test Cases/examples/vefiry-video-autoplay-example will result <project folder>/tmp/video directory. tmpvideo

In the folder you would find 3 image files.

  • Katalon Studio - Quick start_1st.png 1st
  • Katalon Studio - Quick start_2nd.png 2nd
  • Katalon Studio - Quick start_diff(100.00) diff

The 1st screenshot and 2nd screenshot are completely different. Therefore the difference image is painted 100% red. The test case PASSES as it expects the video to autoplay, and the video actually autoplays.

Code description

Custom keywords:

Test Listener

Test Case:

there are a few more Test cases under Test Cases/main/video folder. These are called by the Test Suite Test Suites/Katalon Studio Quick start. This test suite is a component of the Test Suite Collection Execute - verify video. These test cases are almost simmilar to verify-video-autoplay-example though some parameters’ value are different.

What happens? when you encounter a real YouTube trouble

At 10:30AM 17,Oct 2018 JST, I encountered a real YouTube trouble. When I requested https://www.youtube.com/watch?v=Q80JTXYIteU with browser, the page was responed. But the video in the page was black and showed a message ‘sorry for inconvenience’. Wow! What a good chance to evaluate my study I got.

I ran the Test Cases/verify-video-autoplay-example. The test case failed AS EXPECTED. encounted_YouTube_trouble_1d YouTube trouble 1

The test case emitted an ImageDiff as follows: encountered diff

This incident proved, just accidentally, that my approach of verifying YouTube video autoplay or not is capable of detecting actual YoutTube problems when they happen.

Step by step instruction to create your YouTube video test

If you want to create a new Katalon Studio project which verifies a YouTube video embeded in your target web page, please do as follows:

  1. create a new Katalon Studio project for you
  2. import the jar file of aShot into the project. see the Katalon document How to import external library into your automation project . The jar of aShot is here. Or you can reuse the ashot-x.x.x.jar bundled in the demo project’s ./Drivers folder.
  3. import the jar file of Materials. The jar file is donwloadable at Releases page. Or you can reuse the Materials-x.x.x.jar bundled in the demo project’s ./Drivers folder. Drivers
  4. create 3 custom keywords: com.kazurayam.ksbackyard.Assert, com.kazurayam.ksbackyard.ScreenshotDriver and com.kazurayam.ksbackyard.YouTubeVideoVerifier. Copy the Groovy sources from the demo project into your replication. You do not have to modify the source of those keywords. Keywords
  5. create Test Listeners/TL. Copy the Groovy source from the demo project into your replication. You do not have to modify the source.
  6. modify the default Profile and add 2 GlobalVariables: MATERIAL_REPOSITORY of type Null, CURRENT_TESTCASE_ID of type String. No inital value required. The Test Listener TL will resolve appropriate values runtime. GlobalVariables_default
  7. Finally you need to develop test cases and test suites. Create a mimic of Test Cases/examples/verify-video-autoplay-example and modify it as you would like. I have written as much comments in the Groovy source. Read them and find how to customize to meet your requirements.

Conclusion

What have I learned while developing this demo project? I learned that verifying video is a very much complicated task. My study is poor. My Custom Keywords are far short for supporting the complexities of video.

I gave up. I would not go on this issue anymore.

6 Likes

Great work. It could be a good foundation for someone’s future work.