Materials, a library that resolves output file paths in a well-structured folder tree

I have made a demo project on GitHub:

----

In the ‘UsingMaterialsInKatalonStudio’ project, I will show you, step by step, how to write test scripts in Katalon Studio making use of the ‘Materials’ library.

The Groovydoc of the Materials is published here.

===

Problem to solve

What is a material? By the term Material I mean any file created by test scripts on the fly. A typical example of a Material is a PNG file as screenshot of web page taken by WebDriver API. Other examples of Material would include:

  1. PDF files downloaded from web site
  2. Excel/CSV files created by test script
  3. JSON/XML responses from RESTful API call
  4. HTTP Response Header in plain *.txt format

Selenium WebDriver and Katalon Studio provide sound support for interacting with web. But their support falls short of the following problem: which path to save a file as?

Specifying a one-off path is trivial. Say, C:\Users\myname\tmp\sample_screenshot.png would be fine. However if we are to make dozens of materials repeatedly and if we are to reuse the files after interating with web, then it becomes an itchy problem how to resolve paths for all materials appropriately.

How do I want to reuse materials? For example, I want to perform Visual Testing in Katalon Studio. I would take 30 screenshots of my web app in both of the production environment and the development environment. After taking screenshots, I want to compare pairs of images to find out if any differences are found.

Another usecase of materials is just for logging purpose. When I test RESTful API, it is likely I want to save HTTP Respose and Body into files just for logging.

My problem is that I have to design the paths for those files. I want a reusable solution for resolving material paths: a class library which implements a designed repository for the files (materials) with intuitive access methods.

===

Solution

The Materials library implements the com.kazurayam.materials.MaterialRepository object. A test script in Katalon Studio can ask the MaterialRepository object to resolve path for a material for you. The path would be in the format as follows:

${projectDir}/Materials/${testSuiteName}/${testSuiteTimestamp}/${testCaseName}/${subdirs}/${fileName}

For example,

./Materials/TS07_visit_a_web_site/20180919_132138/TC07_visit_a_web_site/1 CURA_Homepage.png

An example of file tree created is like this: TS07_tree

Also the com.kazurayam.materials.MaterialRepository object can make ./Materials/index.html file. This HTML file is a viewer, provides easy access to the files contained in the repository. The following picture shows an example of the ./Materials/index.

index.html screen shot shown in modal
index index_modal

===

How to set up

This project depends on the jar provided by the Materials project on GitHub. The jar files are downloadable from the Releasespage. You can import the jar file into your Katalon Studio project as an External library.

However ${projectDir}/Drivers/Materials-0.17.jar is already bundled in this demo project.

Description of codes

I will describe all test scripts one by one. Please retrieve each pages by clicking the links.

Test Cases

Test Suites

Test Suite Collections

===

Possible usecases

Not only for screen shots but other types of files

The resolveMaterialPath method of MaterialRepository returns a java.nio.file.Path object = the location of a file. It does not read or write java.io.File object. This means that the Materials library does not restrict the types of files to be stored in the MaterialRepository.

This demo project describes how to store screen shots into the <projectDir>/Materials folder. Of course, it is a typical usecase. However you can store other types of files there: PDF files downloaded from web, Excel files createdy by test scripts on the fly, XML and JSON responded Web API services, etc.

4 Likes

cool i think i will need this… thank you sir +1

how can i utilize this to store the json response in a file after executed each API test cases POST something?
how can i call the stored json response file and make use it on next test case or test step to verify the response?

I have updated the Materials library to v0.72.2, and revised the README doc to cover how to use it for API testing. See

Let me quote from the README doc:


Applying the Materials library to RESTFul API Tests

An example Test Suite Test Suites/webservice/openweathermap/TS is available. You can read the source of test cases to find out how you can make use of the Materials library for Web Service/API testing.

Just open the Test Suite Test Suites/webservice/openweathermap/TS and run it specifying the Execution Profile webservice_profile . The test will pass. Please read the log and source code to see what was done.

The Test Suite comprises with 4 Test cases:

  1. Test Cases/webservice/TC10_clearMaterials
  2. Test Cases/webservice/TC11_saveData
  3. Test Cases/webservice/TC12_verifyData
  4. Test Cases/webservice/TC14_makeIndex

The test case TC11_saveData makes a HTTP request to the URL which returns a JSON document. The test case saves the JSON document into a file. The file path would be in the format of:

<projectDir>/Materials/<Test Suite Id>/<Test Suite Timestamp>/<Test Case Id>/<sub dirs>/<file name>

for example

UsingMaterialsInKatalonStudio/Materials/webservice.openweathermap.TS/20190918_090636/webservice.TC11_saveData/weatherData.json

What is the value of this path format resolved by the Materials library?

Please imagine that I execute this test regularly and repeatedly — once a day, for 30 days. Because the path contains yyyyMMdd_hhmmss portion, I can store all 30 JSON files, I can refrain from overwriting the file at each run.

Please imagine that I have 2 or more Test Cases which store JSON files locally. Because the path contains <Test Case Id>/<sub dirs> portion, I can automatically organize the file tree of output files by Test Case Id.

How the code works

The test case uses the Materials library to resolve the file path.

The test case TC11_saveData will save the path value into a GlobalVariable.dataPath.

Next, the test case TC12_verifyData is invoked by the TS.

The test case TC12_verifyData will read the GlobalVariable.dataPath for the path info. The test case will read the weatherData.json file, parse the JSON using groovy.json.JsonSlurper to obtain an ordinary Groovy object.

You can make any verification over the object using ordinary assert statement of Groovy language, or you can use any Assertion libraries of your choice. The test case TC12_verifyData demonstrates how to use com.kazurayam.ksbackyard.Assert class, which works well with the Exception handling and reporting mechanism of Katalon Studio.

1 Like

Hi,

I have a problem while creating index file. Whenever i run a test suite the execution is not being updated for current test suite. On the second run it updates the execution status of previous test suite.

I have added the following code:

MaterialRepository mr = (MaterialRepository)GlobalVariable.MATERIAL_REPOSITORY
assert mr != null

String testCaseId = (String)GlobalVariable.CURRENT_TESTCASE_ID
assert testCaseId != null

Path pngFile = mr.resolveMaterialPath(testCaseId, testCaseId+‘.png’)

WebUI.takeScreenshot(pngFile.toFile().toString())
WebUI.comment(“saved the screenshot into ${pngFile.toAbsolutePath().toString()}”)

mr.makeIndex()

Thanks

You have a Test Suite, right?

The sample project UsingMaterialsInKatalonStudio has a TestListener class named MyTestListener. The #68 has the following fragment:

        // inform the MaterialRespository of the current Test Suite
		mr.markAsCurrent(testSuiteId, testSuiteTimestamp)

This line is important.

Have you implemented a TestListener class equivalent to this in your project? I guess you haven’t.

If you have a Test Suite comprising of 2 or more Test Cases, then you should create a new Test Case , for example named makeIndex, where you want to write shortly

mr.makeIndex()

and append it at the end of the Test Suite.

All Test Cases which take screentshots need NOT call mr.makeIndex(). You should call mr.makeIndex() only once in a Test Suite run. No need to repeat creating the Materials/index.html file.

Yes i have a test suite and i have already implemented mr.markAsCurrent(testSuiteId, testSuiteTimestamp) in test listner as shown below.

I will do that. Thanks

I have abandoned this Materials project. I will no longer maintain this. It was succeeded by another project:

The materialstore project shares the objectives as the Materials project, with completely new API and implementation.