Object Repository Garbage Collector

I published a GitHub repository:

This project provides a Groovy class com.kazurayam.ks.testobject.ObjectRepositoryGarbageCollector class
in a jar file. This class is useful to find out unused TestObjects in the “Object Repository”.

A simple demo

I made a Test Case script as follows:

import com.kazurayam.ks.testobject.combine.ObjectRepositoryGarbageCollector

import groovy.json.JsonOutput

/**
 * A demonstration of ObjectRepositoryGarbageCollector.
 *
 * This TestCase outputs a JSON file which contains a list of garbage Test Objects
 * in the "Object Repository" folder.
 *
 * A "garbage" means a Test Object which is not used by any scripts
 * in the "Test Cases" folder.
 */

// the Garbage Collector instance will scan 2 folders: "Object Repository" and "Test Cases"

ObjectRepositoryGarbageCollector gc = new ObjectRepositoryGarbageCollector.Builder().build()

// gc.jsonifyGarbages() triggers scanning through the 2 folders and analyze the files.
// All forward references from TestCase scripts to TestObject entities are identified.
// Consequently, it can result a list of unused TestObjects.
// Will output the result in a JSON string

String json = gc.jsonifyGarbage()

println JsonOutput.prettyPrint(json)

When I ran this, I got the following output in the console:

2025-05-02 20:53:28.048 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/demo/ObjectRepositoryGarbageCollector/ORGC_jsonifyGarbage
{
    "Garbage": [
        "main/Page_CURA Healthcare Service/a_Foo",
        "main/Page_CURA Healthcare Service/td_28",
        "main/Page_CURA Healthcare Service/xtra/a_Go to Homepage",
        "main/Page_CURA Healthcare Service/xtra/td_28",
        "misc/dummy1"
    ],
    "Run Description": {
        "Project name": "katalon",
        "includeScriptsFolder": [
            
        ],
        "includeObjectRepositoryFolder": [
            "**/*"
        ],
        "Number of TestCases": 27,
        "Number of TestObjects": 16,
        "Number of unused TestObjects": 5
    }
}
2025-05-02 20:53:29.897 INFO  c.k.k.c.keyword.builtin.CommentKeyword   - {"started at":"2025-05-02T20:53:29.438","duration seconds":0.258}

I found that this project contains

  • 27 TestCase scripts
  • 16 TestObjects
  • out of 16, 5 TestObjects are unused by any of TestCase scripts.
  • it took approximately 0.3 seconds to get the result.

Very quick, isn’t it?

This tool just compiles the report. It does NOT remove any TestObject files out of your project.

How to install the library.

  1. Visit the KS_ObjectRepositoryGargabeCollector, Releases page. Identify the latest version. Find a KS_ObjectRepositoryGarbageCollector-x.x.x.jar file attached. Download the jar file, save it into the Drivers folder of your Katalon project.
  2. Visit the MonkDirectoryScanner, Releases. Identify the latest version. Find a MondDirectoryScanner-x.x.x.jar file attached. Download the jar file, save into the Drivers folder of your katalon project.
  3. Close and reopen the project. Confirm that the jars are recognized by Katalon Studio.
  4. Create a Test Case script, which should be similar to the above “GC” script.
  5. You are done. Run it and see how quickly you can get the result.

Dependencies, versions, etc

This library uses only the libraries bundled in Katalon Studio. You don’t have to add any more external libraries other than the aforementioned jar.

This library should run on Katalon Studio Free, Katalon Studio Enterprise, and Katalon Runtime Engine. The jar is compiled by Katalon Studio Free v10.1.0 with JDK17, so the jar requires KS v10.x or above.

More features?

This library supports more:

  1. It can report all Forward References from TestCase scripts to TestObjects.
  2. It can report all Backward References, which is a list of TestObjects associated with list of ForwardReferences to each TestObject.
  3. It can report all Locators (XPath, CSS Selector) associated with list of duplicating TestObjects that implement the same locator
  4. You can specify sub-folder(s) of the “Object Repository” folder to choose the entries from. By this, you can get the report smaller and forcused.

I will write a more details documentation with sample codes later.

Disclaimer

I hope the library reports correctly. But I would not be responsible for the damages when you manually clear away what it found as “garbages”. I would recommend you to set your project backed by Git, and to store the snapshots before cleaning.

3 Likes

I have published the new v0.4.12. I will show you new reports.

List of Locators with container Test Objects

Another Test Case script generates a JSON titled “LocatorIndex”

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kazurayam.ks.reporting.Shorthand
import com.kazurayam.ks.testobject.combine.ObjectRepositoryGarbageCollector
import com.kms.katalon.core.configuration.RunConfiguration

import groovy.json.JsonOutput
import internal.GlobalVariable

/**
 * ObjectRepositoryGarbageCollector#jsonifyCombinedLocatorIndex() demonstration
 */

ObjectRepositoryGarbageCollector gc =
	new ObjectRepositoryGarbageCollector.Builder()
		.includeObjectRepositoryFolder("**/Page_CURA*")
		.build()

String json = gc.jsonifyCombinedLocatorIndex()

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path classOutputDir = projectDir.resolve("build/tmp/testOutput/demo/ObjectRepositoryGarbageCollector")
Path outDir = classOutputDir.resolve("ORGC_jsonifyCombinedLocatorIndex")
Files.createDirectories(outDir)
File outFile = outDir.resolve("garbage.json").toFile()

outFile.text = JsonOutput.prettyPrint(json)

This script produced this (trimmed):

{
    "CombinedLocatorIndex": {
        "Number of Locators": 13,
        "Number of Suspicious Locators": 3,
        "Locators": [
            {
                "Locator": {
                    "value": "(.//*[normalize-space(text()) and normalize-space(.)='Sa'])[1]/following::td[31]",
                    "selectorMethod": "XPATH"
                },
                "Number of Container TestObjects": 2,
                "Container TestObjects": [
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/td_28",
                        "is used": false
                    },
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/xtra/td_28",
                        "is used": false
                    }
                ]
            },
            {
                "Locator": {
                    "value": "//a[@id='btn-make-appointment']",
                    "selectorMethod": "XPATH"
                },
                "Number of Container TestObjects": 1,
                "Container TestObjects": [
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/a_Make Appointment",
                        "is used": true,
                        "References from TestCase": [
                            {
                                "TestObjectId": "main/Page_CURA Healthcare Service/a_Make Appointment",
                                "Number of ForwardReferences": 2,
                                "ForwardReferences": [
                                    {
                                        "TestCaseId": "main/TC0",
                                        "DigestedLine": {
                                            "line": "WebUI.click(findTestObject('Object Repository/main/Page_CURA Healthcare Service/a_Make Appointment'))",
                                            "lineNo": 12,
                                            "pattern": "main/Page_CURA Healthcare Service/a_Make Appointment",
                                            "matchAt": 47,
                                            "matchEnd": 99,
                                            "matched": true,
                                            "regex": false
                                        },
                                        "TestObjectId": "main/Page_CURA Healthcare Service/a_Make Appointment"
                                    },
                                    {
                                        "TestCaseId": "main/TC1",
                                        "DigestedLine": {
                                            "line": "WebUI.click(findTestObject('Object Repository/main/Page_CURA Healthcare Service/a_Make Appointment'))",
                                            "lineNo": 11,
                                            "pattern": "main/Page_CURA Healthcare Service/a_Make Appointment",
                                            "matchAt": 47,
                                            "matchEnd": 99,
                                            "matched": true,
                                            "regex": false
                                        },
                                        "TestObjectId": "main/Page_CURA Healthcare Service/a_Make Appointment"
                                    }
                                ]
                            }
                        ]
                    }
                ]
            },
...

See the full file at here

The CombinedLocatorIndex JSON tells me the following points:

  1. this project contains 12 “Locators” declared in the Object Repository.
  2. A Locator could be found in one or more Test Objects. If “Number of Container TestObjects” has value 2 or more, it means that the Locator is duplicating. You may want to avoid duplication.
  3. A TestObject may be used by zero or more Test Cases. If the number of references from TestCase" of a TestObject is 0, it means the TestObject is unused. "is used": false denotes this. You may want to remove the unused TestObject.
  4. This JSON could be large. As large as mega-bytes. The size depends on the number of TestObjects in your Object Repository. A too-large JSON won’t be very useful.
  5. The ObjectRepositoryGarbageCollector supports includeObjectRepositoryFolder(pattern) and excludeObjectRepositoryFolder(pattern). For example:
ObjectRepositoryGarbageCollector gc =
	new ObjectRepositoryGarbageCollector.Builder()
		.includeObjectRepositoryFolder("**/Page_CURA*")
		.build()

You can specify from which sub-folders of the “Object Repository” to select Test Objects as target. Thus you can make the JSON much smaller and forcused.

Concise list of Suspicious Locators

Final Test Case script generates a JSON titled “Suspicious Locator Index” which would be most useful.

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kazurayam.ks.testobject.combine.ObjectRepositoryGarbageCollector
import com.kms.katalon.core.configuration.RunConfiguration

import groovy.json.JsonOutput

/**
 * ObjectRepositoryGarbageCollector#jsonifySuspiciousLocatorIndex() demonstration
 */

ObjectRepositoryGarbageCollector gc =
	new ObjectRepositoryGarbageCollector.Builder()
		.includeObjectRepositoryFolder("**/Page_CURA*")
		.build()

String json = gc.jsonifySuspiciousLocatorIndex()

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path classOutputDir = projectDir.resolve("build/tmp/testOutput/demo/ObjectRepositoryGarbageCollector")
Path outDir = classOutputDir.resolve("ORGC_jsonifySuspiciousLocatorIndex")
Files.createDirectories(outDir)
File outFile = outDir.resolve("garbage.json").toFile()

outFile.text = JsonOutput.prettyPrint(json)

This script generated a JSON as follows:

{
    "SuspiciousLocatorIndex": {
        "Number of Locators": 3,
        "Number of Suspicious Locators": 3,
        "Locators": [
            {
                "Locator": {
                    "value": "(.//*[normalize-space(text()) and normalize-space(.)='Sa'])[1]/following::td[31]",
                    "selectorMethod": "XPATH"
                },
                "Number of Container TestObjects": 2,
                "Container TestObjects": [
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/td_28",
                        "is used": false
                    },
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/xtra/td_28",
                        "is used": false
                    }
                ]
            },
            {
                "Locator": {
                    "value": "//body",
                    "selectorMethod": "XPATH"
                },
                "Number of Container TestObjects": 1,
                "Container TestObjects": [
                    {
                        "TestObjectId": "misc/dummy1",
                        "is used": false
                    }
                ]
            },
            {
                "Locator": {
                    "value": "//section[@id='summary']/div/div/div[7]/p/a",
                    "selectorMethod": "XPATH"
                },
                "Number of Container TestObjects": 3,
                "Container TestObjects": [
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/a_Foo",
                        "is used": false
                    },
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/a_Go to Homepage",
                        "is used": true,
                        "References from TestCase": [
                            {
                                "TestObjectId": "main/Page_CURA Healthcare Service/a_Go to Homepage",
                                "Number of ForwardReferences": 1,
                                "ForwardReferences": [
                                    {
                                        "TestCaseId": "main/TC1",
                                        "DigestedLine": {
                                            "line": "WebUI.click(findTestObject('Object Repository/main/Page_CURA Healthcare Service/a_Go to Homepage'))",
                                            "lineNo": 36,
                                            "pattern": "main/Page_CURA Healthcare Service/a_Go to Homepage",
                                            "matchAt": 47,
                                            "matchEnd": 97,
                                            "matched": true,
                                            "regex": false
                                        },
                                        "TestObjectId": "main/Page_CURA Healthcare Service/a_Go to Homepage"
                                    }
                                ]
                            }
                        ]
                    },
                    {
                        "TestObjectId": "main/Page_CURA Healthcare Service/xtra/a_Go to Homepage",
                        "is used": false
                    }
                ]
            }
        ]
    },
    "Run Description": {
        "Project name": "katalon",
        "includeScriptsFolder": [
            "main",
            "misc"
        ],
        "includeObjectRepositoryFolder": [
            "main",
            "misc"
        ],
        "Number of TestCases": 5,
        "Number of TestObjects": 16,
        "Number of unused TestObjects": 5
    }
}

Let me explain about this JSON.

  1. A Locator is suspicious if it is declared by a Test Object which is unused (referred by none of Test Cases)
  2. The “SuspiciousLocatorIndex” JSON contains only suspicious Locators. It doesn’t contain trustworthy Locators. Therefore the JSON file would be far smaller than the full list of Locators.
  3. As you work on cleaning your Object Repository, the “SuspiciousLocatorIndex” will become smaller. When the index has got 0 entries, you can be sure you finished cleaning up.
1 Like

The v0.4.13 enables you to write a Test Case script that cleans up unused Test Objects.

Removing unused Test Objects from Object Repository

In the Katalon Community, some people wrote that they have thousands of unused Test Objects in their projects. See this topic for example. They hope to clean up their project. But it is too hard to delete this amount of Test Objects manually. Is it possible to remove the unused Test Objects from Object Repository programatically?

Yes. The following Test Case script shows an example.

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import org.apache.commons.io.FileUtils

import com.kms.katalon.core.configuration.RunConfiguration
import com.kazurayam.ks.testobject.combine.ObjectRepositoryGarbageCollector
import com.kazurayam.ks.testobject.combine.Garbage
import com.kazurayam.ks.testobject.TestObjectId

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path objectRepositoryDir = projectDir.resolve('Object Repository')

//---------------------------------------------------------------------
// We will setup additional test fixture for this testcase

// create `Object Repository/temp` directory.
Path temp = objectRepositoryDir.resolve('temp')
Files.createDirectories(temp)

// copy the files in the `Object Repository/main` directory`into the temp dir
Path main = objectRepositoryDir.resolve('main')
FileUtils.copyDirectory(main.toFile(), temp.toFile())

//---------------------------------------------------------------------
// create ObjectRepositoryGarbageCollector that selects the "temp" subfolder
ObjectRepositoryGarbageCollector gc =
		new ObjectRepositoryGarbageCollector.Builder()
			.includeObjectRepositoryFolder("temp")
			.build()

// find the garbage
Garbage garbage = gc.getGarbage()
Set<TestObjectId> testObjectIds = garbage.getAllTestObjectIds()
	
for (TestObjectId toi : testObjectIds) {
	println toi
}	
// all Test Objects in the "temp" folder are unused by any of Test Cases
assert testObjectIds.size() == 15  

//---------------------------------------------------------------------
// now we will programatically delete the garbage = unused TestObjects
for (TestObjectId toi : testObjectIds) {
	Path relativePath = toi.getRelativePath()
	Path rsFile = objectRepositoryDir.resolve(relativePath)
	assert Files.exists(rsFile)
	// now delete the "*.rs" file
	Files.delete(rsFile)
	// make sure the file has been deleted
	assert ! Files.exists(rsFile)
}

// erase the `Object Repository/temp` directory to tear down
FileUtils.deleteDirectory(temp.toFile())

Once you ran this script, you need to close the project and reopen it so that Katalon Studio recognize the amended Object Repository.

1 Like