How to Highlight Test object in each and every step

I noticed that this feature has been reinvented many times before:

2 Likes

Kudos Kaz :star_struck: for all efforts and time.

Could you add selectOptionByLabel() and selectOptionByIndex(), in GIT … so it would benefit other users if they select dropdown other than value.

Thanks Again :wink:

// selectOptionByLabel()
WebUiBuiltInKeywords.metaClass.static.selectOptionByLabel = { TestObject to, String value, boolean isRegex ->
HighlightElement.on(to)
KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.PLATFORM_WEB, “selectOptionByLabel”, to, value, isRegex)
}

I added WebUI.selectOptionByLabel() and WebUI.selectOptionByIndex()

Get the version tagged as 0.3 or later at
https://github.com/kazurayam/HighlightingElementByTestObjectInEachAndEveryStep/releases

3 Likes

Thanks Kaz :wink:

I love this idea and I praise kazurayam’s ingeniously creative implementation, whose approach will help us solve many more challenges in the future that seemed unthinkable until now, many thanks kaz!

For example, I had long been looking for a way to determine the actual root cause message that is generated when test cases called by WebUI.callTestCase() fail. Because I wanted to send a helpful alarm notification to the responsible colleagues at runtime and not have to wait for the generation and manual analysis of the report (see my original post here). Unfortunately, this root cause message generated during a concrete test step within the nested child test case is not passed on to the parent test case (the one with the WebUI.callTestCase() command). Instead, callTestCase() generates its own, less helpful error notification, which only states that the execution of the child test case failed.

But thanks to kazurayam’s approach, I can now finally catch this original error message at the right moment, without having to overload every single test case with a try-catch block, and write it into a global variable. And not only that: I could easily extend kazurayam’s code to cache all relevant information about the circumstances of the error event in a structured way (keywordName, testObject, testObjectString, inputParams, webElements - see below in the extended pandemic() method) to access them in the parent test case and finally send a meaningful alarm notification with screenshot. Even if you don’t use callTestCase(), with this you can save yourself try-catch blocks in test cases at all!

Finally, I varied the color assignments for the web elements so that you can see at a glance which web element the test case actually failed for, both in the screenshot and in a recorded video. I now distinguish between the following cases and methods, which replace kazurayam’s on() method for me: current (orange coloring), succes (green coloring), failure (red coloring). If a web element is not found at all, it can of course not be colored, but if there is no orange outlined element either, you also know that the error must have occurred after the last green colored element. The supplied detailed data will then only give the last confirmation.

I hope it helps others, too.

public class HighlightElement {

	@Keyword
	public static current(TestObject testObject) {
		return influence(testObject, 'current')
	}

	@Keyword
	public static success(TestObject testObject) {
		return influence(testObject, 'success')
	}

	@Keyword
	public static failure(TestObject testObject) {
		return influence(testObject, 'failure')
	}

	private static influence(TestObject testObject, String status) {
		List<WebElement> elements
		try {
			WebDriver driver = DriverFactory.getWebDriver()
			elements = WebUiCommonHelper.findWebElements(testObject, 5)
			for (WebElement element : elements) {
				JavascriptExecutor js = (JavascriptExecutor) driver
				js.executeScript(
						"arguments[0].setAttribute('style','outline: dashed ${status == 'current' ? 'orange' : (status == 'success' ? 'lime' : 'red')};');",
						element)
			}
		} catch (Exception e) {
			// TODO use Katalon Logging
			e.printStackTrace()
		}
		finally {
			return elements
		}
	}

	private static List<String> influencedKeywords = ['click', 'selectOptionByIndex', 'selectOptionByLabel', 'selectOptionByValue', 'setEncryptedText', 'setText']

	/**
	 * change some of methods of WebUiBuiltInKeywords so that they call HighlightElement.on(testObject)
	 * before invoking their original method body.
	 *
	 * http://docs.groovy-lang.org/latest/html/documentation/core-metaprogramming.html#metaprogramming
	 */
	@Keyword
	public static void pandemic() {
		WebUiBuiltInKeywords.metaClass.'static'.invokeMethod = { String name, args ->
			if (name in influencedKeywords) {
				TestObject to = (TestObject)args[0]
				String toString = args[0].toString().replaceFirst(/^TestObject - '(.*?)'$/, '$1')
				GlobalVariable.G_['lastWebElements'] = (GlobalVariable.G_.containsKey('testStep')) ? GlobalVariable.G_['testStep']['webElements'] : null
				def currentWebElements = HighlightElement.current(to)
				List inputParams = args.collect{ it }.withIndex().findResults{ it, id -> (id > 0) ? it : null }
				GlobalVariable.G_['testStep'] = ['keywordName' : name, 'testObject' : to, 'testObjectString' : toString, 'inputParams' : inputParams, 'webElements' : currentWebElements]
			}
			def result
			try {
				result = delegate.metaClass.getMetaMethod(name, args).invoke(delegate, args)
				if (name in influencedKeywords) {
					TestObject to = (TestObject)args[0]
					HighlightElement.success(to)
				}
			}
			catch(StepFailedException e) {
				if (name in influencedKeywords) {
					TestObject to = (TestObject)args[0]
					HighlightElement.failure(to)
					GlobalVariable.G_['exceptions']['Failure'] << e
				}
				throw e
			}
			catch(StepErrorException e) {
				if (name in influencedKeywords) {
					TestObject to = (TestObject)args[0]
					HighlightElement.failure(to)
					GlobalVariable.G_['exceptions']['Error'] << e
				}
				throw e
			}
			catch(Exception e) {
				if (name in influencedKeywords) {
					TestObject to = (TestObject)args[0]
					HighlightElement.failure(to)
					GlobalVariable.G_['exceptions']['Error'] << e
				}
				throw e
			}
			return result
		}
	}
}
1 Like

Thanks Kaz and Nibel.

Kaz / Nibel could you please push the new enhancement to GIT with sample testcase (positive/ negative scenarios)… so it would benefit fellow Katalon users.

Drunda,

Could you please share your code made for WebUI.callTestCase keyword : what sort of patch you made to the Katalon API? Make a new post in the “Improvement/Feature request” and ask Katalon Team for pulling your patch into the original source code. This will help many others.

OK, I will check.

I don’t use Github actively so far and would have to get used to it first. However, I wouldn’t mind (no, I’d be proud) if you wanted to include my extensions in your next version. :smiley: Thanks!

Edit: I only extended your HilightElement class. What I do exactly in my parent test case would lead too far here and is quite individual in detail and probably less relevant here, since the extensions of the HighlightElement class seem reasonable even without callTestCase().

Drunda,

I wanted you to share your code for WebUI.callTestCase().

I do not need you to work on GitHub for HighlightElement() keyword, as you have already shared your code here.

Oh, now I get it. No, I didn’t make any changes to callTestCase() at all. I use this method as it is in my approach, which I introduced rudimentarily here.

I intended the HighlightElement class for joke. It was just something funny, a toy :frog: that makes our testing tasks a bit enjoyable.

Now we have got Drunda’s development. It’s serious one. No more jokes will be welcomed.

I am wondering which way to go.

1 Like

kazurayam,

I think I understand your concerns. Are they perhaps about the fact that the additional purpose of my code makes it dependent on other project requirements? Please give me two more days. I still have a few small corrections in mind, with which I might be able to improve the code.

1 Like

Just now, duyluong announced a change in the behavior of Katalon Studio, that callTestCase() only generates an error about its own failure, without revealing the root cause message of the child test case, for the next release:

Since we don’t know the release time yet, I’ll revise my above code a little bit anyway …

I have now rewritten my error handling so that hopefully the whole class can be used immediately in any testing project as it is. The only condition should be that the global variable tcExceptionEvents is not needed elsewhere in the Katalon project. But if it is, it can of course also be renamed at every appearance in this class. If the variable does not yet exist, it will be created at runtime and, in any case, will then be filled again and again with a new Map, so that it always contains all currently relevant information about the circumstances of a suddenly occurring error, i.e. keywordName, testObject, testObjectString, inputParams, webElements, exceptions (with type and message) and even the lastWebElements that were recognized in the immediately preceding test step.

Thus, this overall approach does not only permit visual live monitoring of the test execution, but in the event of an error,

  • a screenshot or a recorded video can be used to immediately visualize where exactly the error occurred during the test sequence by means of the colored web elements (orange = current, green = successful, red = faulty; if no red marked web element is visible, the error should usually have occurred immediately after the last green marked element, because the element affected by the error doesn’t seem to exist at all).

  • an alarm notification (e.g. via e-mail and/or Telegram Bot API) could be sent from the test at runtime, containing the meaningful screenshot as well as helpful detailed text information on all important circumstances of the occurred error (see above mentioned variables stored in the tcExceptionEvents Map).

  • if the test case was called in the usual way by a test suite, you don’t even need to overload your test case with an additional try-catch-block surrounding your test steps in order to be able to manage your appropriate teardown actions; just simply edit the catch-blocks centrally in this class; the exceptions Map integrated within tcExceptionEvents will hold all occurred exception events as lists differentiated by exception type; depending on the respective error information, you could then decide, for example, who to alert.

  • if the test case was called using callTestCase(), it could be tried several times to pass (for example in a for loop with nested try-catch-block up to an accepted maximum of attempts); depending on the respective error information, you could then additionally decide whether dependent subsequent test cases are still executable or can finally be cancelled.

      @Keyword
      public static current(TestObject testObject) {
      	return influence(testObject, 'current')
      }
    
      @Keyword
      public static success(TestObject testObject) {
      	return influence(testObject, 'success')
      }
    
      @Keyword
      public static exception(TestObject testObject) {
      	return influence(testObject, 'exception')
      }
    
      private static influence(TestObject testObject, String accessStatus) {
      /**
       * Marks all Web elements that match the given test object, depending on their access status,
       * either orange (current), green (successful), or red (faulty).
       */
      	List<WebElement> elements
      	try {
      		WebDriver driver = DriverFactory.getWebDriver()
      		elements = WebUiCommonHelper.findWebElements(testObject, 5)
      		for (WebElement element : elements) {
      			JavascriptExecutor js = (JavascriptExecutor) driver
      			js.executeScript(
      				"arguments[0].setAttribute('style','outline: dashed ${accessStatus == 'current' ? 'orange' : (accessStatus == 'success' ? 'lime' : 'red')};');",
      				element)
      		}
      	} catch (Exception e) {
      		// TODO use Katalon Logging
      		e.printStackTrace()
      	}
      	finally {
      		return elements
      	}
      }
    
      private static List<String> influencedKeywords = ['click', 'selectOptionByIndex', 'selectOptionByLabel', 'selectOptionByValue', 'setEncryptedText', 'setText', 'scrollToElement']
      /**
       * Change some of methods of WebUiBuiltInKeywords so that they call HighlightElement.current(testObject)
       * before invoking their original method body, call HighlightElement.success(testObject) when passing
       * and call HighlightElement.exception(testObject) when an error occurs.
       *
       * http://docs.groovy-lang.org/latest/html/documentation/core-metaprogramming.html#metaprogramming
       */
      
      @Keyword
      static void addGlobalVariable(String name, def value) {
      /**
       * Adds global variable dynamically at script runtime, i.e. "on the fly".
       * 
       * https://docs.katalon.com/katalon-studio/docs/create-global-variables-on-the-fly.html +++ by Sergii Tyshchenko
       */
      	GroovyShell shell1 = new GroovyShell()
      	MetaClass mc = shell1.evaluate("internal.GlobalVariable").metaClass
      	String getterName = "get" + name.capitalize()
      	mc.'static'."$getterName" = { -> return value }
      	mc.'static'."$name" = value
      }
    
      @Keyword
      public static void pandemic() {
      /**
       * Manipulates all keyword methods contained in the list influencedKeywords when called in the respective test case
       *   * in order to mark the affected web elements before and after each access with different colors and
       *   * in case of an error to temporarily store all relevant information about its circumstances,
       *     i.e. keywordName, testObject, testObjectString, inputParams, webElements, exceptions (with type and message)
       *     and even the lastWebElements that were recognized in the immediately preceding test step,
       *     in the dynamically generated map variable tcExceptionEvents.
       * 
       * This is a joint project by kazurayam and drundanibel
       */
      	WebUiBuiltInKeywords.metaClass.'static'.invokeMethod = { String name, args ->
      		if (name in influencedKeywords) {
      			TestObject to = (TestObject)args[0]
      			String toString = args[0].toString().replaceFirst(/^TestObject - '(.*?)'$/, '$1')
      			if (GlobalVariable.metaClass.hasProperty(GlobalVariable, 'tcExceptionEvents')) {
      				GlobalVariable.tcExceptionEvents['lastWebElements'] = GlobalVariable.tcExceptionEvents.currentTestStep['webElements']
      			}
      			else {
      				addGlobalVariable('tcExceptionEvents', [
      					'exceptions' : ['Failure' : [], 'Error' : [], 'General' : []],
      					'currentTestStep' : ['webElements' : null],
      					'lastWebElements' : null
      				])
      			}
      			def currentWebElements = HighlightElement.current(to)
      			List inputParams = args.collect{ it }.withIndex().findResults{ it, id -> (id > 0) ? it : null }
      			Map currentTestStep = ['keywordName' : name, 'testObject' : to, 'testObjectString' : toString, 'inputParams' : inputParams, 'webElements' : currentWebElements]
      			GlobalVariable.tcExceptionEvents.currentTestStep = currentTestStep
      		}
      		def result
      		try {
      			result = delegate.metaClass.getMetaMethod(name, args).invoke(delegate, args)
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.success(to)
      			}
      		}
      		catch(StepFailedException e) {
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.exception(to)
      				GlobalVariable.tcExceptionEvents.exceptions['Failure'] << e
      			}
      			throw e
      		}
      		catch(StepErrorException e) {
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.exception(to)
      				GlobalVariable.tcExceptionEvents.exceptions['Error'] << e
      			}
      			throw e
      		}
      		catch(Exception e) {
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.exception(to)
      				GlobalVariable.tcExceptionEvents.exceptions['General'] << e
      			}
      			throw e
      		}
      		return result
      	}
      }
    

Important Edit:

Sorry, I renamed the method failure() to exception() yesterday at the last moment and did not correct all places in the code. This is now fixed above and should really work now.

1 Like

Drunda,

Thank you for your greate efforts. I will study your code. It would deserve a few weekends of mine.:wink:

kazurayam,
please note the correction I just made to the code (see above).

Drunda,

I am studying your code. I found, in the pandemic() { ... } method, the following IF statement is repeated 5 times.

        if (name in influencedKeywords) {
            ...
        }

To me, the repetition of IF looks redundant and confusing.

Could you please refactor your code so that the IF statement appears for the minimum number of times (preferably only once) ?

In other word, I have a question about your design intention. How do you want to deal with those keywords which are NOT in the influencedKeywords?

Do you want to save the clinical records of the uninfluenced keywords into GlobalVaraible.tcExceptionEvents as well? Or rather you want to exclude them?

How do you regard the influencedKeywords list?

Is it the List of keywords to be highlighted? — obviously yes.

Is it the list of keywords to be recorded? — not obvious.

is there any reference material for java scripting in katalon, i’m new to katalon.