How to Highlight Test object in each and every step

Product Forums Katalon Studio
,
36

I have now rewritten my error handling so that hopefully the whole class can be used immediately in any testing project as it is. The only condition should be that the global variable tcExceptionEvents is not needed elsewhere in the Katalon project. But if it is, it can of course also be renamed at every appearance in this class. If the variable does not yet exist, it will be created at runtime and, in any case, will then be filled again and again with a new Map, so that it always contains all currently relevant information about the circumstances of a suddenly occurring error, i.e. keywordName, testObject, testObjectString, inputParams, webElements, exceptions (with type and message) and even the lastWebElements that were recognized in the immediately preceding test step.

Thus, this overall approach does not only permit visual live monitoring of the test execution, but in the event of an error,

  • a screenshot or a recorded video can be used to immediately visualize where exactly the error occurred during the test sequence by means of the colored web elements (orange = current, green = successful, red = faulty; if no red marked web element is visible, the error should usually have occurred immediately after the last green marked element, because the element affected by the error doesn’t seem to exist at all).

  • an alarm notification (e.g. via e-mail and/or Telegram Bot API) could be sent from the test at runtime, containing the meaningful screenshot as well as helpful detailed text information on all important circumstances of the occurred error (see above mentioned variables stored in the tcExceptionEvents Map).

  • if the test case was called in the usual way by a test suite, you don’t even need to overload your test case with an additional try-catch-block surrounding your test steps in order to be able to manage your appropriate teardown actions; just simply edit the catch-blocks centrally in this class; the exceptions Map integrated within tcExceptionEvents will hold all occurred exception events as lists differentiated by exception type; depending on the respective error information, you could then decide, for example, who to alert.

  • if the test case was called using callTestCase(), it could be tried several times to pass (for example in a for loop with nested try-catch-block up to an accepted maximum of attempts); depending on the respective error information, you could then additionally decide whether dependent subsequent test cases are still executable or can finally be cancelled.

      @Keyword
  public static current(TestObject testObject) {
  	return influence(testObject, 'current')
  }

  @Keyword
  public static success(TestObject testObject) {
  	return influence(testObject, 'success')
  }

  @Keyword
  public static exception(TestObject testObject) {
  	return influence(testObject, 'exception')
  }

  private static influence(TestObject testObject, String accessStatus) {
  /**
   * Marks all Web elements that match the given test object, depending on their access status,
   * either orange (current), green (successful), or red (faulty).
   */
  	List<WebElement> elements
  	try {
  		WebDriver driver = DriverFactory.getWebDriver()
  		elements = WebUiCommonHelper.findWebElements(testObject, 5)
  		for (WebElement element : elements) {
  			JavascriptExecutor js = (JavascriptExecutor) driver
  			js.executeScript(
  				"arguments[0].setAttribute('style','outline: dashed ${accessStatus == 'current' ? 'orange' : (accessStatus == 'success' ? 'lime' : 'red')};');",
  				element)
  		}
  	} catch (Exception e) {
  		// TODO use Katalon Logging
  		e.printStackTrace()
  	}
  	finally {
  		return elements
  	}
  }

  private static List<String> influencedKeywords = ['click', 'selectOptionByIndex', 'selectOptionByLabel', 'selectOptionByValue', 'setEncryptedText', 'setText', 'scrollToElement']
  /**
   * Change some of methods of WebUiBuiltInKeywords so that they call HighlightElement.current(testObject)
   * before invoking their original method body, call HighlightElement.success(testObject) when passing
   * and call HighlightElement.exception(testObject) when an error occurs.
   *
   * http://docs.groovy-lang.org/latest/html/documentation/core-metaprogramming.html#metaprogramming
   */
  
  @Keyword
  static void addGlobalVariable(String name, def value) {
  /**
   * Adds global variable dynamically at script runtime, i.e. "on the fly".
   * 
   * https://docs.katalon.com/katalon-studio/docs/create-global-variables-on-the-fly.html +++ by Sergii Tyshchenko
   */
  	GroovyShell shell1 = new GroovyShell()
  	MetaClass mc = shell1.evaluate("internal.GlobalVariable").metaClass
  	String getterName = "get" + name.capitalize()
  	mc.'static'."$getterName" = { -> return value }
  	mc.'static'."$name" = value
  }

  @Keyword
  public static void pandemic() {
  /**
   * Manipulates all keyword methods contained in the list influencedKeywords when called in the respective test case
   *   * in order to mark the affected web elements before and after each access with different colors and
   *   * in case of an error to temporarily store all relevant information about its circumstances,
   *     i.e. keywordName, testObject, testObjectString, inputParams, webElements, exceptions (with type and message)
   *     and even the lastWebElements that were recognized in the immediately preceding test step,
   *     in the dynamically generated map variable tcExceptionEvents.
   * 
   * This is a joint project by kazurayam and drundanibel
   */
  	WebUiBuiltInKeywords.metaClass.'static'.invokeMethod = { String name, args ->
  		if (name in influencedKeywords) {
  			TestObject to = (TestObject)args[0]
  			String toString = args[0].toString().replaceFirst(/^TestObject - '(.*?)'$/, '$1')
  			if (GlobalVariable.metaClass.hasProperty(GlobalVariable, 'tcExceptionEvents')) {
  				GlobalVariable.tcExceptionEvents['lastWebElements'] = GlobalVariable.tcExceptionEvents.currentTestStep['webElements']
  			}
  			else {
  				addGlobalVariable('tcExceptionEvents', [
  					'exceptions' : ['Failure' : [], 'Error' : [], 'General' : []],
  					'currentTestStep' : ['webElements' : null],
  					'lastWebElements' : null
  				])
  			}
  			def currentWebElements = HighlightElement.current(to)
  			List inputParams = args.collect{ it }.withIndex().findResults{ it, id -> (id > 0) ? it : null }
  			Map currentTestStep = ['keywordName' : name, 'testObject' : to, 'testObjectString' : toString, 'inputParams' : inputParams, 'webElements' : currentWebElements]
  			GlobalVariable.tcExceptionEvents.currentTestStep = currentTestStep
  		}
  		def result
  		try {
  			result = delegate.metaClass.getMetaMethod(name, args).invoke(delegate, args)
  			if (name in influencedKeywords) {
  				TestObject to = (TestObject)args[0]
  				HighlightElement.success(to)
  			}
  		}
  		catch(StepFailedException e) {
  			if (name in influencedKeywords) {
  				TestObject to = (TestObject)args[0]
  				HighlightElement.exception(to)
  				GlobalVariable.tcExceptionEvents.exceptions['Failure'] << e
  			}
  			throw e
  		}
  		catch(StepErrorException e) {
  			if (name in influencedKeywords) {
  				TestObject to = (TestObject)args[0]
  				HighlightElement.exception(to)
  				GlobalVariable.tcExceptionEvents.exceptions['Error'] << e
  			}
  			throw e
  		}
  		catch(Exception e) {
  			if (name in influencedKeywords) {
  				TestObject to = (TestObject)args[0]
  				HighlightElement.exception(to)
  				GlobalVariable.tcExceptionEvents.exceptions['General'] << e
  			}
  			throw e
  		}
  		return result
  	}
  }

Important Edit:

Sorry, I renamed the method failure() to exception() yesterday at the last moment and did not correct all places in the code. This is now fixed above and should really work now.

1 Like