How to Highlight Test object in each and every step

Drunda,

I wanted you to share your code for WebUI.callTestCase().

I do not need you to work on GitHub for HighlightElement() keyword, as you have already shared your code here.

Oh, now I get it. No, I didn’t make any changes to callTestCase() at all. I use this method as it is in my approach, which I introduced rudimentarily here.

I intended the HighlightElement class for joke. It was just something funny, a toy :frog: that makes our testing tasks a bit enjoyable.

Now we have got Drunda’s development. It’s serious one. No more jokes will be welcomed.

I am wondering which way to go.

1 Like

kazurayam,

I think I understand your concerns. Are they perhaps about the fact that the additional purpose of my code makes it dependent on other project requirements? Please give me two more days. I still have a few small corrections in mind, with which I might be able to improve the code.

1 Like

Just now, duyluong announced a change in the behavior of Katalon Studio, that callTestCase() only generates an error about its own failure, without revealing the root cause message of the child test case, for the next release:

Since we don’t know the release time yet, I’ll revise my above code a little bit anyway …

I have now rewritten my error handling so that hopefully the whole class can be used immediately in any testing project as it is. The only condition should be that the global variable tcExceptionEvents is not needed elsewhere in the Katalon project. But if it is, it can of course also be renamed at every appearance in this class. If the variable does not yet exist, it will be created at runtime and, in any case, will then be filled again and again with a new Map, so that it always contains all currently relevant information about the circumstances of a suddenly occurring error, i.e. keywordName, testObject, testObjectString, inputParams, webElements, exceptions (with type and message) and even the lastWebElements that were recognized in the immediately preceding test step.

Thus, this overall approach does not only permit visual live monitoring of the test execution, but in the event of an error,

  • a screenshot or a recorded video can be used to immediately visualize where exactly the error occurred during the test sequence by means of the colored web elements (orange = current, green = successful, red = faulty; if no red marked web element is visible, the error should usually have occurred immediately after the last green marked element, because the element affected by the error doesn’t seem to exist at all).

  • an alarm notification (e.g. via e-mail and/or Telegram Bot API) could be sent from the test at runtime, containing the meaningful screenshot as well as helpful detailed text information on all important circumstances of the occurred error (see above mentioned variables stored in the tcExceptionEvents Map).

  • if the test case was called in the usual way by a test suite, you don’t even need to overload your test case with an additional try-catch-block surrounding your test steps in order to be able to manage your appropriate teardown actions; just simply edit the catch-blocks centrally in this class; the exceptions Map integrated within tcExceptionEvents will hold all occurred exception events as lists differentiated by exception type; depending on the respective error information, you could then decide, for example, who to alert.

  • if the test case was called using callTestCase(), it could be tried several times to pass (for example in a for loop with nested try-catch-block up to an accepted maximum of attempts); depending on the respective error information, you could then additionally decide whether dependent subsequent test cases are still executable or can finally be cancelled.

      @Keyword
      public static current(TestObject testObject) {
      	return influence(testObject, 'current')
      }
    
      @Keyword
      public static success(TestObject testObject) {
      	return influence(testObject, 'success')
      }
    
      @Keyword
      public static exception(TestObject testObject) {
      	return influence(testObject, 'exception')
      }
    
      private static influence(TestObject testObject, String accessStatus) {
      /**
       * Marks all Web elements that match the given test object, depending on their access status,
       * either orange (current), green (successful), or red (faulty).
       */
      	List<WebElement> elements
      	try {
      		WebDriver driver = DriverFactory.getWebDriver()
      		elements = WebUiCommonHelper.findWebElements(testObject, 5)
      		for (WebElement element : elements) {
      			JavascriptExecutor js = (JavascriptExecutor) driver
      			js.executeScript(
      				"arguments[0].setAttribute('style','outline: dashed ${accessStatus == 'current' ? 'orange' : (accessStatus == 'success' ? 'lime' : 'red')};');",
      				element)
      		}
      	} catch (Exception e) {
      		// TODO use Katalon Logging
      		e.printStackTrace()
      	}
      	finally {
      		return elements
      	}
      }
    
      private static List<String> influencedKeywords = ['click', 'selectOptionByIndex', 'selectOptionByLabel', 'selectOptionByValue', 'setEncryptedText', 'setText', 'scrollToElement']
      /**
       * Change some of methods of WebUiBuiltInKeywords so that they call HighlightElement.current(testObject)
       * before invoking their original method body, call HighlightElement.success(testObject) when passing
       * and call HighlightElement.exception(testObject) when an error occurs.
       *
       * http://docs.groovy-lang.org/latest/html/documentation/core-metaprogramming.html#metaprogramming
       */
      
      @Keyword
      static void addGlobalVariable(String name, def value) {
      /**
       * Adds global variable dynamically at script runtime, i.e. "on the fly".
       * 
       * https://docs.katalon.com/katalon-studio/docs/create-global-variables-on-the-fly.html +++ by Sergii Tyshchenko
       */
      	GroovyShell shell1 = new GroovyShell()
      	MetaClass mc = shell1.evaluate("internal.GlobalVariable").metaClass
      	String getterName = "get" + name.capitalize()
      	mc.'static'."$getterName" = { -> return value }
      	mc.'static'."$name" = value
      }
    
      @Keyword
      public static void pandemic() {
      /**
       * Manipulates all keyword methods contained in the list influencedKeywords when called in the respective test case
       *   * in order to mark the affected web elements before and after each access with different colors and
       *   * in case of an error to temporarily store all relevant information about its circumstances,
       *     i.e. keywordName, testObject, testObjectString, inputParams, webElements, exceptions (with type and message)
       *     and even the lastWebElements that were recognized in the immediately preceding test step,
       *     in the dynamically generated map variable tcExceptionEvents.
       * 
       * This is a joint project by kazurayam and drundanibel
       */
      	WebUiBuiltInKeywords.metaClass.'static'.invokeMethod = { String name, args ->
      		if (name in influencedKeywords) {
      			TestObject to = (TestObject)args[0]
      			String toString = args[0].toString().replaceFirst(/^TestObject - '(.*?)'$/, '$1')
      			if (GlobalVariable.metaClass.hasProperty(GlobalVariable, 'tcExceptionEvents')) {
      				GlobalVariable.tcExceptionEvents['lastWebElements'] = GlobalVariable.tcExceptionEvents.currentTestStep['webElements']
      			}
      			else {
      				addGlobalVariable('tcExceptionEvents', [
      					'exceptions' : ['Failure' : [], 'Error' : [], 'General' : []],
      					'currentTestStep' : ['webElements' : null],
      					'lastWebElements' : null
      				])
      			}
      			def currentWebElements = HighlightElement.current(to)
      			List inputParams = args.collect{ it }.withIndex().findResults{ it, id -> (id > 0) ? it : null }
      			Map currentTestStep = ['keywordName' : name, 'testObject' : to, 'testObjectString' : toString, 'inputParams' : inputParams, 'webElements' : currentWebElements]
      			GlobalVariable.tcExceptionEvents.currentTestStep = currentTestStep
      		}
      		def result
      		try {
      			result = delegate.metaClass.getMetaMethod(name, args).invoke(delegate, args)
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.success(to)
      			}
      		}
      		catch(StepFailedException e) {
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.exception(to)
      				GlobalVariable.tcExceptionEvents.exceptions['Failure'] << e
      			}
      			throw e
      		}
      		catch(StepErrorException e) {
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.exception(to)
      				GlobalVariable.tcExceptionEvents.exceptions['Error'] << e
      			}
      			throw e
      		}
      		catch(Exception e) {
      			if (name in influencedKeywords) {
      				TestObject to = (TestObject)args[0]
      				HighlightElement.exception(to)
      				GlobalVariable.tcExceptionEvents.exceptions['General'] << e
      			}
      			throw e
      		}
      		return result
      	}
      }
    

Important Edit:

Sorry, I renamed the method failure() to exception() yesterday at the last moment and did not correct all places in the code. This is now fixed above and should really work now.

1 Like

Drunda,

Thank you for your greate efforts. I will study your code. It would deserve a few weekends of mine.:wink:

kazurayam,
please note the correction I just made to the code (see above).

Drunda,

I am studying your code. I found, in the pandemic() { ... } method, the following IF statement is repeated 5 times.

        if (name in influencedKeywords) {
            ...
        }

To me, the repetition of IF looks redundant and confusing.

Could you please refactor your code so that the IF statement appears for the minimum number of times (preferably only once) ?

In other word, I have a question about your design intention. How do you want to deal with those keywords which are NOT in the influencedKeywords?

Do you want to save the clinical records of the uninfluenced keywords into GlobalVaraible.tcExceptionEvents as well? Or rather you want to exclude them?

How do you regard the influencedKeywords list?

Is it the List of keywords to be highlighted? — obviously yes.

Is it the list of keywords to be recorded? — not obvious.

is there any reference material for java scripting in katalon, i’m new to katalon.

Katalon documentation is not a good place to look for tutorials on Java programming in general.

You should buy a book on Java.

kazurayam,

the IF statements may look redundant, but I don’t think they are. :wink: What I’m trying to do with this construction is this: All built-in keywords that require a test object as an argument and are to be specially monitored due to a design decision of the test architect (to be maintained in the list influencedKeywords) lead to the coloring of the affected web elements on the web page and are specially error handled by caching all circumstances of their occurrence.

However, all other built-in keywords should (at least in my use case) also be error-monitored. But because I can’t be sure that they have a test object argument at all and also because they don’t fit to the context of the highlighted elements in the tcExceptionEvents object, I need the IF statement to differentiate everywhere, in the try as well as in each of the exception blocks.

But it’s certainly true that this is still not a complete solution. Because the possibility for error monitoring should actually exist for every single step in the test case, thus also for custom keywords and other commands. As a further development of the simple built-in error handling of Katalon Studio (with the options STOP_ON_FAILURE, CONTINUE_ON_FAILURE and OPTIONAL) you should be able to decide at a central point how to proceed due to the circumstances of an occurred error. So this can only be a start pointing in a possible direction.

I can only speak for myself, but in our company the stakeholders want to be informed immediately if a relevant error occurs. Speed and relevance are equally important, because false alarms are extremely annoying when they accumulate. That’s why I need ways to correctly address (stakeholders vs. test engineers) and channel (Telegram message vs. email) alarm notifications. Maybe I’ll tinker with it a little more …

I understand that you want to monitor all built-in keywords which take TestObject as args; not only the keywords listed in the influencedKeywords list.

I want to find out a way to do this. I would try
if (args[0] instanceof com.kms.katalon.core.testobject.TestObject)
to find if the keyword was invoked with a TestObject as the 1st argument.

I would continue studying…

In Feb I worked on my project to include the proposal by @Drunda_Nibel. In my GitHub project, I mad a branch named Drunda_Niebel. The most current code set is accessible at the following URL:

Unfortunately I encountered a blocking problem. Therefore I hesitated announcing this modified version.

What was the blocking problem?

  • git clone https://github.com/kazurayam/HighlightingElementByTestObjectInEachAndEveryStep
  • git checkout Drunda_Nibel
  • start Katalon Studio (version 6.1.1 or under), open the project HighlightingElementByTestObjectInEachAndEveryStep
  • run Test Cases/main/TC2_failing
  • then you will see the following result:

The LogView shows that I got the last step failed. It meaned closeBrowser() failed!. But in fact, invocation of closeBrowser() was just working. The Log was just misleading. This must be a bug in Katalon Studio!


Katalon Studio 6.1.2.rc2 was announced: Katalon Studio version 6.1.2.rc2 with Settings support for custom keyword plugins

This version includes

I tried version 6.1.2 with the above mentioned case. I confirmed that the problem was fixed.

1 Like

わあ すごいですね ありがどう ございました

@1521352988

Thank you.

Please be advised, we are expected to write posts in English.

That’s true. I am a Chinese, working in a Japanese company which in china dalian.and my English is poor.i am very admire the English who can good learing.very glad you can reply me.Foreign friend! Exciting!

Wow, as a newbie to Katalon and groovy, this really excited me!!! Really really great job @kazurayam.
If I may ask, this is me being lazy:
Can one get intellisense \ autocomplete to pick up new methods and its parameters when you use metaClass to add custom methods?

I am not sure what you want.

I guess you want to know the signatures of methods of com.katalon.core.webui.keyword.WebUiBuiltInKeywords class. The most obvious way is to read the source code of the class. You can find the source code here:

If you want something more magical (you can do any by using Java Reflection mechanism), please describe what sort of problem you want to solve.