The World Wide Wait

Sorry to flood this topic, but I can’t help myself… I got curious about the SmartWait implementation:

public static void doSmartWait() {
        WebDriver main = DriverFactory.getWebDriver();
        JavascriptExecutor js = (JavascriptExecutor) main;
        try {
            js.executeAsyncScript(WAIT_AJAX_SCRIPT);
            js.executeAsyncScript(WAIT_DOM_SCRIPT);
        } catch (Exception e) {
            // Ignore exceptions to avoid clogging user's console log
        }
    }

    private static String WAIT_AJAX_SCRIPT = "\tvar callback = arguments[arguments.length - 1].bind(this);\r\n\twindow.katalonWaiter.katalon_smart_waiter_do_ajax_wait(callback);";

    private static String WAIT_DOM_SCRIPT = "\tvar callback = arguments[arguments.length - 1].bind(this);\r\n\twindow.katalonWaiter.katalon_smart_waiter_do_dom_wait(callback);";
}

Basically it executes some custom JS in the katalonWaiter class to wait for certain things (AJAX, DOM changes I’m assuming), but I’m not able to find where this katalonWaiter class is sourced from in the repo :expressionless:

More to the original topic, I do have somewhat of a catch all wait condition that I use every now and then:

public static void waitForElementRendering() {
	By locator = By.xpath("//*");
	int elementCount = driver.findElements(locator).size();
	while(driver.findElements(locator).size() != elementCount) {
		elementCount = driver.findElements(locator).size();
	}
}

Hopefully what I’m doing here is obvious: I’m just waiting for the number of elements in the DOM to stop changing. This actually works pretty well, but has a couple of caveats:

1.) It doesn’t consider changing attributes. Only the actual number of elements in the DOM at any given time.
2.) For very large pages, this is quite inefficient. It will often wait 2 or 3 seconds or even more, simply because locating 1000’s of elements in a loop is expensive.

2 Likes

Go to <katalon-install>\configuration\resources\extensions\Chrome\Smart Wait\content\wait.js

The code you have is the injection to invoke the wait object from Groovy. When you open that (with your supersharp JS hat on) you’re going to “Heh” and smirk to yourself. :sunglasses:

1 Like

OMG :joy:

    var domCount = 0;
    var domTime = "";
    function katalon_smart_waiter_do_dom_wait() {
      setTimeout(() => {
        if (domTime && (Date.now() - domTime > 30000)) {
          domTime = "";
          callback(true);
        } else if (
          window.katalonWaiter.domModifiedTime &&
          (Date.now() - window.katalonWaiter.domModifiedTime < 400)
        ) {
          domCount++;
          if (domCount === 1) {
            domTime = Date.now();
          }
          return katalon_smart_waiter_do_dom_wait();
        } else {
          domTime = "";
          callback(true);
        }
      }, 100);
    }
    return katalon_smart_waiter_do_dom_wait();

Told ya. Take a bow. :bowing_man:

Well that explains why smartwait can be inefficient at times. It looks like it would take the same performance hit from very large DOMs.

It is currently inefficient, but not because of the large DOMs. Processing ten thousands of elements wouldn’t incur too much of a hit on the performance. :rofl: ( I tried). Well anyway we have made an improvement on performance and will roll this out in the future.

1 Like

Sorry, Brandon, I meant to come back to this.

First a note to the unwary:

This thread, and certainly this particular post, is advanced in nature, theoretical and very likely a cause for serious headaches. We are discussing pre-alpha ideas. You have been warned.

You could inject a mutation observer – assuming your target is running in a modern browser (certainly old IE is going to let you down). A MutationObserver perhaps monitored by a timer (setInterval perhaps) with suitable resolution (I’m thinking ~250ms) could provide exactly what you (we) are looking for here.

Not sure inefficiency counts here. Debating with myself as to how and where the counting takes place. Something needs to loop, no question. As long as the loop’s resolution is fine-grained enough, I’d be happy. Said another way, we’re already reconciled to having to wait (for the page to load and finish doing its thing) so as long as the resolution/timer is fine enough (without being too fine) I’d be happy.

Pseudocodally (that’s a word - I’ve used it twice now, so get over it! :stuck_out_tongue_winking_eye: ) something like:

Groovy

//Injects the JS startMutationObserver code below
startMutationObserverInBrowserJS()

pollForMutationsHaveEndedResult() // looping method

JavaScript

// Sums mutation events writing result to window.uniqueId.mutationsCount
// Perhaps this is wrapped/called by setInterval? Not sure...
startMutationObserver()

The Groovy method PollForMutationsHaveEndedResult() needs to wait for a consistent value being returned from window.uniqueId.mutationsCount (i.e. the summing value has not changed over the resolution period). It might be beneficial and a little more robust to check one last time, if it’s a one-liner call - in terms of clock-cycles, just that call (outgoing from groovy and into browser JS) is an eternity.

Problem: Let’s say the TC is running live in KS. Human user moves the mouse and causes dom mutations (may not even be visible on screen).

Obviously, not an issue when running headless or anywhere out of a human’s reach.


Problem: Some page effects/animations are inherently mutable. Your JS mutationObserver code would need to skip those.


Like I said, this is very theoretical. There are a few potential flakes in the code I’m imagining. I’m not even sure it’s worth the effort. :confused:

2 Likes

Eureka!!! I’ve implemented my own “universal wait” condition, inspired by the SmartWait implementation (perhaps it can even provide some insights that could improve SmartWait). While it’s still a brand-new solution, and I need to do some extensive testing, the initial results look promising.

1.) This approach uses the MutationObserver javascript interface, which is meant to be a replacement for the deprecated Mutation Events feature (which SmartWait is currently using). Here’s the script:

if(typeof observer === "undefined") {
	window.domModifiedTime = Date.now();
	var targetNode = document.querySelector("body");
	var config = { attributes: true, childList: true, subtree: true };
	var callback = () => {
		window.domModifiedTime = Date.now();
	};
	var observer = new MutationObserver(callback);
	observer.observe(targetNode, config);
}

What this does is set up a MutationObserver object, which will listen for any changes to the DOM, as specified in the config var (in my case, I’m listening for node additions/removals, as well as attribute changes).

As with SmartWait, the general idea here is to let the javascript listen for DOM changes, and update the domModifiedTime value whenever a change is observed. Then, as you’ll see below, we can compare this value with the current time to see if we need to keep waiting.

Note: You could manage this script in your keyword directly, but I chose to save it as a separate wait.js file within my project, then read it as a String (using org.apache.commons.io.FileUtils), as demonstrated below.

2.) Create a keyword that will inject the above script, then continually check the domModifiedTime against the current time:

public static void waitForDomModifications() {
	KeywordUtil.logInfo("Waiting for DOM modifications...");
    WebDriver driver = DriverFactory.getWebDriver();
	JavascriptExecutor jsExecutor = jsExecutor = (JavascriptExecutor)driver;
	File file = new File(RunConfiguration.getProjectDir() + "/Keywords/base/util/wait.js"); 
	String script = FileUtils.readFileToString(file, "UTF-8");
	jsExecutor.executeScript(script);
	while(System.currentTimeMillis() - jsExecutor.executeScript("return window.domModifiedTime") < 500) {
		Thread.sleep(30);
	}
}

In other words, if the domModifiedTime is within 500 ms of the current time, wait 30 ms and check again.

Again, this is a very very early prototype, so I will be doing some testing over the coming weeks, but initial results look very promising. Hopefully this will prove useful to some of you looking for that elusive “silver bullet”. I’m open to any suggestions, comments. Happy testing! :slight_smile:

2 Likes

That may be the case, but I suspect on initial reading you’ll only be tweaking perhaps to suit your own preferences/network sensibilities.

Bravo, @Brandon_Hein, nice work!

( And praises for the JavaScript fu! ) :sunglasses:

Lastly, this should be a TIps article, methinks.

1 Like