David Walter
Katalon Apprentice
11/07/2018

Katalon Studio and BrowserMob Proxy

Greetings, 

I am currently working on a automation project that requires the ability to manipulate request headers to mock authentication (our platform allows access to content based on IP ranges, referer's, etc...). 

I understand that Selenium by design does not allow the modification of headers, but have seen various implementations that use a proxy such as BrowserMob Proxy to intercept the requests for this to be done.  

Has anyone successfully integrated BrowserMob Proxy with Katalon to achieve this? 

Below is some example code that I've tried, both with Firefox and Chrome. I'm not an expert in Java or Groovy, but it seems like for Firefox at least, its not respecting the port  I'm trying to start on. 

For Firefox, it starts the browser and navigates to the web page, but it looks like it's listening on a different port than the proxy is on.
Port started:34531
172.254.253.150
172.254.253.150:34531
1541618735347	geckodriver	INFO	Listening on 127.0.0.1:33137

For Chrome, I've read that it won't respect the DesiredCapabilties.Proxy setting and you have to pass it in as a option. When I try that it fails with the below error:
Port started:33537
172.254.253.150
172.254.253.150:33537
Starting ChromeDriver 2.29.461591 (62ebf098771772160f391d75e589dc567915b233) on port 22277
Only local connections are allowed.
14:12:28.370 [LittleProxy-0-ClientToProxyWorker-1] ERROR org.littleshoot.proxy.impl.ClientToProxyConnection - (AWAITING_INITIAL) [id: 0xfee38cc1, L:/172.254.253.150:33537 - R:/172.254.253.150:33553]: Caught an exception on ClientToProxyConnection
java.lang.NoSuchMethodError: com.google.common.net.HostAndPort.getHostText()Ljava/lang/String;
And my Code:
package bmp
import org.openqa.selenium.Proxy
import org.openqa.selenium.WebDriver
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.remote.CapabilityType
import org.openqa.selenium.remote.DesiredCapabilities

import com.kms.katalon.core.annotation.Keyword

import io.netty.handler.codec.http.HttpRequest
import io.netty.handler.codec.http.HttpResponse
import net.lightbody.bmp.BrowserMobProxyServer
import net.lightbody.bmp.client.ClientUtil
import net.lightbody.bmp.filters.RequestFilter as RequestFilter
import net.lightbody.bmp.filters.ResponseFilter as ResponseFilter
import net.lightbody.bmp.util.HttpMessageContents
import net.lightbody.bmp.util.HttpMessageInfo



public  class bmp {

	public static WebDriver driver
	public static BrowserMobProxyServer proxy

	@Keyword
	public  proxySetup() {

		proxy = new BrowserMobProxyServer()
		proxy.setTrustAllServers(true)
		proxy.start()
		int port = proxy.getPort()

		System.out.println("Port started:" + port)

		proxy.addRequestFilter(new RequestFilter() {
					@Override
					public HttpResponse filterRequest(HttpRequest request, HttpMessageContents contents, HttpMessageInfo messageInfo) {
						//proxy.addHeader("X-Forwarded_for", "66.249.69.245")
						request.headers().add("X-Forwarded_for", "66.249.69.245")
						String q = URLDecoder.decode(messageInfo.getOriginalUrl(), "UTF-8");
						System.out.println("Request: "+ q );
						return null;
					}
				});

		proxy.addResponseFilter(new ResponseFilter() {
					@Override
					public void filterResponse(HttpResponse response, HttpMessageContents contents, HttpMessageInfo messageInfo) {
						headers = proxy.getAllHeaders()
						System.out.println("Headers: " + headers)
					}
				});

		Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
		
/**
 * Chrome Spike with options
 * 
 **/
/*
		ChromeOptions options = new ChromeOptions();
		// Add the WebDriver proxy capability.
		String hostIp = Inet4Address.getLocalHost().getHostAddress();
			println(hostIp)
		seleniumProxy.setHttpProxy(hostIp+":"+ proxy.getPort());
			println(hostIp+":"+ proxy.getPort())
		seleniumProxy.setSslProxy(hostIp+":"+proxy.getPort());;
		options.setCapability("proxy", seleniumProxy);

		ChromeDriver driver = new ChromeDriver(options);
*/	

		/**
		 * Chrome Spike with desired capabilites
		 *
		 **/
/*		
		 ArrayList<String> switches = new ArrayList<String>();
		 String hostIp = Inet4Address.getLocalHost().getHostAddress();
		 switches.add("--proxy-server=" + hostIp+":"+proxy.getPort());
		 seleniumCapabilities.setCapability("chrome.switches", switches);
		 driver  = new ChromeDriver(seleniumCapabilities)
 */
		
		try {
			String hostIp = Inet4Address.getLocalHost().getHostAddress();
			println(hostIp)
			seleniumProxy.setHttpProxy(hostIp+":"+proxy.getPort());
			println(hostIp+":"+proxy.getPort())
			seleniumProxy.setSslProxy(hostIp+":"+proxy.getPort());
		} catch (UnknownHostException e) {
			e.printStackTrace();
			println("invalid Host Address")
		}
		
		DesiredCapabilities seleniumCapabilities = new DesiredCapabilities()

		seleniumCapabilities.setCapability(CapabilityType.PROXY, seleniumProxy)
		seleniumCapabilities.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true)

		driver = new FirefoxDriver(seleniumCapabilities)

		driver.get("xxx_Sample_URL_XXX")
	}


	@Keyword
	public static void proxyShutdown() {
		//driver.quit()
		proxy.stop()
	}
}
Any help would be greatly appreciated, This is the last hurdle to get wide scale adoption of Katalon at my workplace, as this is a critical aspect of our testing. 

Other alternatives appreciated, but I have to be able to set the headers in Katalon. So no external proxies. 

Thanks in advance. 


Upvote
Quote

Comments

  • David Walter
    Katalon Apprentice
    11/08/2018
    Chrome error was fixed by upgrading the the latest BrowserMobProxy jar (2.1.5)

    Any thoughts on the driver not respecting the proxy settings?
    Upvote
    Quote
Sign In or Register to comment.