Greetings,
I am currently working on a automation project that requires the ability to manipulate request headers to mock authentication (our platform allows access to content based on IP ranges, referer’s, etc…).
I understand that Selenium by design does not allow the modification of headers, but have seen various implementations that use a proxy such as BrowserMob Proxy to intercept the requests for this to be done.
Has anyone successfully integrated BrowserMob Proxy with Katalon to achieve this?
Below is some example code that I’ve tried, both with Firefox and Chrome. I’m not an expert in Java or Groovy, but it seems like for Firefox at least, its not respecting the port I’m trying to start on.
For Firefox, it starts the browser and navigates to the web page, but it looks like it’s listening on a different port than the proxy is on.
Port started:34531172.254.253.150172.254.253.150:345311541618735347 geckodriver INFO Listening on 127.0.0.1:33137
For Chrome, I’ve read that it won’t respect the DesiredCapabilties.Proxy setting and you have to pass it in as a option. When I try that it fails with the below error:
Port started:33537172.254.253.150172.254.253.150:33537Starting ChromeDriver 2.29.461591 (62ebf098771772160f391d75e589dc567915b233) on port 22277Only local connections are allowed.14:12:28.370 [LittleProxy-0-ClientToProxyWorker-1] ERROR org.littleshoot.proxy.impl.ClientToProxyConnection - (AWAITING_INITIAL) [id: 0xfee38cc1, L:/172.254.253.150:33537 - R:/172.254.253.150:33553]: Caught an exception on ClientToProxyConnectionjava.lang.NoSuchMethodError: com.google.common.net.HostAndPort.getHostText()Ljava/lang/String;
And my Code:
package bmpimport org.openqa.selenium.Proxyimport org.openqa.selenium.WebDriverimport org.openqa.selenium.firefox.FirefoxDriverimport org.openqa.selenium.remote.CapabilityTypeimport org.openqa.selenium.remote.DesiredCapabilitiesimport com.kms.katalon.core.annotation.Keywordimport io.netty.handler.codec.http.HttpRequestimport io.netty.handler.codec.http.HttpResponseimport net.lightbody.bmp.BrowserMobProxyServerimport net.lightbody.bmp.client.ClientUtilimport net.lightbody.bmp.filters.RequestFilter as RequestFilterimport net.lightbody.bmp.filters.ResponseFilter as ResponseFilterimport net.lightbody.bmp.util.HttpMessageContentsimport net.lightbody.bmp.util.HttpMessageInfopublic class bmp { public static WebDriver driver public static BrowserMobProxyServer proxy @Keyword public proxySetup() { proxy = new BrowserMobProxyServer() proxy.setTrustAllServers(true) proxy.start() int port = proxy.getPort() System.out.println("Port started:" + port) proxy.addRequestFilter(new RequestFilter() { @Override public HttpResponse filterRequest(HttpRequest request, HttpMessageContents contents, HttpMessageInfo messageInfo) { //proxy.addHeader("X-Forwarded_for", "66.249.69.245") request.headers().add("X-Forwarded_for", "66.249.69.245") String q = URLDecoder.decode(messageInfo.getOriginalUrl(), "UTF-8"); System.out.println("Request: "+ q ); return null; } }); proxy.addResponseFilter(new ResponseFilter() { @Override public void filterResponse(HttpResponse response, HttpMessageContents contents, HttpMessageInfo messageInfo) { headers = proxy.getAllHeaders() System.out.println("Headers: " + headers) } }); Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy); /** * Chrome Spike with options * **//* ChromeOptions options = new ChromeOptions(); // Add the WebDriver proxy capability. String hostIp = Inet4Address.getLocalHost().getHostAddress(); println(hostIp) seleniumProxy.setHttpProxy(hostIp+":"+ proxy.getPort()); println(hostIp+":"+ proxy.getPort()) seleniumProxy.setSslProxy(hostIp+":"+proxy.getPort());; options.setCapability("proxy", seleniumProxy); ChromeDriver driver = new ChromeDriver(options);*/ /** * Chrome Spike with desired capabilites * **//* ArrayList<String> switches = new ArrayList<String>(); String hostIp = Inet4Address.getLocalHost().getHostAddress(); switches.add("--proxy-server=" + hostIp+":"+proxy.getPort()); seleniumCapabilities.setCapability("chrome.switches", switches); driver = new ChromeDriver(seleniumCapabilities) */ try { String hostIp = Inet4Address.getLocalHost().getHostAddress(); println(hostIp) seleniumProxy.setHttpProxy(hostIp+":"+proxy.getPort()); println(hostIp+":"+proxy.getPort()) seleniumProxy.setSslProxy(hostIp+":"+proxy.getPort()); } catch (UnknownHostException e) { e.printStackTrace(); println("invalid Host Address") } DesiredCapabilities seleniumCapabilities = new DesiredCapabilities() seleniumCapabilities.setCapability(CapabilityType.PROXY, seleniumProxy) seleniumCapabilities.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true) driver = new FirefoxDriver(seleniumCapabilities) driver.get("xxx_Sample_URL_XXX") } @Keyword public static void proxyShutdown() { //driver.quit() proxy.stop() }}
Any help would be greatly appreciated, This is the last hurdle to get wide scale adoption of Katalon at my workplace, as this is a critical aspect of our testing.
Other alternatives appreciated, but I have to be able to set the headers in Katalon. So no external proxies.
Thanks in advance.