How to capture network traffic and retrieve http requests info in Katalon Studio Projects.?

rakesh.m · November 17, 2022, 6:17am

Hi All,

i’m trying to capture http requests in runtime and use the specific data from api response in katalon studio projects.

I tried to implement mobproxy in the test suite but thats wasn’t helpful.
is there a way we can achieve this.?

Thanks
Rakesh

Russ_Thomas · November 17, 2022, 1:50pm

@bionel is your man here methinks.

bionel · November 17, 2022, 7:56pm

@Russ_Thomas
not yet.
i don’t understand the question.
it is about sending a certain http request (api call) and parse the response, or it is about loading a webpage and capture all api calls under the hood of the webbrowser?
but let’s wait for @rakesh.m input

kazurayam · November 18, 2022, 12:27am

This looks something similar to

in

which used Chrome DevTools Protocol, Network Domain

rakesh.m · November 18, 2022, 1:07am

Usecase : in testscript execution at some point there is a save operation where an api is triggered to save the details. We need to capture that api response and use its parameters in next steps for data validation.

rakesh.m · November 18, 2022, 1:10am

Also we are not supposed to use the alpha versions of katalon studio as the tests need to be executed in testcloud. Only official release versions.

kazurayam · November 18, 2022, 1:25am

Are you going to involve a web browser (Chrome, FireFox, …) during your test?

Or are you goint to make HTTP GET/POST request to a URL using RESTFull/SOAP API?

(you put this post with “web-testing” category, but your description seems suggesting it is not “web-testing”)

kazurayam · November 18, 2022, 1:35am

Are you asking us how to get a JSON/XML text out of the Request object?

The following post may interest you:

rakesh.m · November 18, 2022, 3:42am

Yes i do post request for getting the response.

as mainly this usecase is about intercepting the http calls and then utilizing the response. so i have put this in web category.

bionel · November 18, 2022, 3:58pm

i am tired of this mess.
trying to look into the documentation, where code snipets used to be, nothing there.
anyway …

@rakesh.m i still don’t understand your question.
what traffic to intercept?
you are sending a request, therefore, from the repsonse object grab whatever you like:
see: ResponseObject (Katalon Studio API Specification)

unfortunately, I am not able to give you quick examples on how to build a request quickly from a test object, how to get the data from it, how to send it and how to parse the response.

the new doc is confusing and nothing is where it used to be, therefore kindly don’t ping me anymore on this topic.

attn @Russ_Thomas

Russ_Thomas · November 18, 2022, 4:13pm

Here is my take.

As a normal part of the AUT’s execution, it submits an API call to some endpoint.

In your test you want to somehow grab a copy of the response from the API, get some of the data from it and use that data in subsequent steps in your test.

Correct?

I don’t think there is a way using Katalon’s WS API set to grab a copy of an API call to do what you want to do.

My advice:

Make the API call in a separate test case using Katalon’s WS API. Prove the API is working correctly independent of the AUT. If not, you have a failure.
Use Katalon’s WebUI APIs to test the outcomes of a successful submission. That is, check the AUT behaves correctly when an API call is successful. If it doesn’t, you have a failure.

If I’ve misunderstood, then you need to state your problem/question more clearly. As you can see, you are already causing some considerable frustration among seasoned test engineers.

Thanks.

bionel · November 19, 2022, 10:23am

Provided this is true, and I am guessing right that you have a certain test object defined to send the Post request, you may have to read here:

https://docs.katalon.com/docs/legacy/katalon-studio-enterprise/test-design/web-services-test-design/using-web-services-in-a-test-case#in-test-case-editors-script-view

so, everything you need is in the response object if you do it like:

def response = WS.sendRequest(findTestObject([]))

otherwise, you have to bring more details.
can you show us your test-case code?

kazurayam · November 19, 2022, 1:36pm

If this is the case, the URL of the endpoint is hidden somewhere in the web user interface on browser. The URL will not be exposed to users. @rakesh.m would never know the URL. Therefore he would not be able to make the API call using Katalon’s WS API.

This difficulty might be resolved by CDP Network feature, like this:

In this figure, the Tester knows only the URL of a HTML page. He does not know the URL of CSS and JavaScript that are linked to from the HTML page. Chrome browser can find all URLs. So, via CDP, you can ask Chrome browser to inform you of the URLs of HTML + CSS + JS and other details exchanged over network (such as HTTP Response Status code) . Possbily CDP Network feature supports the backend API endpoits as well.

Of course, CDP requires a serious custom programming efforts. You need to be highly skilled. Katalon Studio provides no support for it.

bionel · November 19, 2022, 2:48pm

agreed we disagree.
the user said he send the request.
therefore, before moving forward, let us wait for a sane input on this matter, otherwise is just waste of our time.

rakesh.m · November 21, 2022, 3:57am

The HTTP response(network XHR Calls/responses) i need to intercept.

rakesh.m · November 21, 2022, 4:00am

Perfect Understanding. but the thing while im running the tests, if i could capture those https requests/responses from network tab then it would be very useful for me to validate through api’s as we tried doing via web but its relatively slower and it takes more time for validation.

kazurayam · November 21, 2022, 4:43am

You can do it if you could write a script that drivers Chrome DevTools Protocol, Network domain.

A sample is here

github.com

kklisura/chrome-devtools-java-client/blob/master/cdt-examples/src/main/java/com/github/kklisura/cdt/examples/TracingExample.java

package com.github.kklisura.cdt.examples;

/*-
 * #%L
 * cdt-examples
 * %%
 * Copyright (C) 2018 - 2021 Kenan Klisura
 * %%
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 * #L%

This file has been truncated. show original

or read an article

Unfortunately Katalon Studio does not provide built-in support CDP at all.

But you can develop your custom Groovy code in Katalon Studio that utilizes kklisura’s lib and use CDP features.

bionel · November 28, 2022, 8:58am

@rakesh.m sorry for late reply, was a bit busy those days …
Well, now I understand what do you need.
You can follow the path proposed by @kazurayam … or show us what you tried with mobproxy and where did you failed.

Mobroxy it is used under the hood by Katalon to intercept API responses as .har files when doing Webservice tests, as far as I know.
So, most of the needed libraries should be already present.

E.g something like this should work: BrowserMob Proxy Selenium Java Tutorial with an Example!

Of-course, the code sample there have to be adapted to Katalon …
Unfortunately, I am very rusty with WebUI testing (and with testing in general, at my current work I am doing different stuff now) …
But, if you show us your code, and what failures you have, perhaps somebody more ‘up-to-date’ and more patient from the community can help you, e.g Kazu or @Russ_Thomas

For now, focus on just getting a .har file to be generated when you navigate to a certain page.
How to parse it and extract the data needed … later. Although the har file is just a json, such tend to be very large with WebUI tests, depending on the resources loaded by your AUT, compared with .har files generated by API calls with Webservice testing.

kazurayam · February 10, 2023, 9:53am

I developed an example of using CDP in Katalon Studio. See