Unable to get data from Gmail API

plaidshirtakos · March 29, 2021, 12:44pm

I use this tutorial to connect to Gmail API: Java Quickstart | Gmail API | Google Developers I would like to make a keyword in Katalon Studio, which depends on Gmail API. I modified from sample code that line:

InputStream in = GmailQuickstart.class.getResourceAsStream(CREDENTIALS_FILE_PATH);

to this:

InputStream ins = new FileInputStream(CREDENTIALS_FILE_PATH);

I have added also following line:
import com.google.api.client.http.HttpRequest;
JAR files are imported, project is running and browser window is opened to get token. After successful authorization I got error message:

Caused by: java.lang.NoSuchMethodError: com.google.api.client.http.HttpRequest.setResponseReturnRawInputStream(Z)Lcom/google/api/client/http/HttpRequest;

List of imported dependencies:

commons-codec-1.15.jar
commons-logging-1.2.jar
google-api-client-1.31.3.jar
google-api-client-extensions-1.6.0-beta.jar
google-api-client-jackson2-1.31.3.jar
google-api-client-java6-1.31.3.jar
google-api-services-gmail-v1-rev110-1.25.0.jar
google-http-client-1.39.1.jar
google-http-client-jackson2-1.39.1.jar
google-oauth-client-java6-1.31.4.jar
google-oauth-client-jetty-1.31.4.jar
guava-30.1.1-jre.jar
httpclient-4.5.13.jar
httpcore-4.4.14.jar
j2objc-annotations-1.3.jar
jackson-core-2.12.2.jar
jsr305-3.0.2.jar

kazurayam · March 30, 2021, 6:38am

I tried the quickstart tutorial outside Katalon Studio (Emacs + Gradle + JDK11). It worked fine without any change.

@plaidshirtakos

I assume that

you are using Katalon Studio 7.9.1
you downloaded the aforementioned jar files and located in the <projectdir>/Drivers folder

You mentioned that you chose the following jar.

On the other hand in any Katalon Studio project of v7.9.1 has <projecitdir>/.classpath file. In the .classpath file I found a line:

	<classpathentry kind="lib" path="/Applications/Katalon Studio.app/Contents/Eclipse/plugins/com.google.http-client.google-http-client_1.22.0.jar"/>

This means that com.google.http-client.google-http-client_1.22.0.jar file is bundled in Katalon Studio. The bundled jar will have precedence over the jar files in the <projectdir>/Drivers folder. To be precise, the precedence is determined by the order of <classpathentriy> lines in the .classpath file. Earlier wins.

Caused by: java.lang.NoSuchMethodError: com.google.api.client.http.HttpRequest.setResponseReturnRawInputStream(Z)Lcom/google/api/client/http/HttpRequest;

I suppose that this Exception occured becase the com.google.api.client.http.HttpRequest class was loaded from the older version 1.22.0.

How to workaround?

The .classpath file is given by Katalon to you. You can not change it at all. You are suppose to live with the versions of external libs bundled in the Katalon Studion ver x.x.x if you are to develop a Custom Keyword for Katalon Studio.

In orther words, you can not take initiative which version of google-api-client-x.x.x.jar to choose. Rather, you have to find out which version of google-api-client-x.x.x.jar can coexist with the com.google.http-client.google-http-client_1.22.0.jar which is bundled in KS, and use that version to build your Custom Keyword.

kazurayam · March 30, 2021, 7:19am

Older version Gmail API jar might not work well with the recent version of Gmail services. I am not sure, but the google-api-client-1.23.0 has already been updated to 1.31.3 until now. I suppose there have been many changes in the Gmail service. Therefore I guess @plaidshirtakos may find that he really need the recent version of google-api-client-x.x.x.jar such as 1.31.3. In this case he will get stuck, as he can not take the initiative to choose the versions of Google API.

@ThanhTo
@duyluong

The Gmail API used by the “Read and Write Gmail Messages plugin” might be too old to work with the recent version of Gmail service. This might be the reason why many people report problems about the plugin. For example,

kazurayam · March 30, 2021, 7:26am

@ThanhTo

Why this jar is bundled in Katalon Studio 7.9.1?
For what use case is it used?
Is it really necessary to be bundled?

plaidshirtakos · March 30, 2021, 8:15am

Thanks @kazurayam ! HttpRequest.setResponseReturnRawInputStream was introduced in google-http-client 1.29.0, so I need at least that version for that keyword.

kazurayam · March 30, 2021, 9:17am

@ThanhTo
@duyluong

Is there any way for @plaidshirtakos to change the build path of KS so that his custom Groovy code refer to the google-http-client-1.39.1.jar in the Drivers folder rather than the com.google.http-client.google-http-client_1.22.0.jar bundled in the KS 7.9.1?

ThanhTo · March 31, 2021, 3:49am

I am not sure if it would help, but Katalon Studio offers a way to exclude built-in libraries from the build path.

kazurayam · March 31, 2021, 5:59am

@ThanhTo

Thank you for your reply. I believe that the Excluding built-in jars would solve his problem.

@plaidshirtakos

I suppose you would want to check from which jar file the com.google.api.client.http.HttpRequest is loaded. You can use the following Test Case script.

import com.google.api.client.http.HttpRequest

URL u = HttpRequest.class.getResource("/" + HttpRequest.class.getName().replaceAll("\\.", "/") + ".class");
System.out.println(u);

When I ran this, I got the following output:

14:51:35.222 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2021-03-31 14:51:35.225 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TCi
2021-03-31 14:51:36.365 DEBUG testcase.TCi                             - 1: u = HttpRequest.getResource("/" + getName().replaceAll("\.", "/") + ".class")
2021-03-31 14:51:36.394 DEBUG testcase.TCi                             - 2: out.println(u)

jar:file:/Applications/Katalon%20Studio.app/Contents/Eclipse/plugins/com.google.http-client.google-http-client_1.22.0.jar!/com/google/api/client/http/HttpRequest.class

2021-03-31 14:51:36.447 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TCi

plaidshirtakos · March 31, 2021, 6:50am

I use free edition of KS, so I can’t use the excluding option, because it is deactivated.

plaidshirtakos · March 31, 2021, 7:05am

Thanks, I checked it, /Katalon_Studio_Windows_64-7.9.0/plugins/com.google.http-client.google-http-client_1.22.0.jar is in use, so yes, NoSuchMethodError is caused by this.

kazurayam · March 31, 2021, 7:12am

The last resort for you would be removing the /Katalon_Studio_Windows_64-7.9.0/plugins/com.google.http-client.google-http-client_1.22.0.jar file. You want to remove it manually by rm file in command line or DEL in Windows Explorer. Just try to see what sort of difficulties you get next.

plaidshirtakos · March 31, 2021, 12:38pm

I have tried to remove it and to change it too. There is same error message in log, when I try to execute KS.

org.osgi.framework.BundleException: Could not resolve module: com.kms.katalon.activation [26]
Unresolved requirement: Require-Bundle: com.kms.katalon.application
-> Bundle-SymbolicName: com.kms.katalon.application; bundle-version=“1.0.0.202101180916”; singleton:=“true”
com.kms.katalon.application [28]
Unresolved requirement: Require-Bundle: com.google.http-client.google-http-client
Unresolved requirement: Require-Bundle: com.kms.katalon.controller
-> Bundle-SymbolicName: com.kms.katalon.controller; bundle-version=“1.0.0.202101180916”; singleton:=“true”
com.kms.katalon.controller [72]
Unresolved requirement: Require-Bundle: com.kms.katalon.core.webservice
-> Bundle-SymbolicName: com.kms.katalon.core.webservice; bundle-version=“1.0.0.202101180916”; singleton:=“true”
com.kms.katalon.core.webservice [78]
Unresolved requirement: Require-Bundle: com.google.http-client.google-http-client

kazurayam · April 1, 2021, 2:06am

It seems that Katalon Studio really requires the /Katalon_Studio_Windows_64-7.9.0/plugins/com.google.http-client.google-http-client_1.22.0.jar file. Now we know we can not remove the jar.

So, @plaidshirtakos, you are stuck. It seems that you can not work on the Gmail API in Katalon Studio unless Katalon Team works on upgrading the Gmail API-related bundled dependencies.

plaidshirtakos · April 1, 2021, 6:52am

@ThanhTo , @duyluong : Are you planning to update it in the near future?

Jass · May 4, 2021, 8:59am

duyluong · May 12, 2021, 9:18am

Hi @plaidshirtakos,

To prevent adding com.google.http-client.google-http-client_1.22.0.jar to project classpath, you should go to Project Settings/Library Management and add this jar name to Exclude field then reopen your project.

References: https://docs.katalon.com/katalon-studio/docs/external-libraries.html#exclude-built-in-libraries

duyluong · May 12, 2021, 9:27am

@kazurayam

Why this jar is bundled in Katalon Studio 7.9.1?
For what use case is it used?
Is it really necessary to be bundled?

This jar is a dependency of Katalon Studio Web Service engine, we will consider upgrading this library if it doesn’t affect too much with our Web Service engine.