The Diagnosis
The root cause of the SSLHandshakeException is that Katalon Studio’s built-in HTTP client (which powers the Object Repository UI) utilizes the default Java Virtual Machine (JVM) truststore and keystore. When an endpoint demands mutual authentication, the server requests a client certificate. Because Katalon’s default network execution profile is unaware of your custom client certificate (.p12, .pfx, or .jks), the TLS handshake fails immediately.
To solve this globally and seamlessly for both the Object Manager UI and test executions, we must configure Katalon’s execution properties to initialize the JVM with your specific keystore parameters.
The Solution: Global JVM Arguments
The most scalable, industry-standard approach doesn’t require modifying individual Object Repository objects. Instead, you inject the keystore configuration directly into Katalon’s execution profile via Project Settings.
-
Open your project in Katalon Studio.
-
Navigate to Project > Settings > Execution > Default > Web Service (or External depending on your version’s execution paths).
-
Add the following JVM arguments to the execution parameters:
Plaintext
-Djavax.net.ssl.keyStore=C:/path/to/your/certificate.p12
-Djavax.net.ssl.keyStorePassword=your_keystore_password
-Djavax.net.ssl.keyStoreType=PKCS12
Note: If you are using a .jks file instead of .p12, change the keyStoreType value to JKS.
Custom Keyword Helper (For Dynamic/Multi-Certificate Environments)
If your framework requires switching between different client certificates dynamically during a test run (which cannot be handled by static JVM arguments), you must override the engine’s default system properties programmatically before the request is invoked.
Below is a robust Custom Keyword to handle this programmatically.
Groovy
package com.architecture.tls
import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.util.KeywordUtil
public class SSLContextHelper {
/**
* Dynamically sets the SSL Client Certificate for mTLS handshakes
* @param keystorePath Absolute path to the .p12 or .jks file
* @param keystorePassword The password for the keystore
* @param keystoreType Default is "PKCS12", use "JKS" if applicable
*/
@Keyword
public static void setClientCertificate(String keystorePath, String keystorePassword, String keystoreType = "PKCS12") {
try {
System.setProperty("javax.net.ssl.keyStore", keystorePath)
System.setProperty("javax.net.ssl.keyStorePassword", keystorePassword)
System.setProperty("javax.net.ssl.keyStoreType", keystoreType)
KeywordUtil.logInfo("Successfully loaded mTLS client certificate profile: " + keystorePath)
} catch (Exception e) {
KeywordUtil.markFailed("Failed to set SSL System Properties: " + e.getMessage())
}
}
}
How to use it in a Test Case:
Groovy
// Call this at the very beginning of your test script before hitting the Object Repository request
CustomKeywords.'com.architecture.tls.SSLContextHelper.setClientCertificate'('C:/certs/client.p12', 'SecurePass123!')
// Now send your object manager request safely
WS.sendRequest(findTestObject('Object Repository/Secure_API_Endpoint'))