Hi,
Can we get statistics summary e.g. average throughput, error rate, etc. after test execution for app testing?
Hi @mohit.singh9512,
To get statistics summary, you could use driver.getPerformanceData(…) like below:
(To understand what those statistics mean, you could refer to this document: dumsys command)
import com.kms.katalon.core.mobile.keyword.internal.MobileDriverFactory as MobileDriverFactory
import io.appium.java_client.AppiumDriver as AppiumDriver
import io.appium.java_client.MobileElement as MobileElement
String packageName = 'com.hmh.api'
String dataType = 'memoryinfo' // one of 'cpuinfo', 'batteryinfo', 'networkinfo' or 'memoryinfo'
int dataReadTimeout = 20 // The number of attempts to read
AppiumDriver<MobileElement> driver = MobileDriverFactory.getDriver()
List<List<Object>> data = driver.getPerformanceData(packageName, dataType, dataReadTimeout);
println "\n --- "
println data
println " --- \n"
/* Look at the console log, you will get something like...
---
[[totalPrivateDirty, nativePrivateDirty, dalvikPrivateDirty, eglPrivateDirty, glPrivateDirty, totalPss, nativePss, dalvikPss, eglPss, glPss, nativeHeapAllocatedSize, nativeHeapSize], [51151, 3336, 1360, 18971, 23424, 56347, 3371, 1417, 18971, 23424, 7382, 10240]]
---
/*
The result is not readable, right? /=)
So I suggest the following step to mapping the result into a more readable form
List<String> data = driver.getPerformanceData(packageName, dataType, dataReadTimeout)
HashMap<String, Integer> readableData = mappingData(data)
def mappingData(List<String> data) {
HashMap<String, Integer> readableData = new HashMap<String, Integer>()
List<String> keys = data.get(0)
List<String> values = data.get(1)
int length = keys.size()
for (int i = 0; i < length; i++) {
String key = keys.get(i)
Object value = values.get(i)
int intValue = value != null ? Integer.parseInt(value.toString()) : 0
readableData.put(key, intValue)
}
return readableData
}
Besides, to be sure what kind of performance data you can get. You could use driver.getSupportedPerformanceDataTypes()
List<String> performanceDataTypes = driver.getSupportedPerformanceDataTypes()
println('\n --- ')
println('> Supported performance data types: ' + performanceDataTypes)
println(' --- \n')
Put them all together, and we come to this one
import com.kms.katalon.core.mobile.keyword.internal.MobileDriverFactory as MobileDriverFactory
import io.appium.java_client.AppiumDriver as AppiumDriver
import io.appium.java_client.MobileElement as MobileElement
import com.kms.katalon.core.util.internal.JsonUtil as JsonUtil
Mobile.startApplication(GlobalVariable.G_AndroidApp, false)
AppiumDriver<MobileElement> driver = MobileDriverFactory.getDriver()
String packageName = 'com.hmh.api'
int dataReadTimeout = 20
List<String> performanceDataTypes = driver.getSupportedPerformanceDataTypes()
println('\n --- ')
println('> Supported performance data types: ' + performanceDataTypes)
println(' --- \n')
performanceDataTypes.stream().forEach({ def dataType ->
try {
List<String> data = driver.getPerformanceData(packageName, dataType, dataReadTimeout)
HashMap<String, Integer> readableData = mappingData(data)
println('\n --- ' + dataType)
println(JsonUtil.toJson(readableData, true))
println(' --- \n')
} catch (def error) {
println('\n Unable to get performance data of type "' + dataType + '"\n')
}
})
Mobile.closeApplication()
// - - -
def mappingData(List<String> data) {
HashMap<String, Integer> readableData = new HashMap<String, Integer>()
List<String> keys = data.get(0)
List<String> values = data.get(1)
int length = keys.size()
for (int i = 0; i < length; i++) {
String key = keys.get(i)
Object value = values.get(i)
int intValue = value != null ? Integer.parseInt(value.toString()) : 0
readableData.put(key, intValue)
}
return readableData
}
Sample results:
---
> Supported performance data types: [cpuinfo, memoryinfo, batteryinfo, networkinfo]
---
--- cpuinfo
{
"kernel": 0,
"user": 0
}
---
--- memoryinfo
{
"eglPss": 18971,
"glPrivateDirty": 23424,
"totalPrivateDirty": 51159,
"dalvikPss": 1540,
"nativePrivateDirty": 3344,
"glPss": 23424,
"totalPss": 56633,
"nativeHeapAllocatedSize": 7385,
"dalvikPrivateDirty": 1484,
"nativeHeapSize": 10240,
"eglPrivateDirty": 18971,
"nativePss": 3376
}
---
--- batteryinfo
{
"power": 100
}
---
--- networkinfo
{
"rb": 17372774,
"st": 1585699200,
"op": 0,
"bucketDuration": 3600,
"activeTime": 0,
"tp": 8545,
"rp": 15148,
"tb": 1737439
}
---
This post is based on a very useful article from appiumpro.com. Very thanks to Appium Pro Team!
hi @thongnmtran
I am specifically looking for network related statistics, for eg. count of network calls, average response time of calls etc. Is it possible to get such information?
Katalon is strictly a functional testing tool that works on the GUI layer, primarily. If you want detailed statistics, you’d be much better served using a designated performance testing tool that works on the API layer instead.
1 Like
Hi @Brandon_Hein,
Yeah, I very agree with you that Katalon should provide a higher-level interface. I will discuss this with the Katalon team to see if we could provide a new mobile keyword, or maybe a new GUI for this feature /=)
Hi @mohit.singh9512,
as the previous answer, you could get the network statistic from the networkinfo. Those are raw information, so you have to calculate the average response time of calls… by yourself.
String packageName = 'com.hmh.api'
int dataReadTimeout = 20
String dataType = 'networkinfo'
List<String> data = driver.getPerformanceData(packageName, dataType, dataReadTimeout)
/* Network information:
{
"rb": 17372774,
"st": 1585699200,
"op": 0,
"bucketDuration": 3600,
"activeTime": 0,
"tp": 8545,
"rp": 15148,
"tb": 1737439
}
*/
with…
- op: number of operations
- st: bucket start time
- bucketDuration: bucket duration time
- activeTime: active time
- rb: rx bytes
- tb: tx bytes
- tp: tx packages
- rp: rx packages