@kazurayam interresting …
anyway, I saw this sentence in your git project:
It is pointless to give the maximum number of threads for TestClosureCollectionExceutor
with a value larger (8, 16, 32) than the number of cores you have.
Is this based on experiments, or an assumption?
Why do I ask? Well, there are two known methods for parallelization in Java (and not only in Java).
Processes and threads, a short description about can be found here:
Process vs Thread: What's the Difference? - javatpoint.
Not sure exactly how ExecutorService actually works, but it seems to be an interface to help create and execute threads (and async top of that, which is yet another clever mechanism for this case as there is no obvious reason for inter-thread communication)
In the case of processes, it make sense to limit the number of them to numproc
(or numproc -1
sometime) because those are handled by the kernel scheduler. So having more processes than the number of cores, some will be queued until resources are released.
However, with threads, the limitations usually came from other factors (mem available / heapsize, iowaits etc)
So, if i am guessing right and the ExecutorService is using just threading, you can try to experiment a bit by going with the pool size with larger values, until you find an optimum (it is not a guarantee it will perform better, since, as I mentioned, there are lot of other factors involved, but worth to try)
You can also keep an eye on the taskmanager and see how many java processes are running when you exectute the tests.
If you see just a single process (but with plenty threads, ofcourse), then everything is just threaded, if you see more than one, it may be that under-the-hood it is forking processes, so the limitation to numproc
start to make sense.