Call System.gc between work requests #558
Conversation
Bencodes
requested review from
cgruber,
jin,
restingbull and
timpeut
as code owners
| @@ -73,6 +73,8 @@ class PersistentWorker( | |||
| .forEach { request -> | |||
| launch { | |||
| compileWork(request, io, writeChannel, execute) | |||
| //Be a friendly worker by performing a GC between compilation requests | |||
| System.gc() | |||
There was a problem hiding this comment.
Mirroring what's being done in the generic worker implementation inside of Bazel might actually be a better approach here that doesn't incur too much gc overhead. I can make those changes if folks think this is the right direction.
There was a problem hiding this comment.
I've found in other systems calling System.gc frequently only hurts instead of helping. Calling periodically would certainly be better, but is there other JVM GC settings that can help here instead of manually invoking GC?
There was a problem hiding this comment.
I am curious why bazel has decided to call System.gc as well - id think tuning the GC a bit more worthwhile. That said the approach you linked @Bencodes seems like a decent compromise.
| import java.time.Duration | ||
| import java.util.concurrent.atomic.AtomicReference | ||
|
|
||
| class CpuTimeBasedGcScheduler( |
There was a problem hiding this comment.
Mirroring what's being done inside the generic Bazel worker
There was a problem hiding this comment.
It maybe worthwhile for future readers to inline the link to bazel source and part of the PR description in a class comment :)
These long running Kotlin workers are consuming lots of memory, eventually causing the Linux out-of-memory killer to step in and kill a random java process resulting in this error message:
Example crash output (the out file is empty):
Calling
System.gcbetween these Kotlin work requests has improved stability for us, and it seems to be the standard in Bazel core (seen here and here).This implementation is intended to closely mirror what Bazel is doing inside it's worker implementation seen here.