The JVM ate my variable!

Consider this code:

Object obj = new Object();
WeakReference<Object> ref = new WeakReference<Object>(obj);

List<byte[]> filler = new LinkedList<byte[]>();
while (ref.get() != null) {
    filler.add(new byte[1000]);
}
System.out.println("Filler size " + filler.size());

When executed it should always run out of memory: ref.get() will not return null because the referenced object cannot be garbage collected. The local variable obj still holds a reference to it. The filler increases in size indefinitely, and the program will ultimately crash with an OutOfMemoryError.

What really happens is this:

$ java test
Filler size 28186

Wait, what?!

It turns out that the Hotspot virtual machine analyzes the code, sees that the variable obj is not used after the while-loop, and rewrites the method while it is running so that the local variable is effectively removed. This is called OSR (On Stack Replacement) compilation in the Hotspot VM. If you added something like print(obj) after the loop you would get the expected OOME. Quoting Kris Mok in About Printcompilation:

OSR in HotSpot is used to help improve performance of Java methods stuck in loops [6]. Without OSR, a method running in the interpreter can’t transfer to its compiled version even if there is one available, until the next time this method is invoked. With OSR, though, a Java method with long-running loops can run in the interpreter, trigger an OSR compilation in one of its loops, keep running in the interpreter until the compilation completes, and jump right into the compiled version without having to wait for “the next invocation”.

The process of “jumping right into the compiled version” may sound simple but in reality is anything but. The new method body does not start from the beginning but from the “back edge” of the running loop. The stack frame created by the interpreter is replaced by the one created by the JIT compiler. It is this process that is capable of removing local variables from the method.

JLS 12.6.1 especially allows this:

Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. For example, a compiler or code generator may choose to set a variable or parameter that will no longer be used to null to cause the storage for such an object to be potentially reclaimable sooner.

This behaviour is really unreliable: the thresholds for compiling code depend on the hardware and VM options. If you make the filler grow faster by adding bigger arrays of bytes you get you get an OOME: you have to give the compiler enough loop iterations to be triggered.

On-Stack-Replacement can affect benchmarks in unexpected ways: the code you are running changes during the test. Cliff Click has written on this subject.

It also influences finalizers: Since obj is a local variable you would expect that its finalizer would not be called at least until after the method. But since the object is GC’d before the method ends, it is finalized as well. The lesson is you can’t depend in any way on when objects are finalized – it may be sooner than you think!

If you are wondering if you can see when OSR happens, you can use the -XX:+PrintCompilation flag:

$ java -XX:+PrintCompilation Test
    125   1       java.lang.String::hashCode (60 bytes)
    133   2       sun.nio.cs.UTF_8$Encoder::encodeArrayLoop (490 bytes)
    158   3       java.lang.String::charAt (33 bytes)
    159   4       java.lang.String::indexOf (151 bytes)
    174   5       java.lang.Object::<init> (1 bytes)
    182   6       java.util.LinkedList$Entry::<init> (20 bytes)
    183   7       java.util.LinkedList::add (12 bytes)
    185   8       java.util.LinkedList::addBefore (52 bytes)
    348   1 %     Test::main @ 25 (78 bytes)
Filler size 28082

The last line tells us that Hotspot compiled the Test.main method with OSR (the %-flag) 348ms into the execution.

(Inspired by the Stackoverflow question Are WeakHashMap Cleared During A Full GC? Thanks to berry120 and jalopaba for contributing detailed answers.)