Return barriers and their application to stack tracing on modern VMs

Sammanfattning: Virtual machines are today at the core of a very large portion of the code that runs our everyday lives. Their relative performance and characteristics are of high significance for society indirectly and few other are as impactful as the JVM. In this thesis, we examine the possibility to speed up stack tracing on HotSpot and OpenJDK by implementing a return barrier. The barrier is implemented by overwriting the return program counter on the top frame with a function that re-installs itself on the frame below every time it’s executed. Because of this, the barrier acts as a “low water mark” and we then leverage that fact to enable Java Flight Recorder to cache the section of the stack that has already been traversed before. While the first implementation of such a cache did not prove successful in terms of performance improvement we look at costs and benefits of different steps in the implementation and highlight where future use the barrier might be attempted.