Hot code heap proposal would speed Java execution

OpenJDK draft plan would decrease fragmentation in the JVM code cache and reduce application execution time.

To reduce application execution time, Java’s segmented code cache would be extended with an optional “hot” code heap to compactly accommodate a part of non-profiled methods, under a draft JDK enhancement proposal now floating in the OpenJDK community. Another success metric of the plan is reducing the fragmentation of highly-optimized code in the code cache.

By extending Java’s compiler control mechanism to mark certain methods as hot so they could compile into the hot code heap, the proposal would separate code that is known to be hot, decrease fragmentation of selected code, and reduce the negative impact of compiled code hotness scattering on the performance of Java applications. It also would provide method-context dependent control for placing C2-compiled blobs into the hot code heap and provide a basis for the possibility of profiling and compacting the code using the virtual machine itself.

The hot code heap proposal is based on older proposals for a segmented code cache and compiler control. Both are from Java Development Kit 9, which was released in September 2017. Motivating the new proposal is the premise that sparse hot code is slower. Some applications may lose their performance due to the huge JVM code cache. This happens under certain conditions such as a lot of code being JIT-compiled, the presence of a vast amount of hot code, and hot code being scattered throughout the code cache.

The CPU itself imposes penalties for executing large amounts of scattered code, the proposal states. On systems where this problem is significant, it cannot be solved by other means, such as large pages. The degree of the slowdown depends on the amount of hot code, the sparseness of that code, and the processor type.

The proposal also emphasizes that not all compiled code is called frequently all the time and that co-locating hot code using profile information can boost performance. It also is possible to enhance compilation policy for hot code for better performance and to perform more aggressive sweeping of colder code to reduce code cache.

The hot code heap proposal has not been designated for a specific release of Java. The current release, JDK 22, was released last week while JDK 23 is due in September. JDK 23 would be a potential target for the hot code heap proposal, although the plan could slip out to a further release.