Let's Take a Look at JEP 483: Ahead-of-Time Class Loading and Linking

3/28/2025 at 1:47:01 PM

I think the big problem here is conceptual. The JDK folks are looking at this akin to PGO when, IMHO, they should be looking at this as an AOT cache (yes, the flag names make this even more confusing). How do those two differ, you ask?

With PGO you do a lot of deliberate work to profile your application under different conditions and feed that information back to the compiler to make better branch/inlining decisions. With a AOT cache, you do nothing up front, and the JVM should just dump a big cache to disk every time it exits just in case it gets stared again on the same host. In this case, training runs would just be a” run you did to create the cache". With that said, the big technical challenge right ow is that building the AOT cache is expensive hence performance impacting and cannot really be done alongside a live application - but that’s where I think the focus should be, making filling the aot cache something less intensive and automatic.

Another aspect this strategy would help with is “what to do with these big AOT cache files”, if the AOT cache really starts caching every compiled method, it will become essentially another so file possibly of a size greater than the original JAR it started off with. Keeping this is in a docker image will double the size of the image slowing down deployments. Alternatively, with the aot cache concept, you just need to ensure there is some form of persistent disk cache across your hosts. The same logic also significantly helps CLIs, where I dont’ want to ship a 100MB CLI + Jlink bundle and have to add another 50MB of aot cache in it - what I do want is every time the client uses my CLI the JVM keeps improving the AOT cache.

by cleverfoo

3/28/2025 at 2:55:44 PM

It would be nice to be able to trigger AOT somehow, e.g. as part of a Docker build, or as part of an app startup as you say. Then the software deployment can decide what to do.

by robertlagrant

3/28/2025 at 1:55:43 PM

Is there any reason to think Java code can’t be statically linked, and then dead code eliminated (for that specific build of the app)?

I’m not asking if the tooling currently exists, I’m curious if there’s something inherent in .class files that would prevent static linking.

by jagged-chisel

3/28/2025 at 2:16:45 PM

> I’m not asking if the tooling currently exists, I’m curious if there’s something inherent in .class files that would prevent static linking.

It's not so much a problem with the .class files, instead it's a problem with reflection.

I can write `var foo = Class.forName("foo.bar.Baz")` which will cause the current class loader to look up and initialize the `foo.bar.Baz` class if it's available. I can then reflectively initialize an instance of that class by calling `foo.newInstance()`

Java has a ton of really neat meta-programming capabilities (and those will increase with the new ClassFile api). Unfortunately, those make static compilation and dead code elimination particularly hard. Tools that allow for static compilation (like graal) basically push the dev to declare upfront which classes will be accessed via reflection.

by cogman10

3/28/2025 at 2:09:08 PM

Well, assuming that by "statically linking" you mean in the c sense, that's exactly what GraalVM native image does today, it statically analyzes the JAR for reachability only compiling the methods/classes in use. This works but it's also what makes native-image difficult to use and brittle.

It's hard, and some might argue impossible, to statically analyze reachability in a dynamic language like java that allows for runtime class loading and redefinition. As it turns out, Java is much closer to javascript than C++ in terms of dynamic runtime behavior.

by cleverfoo

3/28/2025 at 5:16:35 PM

In the Java sense, if you properly utilize JPMS, JLink can cut dead modules and reduce your image size drastically. This obviously of course like you said depends on how "open" your runtime model is. If you're not dynamically loading jars it works really well.

by vips7L

3/28/2025 at 2:05:42 PM

Native Image does exactly that already, but producing an AOT-compiled-and-linked native executable is not the goal, it's just a means to some goal. The real question is what is it that you want to optimise? Is it startup/warmup time? Is it the size of the binary? Peak performance? Developer productivity? Program functionality? Ops capabilities?

AOT compilation certainly doesn't give you the best outcome for all of these concerns (regardless of the language).

by pron

3/28/2025 at 5:16:28 PM

The Leyden team are looking to do exactly what you're looking for. There will be further JEPs.

by pron

3/28/2025 at 12:30:51 PM

> The AOT cache file has a size of 66 MB in this case. It is considered an implementation detail and as such is subject to change between Java versions.

Does this mean this is mostly focused on containerised environments? I assume it means you couldn't upgrade the JVM on a server and know that this will keep working?

by robertlagrant

3/28/2025 at 6:48:29 PM

Leyden's AOT support is not specifically geared towards containerized workloads, AFAICS. It's a general trend in the Java world though to bundle the JVM alongside applications, potentially creating a tailored JVM distribution with just a subset of all the available modules of the SDK. The model where one JVM is installed on a server and multiple applications are using it is becoming more and more rare. In that current model, a JVM upgrade would happen when distributing a new version of the application using it.

by gunnarmorling

3/31/2025 at 8:33:10 AM

I've been out of the Java world a while - you spotted exactly what I was thinking of, which was a global JVM deployment that everything used. Or at least an app container that lots of things used. Thanks for the update.

by robertlagrant

3/28/2025 at 12:43:02 PM

> While start-up times don't matter that much for long running workloads, they can make a huge difference in cloud-native scenarios where applications are dynamically scaled out, spinning up new instances on demand as the load of incoming requests increases.

"App Images" are what these are called in Android afaik. ART (the Android Run Time) has been capable of "AoT Class Loading & Linking" since 2016/7 (and ~pauseless GC since 2019). Seems like the desktop/server grade JVMs have a lot of ground to cover.

by ignoramous

3/28/2025 at 1:38:02 PM

Both the requirements and the workload challenges of server-side applications differ greatly from client applications. The JDK has had a pauseless GC (i.e. <1ms pauses) for 18 months now. The need for fast startup and warmup in server-side apps is newer than for client apps, and even with it, optimising for peak performance is still a higher priority than for client apps.

You should expect a runtime that is used more on the server side to be "behind" when it comes to concerns that are more prevalent on the client and vice-versa, and that's exactly what you see.

by pron

3/28/2025 at 2:52:47 PM

I was configuring Shenandoah GC on OpenJDK 8 before COVID, so I can say for sure that it's been available for a lot longer than 18 months. However, ZGC (the other low-pause-time collector) has only been the default for a little while.

by kbolino

3/28/2025 at 3:59:35 PM

> You should expect a runtime that is used more on the server side to be "behind" when it comes to concerns that are more prevalent on the client

True, but the Client x Server concerns seem to be overlapping now, across the board. For instance, tech built for ChromeOS (a desktop class OS) is also used by GCP ("Container-optimized OS") and AWS ("Firecracker").

While ART has had tiered compilation (C1/C2 in the OpenJDK world) for quite a while. I also mentioned ART has long had Copying Collector (almost pauseless GC), which could help Server-grade apps just the same.

All that said, for ART, things are relatively "simpler" as it has to only target Linux+Fuschia on ARM32/64, x86/amd64, RISC-V ISAs.

An interesting side note about Dalvik/ART (Zygote) is that it uses the old Server-side technique of forking (but without exec; like httpd's "fork mode") to share pages and resources with its children (ie, with all app processes on Android).

by ignoramous

3/28/2025 at 6:38:27 PM

> True, but the Client x Server concerns seem to be overlapping now, across the board.

Sure, which is why you're seeing this work.

> While ART has had tiered compilation (C1/C2 in the OpenJDK world) for quite a while. I also mentioned ART has long had Copying Collector (almost pauseless GC), which could help Server-grade apps just the same.

I hope you're not suggesting that ART has a similar quality of compiler optimisations and GC as the JDK. But if you are, you're welcome to try and run significant server workloads on ART and see how they compare to JDK 24.

BTW, the last (Go-style) non-copying collector was deprecated in the JDK in 2017 and completely removed in 2020: https://openjdk.org/jeps/363. The reason why the low-latency GCs (what you call "almost pauseless") aren't the default in the JDK is that minimising latency is not (yet?) the top requirement for server apps. Some value throughput more.

by pron

3/28/2025 at 3:50:48 PM

Please don't write pauseless if there are short pauses. Pauseless in the Java GC context is a marketing scam.

by pebal

3/28/2025 at 4:11:05 PM

We call them low-latency GCs, but I was responding to a comment that called the same thing "pausless". However, even programs without any GC pauses can have pauses of a similar duration due to OS activity, so programs on regular OS aren't generally pauseless anyway and the term is mostly meaningless in most contexts. As a marketing term (which we don't use) it merely means the application won't experience GC-related pauses that are longer than OS-related pauses.

by pron

3/28/2025 at 4:28:52 PM

This is some weird way of counting. A system pause plus a GC pause is two pauses. Just because one pause can't be avoided doesn't mean you can introduce more pauses.

by pebal

3/28/2025 at 6:20:52 PM

Just to be clear, the pause in ZGC is done as an optimisation of thread synchronisation (there's no GC work done in the pause). It's just as easy to do things without anything you would consider a pause, only it's less efficient. Momentarily stopping all program threads is just an efficient way to deliver a message to all threads considering that the OS might have some threads descheduled.

At the end of the day, we care about the latency for some particular thread(s), and truly minimising that at this level requires a realtime kernel (that comes with significant performance disadvantages).

by pron

3/28/2025 at 4:39:58 PM

> I was responding to a comment that called the same thing "pausless"

In my defense, I called it "~pauseless" (as in, roughly pauseless).

by ignoramous

3/28/2025 at 5:56:31 PM

Pebal is really pedantic about the term in every thread. Because apparently only he has ever achieved pause free GC for his C++ gc.

by vips7L

3/28/2025 at 4:25:52 PM

Unless you are on a realtime OS, you may see longer pauses from the OS due to scheduling other processes, so I think it's fair to consider the new low-latency collectors (Shenandoah, ZGC) pauseless from a practical point of view.

by gunnarmorling

3/28/2025 at 4:46:59 PM

You can't call GC pauseless if it introduces pauses. We don't say something is free if you have to pay little for it. We say it's cheap.

by pebal

3/28/2025 at 8:17:30 PM

I think as a technical term, pausless (as used here) has a specific meaning, just like real-time has a specific technical meaning (not literally 0 latency since that's generally physically impossible).

by adambatkin

3/28/2025 at 12:58:50 PM

On the contrary, lots of this work was already present in commercial products long before Android came to be,

https://www.ptc.com/en/products/developer-tools/perc

https://www.aicas.com/products-services/jamaicavm/

https://eclipse.dev/openj9/docs/aot/

https://www.ibm.com/docs/en/sdk-java-technology/8?topic=refe...

Now gone, https://en.wikipedia.org/wiki/Excelsior_JET

What you are getting now is commercial work being offered as free beer, also why Excelsior JET is no longer relevant as company.

by pjmlp

3/28/2025 at 3:53:49 PM

Thanks.

> On the contrary, lots of this work was already present in commercial products

I should have been clearer that I meant other OSS JVMs have catching up to do.

> long before Android

True. The evolution of Dalvik/ART remind me of HP Dynamo: https://archive.arstechnica.com/reviews/1q00/dynamo/dynamo-2...

> What you are getting now is commercial work being offered as free beer

... AOSP is free as in beer.

by ignoramous

3/28/2025 at 6:20:54 PM

> AOSP is free as in beer.

True, however it isn't Java proper, and apparently Google has started to rewrite subsystems in Kotlin as well, when initially they said the OS layers would still be Java (language).

Apparently the updates to Java 11 and 17 LTS subsets in ART, were motivated to keep up with Maven Central more than anything else.

by pjmlp

3/30/2025 at 8:08:20 AM

Java has had AOT via GCJ or some commercial compiler which starts with a E and not Eclipse but I cant remember its name, since early 00s. The Android Run Time only supports a specific version of the JDK and a subset of its available Java APIs so having official AOT support is an entirely different story. And that is not even mentioning Graal AOT.

by ksec