How I cut AWS Lambda Java Cold Start Times in Half

It is rare that a simple JVM argument change can have a dramatic impact on execution times, but in the case of AWS Lambda adjusting the Tiered Complication settings can have a really big impact on performance in many (but not all) cases.

The change I made was to add the JVM arguments:

-XX:+TieredCompilation -XX:TieredStopAtLevel=1

On AWS Lambda you can do this by simply adding the environment variable:

JAVA_TOOL_OPTIONS = -XX:+TieredCompilation -XX:TieredStopAtLevel=1

This chart below shows the average execution time of a Java (more specifically written in CFML and using FuseLess) Lambda Function over the course of two days. I've annotated where I made the change and you can see average execution times cut in half, cold start times also cut in half (though the chart doesn't show that). Before enabling the flag I was seeing cold start times of about 3.6 seconds, after the flag average cold start time was around 1.2 seconds.

Note that this particular java lambda function that I used for testing does have some highly intensive invocations that might take up to 30 seconds to execute, after the change I didn't see any executions going above 15 seconds.

What is TieredCompilation and what is TieredStopAtLevel?

That graph is probably leading you to wonder what is TieredCompilation in java. As you may know the JVM translates your bytecode (class files) into machine code at runtime. This step can take a lot of resources if the compiler wants to make it optimal, it might add counters so it can figure out where the hot spots are and then perform further optimizations. This works great and leads to awesome performance on the JVM when your server is running for a long time. In a way the JVM self tunes itself over time, at runtime.

When you are running on a serverless environment like AWS Lambda your runtime will only survive for a short period of time (as little as a few minutes, but no more than a few hours) before being destroyed. The assumption that the JVM default configuration makes is servers will run for a long time, and it should optimize the parts of the code that run the most over time. By setting the TieredStopAtLevel flag to 1 we are telling the JVM not to insert profiling code that is used for runtime measurements and optimizations. In essence we are telling the JVM, don't try to be too smart, don't optimize for the future - this code may never run agin, just execute the code, and in the process we are saving CPU cycles and yielding faster execution.

I learned about this technique from AWS, so it makes me wonder why they don't just enable it by default on all Java Lambda Functions? I think all Java Lambda functions could see a lower cold start time with these flags set, but possibly not a lower overall average execution time like I saw. It really depends on how well your code can be optimized by the jvm, and how many executions run before the lambda container is recycled. It is certainly worth your time to experiment and see if this flag will improve your Java Lambda Cold Start and Average Duration execution times. Such large performance gains are often not so easily found.

While my experience was specific to AWS Lambda running Java, I'm sure if you are running Azure Cloud Functions on Java, or Google Cloud Functions on Java you could use the same tip to cut your cold start times and hopefully your average duration as well.

Other ways to improve Java Cold Start Times

• Deployment Size - I've found that the total size of your deployment can make a big difference in the Java Cold Start execution time. Going from a 50mb deployment zip to a 15mb zip can make a big difference. Upon Cold Start, Lambda has to download your zip from the network, and then extract the zip to the file system.
• RAM - Increasing the amount of RAM that your Java Lambda has will reduce the ColdStart time, for Java I think the Lambda should have at least 1GB of memory (I usually run around 2-3GB). But it's not just about the RAM, because increasing the RAM will also give the Lambda instance more CPU power, which is helpful on Cold Starts.
• Avoid Nested Jars - Avoid nesting jars within jars within jars if you can. This is just creating busy work for the CPU to extract upon cold start.