Fighting cold startup issues for your Kotlin Lambda with GraalVM
In this post we will see how we can tackle high cold startup times for Kotlin functions on AWS Lambda using GraalVM and the AWS custom Lambda Runtimes. Although we use Kotlin here, the article also applies to plain Java.
Kotlin is an awesome language and gains more and more traction. Thus it does not surprise that developers and companies would also like to use this great language in a serverless scenario. AWS Lambda offers a Java11 runtime and it is possible to also run applications written in Kotlin there. This works great in general but cold startup is really an issue here.
Tim Bray summarized it really nicely in his talk Inside AWS: Technology Choices for Modern Applications: The java community optimized for runtime performance over decades. And this was right in the application server days where application startup was not much of an issue. In the serverless era the bad startup performance of the JVM becomes a real deal-breaker for JVM languages.
But there is hope - GraalVM to the rescue.
GraalVM gives you enhanced performance for JVM-based applications written in languages such as Java, Scala, Groovy, Clojure or Kotlin. It uses Graal, a dynamic compiler that dramatically improves efficiency and speed of applications through unique approaches to code analysis and optimization.
This sounds like GraalVM can help us to fight the cold startup performance issues.
Running your application inside a Java VM comes with startup and footprint costs. GraalVM has a feature to create native images for existing JVM-based applications. The image generation process employs static analysis to find any code reachable from the main Java method and then performs full ahead-of-time (AOT) compilation. The resulting native binary contains the whole program in machine code form for its immediate execution.
This was not helping us on AWS Lambda until recently. AWS only provides a standard Java8 and Java11 runtime and it was very hard to run a GraalVM native image there (see Using GraalVM to run Native Java in AWS Lambda with Golang).
At the re:Invent 2018 AWS announced custom runtimes for AWS Lambda. So now we can get a bit more creative on the runtimes of our lambdas.
Let’s have a look if we can manage to run a lambda implemented in Kotlin in a GraalVM native image using a custom runtime.
Implementing a custom runtime in Kotlin
First we need to implement a custom lambda runtime in Kotlin. Fortunately the custom runtime API is really simple. We just need to poll for new lambda invocations using a HTTP endpoint. After an invocation was returned we call the actual handler and POST the result back to another HTTP endpoint (see AWS Lambda Runtime Interface)
All the event loop function does is to loop and invoke a function. Also it takes care of errors when executing the function and reports it to the API endpoint responsible to handle initialization errors. The environment variable AWS_LAMBDA_RUNTIME_API contains the host and port of the runtime API.
Let’s look the getInvocation function next. It retrieves an invocation via the /runtime/invocation/next endpoint. The json returned from the endpoint is the same that a normal lambda handler would get as an input. Our sample code can just handle APIGateway events. The request id is needed to report the function result later and can be retrieved from a header named lambda-runtime-aws-request-id.
Now we can look at the next step — the executeHandlerAndPostResult function. All this function does is to invoke the actual handler and POST the result to the runtime API. The request id is used to correlate the invocation to the result.
The actual handler logic is implemented as a http4k handler function. Http4k is a lightweigt HTTP toolkit written in Kotlin. A http4k router can also be invoked directly without another HTTP request and thus gives us a nice way to choose the actual handler function and also lets us implement the handler in a much friendlier way than a usual AWS Lambda function.
Alternatively the handler could also be invoked using reflection like the standard Java runtime does it. The name of the handler can be obtained via the _HANDLER environment variable.
We are also responsible here to notify the runtime API about errors.
That is all we need for the custom runtime. The next step is to build a native image out of our jar file and to build the deployment package.
Build and deploy the native image
We use the GraalVM Community Edition docker image to invoke the native image. (see package.sh in the sample application)
The result of the command above is a native binary. This is one part of our deployable package. We need to add a shell script named bootstrap to the package. This script is the entry point that is called to start the runtime. It becomes really simple in our case (server is the name of our native image).
We are using the serverless framework to manage and deploy our project. See the configuration file below — note the runtime: provided attribute on the function configuration. This is instructing the lambda runtime to invoke the bootstrap script rather than to ramp up the usual java runtime.
Where are the numbers? Is it worth it?
So what do we gain by all this?
It is hard to provide accurate measures for the cold startup cases. So I just looked at a few first calls after a deployment. The values showed some variance but still give a good indication on how much faster the GraalVM version is.
Looking at the logs of a invocation that triggered a cold start we see the following:
REPORT RequestId: 155aa77e-6f07–4f31-a468–6de06edfd98b Duration: 23.85 ms Billed Duration: 300 ms Memory Size: 1024 MB Max Memory Used: 66 MB Init Duration: 178.22 ms
The “Init Duration” contains everything need on AWS side to bring up the runtime. The “Duration” is the time the custom runtime took to initialize and the function logic took to execute.
To compare I used a reference function running on the standard java11runtime. The measurements are taken from the Cloud Watch duration metric of the lambda.
The following report log line can be observed from a cold start of the java11 runtime.
REPORT RequestId: bbb722f5-d7ae-451a-aac2–416c65400bb2 Duration: 2282.02 ms Billed Duration: 2300 ms Memory Size: 1024 MB Max Memory Used: 133 MB Init Duration: 391.63 ms
We can see that both the “Init Duration” and the “Duration” are significantly higher.
The GraalVM version showed cold startup times around 200ms while the standard java11 runtime was somewhere near 2500ms. This effect is quite amazing. Also in a “warm” state the GraalVM version seems to be faster than the standard runtime (median 1.1ms vs 2.1ms). But the difference is so small that it can be neglected. It would be interesting to compare a more complex function here.
To the not so bright side of the story. Building a native image using GraalVM is still not straight forward — a lot of limitations exist. I did not manage to use OkHttp as a HTTP client. The GraalVM compiler failed there so I fell back to the Java Http client. Also reflection can be an issue. Using Jackson for serialization and deserialization from and to JSON brings in reflection and I had to list the classes that need to be serialized and deseralized in a json file and provide the native image builder with this file (-H:ReflectionConfigurationFiles=/working/reflect.json). This reflection configuration can also be provided programmatically. This can be really cumbersome in a bigger project, because these reflection errors I encountered only showed on runtime.
Also the GraalVM native image compilation takes a really long time. For the simple test project it took up to 90 seconds
So GraalVM is really promising. It could be the tool to overcome cold startup issues for JVM-based languages on AWS Lambda. But also it is questionable if this is already usable in a real-world project given the pitfalls and limitations mentioned earlier. But it is definitely worth to have an eye on GraalVM in the future.
The sample project described in this post can be found on github.
