Kaleidoscope Performance with MCJIT

In a previous post I described the process of porting the LLVM Kaleidoscope tutorial program to use MCJIT as its execution engine.  After navigating through a serious of road blocks we ended up with an implementation that was working as expected.

So it works, but the next question is, “Is it any good?”

A lot of people considering the transition from the older JIT execution engine to MCJIT have concerns about the possible performance implications, particularly related to the fact that MCJIT doesn’t support lazy compilation.  The older JIT engine will generate code for functions in an LLVM module one function at a time, delaying compilation of each function until it is about to be executed.  The MCJIT engine operates on entire modules, generating code for all functions in a module at once.  In the previous post, we modified the Kaleidoscope interpreter to create multiple modules as needed, but we’re still compiling the entire current module when a function is executed.

So what does that look like in terms of performance?

Using MCJIT with the Kaleidoscope Tutorial

You may have noticed that there are two different JIT execution engines in the LLVM project.  The older implementation (llvm::JIT) is a sort of ad hoc implementation that brings together various pieces of the LLVM code generation and adds its own glue to get dynamically generated code into memory one function at a time.  The newer implementation (llvm::MCJIT) is heavily based on the core MC library and emits complete object files into memory then prepares them for execution.

MCJIT has several advantages, including broader platform support and better tool integration.  However, because it is designed to compile entire modules into object images the MCJIT engine doesn’t directly support some key features of the older JIT implementation, such as lazy compilation.  By lazy compilation, I mean deferring compilation of individual functions until just before the function is going to be executed.

At this point you may find yourself saying, “Wait a minute?  Are you saying MCJIT doesn’t do ‘just-in-time’ compilation?!?”  Well…sort of.  It’s more of a dynamic code emitter than a true just-in-time compiler.  That said we’d like it to become a long term replacement for the old JIT so that we can reap the benefits of ongoing development in core MC code generation.

So the question becomes, can we make MCJIT do what the older JIT engine does?  The current answer is, “I hope so.”  As a means of exploring this question, I decided to try to convert the Kaleidoscope tutorial to use MCJIT.