LuaJ
I was reading over the list of features that CurioDB lacks compared to Redis , that I’d previously documented. It contained the item “no Lua scripting”, which I’d written confidently at the time, certain that I wouldn’t possibly be able to implement something so serious in my toy project. But then I thought to myself, “why not?”, and after a bit of research, I discovered the fantastic LuaJ library , which transformed the task from a significant engineering feat, into yet another case of merely gluing some libraries together.
LuaJ provides a complete API for compiling and running Lua scripts, and exposing JVM code to them as Lua functions. LuaJ also makes a complete range of Lua data types available to your JVM code. I purposely refer to the JVM ( Java Virtual Machine ) here rather than Java the language, as in my case, it’s actually Scala being used to code against LuaJ, which makes the overall accomplishment seem even more death-defying.
Obligatory Blog Snippets
Let’s take a look. First up, we’ll compile and run a snippet of Lua (sans imports and class scaffolding, which you can pick up from the final result in CurioDB if you wish):
// The global namespace we‘ll run the script in, which we could preload
// variables into if needed.
val globals = JsePlatform.standardGlobals()
// A tiny string of Lua code - it returns the second string from a
// table called "names" (Lua tables use 1-based indexes).
val script = "names = {‘foo‘, ‘bar‘, ‘baz‘}; return names[2]"
// Running the script returns a LuaValue object, which from our Lua
// code above, we know to be a string, so we cast it to one and print
// the value, which should be "bar".
val result = globals.load(script).call()
println(result.asjstring)As simple as that. Now let’s look at passing Lua variables from JVM land into a Lua script, and pulling Lua variables back out as JVM objects:
// The same table declared above in Lua, but this time in Scala.
val names = LuaValue.listOf(Array("foo", "bar", "baz").map(LuaValue.valueOf))
// Create the global namespace again, and pass the Lua table into it.
val globals = JsePlatform.standardGlobals()
globals.set("names", names)
// Our Lua code is inline here, and just returns the variable "names".
val result = globals.load("return names").call()
// The result is our original table back in Scala, where we can access
// items by index, just as we previously did in Lua.
println(result.get(2).asjstring)There you have it - JVM objects representing Lua variables, moving into and back out of Lua code. Now let’s look at the reverse, calling JVM code from inside Lua. To do this, we implement a class representing a Lua function:
// Simple function that takes a string, and prefixes it with "Hello ".
class SayHelloFunction extends OneArgFunction {
override def call(name: LuaValue) =
LuaValue.valueOf("Hello " + name.tojstring)
}
// Again, build the global namespace, this time adding the function
// object to it with a given name, and calling it from within Lua.
val globals = JsePlatform.standardGlobals()
globals.set("say_hello", new SayHelloFunction())
val result = globals.load("return say_hello(‘grandma‘)").call()
println(result.asjstring) // prints "Hello grandma".That’s it, the full round trip - Lua calling JVM code, and vice versa. With that working, the rest is up to your imagination. I’ve only scratched the surface of what LuaJ provides - all of the data types found in Lua, its standard library, and much more.
The final result was a little more involved than the above implies. CurioDB is carefully designed to be non-blocking, using tell rather than ask , where actors only ever send messages forwards, without expectation of a reply. The challenge here was introducing the synchronous redis.call API into a fundamentally asynchronous system. The solution involved modelling the scripting API as a client actor, with a controlled amount of blocking, much like the way TCP and HTTP connections are managed in the system.
Call-ception
A really fun and whacky side effect of implementing this possibly a little too correctly(for lack of a better term), is that it unintentionally allows the Lua API to recursively call itself:
$ redis-cli EVAL "return redis.call(‘EVAL‘, ‘return redis.call(\‘EVAL\‘, \‘return redis.call(\\\\\‘TIME\\\\\‘)\‘, 0)‘, 0)" 0
1) (integer) 227734
2) (integer) 541653Is this a feature, or a bug? The scripting API in Redis specifically disallows this, most likely for good reason.
http://www.tuicool.com/articles/M36byiy
http://stackoverflow.com/questions/32573748/multiple-return-values-in-luaj
http://levelup.sinaapp.com/
http://www.luaj.org/luaj/3.0/README.html
http://luaj.org/luaj/3.0/api/org/luaj/vm2/LuaValue.html
http://stackoverflow.com/questions/12358047/how-can-i-pass-objects-to-an-exposed-luaj-function