How can I serialize functions at runtime in Clojure?
Is there a way to serialize functions at runtime in Clojure? I would like to be able to send stateless (but not pure) functions over the wire in a serialized format (edn perhaps, but I'm open to anything).
For example...
If I run prn-str
for a function, I don't get what was expected or expected.
user=> (def fn1 (fn [x] (* x 2)))
#'user/fn1
user=> (def data {:test 1 :key "value"})
#'user/data
user=> (defn fn2 [x] (* x 2))
#'user/fn2
user=> (prn-str fn1)
"#object[user$fn1 0x28b9c6e2 \"user$fn1@28b9c6e2\"]\n"
user=> (prn-str data)
"{:test 1, :key \"value\"}\n"
user=> (prn-str fn2)
"#object[user$fn2 0x206c48f5 \"user$fn2@206c48f5\"]\n"
user=>
I would like / expect something like this:
user=> (prn-str fn2)
"(fn [x] (* x 2))\n"
or maybe,
user=> (prn-str fn2)
"(defn fn2 [x] (* x 2))\n"
source to share
At some point, it stops being Clojure, so the expectation that we can arbitrarily bypass the journey from source to machine instructions and back doesn't work a bit.
We need to be able to serialize the function to a byte array and send it over the wire. I suspect that you will need to grab the function's java.lang.Class object and then pass that through the java.lang.instrument.ClassFileTransformer to get the bytes. After that, you can pass them to the friendly java.lang.ClassLoader on the remote jvm.
source to share
You basically have two options:
- pass the source code (s-expressions stored as clojure data)
- transfer jar files and download them from the other side.
for the first option, you keep the original during compilation of the function (almost always when it is defined), and then pass the same original expression to another computer and let it compile the same thing. so first you can create a vector of expressions:
(domain-functions '[(defn foo [x] x) (defn bar [y] (inc y)]
then you can store this in the database and every client can pass it to read
and then they all have the same functionality.
The second option depends on the fact that every time you define a function, it creates a class file in the / target directory and then loads it. Then you can sync this directory and download them from the other side. This approach is, of course, completely crazy, although people are crazy here. I recommend the first approach
And as a personal note:
I am doing this now with datomic and I have adopted the practice of injecting git -hash into a function name using a macro, so I am absolutely sure that when I call the function I get the same function I see in the editor. It brings peace of mind when running many cases that are all pulling from the same DB.
source to share
Flambo, a Clojure wrapper for Spark, uses the serializable-fn library to serialize functions (which Spark requires). Sparkling, another wrapper for Spark, uses the native functionality of Clojure through this abstract Java class that implements the Java Serializable interface.
source to share
This is actually not a very good way, and for some reason, just sending a function to another computer or storing it in a DB can cause a lot of problems, not least of which other functions may be required for that function.On the other end ...
A much better idea is to stick to data. Instead of writing the function, write the function name as an event, and then this can be translated later by whatever reads your data. Stick to data, which is idiomatic.
source to share