OCaml integration in python - How to host ocaml session from python?
I would like the python GUI to execute an OCAMl process in the background. I would like to keep one session for the entire life cycle of the program and depending on user inputs, call some OCaml commands and get OCaml output. Some OCaml variables and structures can be defined along the way, so I would like to maintain one current session.
My solution was to navigate the OCaml toplevel process with popen and interact with its stdin and stdout. This works exclusively for me for several reasons: 1. I do not know when OCaml is calculating and cannot determine if its output is complete or is still there (especially if the evaluation takes some time and if several OCaml commands were called). 2. I have no inherent way of telling if the OCaml command ran smoothly or if there were OCaml warnings or errors. 3. I am losing the OCaml output structure. For example, if the output is spread across multiple lines, I cannot determine which lines were split due to the size of the line and which were originally separate lines.
I know there are several discussions and some packages for combining python with OCaml, but they all run python commands from OCaml and I need the opposite.
source to share
If you want to run OCaml in a separate process, you need to wrap your OCaml calls in a function that collects all the results and returns them in a useful serialization format like. JSON. Alternatively, you could try writing a Python C extension that uses OCaml toplevel as a library , but that's probably a fair job.
source to share
If you want a tight coupling between your Python code and OCaml, there really should be two separate "core" processes (think of them as network nodes).
As @Sven Marnach already mentioned, a good option to implement this is to connect the two processes using a JSON based protocol.
A more convenient approach would be to use Google's gRPC framework ( https://grpc.io/ ) and exchange Protobuf messages ( https://developers.google.com/protocol-buffers/ ). The structure is very tame. Unfortunately there is no support for OCaml yet, but I think you can thinly trim OCaml main
to a thin Python layer, or translate it to JS. Then all you have to do is just wire your functions to the gRPC interfaces.
This is what the system looks like:
+----------+ +------+ +---Thin Python wrapper / JS wrapper---+
| Your | | | | +--------------------------------+ |
| Python |<->| gRPC |<->| | Your OCaml app | |
| app | | | | +--------------------------------+ |
+----------+ +------+ +--------------------------------------+
PS I am using the same approach in a problem similar to yours (but the GUI is in Java). I would say that it is very convenient, it develops quickly and easily expands.
PPS You are not alone in this :). Here's an interesting talk from an article by a (former?) Google employee ( https://arxiv.org/abs/1702.01715 ):
Google software developers are strongly encouraged to program in one of the five officially approved programming languages at Google: C ++, Java, Python, Go, or JavaScript.
Interaction between these different programming languages is done primarily using the Buffers protocol. Protocol buffers are a way to encode structured data in an efficient yet extensible way. It includes a domain-specific language for specifying structured data, as well as a compiler that takes such descriptions and generates code in C ++, Java, Python to build, access, serialize and deserialize these objects. Googles version Protocol buffers are integrated with the Googles RPC libraries to provide a simple cross-language RPC, with requests and responses serialized and deserialized automatically by the RPC framework.
source to share
As a side note, it is entirely possible to start a separate top-level process and send input phrases to it and read the corresponding output. The trick to detect the end of the top-level output is to add a guard phrase after each input phrase: instead of sending only f ();;
to the top-level process, one can send f ();; "end_of_input";;
and then watch the top-level output corresponding to "end_of_input";;
(aka - : string = "end_of_input"
). It is my experience that errors and warnings are usually fairly easy to detect or analyze from the output stream; so the only missing point is the code formatting.
source to share