Reading C ++ structures from socket to LISP
We have an application protocol defined as C ++ classes that are passed over the network. I want to connect to a server that is sending data in this format. I want to write a client in lisp (preferably sbcl) to communicate with this server. I would prefer it written in pure lisp instead of using CFFI to wrap C ++ dlls. The sampling frames would look something like this:
class Header
{
public:
int MsgType;
uint64_t Length;
}
class SampleMsg
{
public:
Header MsgHeader;
char Field1[256];
bool Field2;
double Field3;
SomeOtherClass Field4;
}
I want to know how to map these structures in lisp so that they are binary compatible and how to read / write such structures. Is there an easier way than packing / unpacking each field in the structure?
For example, in C #, you can map a binary structure like this and read it directly from a byte array:
[StructLayout(LayoutKind.Sequential)]
public struct Header
{
public int MsgType;
public ulong Length;
}
[StructLayout(LayoutKind.Sequential)]
public struct SampleMsg
{
public:
public Header MsgHeader;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 256)]
public string Field1;
public bool Field2;
public double Field3;
public SomeOtherClass Field4;
}
If a similar method is possible in lisp, that would be ideal. If not, I'm ready to do some plumbing while it's manageable.
EDIT:
Tried Svante's suggestion:
(ql:quickload "userial")
(in-package :sb-bsd-sockets)
(defun read-buffer (host port)
(let ((socket (make-instance 'inet-socket :type :stream :protocol :tcp)))
(socket-connect socket host port)
(let ((buf (socket-receive socket nil 1024 :element-type '(unsigned-byte 8))))
(socket-close socket)
buf)))
(defstruct header
msg-type
length)
(userial:make-slot-serializer (:header header (make-header))
:int64 msg-type
:uint64 length)
(defvar *buffer*)
(defvar *b*)
(setq *buffer* (read-buffer #(10 1 2 75) 5003))
(setq *b* (make-array 2048 :element-type '(unsigned-byte 8) :fill-pointer 0 :adjustable t))
(map 'vector #'(lambda (x) (vector-push x *b*)) *buffer*)
(setf (fill-pointer *b*) 0)
At this point, *b*
it runs something like this: #(7 0 0 0 0 0 0 0 176 2 0 0 0 0 0 0 45 71 253 83 0 0 0 0 165 30 11 11 0 0 0 ...)
. The first 7 correspond to type msg, which should be 7. The length should be 688 (176 + 2 * 256).
Now I am
(userial:with-buffer *b* (userial:unserialize :header))
. It gives me
#S(HEADER :MSG-TYPE 504403158265495552 :LENGTH 12682699500628738048)
#(7 0 0 0 0 0 0 0 176 2 0 0 0 0 0 0)
Sounds like an entiancy problem. How to fix it? I can't seem to find a way to deal with content inside a custom library.
EDIT2:
Finally ended up dropping the custom one and writing these (the following are practical common lisps):
(defun read-64 (buf)
(let ((u 0))
(setf (ldb (byte 8 56) u) (aref buf 7))
(setf (ldb (byte 8 48) u) (aref buf 6))
(setf (ldb (byte 8 40) u) (aref buf 5))
(setf (ldb (byte 8 32) u) (aref buf 4))
(setf (ldb (byte 8 24) u) (aref buf 3))
(setf (ldb (byte 8 16) u) (aref buf 2))
(setf (ldb (byte 8 8) u) (aref buf 1))
(setf (ldb (byte 8 0) u) (aref buf 0))
u))
(defun read-32 (buf)
(let ((u 0))
(setf (ldb (byte 8 24) u) (aref buf 3))
(setf (ldb (byte 8 16) u) (aref buf 2))
(setf (ldb (byte 8 8) u) (aref buf 1))
(setf (ldb (byte 8 0) u) (aref buf 0))
u))
(defun read-16 (buf)
(let ((u 0))
(setf (ldb (byte 8 8) u) (aref buf 1))
(setf (ldb (byte 8 0) u) (aref buf 0))
u))
Now I can write (read-uint64 (subseq *buffer* 8 16))
to get the msg length. Thanks for the help.
source to share
You can use the userial
one available in Quicklisp.
However, I would try really hard to get rid of the need to synchronize the two definitions (one in C ++, one on the Lisp side).
Edit: Here's what I meant. I have done some very small tests, so no guarantees. In particular, I have not tested with C ++ output and you will likely have a lot to tweak to align.
(defstruct header msg-type length) ;; Msg-type might be best handled with an enum unserializer: ;; (make-enum-unserializer :msg-type (:foo :bar)), but I don't know ;; what your values are. (defstruct sample-msg msg-header field-1 field-2 field-3 field-4) ;; You might need to use a different serializer for msg-type for ;; alignment. (make-slot-serializer (:header header (make-header)) :int msg-type :uint64 length) (make-vector-serializer :vector-256-char :uint8 256) ;; I have no idea how a boolean is serialized and aligned on the C++ ;; side, so I'll just use :boolean for field-3 here as a first ;; attempt. (make-slot-serializer (:sample-msg sample-msg (make-sample-msg)) :header msg-header :vector-256-char field-1 :boolean field-2 :float64 field-3 :some-other field-4) ;; You can serialize and unserialize now: (serialize :sample-msg some-sample-msg) (rewind-buffer) (unserialize :sample-msg) ;; Userial operates on an adjustable vector with fill-pointer in the ;; special variable *buffer*, so you'll need to fill that with content ;; from wherever you read that from. (with-buffer (read-my-content) (unserialize :sample-msg))
source to share