Working with raw bytes from the network in go
(Sorry long question!) I recently tried Go as opposed to C ++ for a game server emulator that I am working on as a side project, and wondering if I am implementing it in a reasonable Go environment. As you would expect, the server communicates with one or more game clients by sending raw (TCP) packets that adhere to a specific protocol specification. The relevant part looks something like this:
get header -> decrypt it -> recv bytes until header length is reached -> decrypt remainder of packet -> send to handler -> decode packet if necessary -> -> send response
The protocol is defined in bytes in small trailing order, so in my C ++ implementation, the package header looks like this (I know it only works on LE machines):
struct pkt_header {
uint16_t length;
uint16_t type;
uint32_t flags;
};
After recv () and decrypting that header, I'll extract the fields:
// client->recv_buffer is of type u_char[1024]
header = (pkt_header*) client->recv_buffer;
if (client->recv_size < header->length) {
// Recv some more
}
// Decrypt and so on
In the handlers themselves, I can nest the above header structure in other package structure definitions and apply them to buffer [] arrays to directly access the fields. From what I've read, struct alignment is (unsurprisingly) difficult or impossible, and very frustrating with Go.
Not knowing what else to do, I wrote this function to go from arbitrary Struct -> [] byte:
// Serializes the fields of a struct to an array of bytes in the order in which the fields are
// declared. Calls panic() if data is not a struct or pointer to struct.
func StructToBytes(data interface{}) []byte {
val := reflect.ValueOf(data)
valKind := val.Kind()
if valKind == reflect.Ptr {
val = reflect.ValueOf(data).Elem()
valKind = val.Kind()
}
if valKind != reflect.Struct {
panic("data must of type struct or struct ptr, got: " + valKind.String())
}
bytes := new(bytes.Buffer)
for i := 0; i < val.NumField(); i++ {
field := val.Field(i)
switch kind := field.Kind(); kind {
case reflect.Struct:
binary.Write(bytes, binary.LittleEndian, StructToBytes(field.Interface()))
case reflect.Array, reflect.Slice:
binary.Write(bytes, binary.LittleEndian, field.Interface())
case reflect.Uint8:
binary.Write(bytes, binary.LittleEndian, uint8(field.Uint()))
case reflect.Uint16:
binary.Write(bytes, binary.LittleEndian, uint16(field.Uint()))
// You get the idea
}
}
return bytes.Bytes()
}
And I would do it in the handler:
type Header struct {
length uint16
size uint16
flags uint32
}
newHeader := new(Header)
// Initialization, etc
client.Conn.Write(StructToBytes(newHeader)) // ex. [C8 00 03 00 00 00 01 00]
As a beginner newbie, feedback on how I can implement this more efficiently is more than welcome. This worked well so far, but now I am faced with the problem of how to do the opposite: go from [] byte-> Struct (eg [C8 00 03 00 00 01 00 00] to the header {length = C8, size = 03, flags = 0100}
Do I just need to implement the opposite of this or is there a better way to go from a byte array to an arbitrary structure (or vice versa, as opposed to my function)? Please let me know if there was any additional clarity.
source to share
The way to go would be to use encoding / binary , which internally does pretty much what you wrote above.
( playground )
package main
import (
"bytes"
"encoding/binary"
"fmt"
"log"
)
type Header struct {
Length uint16
Size uint16
Flags uint32
}
func main() {
header := &Header{Length: 0xC8, Size: 3, Flags: 0x100}
fmt.Printf("in = %#v\n", header)
buf := new(bytes.Buffer)
err := binary.Write(buf, binary.LittleEndian, header)
if err != nil {
log.Fatalf("binary.Write failed: %v", err)
}
b := buf.Bytes()
fmt.Printf("wire = % x\n", b)
var header2 Header
buf2 := bytes.NewReader(b)
err = binary.Read(buf2, binary.LittleEndian, &header2)
if err != nil {
log.Fatalf("binary.Read failed: %v", err)
}
fmt.Printf("out = %#v\n", header2)
}
What a seal
in = &main.Header{Length:0xc8, Size:0x3, Flags:0x100}
wire = c8 00 03 00 00 01 00 00
out = main.Header{Length:0xc8, Size:0x3, Flags:0x100}
source to share