Golang: calculating the amount of memory (or byte length) of a card
I want to limit the map to the maximum bytes. There seems to be no easy way to calculate the byte length of the map.
"encoding/binary"
the package has a nice feature Size
, but it only works for slices or "fixed values", not a map.
I could try to get all the key / value pairs from the map, infer their type (if it is a map[string]interface{}
), and calculate the length - but that would be cumbersome and probably wrong (since that would rule out golang's "internal" cost of the map itself - managing pointers on elements, etc.).
Any suggested way to do this? code example is preferred.
source to share
This is the definition for the map title:
// A header for a Go map.
type hmap struct {
// Note: the format of the Hmap is encoded in ../../cmd/gc/reflect.c and
// ../reflect/type.go. Don't change this structure without also changing that code!
count int // # live cells == size of map. Must be first (used by len() builtin)
flags uint32
hash0 uint32 // hash seed
B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing
nevacuate uintptr // progress counter for evacuation (buckets less than this have been evacuated)
}
Calculating its size is pretty straightforward (unsafe.Sizeof).
This is the definition for each individual bucket that the map point points to:
// A bucket for a Go map.
type bmap struct {
tophash [bucketCnt]uint8
// Followed by bucketCnt keys and then bucketCnt values.
// NOTE: packing all the keys together and then all the values together makes the
// code a bit more complicated than alternating key/value/key/value/... but it allows
// us to eliminate padding which would be needed for, e.g., map[int64]int8.
// Followed by an overflow pointer.
}
bucketCnt
- constant, defined as:
bucketCnt = 1 << bucketCntBits // equals decimal 8
bucketCntBits = 3
The final calculation will be as follows:
unsafe.Sizeof(hmap) + (len(theMap) * 8) + (len(theMap) * 8 * unsafe.Sizeof(x)) + (len(theMap) * 8 * unsafe.Sizeof(y))
Where theMap
is your map value is the map x
key y
type value and the map value type value.
You will need to share the structure hmap
with your package through an assembly, similarly thunk.s
in the runtime.
source to share