How does type conversion work internally? What's the memory use for the same?
How does Go type conversion internally work?
What is the memory usage for type casting? For example:
var str1 string
str1 = "26MB string data"
byt := []byte(str1)
str2 := string(byt)
whenever i print any variable, will it take up more memory?
I am concerned about this because when I try to unmarshall I get " fatal error: runtime: out of memory "
err = json.Unmarshal([]byte(str1), &obj)
The str1 value is taken from the HTTP response, but read using ioutils.ReadAll, hence it contains the complete response.
source to share
This is called a conversion to Go (not a cast) and is covered in Spec: Conversions:
Special rules apply to (non-const) conversions between numeric types or to string types. These transformations can change the view
x
and incur runtime costs. All other conversions change only the type, not the representationx
.
Thus, usually the conversion does not make a copy, but only changes the type. Conversion to / from is string
usually done because the string
values are immutable and, for example, if converting string
to []byte
would not make a copy, you can change the content string
by changing the elements of the resulting byte chunk.
See the related question: Does conversion between types of aliases in Go create copies?
There are some exceptions (compiler optimizations) when converting to / from string
does not make a copy, see Golang for details : [] byte (string) vs [] byte (* string) .
If you already have JSON content as a string
value that you want to unmarshal, you shouldn't convert it to []byte
just for the sake of unmarshalling. Instead, use to get which one reads from the passed value, and pass that reader in , so you can unpack without having to make a copy of your large JSON input string. strings.NewReader()
io.Reader
string
json.NewDecoder()
This is how it might look:
input := "BIG JSON INPUT"
dec := json.NewDecoder(strings.NewReader(input))
var result YourResultType
if err := dec.Decode(&result); err != nil {
// Handle error
}
Also note that this solution can be further optimized if a large JSON string is being read from io.Reader
, in which case you can skip reading altogether at the beginning, just pass that json.NewDecoder()
in json.NewDecoder()
, for example:
dec := json.NewDecoder(jsonSource)
var result YourResultType
if err := dec.Decode(&result); err != nil {
// Handle error
}
source to share