How can I convert a boto3 Dynamo DB item to a regular dictionary in Python?
In Python, when an item is fetched from DB Dynamo using boto3, a schema similar to the following is obtained.
{
"ACTIVE": {
"BOOL": true
},
"CRC": {
"N": "-1600155180"
},
"ID": {
"S": "bewfv43843b"
},
"params": {
"M": {
"customer": {
"S": "TEST"
},
"index": {
"N": "1"
}
}
},
"THIS_STATUS": {
"N": "10"
},
"TYPE": {
"N": "22"
}
}
Also, when inserting or scanning, the dictionaries must be converted this way. I have not been able to find a wrapper that takes care of such a conversion. Since apparently boto3 doesn't support this, are there any better alternatives than implementing a code for it?
source to share
To understand how to solve this, it is important to recognize that boto3 has two main modes of operation: one that uses the low-level Client API, and one that uses higher-level abstractions such as Table . The data structure shown in the question is an example of what is being consumed / created by a low-level API that is also used by AWS CLI and dynamodb web services.
To answer your question - if you can only work with high-level abstractions like Table when using boto3 then it will be a little easier for you, as the comments show. Then you can work around the whole problem - python types are marshaled to and from the low-level data format for you.
However, there are several times when it is not possible to use only those high-level designs. I specifically ran into this issue while working with DynamoDB streams linked to Lambdas. Lambda inputs are always in low-level format and this format is harder to work with IMO.
After some digging, I discovered that boto3 itself has some great features hidden away for making conversions. These functions are used implicitly in all internal conversions mentioned earlier. To use them directly, import the TypeDeserializer / TypeSerializer classes and combine them with dict understandings like so:
import boto3
low_level_data = {
"ACTIVE": {
"BOOL": True
},
"CRC": {
"N": "-1600155180"
},
"ID": {
"S": "bewfv43843b"
},
"params": {
"M": {
"customer": {
"S": "TEST"
},
"index": {
"N": "1"
}
}
},
"THIS_STATUS": {
"N": "10"
},
"TYPE": {
"N": "22"
}
}
# Lazy-eval the dynamodb attribute (boto3 is dynamic!)
boto3.resource('dynamodb')
# To go from low-level format to python
deserializer = boto3.dynamodb.types.TypeDeserializer()
python_data = {k: deserializer.deserialize(v) for k,v in low_level_data.items()}
# To go from python to low-level format
serializer = boto3.dynamodb.types.TypeSerializer()
low_level_copy = {k: serializer.serialize(v) for k,v in python_data.items()}
assert low_level_data == low_level_copy
source to share