OutOfMemoryError when using Gson to parse a large JSON response

URL url = new URL("http://pubapi.cryptsy.com/api.php?method=orderdatav2");
CryptsyCurrencyPairsReturn response = gson.fromJson(new InputStreamReader(url.openStream()), CryptsyCurrencyPairsReturn.class);

      

This is throwing an OutOfMemoryException for some of my users on older Android devices. How can I parse this great answer without running out of memory?

+3


source to share


3 answers


Analyzing big data in one-step mode is always tricky and complex. However, Gson comes with some nice features to support this.

You should be looking com.google.gson.stream.JsonReader

for doing json parsing using streams. This will allow you to parse the data in an incremental order as it is loaded and free devices from OutOfMemory errors.



Read this for more information .

+4


source


In my experience, you need to pass JSON data.
And yes, you can use google GSON to stream JSON data, this is an example on how to do it:

APIModel result = new APIModel();
        try {
            HttpResponse response;
            HttpClient myClient = new DefaultHttpClient();
            HttpPost myConnection = new HttpPost(APIParam.API_001_PRESENT(
                    serial_id, api_key));
            try {
                response = myClient.execute(myConnection);
                Reader streamReader = new InputStreamReader(response
                        .getEntity().getContent());
                JsonReader reader = new JsonReader(streamReader);
                reader.beginObject();
                while (reader.hasNext()) {

                    String name = reader.nextName();

                    if (name.equals("result")) {
                        if (reader.nextString() == "NG") {
                            result.setResult(Util.API_001_RESULT_NG);
                            break;
                        }
                    } else if (name.equals("items")) {
                        result = readItemsArray(reader);
                    } else {
                        reader.skipValue(); // avoid some unhandle events
                    }
                }

                reader.endObject();
                reader.close();
            } catch (Exception e) {
                e.printStackTrace();
                result.setResult(Util.API_001_RESULT_NG);
            }
        } catch (Exception e) {
            e.printStackTrace();
            result.setResult(Util.API_001_RESULT_NG);
        }

      

readItemsArray:

// read items array
    private APIModel readItemsArray(JsonReader reader) throws IOException {
        APIModel result = new APIModel();
        String item_name, file_name, data;
        result.setResult(Util.API_001_RESULT_OK);

        reader.beginArray();
        while (reader.hasNext()) {
            item_name = "";
            file_name = "";
            data = "";
            reader.beginObject();
            while (reader.hasNext()) {
                String name = reader.nextName();
                if (name.equals("name")) {
                    item_name = reader.nextString();
                } else if (name.equals("file")) {
                    file_name = reader.nextString();
                } else if (name.equals("data")) {
                    data = reader.nextString();
                } else {
                    reader.skipValue();
                }
            }
            reader.endObject();
            result.populateModel("null", item_name, file_name, data);
        }
        reader.endArray();
        return result;
    }

      



API class:

public class APIModel {
    private int result;
    private String error_title;
    private String error_message;
    private ArrayList<String> type;
    private ArrayList<String> item_name;
    private ArrayList<String> file_name;
    private ArrayList<String> data;

    public APIModel() {
        result = -1;
        error_title = "";
        error_message = "";
        setType(new ArrayList<String>());
        setItem_name(new ArrayList<String>());
        setFile_name(new ArrayList<String>());
        setData(new ArrayList<String>());
    }

    public void populateModel(String type, String item_name, String file_name, String data) {
        this.type.add(type);
        this.item_name.add(item_name);
        this.file_name.add(file_name);
        this.data.add(data);
    }

    public int getResult() {
        return result;
    }

    public void setResult(int result) {
        this.result = result;
    }

    public String getError_title() {
        return error_title;
    }

    public void setError_title(String error_title) {
        this.error_title = error_title;
    }

    public String getError_message() {
        return error_message;
    }

    public void setError_message(String error_message) {
        this.error_message = error_message;
    }

    public ArrayList<String> getType() {
        return type;
    }

    public void setType(ArrayList<String> type) {
        this.type = type;
    }

    public ArrayList<String> getItem_name() {
        return item_name;
    }

    public void setItem_name(ArrayList<String> item_name) {
        this.item_name = item_name;
    }

    public ArrayList<String> getFile_name() {
        return file_name;
    }

    public void setFile_name(ArrayList<String> file_name) {
        this.file_name = file_name;
    }

    public ArrayList<String> getData() {
        return data;
    }

    public void setData(ArrayList<String> data) {
        this.data = data;
    }


}

      

before i use the streaming API from google GSON i also got an OOM error because the JSON data i got is very big data (lots of Base64 encoded images and sounds) but with GSON streaming i can get over this error because it does not read data for the token all at once. And for the Jackson JSON library, I think it also has a streaming API and how to use it is almost the same as with my google GSON implementation. I hope my answer can help you, and if you have another question about my answer feel free to ask in the comment :)

+3


source


I would recommend looking at JsonReader (API 11+) or other options recommended.

+1


source







All Articles