Google Cloud Storage Unexpected end of ZLIB input stream when uploading file larger than chunksize using java api
I am trying to download reviews from a play store that are available in a bucket using the java api (details at the end of the post). For browse files that are larger than the block size when compressed, I get the following exception:
java.io.EOFException: Unexpected end of ZLIB input stream
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:116)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at com.google.api.client.util.ByteStreams.copy(ByteStreams.java:51)
at com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
at com.google.api.client.util.IOUtils.copy(IOUtils.java:63)
at com.google.api.client.googleapis.media.MediaHttpDownloader.executeCurrentRequest(MediaHttpDownloader.java:247)
at com.google.api.client.googleapis.media.MediaHttpDownloader.download(MediaHttpDownloader.java:199)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMediaAndDownloadTo(AbstractGoogleClientRequest.java:553)
at com.google.api.services.storage.Storage$Objects$Get.executeMediaAndDownloadTo(Storage.java:4540)
at com.google.api.services.storage.Storage$Objects$Get$executeMediaAndDownloadTo.call(Unknown Source)
The code I used is:
private void saveObject(StorageObject object) {
try {
System.out.println("Getting object data");
FileOutputStream fos = new FileOutputStream (new File(object.getName()));
Storage.Objects.Get getObject =
storage.objects().get("pubsite_prod_rev_XXXXX", object.getName());
// setting the chunksize to a small value fails pretty much on all files
getObject.getMediaHttpDownloader().setDirectDownloadEnabled(false).setChunkSize(MediaHttpUploader.MINIMUM_CHUNK_SIZE)
// setting the chunksize to larger value allows me to download some of the files but fails on larger ones
//getObject.getMediaHttpDownloader().setDirectDownloadEnabled(false).setChunkSize(MediaHttpDownloader.MAXIMUM_CHUNK_SIZE)
getObject.executeMediaAndDownloadTo(fos);
fos.close();
} catch(Exception e) {
System.err.println("Error downloading " + object.getName());
e.printStackTrace();
}
I can upload files using gsutil, so it seems like the chunking implementation is inside the MediaHttpDownloader.
I am using java 7 and the following libs
compile group: 'com.google.api-client', name: 'google-api-client', version: '1.19.0'
compile group: 'com.google.oauth-client', name: 'google-oauth-client-jetty', version: '1.19.0'
compile group: 'com.google.apis', name: 'google-api-services-storage', version: 'v1beta2-rev56-1.19.0'
Also I am using groovy but don't see how it will affect the problem as the error is inside google lib.
thanks for any help
Andre
+3
source to share
No one has answered this question yet
Check out similar questions: