Moving Cassandra 1.2: Broken Pipe

We have migrated our cluster to Cassandra 1.2 (from 1.1.7) and we have tried post-migration repairs (we run them periodically as recommended).

The repair failed because the SSTable version is different, so we used nodetool upgradesstables

and nodetool scrub

and tried again.

But then the restore ended with this exception:

java.io.IOException: Broken pipe
    at com.google.common.base.Throwables.propagate(Throwables.java:160)
    at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
    at sun.nio.ch.FileChannelImpl.transferToDirectly(Unknown Source)
    at sun.nio.ch.FileChannelImpl.transferTo(Unknown Source)
    at org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream(CompressedFileStreamTask.java:90)
    at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
    at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)

      

We also noticed that it nodetool netstats

shows strange stats before dying (SSTable transmission is 756845% ...).

EDIT

There seems to be a problem with passing compressible SSTables, I'll try to disable compression on that ColumnFamily and try again.

RE-EDIT

Disabling compression fixed the problem, so we tried to re-enable it and got

java.lang.RuntimeException:
  Last written key DecoratedKey(97115[...]3136) >= current key DecoratedKey(8...

      

Any idea?

+3


source to share


1 answer


The issue was caused by a bug in Cassandra 1.2 affecting the transfer of compressed column families between nodes. This was fixed in 1.2.2 => Jira entry for this bug .



+2


source







All Articles