What if Cassandra reports a rejection, but made a partial entry?
Cassandra does not guarantee atomic behavior, so there is a small chance that one replica will fail, but the other will survive.
Is there any information on how to defend against this and what to do to heal it if it does? Is Cassandra healing herself in this regard?
[Update]
I specifically focused on the case where you send a write request to say that all replicas and only one replica have a write error. The node error while writing persists and works. According to the Cassandra documentation, a write request will fail even if the other two (if you have a replication factor of 3) succeed.
According to the documentation, in this case, two copies have been modified and one remains original. It was stated that in this case its an inconsistent state, since the other two would not be able to discard the recorded change.
So the question is how you can protect yourself from this.
source to share
In cassandra, a timeout like this is not considered a failure. See this blog post describing how Cassandra handles various conditions when it comes to writing:
Remember, for recording, a timeout is not an error.
How can we tell that, since we don't know what happened before the replica failed? The coordinator can enforce the results to a pre-update or post-update state. This is what Cassandra is doing with the scheduled relay.
... the coordinator saves the update locally and resubmits it to the failed replica when it recovers, thereby forcing it to the post-update state that the client originally requested.
So, to answer your question, yes, cassandra will heal itself using the intended handoff and when that process fails (i.e. max_hint_window_in_ms is exceeded before the replica is online) the repair should bring things to consistent state. This is one of the reasons why regular repairs are recommended.
This article explains handoffs in detail.
source to share