Storm exceptions - handling in bolt.execute
Im using storm to handle a thread in which one of the bolts writes cassandra . The cassandra command session.execute()
can throw an exception, and I'm wondering how to catch this so that the tuple "doesn't fire" so that it can be repeated.
The docs for IRichBolt don't show , it throws anything, so I'm wondering how the exception cases are handled.
The main question is, should I wrap the cassandra call in a try / catch, or will it handle this case for me?
source to share
Multi-page answer:
1) Definitely bundle your code with a try-catch block.
2) How to deal with the error depends on the Storm topology and the type of failure:
If an exception indicates that an immediate retry might work, you can loop a small, finite number of times until the retry succeeds or you end up trying.
If the tuple you are executing is a tuple that was spout out, then your bolt can disable the tuple. This will force the Storm to repeat (i.e. call the fail () method on the spout and you can program the repeat)
If there has already been at least one side effect as a result of processing this tuple, and you do not want to repeat that side effect as a result of reloading the tuple, then you need to be a little more creative. Your Cassandra bolt might emit a bad tuple onto a bad crust stream where it might persist somewhere (HBase, filesystem, Kafka) until you're ready to try again. To try again, you can add another Spout to your topology that reads from this store of failed tuples and streams them back to the Cassandra bolt to try again. This gives you the ability to continuously loop your reps with extended times between attempts. If along with the bad tuple, you also keep / log the Cassandra exception, you can view / track the log to see ifare there any issues your admin should be aware of.
source to share