Clever Geek Handbook

Google Cloud Pub / Sub Retry Count

We're moving from a volatile messaging service to Google's Sub Pub in NodeJS. It seems to work well, but we would like to enable error handling.

We would like to limit the number of retries for a particular message to, say, 10 times in our test environment and 100 times in production. Now, if a message doesn't fire 10 times (in a test), instead of sitting in our queue and continuing to be processed and failing for 7 days, we would like to move it to a separate error queue and send us an email.

We are currently all set up in our previous message queue, but we have not yet been able to find a count attribute to retry Pub Subs for every message. Does anyone know if this exists?

We use task queues in Google App Engine and they have everything we need, but Google pub sub seems to be missing a lot. We require any solution to reside in Node.

+2


source to share


2 answers


Cloud Pub / Sub has no limit on the number of retry attempts to deliver a message to a subscriber. If your caller never confirms the message within the ack deadline, it will be re-sent before the message expires after 7 days.

If you want to stop receiving these messages, you will need to pick them up at some point. If you want to protect against "death messages" that cannot be handled by your subscribers, I recommend the following:



  • Track the number of failures in the database using the message ID. Hopefully it doesn't crash frequently, so this database shouldn't be too large and will only be queried if it crashes.

  • When the message fails, query the database and see how many of them happened first. Increase the counter and do not acknowledge the message if the counter is below your threshold.

  • If a message fails for more than your threshold, please post the message in a separate "failed messages" section, send an email and confirm the message.

  • If necessary, you have a tool that you can use to post messages from the Failed Messages topic to your main question if the issues that caused the message to be lost in the first place have been fixed.

You now have a post saved in a separate topic (for 7 days or until you confirm it) and no post will be sent to subscribers on your main topic.

+5


source


In python see the 'num_retries' parameter in .execute ():

pubsub_client.projects().topics().publish(topic='projects/xxxx',body=body).execute(num_retries=0)



Not sure if the same exists in Node.JS, but I hope this points you in the right direction.

0


source






More articles:


All Articles