Archiving a custom user notification service
I am creating an application that needs to send notifications to users at specific times of the day. Users can choose what time of day they would like to be notified and on what days they would like to be notified. For example, a user might receive a notification at 6 am every day or 7 am on weekdays only.
On the back-end server, I'm not sure how to archive the service sending these notifications. The solution should be handled:
- concurrency so I can scale my servers (notifications shouldn't be duplicated)
- system reboots
- If the user changes their preferences, pending notifications should be rescheduled
source to share
Using a message broker like RabbitMQ and a task scheduler like Celery might suit your needs.
Asynchronous or non-blocking processing is a method of separating the execution of certain tasks from the main thread of a program. This gives you several benefits, including allowing you to run your user-centric code seamlessly.
Message passing is a technique that program components can use to communicate and exchange information. It can be implemented synchronously or asynchronously and can allow discrete processes to communicate seamlessly. Message passing is often implemented as an alternative to traditional databases for this type of use, as message queues often provide additional functionality, provide improved performance, and can be entirely in memory.
Celery is a task queue that is built on an asynchronous messaging system. It can be used as a bucket where programming tasks can be dumped. The program that submitted the task can continue to execute and respond quickly, and then later, it can poll celery to see if the computation and data retrieval is complete.
While celery is written in Python, its protocol can be implemented in any language. worker is a Python implementation of Celery. If the language has an AMQP client, there shouldn't be a lot of work to create a working one in your language. The celery worker is simply a program that connects a broker to handle messages.
Also, theres another way to be language independent, and thats using REST tasks, not your tasks being functions, theyre url. With this information, you can even create simple web servers that allow you to pre-install code. Just output the endpoint that does the operation and create a task that just makes an HTTP request to that endpoint.
Here is a python example from the official documentation :
from celery import Celery
from celery.schedules import crontab
app = Celery()
@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
# Calls test('hello') every 10 seconds.
sender.add_periodic_task(10.0, test.s('hello'), name='add every 10')
# Calls test('world') every 30 seconds
sender.add_periodic_task(30.0, test.s('world'), expires=10)
# Executes every Monday morning at 7:30 a.m.
sender.add_periodic_task(
crontab(hour=7, minute=30, day_of_week=1),
test.s('Happy Mondays!'),
)
@app.task
def test(arg):
print(arg)
source to share
As I see it, you need to have 3 types of entities: users (to store email or in some other way for the user), notifications (to store what you want to send to the user - text, etc.) and schedules to store. when the user wants to be notified). You need to store objects of these types in some kind of database. The schedule must be connected to the user, the notification must be connected to the user and the schedule.
Let's say you have a cron job that runs some script every minute. This script will try to get all schedule related notifications for the current time (start time). Remember to implement some type of overlapping prevention .
After that, the script will put the tasks (with all the required data: type of notification, users you want to notify, etc.) in the queue (beanstalkd or whatever). You can create as many workers (even in different physical instances) as you want to serve this queue (without thinking about duplication) - this will give you powerful scalability...
In case the user changes his schedule, he will affect all his notifications at the same moment. There is no pending notification as they will only be served when they really need to be sent.
This is a very high level description. Many things depend on the language, database, queue server, wokers implementation.
source to share