Can you get Quartz.Net to use a priority queue?
Quart.Net by default only uses the priority of a job if it is started at the same time as another job. If two jobs start at different times, the earliest job will always start first in the thread pool. I have a scenario, although I need the highest priority job in the queue to always run on the next thread. Is there an easy way to do this with Quartz.Net, or will I have to implement my own scheduler if I want to do this? (Or switch to another technology).
The specific scenario I mean is: Periodically, jobs will run at high priority, which may cause some exit for another process. Minimizing latency is important here. However, I also have moments where I can run batches of up to several thousand jobs at once with a much lower priority. I am worried that when these "batches" are launched that much more important work will wait too long.
Is there an easy way to do this with Quartz.Net or a competing technology?
source to share
Have you seen trigger priorities? I don't think it will run it on a different thread, but it will force the job to start before another job that started earlier with a lower priority.
Trigger trigger = TriggerBuilder.Create()
.WithIdentity("TestTrigger")
.StartNow()
.WithPriority(10)
.Build();
source to share
As you remember, this is not directly supported by Quartz.Net. You can create your own scheduler (see below), or you can work around it if you are using database storage by updating the NEXT_FIRE_TIME column in the QRTZ_TRIGGERS table. If you update NEXT_FIRE_TIME to be prioritized, then the next time the scheduler checks for a schedule change, your jobs will run in priority order.
Exist
public void SignalSchedulingChange(DateTimeOffset? candidateNewNextFireTimeUtc)
on the QuartzScheduler , which you could have called after updating the fire time, but unfortunately it was not opened by StdScheduler . If you go down the "my own planner" route, you would call it directly. Otherwise, calling the ScheduleJob method in the scheduler (and scheduling the job) causes this as well, so you can use that and schedule the job after updating the next firing time.
source to share
The easiest way to implement this behavior is to subclass StdAdoDelegate
(or one of the concrete DB implementations) and override GetSelectNextTriggerToAcquireSql
to sort instead of PRIORITY
before NEXT_FIRE_TIME
.
The following class extracts the default SQL query and swaps the clause order using a null character temporary placeholder:
public class CustomAdoDelegate : StdAdoDelegate
{
protected override string GetSelectNextTriggerToAcquireSql(int maxCount)
{
return base.GetSelectNextTriggerToAcquireSql(maxCount)
.Replace(ColumnNextFireTime + " ASC", "\0")
.Replace(ColumnPriority + " DESC", ColumnNextFireTime + " ASC")
.Replace("\0", ColumnPriority + " DESC");
}
}
Be aware that this can lead to long delays for lower priority jobs if the scheduler is overloaded with higher priority jobs.
I haven't tried it myself, but it might be worth considering some hybrid approach that somehow balances the values of PRIORITY
and NEXT_FIRE_TIME
. For example, you can subtract the priority in seconds from the next fire time to create a variable-sized priority window.
order by dateadd(ss, -PRIORITY, NEXT_FIRE_TIME)
Thus, a priority = 10 trigger will only beat a priority = 5 trigger if it was fired no more than 5 seconds later. Just a thought.
source to share