Scheduling Workers and Enqueuers
A queue worker is a long-running process. In production, a process manager such as Supervisor keeps it alive. See Keeping the Worker Running.
Some hosts have no process manager, or you want the scheduler to own the worker. ScheduleWorker and ScheduleEnqueuer let a scheduled event start the queue commands for you. Each tick starts a worker or enqueuer, which runs until its lifetime ends or it has nothing left to do.
ScheduleWorkerstartsqueue:worker.ScheduleEnqueuerstartsqueue:enqueuer.
INFO
These services need the scheduler to be set up first. See Scheduler. Install windwalker/queue as well.
Keep a Worker Running
Register an event in your schedules file. Type-hint ScheduleWorker in the handler, and call runByCommand():
// resources/registry/schedules.php
use Windwalker\Core\Queue\ScheduleWorker;
use Windwalker\Core\Queue\ScheduleWorkerOptions;
use Windwalker\Core\Schedule\Schedule;
/** @var Schedule $schedule */
$schedule->task('queue-worker')
// Every 5 minutes at minutes 4, 9, 14, ... (4-59/5 * * * *)
->everyMinutes(5, 4, 59)
->handler(
fn (ScheduleWorker $worker) => $worker->runByCommand(
lifetime: 180,
channels: ['mail', 'report'],
options: new ScheduleWorkerOptions(tries: 3),
)
);The handler runs php windwalker queue:worker with the channels, the --lifetime value, and the non-zero options. It waits until the worker exits.
Every tick checks the task lock first. While a worker is running, later ticks skip the event. When the worker exits, the next tick starts a new one, so up to about two minutes pass without a worker.
WARNING
Set lifetime above zero. With lifetime: 0, the worker never exits. The handler never returns, and the other events in that run never start.
Events run one after another. A running worker therefore occupies the scheduler for its whole lifetime.
Run an Enqueuer
ScheduleEnqueuer::runByCommand() works the same way. Use it to create jobs from a source on each tick:
use Windwalker\Core\Queue\ScheduleEnqueuer;
use Windwalker\Core\Queue\ScheduleEnqueuerOptions;
$schedule->task('queue-enqueuer')
// Every 5 minutes at minutes 2, 7, 12, ... (2-59/5 * * * *)
->everyMinutes(5, 2, 59)
->handler(
fn (ScheduleEnqueuer $enqueuer) => $enqueuer->runByCommand(
lifetime: 60,
channels: 'mail',
options: new ScheduleEnqueuerOptions(
stopWhenEmpty: true,
),
)
);With stopWhenEmpty, the process exits as soon as every handler returns null. It does not idle until the lifetime ends.
The command loads its channel handlers in the new process. See Enqueuer for how to write them.
Options
The first argument of runByCommand() is the lifetime in seconds. 0 means unlimited. The options object sets the rest. These are the worker options and their CLI flags:
new ScheduleWorkerOptions(
once: false, // --once: run one job, then exit
backoff: 0, // --backoff: seconds before a retry
force: false, // --force: run while the queue is paused
sleep: 1.0, // --sleep: seconds to wait when empty
tries: 5, // --tries: maximum attempts per job
timeout: 60, // --timeout: seconds allowed per job
maxRuns: 0, // --max-runs: stop after N jobs
shuffleChannels: false, // --shuffle-channels
stopWhenEmpty: false, // --stop-when-empty
enqueuer: false, // --enqueuer: also run the enqueuer
);The enqueuer options are similar. They have no tries or backoff, because the worker applies those:
new ScheduleEnqueuerOptions(
once: false, // --once: run one round, then exit
force: false, // --force: run while the queue is paused
memoryLimit: 128, // --memory: limit in megabytes
sleep: 1.0, // --sleep: seconds after each round
timeout: 60, // --timeout: seconds a round may run
maxRuns: 0, // --max-runs: stop after N loops
shuffleChannels: false, // --shuffle-channels
stopWhenEmpty: false, // --stop-when-empty
);INFO
runByCommand() does not pass memoryLimit to the worker. The worker uses its own default. memoryLimit applies to the in-process run() methods below.
TIP
enqueuer: true runs the enqueuer inside the worker process. The worker prints a warning, because a separate process performs better. Use two scheduled events instead when throughput matters.
Run in the Scheduler Process
run() runs the worker loop inside the scheduler process. Unlike runByCommand(), it does not start a separate queue:worker process. Compared with that process, the loop shares memory and services with the scheduler, so memory grows across the whole loop. memoryLimit still stops the loop when it is reached. A fatal error or exit() in a job also ends the scheduler, and the events after it in that run do not start.
run() keeps processing until the lifetime ends:
$schedule->task('queue-worker-inline')
// Every 5 minutes at minutes 4, 9, 14, ... (4-59/5 * * * *)
->everyMinutes(5, 4, 59)
->handler(
fn (ScheduleWorker $worker) => $worker->run(
lifetime: 180,
channels: 'default',
options: new ScheduleWorkerOptions(
stopWhenEmpty: true,
tries: 3,
),
)
);runOnce() is a similar method. It processes the queue until it is empty, then returns. Both methods throw a DomainException when windwalker/queue is not installed.
WARNING
Jobs run inside the scheduler process. A slow job delays the other events, and a job that leaks memory affects the whole process. For long work, use runByCommand().
Simulate a Queue with Cron
Without a background process, cron can run short batches instead. We recommend a batch every five minutes at minutes 2, 7, 12, and so on (2-59/5 * * * *). Each batch processes the queue for a fixed time and exits. See Choose the Interval for the reasons.
Limited throughput
This pattern gives you only one worker. Each task runs one process at a time, and the tasks run only a few minutes in every five. It suits small scheduled jobs and work that does not need to run right away. For a large volume of real-time background work, use proper queue workers under a process manager, as described in Keeping the Worker Running.
Set Up the Cron Line
Start the scheduler with the crontab entry described in Installing the Crontab. The batch below is one of the events it runs.
A Queue Batch
Register two tasks in your schedules file. The enqueuer task runs at minutes 2, 7, and so on. The worker task runs at minutes 4, 9, and so on, so the two never overlap. The enqueuer task comes first in the file:
use Windwalker\Core\Queue\ScheduleEnqueuer;
use Windwalker\Core\Queue\ScheduleEnqueuerOptions;
use Windwalker\Core\Queue\ScheduleWorker;
use Windwalker\Core\Queue\ScheduleWorkerOptions;
$schedule->task('queue-enqueuer')
// Every 5 minutes at minutes 2, 7, 12, ... (2-59/5 * * * *)
->everyMinutes(5, 2, 59)
->handler(
fn (ScheduleEnqueuer $enqueuer) => $enqueuer->run(
lifetime: 60,
channels: 'mail',
options: new ScheduleEnqueuerOptions(
stopWhenEmpty: true,
),
)
);
$schedule->task('queue-worker')
// Every 5 minutes at minutes 4, 9, 14, ... (4-59/5 * * * *)
->everyMinutes(5, 4, 59)
->handler(
fn (ScheduleWorker $worker) => $worker->run(
lifetime: 180,
channels: ['mail', 'default'],
options: new ScheduleWorkerOptions(
stopWhenEmpty: true,
tries: 3,
),
)
);Each task has its own task lock and lifetime. Keep each lifetime below the five-minute interval, so a task ends before its next tick starts it again. stopWhenEmpty makes each part exit as soon as its work is done, so an idle queue does not hold the batch open. The enqueuer task also needs its channel handlers. See Enqueuer.
TIP
Keep each lifetime below the interval. A task that ends early leaves the queue idle until the next tick.
Choose the Interval
The pattern 2-59/5 * * * * runs the enqueuer at minutes 2, 7, 12, and so on. The worker uses 4-59/5 * * * *, two minutes later. Use everyMinutes(5, 2, 59) and everyMinutes(5, 4, 59) to write them. These are a good default for queue tasks:
- It avoids minute
0. Hourly, daily, and most system cron jobs start at:00, so the batch does not compete with them. - It avoids every minute used by
*/5,*/10,*/15, and*/30. Those expressions all run at multiples of 5, and this pattern never lands on one. - It leaves room for the task. Each lifetime ends before the next run at the five-minute mark.
Other step values hit some of the same minutes. For example, */3 also runs at minutes 12 and 57. A one-minute interval also works, but then lifetime must stay below 60 seconds.
INFO
The worker checks lifetime between jobs. A job that starts just before the limit can run past it. Keep a margin of at least the job's timeout below the interval.
Place the Batch Last
Events due in the same minute run one after another in the scheduler process. A queue task blocks the events that come after it in that minute, so register the queue tasks last in the file. Register other events above them.
$schedule->daily('purge-sessions')
->handler(fn () => print("Sessions purged\n"));
// Last: the queue tasks block the events that follow them.
// Every 5 minutes at minutes 4, 9, 14, ... (4-59/5)
$schedule->task('queue-worker')
->everyMinutes(5, 4, 59)
/* ... */;WARNING
An event registered after the batch can start late. If the batch runs past its minute, the scheduler may skip the event for that minute.
Summary
ScheduleWorker::runByCommand()startsqueue:workeron each tick.ScheduleEnqueuer::runByCommand()startsqueue:enqueueron each tick.- Set a
lifetimeabove zero, so the worker recycles and the scheduler continues. - Register enqueuer handlers in
queue.enqueuer.handlers, because the command runs in a new process. - Use
run()to run the loop in the scheduler process. - Without a background process, use the cron tasks with the intervals
2-59/5 * * * *(enqueuer) and4-59/5 * * * *(worker), and register the queue tasks as the last events.