Howto move from Resque to Solid Queue
Reading time: 4 minutes
Resque has done its job for a lot of
Rails applications, but it needs a Redis instance just to run background
jobs, and anything cron-like needs the separate resque-scheduler gem on
top. Solid Queue is Rails’ own
database-backed Active Job backend - jobs live in your existing database, and
recurring jobs are built in. Here is how we move a project from Resque
(including its scheduled jobs) to Solid Queue.
Prerequisites
- Ruby on Rails 7.1 or newer (Solid Queue ships as the default Active Job backend since Rails 8, and works as a gem on 7.1/7.2)
- A working test suite, so you can tell the migration didn’t silently break a job - see our approach to Test Driven Development
- A list of every Resque job class and every entry in your
resque-schedulercron config, so nothing gets forgotten in the move
Add Solid Queue
bundle add solid_queue
bin/rails solid_queue:install
The installer adds a config/queue.yml, a db/queue_schema.rb, and wires up
a separate queue database connection (recommended, so job traffic doesn’t
compete with your primary database on locks and connections).
Add the queue database to config/database.yml:
production:
primary:
<<: *default
database: myapp_production
queue:
<<: *default
database: myapp_production_queue
migrations_paths: db/queue_migrate
Run the queue database migrations:
bin/rails db:prepare
Point Active Job at Solid Queue:
# config/environments/production.rb
config.active_job.queue_adapter = :solid_queue
Convert your job classes
Resque jobs are plain classes with a class-level perform:
class HardWorker
@queue = :default
def self.perform(user_id)
User.find(user_id).send_welcome_email
end
end
Resque.enqueue(HardWorker, user.id)
Active Job jobs (which Solid Queue runs) are instance-level and look like this instead:
class HardWorkerJob < ApplicationJob
queue_as :default
def perform(user_id)
User.find(user_id).send_welcome_email
end
end
HardWorkerJob.perform_later(user.id)
Go through every @queue/self.perform job, turn it into an ApplicationJob
subclass, and replace every Resque.enqueue(...) call site with
.perform_later(...). Keep the queue names the same (queue_as :default,
queue_as :mailers, …) so priority and concurrency settings still apply
where you expect them.
Move cron/recurring jobs
This is the part that’s easy to lose in a migration. With
resque-scheduler, recurring jobs live in a YAML file, typically
config/resque_schedule.yml:
cleanup_expired_sessions:
cron: "0 3 * * *"
class: CleanupExpiredSessionsJob
queue: default
description: "Removes expired sessions every night at 3am"
send_daily_digest:
cron: "0 8 * * *"
class: SendDailyDigestJob
queue: mailers
args: "daily"
Solid Queue has recurring tasks built in - no extra gem needed. They live in
config/recurring.yml:
production:
cleanup_expired_sessions:
class: CleanupExpiredSessionsJob
schedule: "0 3 * * *"
queue: default
send_daily_digest:
class: SendDailyDigestJob
schedule: "0 8 * * *"
queue: mailers
args: ["daily"]
The schedule key accepts the same cron syntax resque-scheduler used, so
most entries move over line by line. Solid Queue also accepts human-readable
schedules (every hour, every day at 9am) if you’d rather not keep the
cron strings.
ATTENTION: resque-scheduler re-reads its YAML file while running and
enqueues jobs as class + args, same as Resque.enqueue. Solid Queue’s
recurring tasks are loaded when the Solid Queue process starts, so a change
to config/recurring.yml needs a restart of the Solid Queue supervisor (the
same as any other config file that’s loaded once at boot) to take effect.
Run the worker process
Resque workers are started through a rake task
(bundle exec rake resque:work). Solid Queue ships its own supervisor:
bin/rails solid_queue:start
or, using the executable added by the gem:
bin/jobs
Both start dispatcher, scheduler, and worker threads according to
config/queue.yml, including your recurring tasks - there is no separate
process to run for the cron-style jobs anymore.
If you supervise Resque with systemd (see
Control Resque through Capistrano using systemd),
swap the ExecStart line in the unit file:
# ~/.config/systemd/user/myapp-resque-workers.service
[Service]
ExecStart=/usr/bin/env bundle exec rake resque:work
becomes:
# ~/.config/systemd/user/myapp-solid-queue.service
[Service]
ExecStart=/usr/bin/env bin/jobs
Everything else in that setup - running it as a systemctl --user service,
enabling lingering, and hooking start/stop/restart into your Capistrano
deploy.rb - carries over unchanged. Just rename the unit and the
Capistrano task from resque:* to solid_queue:*.
Verify before switching over
- Run the full test suite against the converted jobs
- Enqueue each converted job manually in a console on staging and confirm it runs
- Let the recurring tasks run through at least one full cycle on staging and
check
SolidQueue::RecurringExecution(or the job’s own logging) to confirm it fired on schedule - Only then remove the
resque,resque-scheduler, andredis(if nothing else needs it) gems from theGemfile
Summary
Moving from Resque to Solid Queue means converting job classes from
class-level Resque jobs to instance-level Active Job jobs, moving
resque-scheduler’s cron YAML into config/recurring.yml (same cron syntax,
no extra gem), and swapping the worker process in your systemd/Capistrano
setup. Once it’s running, Redis is no longer a dependency for background
jobs - the queue lives in the same database as everything else.
Newsletter
See Also
- Control Resque through Capistrano using systemd
- Identify unused Routes in Ruby on Rails
- Use Geocoder and MaxMind DB to Geocode IP Addresses
- Working with Legacy Ruby on Rails: spring.gem fork() Crash
- Analyzing SassC::SyntaxError in Ruby on Rails 7.0