Hello Artisan

In this tutorial i will show you a simple laravel app that interpret how to use Laravel Queues and how to run Laravel Queue with redis and Monitoring with Laravel Horizon. I will show you database as well as Redis queue connections. We’ll see how Horizon can provide us a beautiful configuration and monitoring capabilities to queues. 

We can simply send email in Laravel. But in this tutorial we will see how we can send email via Laravel Queue with redis connection and monitor our failed job via Laravel Horizon.

In this laravel redis queue tutorial you will learn a bit more about queue and how to work with redis connection. Horizon will be bonus for you in this tutorial. In this example tutorial laravel horizon will be bonus part for you.

In this tutorial i’ll create a simple demo app that shows us how we can use queues in Laravel using the database connection. Then i’ll take a look at the Redis queue connection. Next, i’ll use Horizon to add some additional configuration and monitoring capabilities to our queues.

Now lets start our Laravel Horizon and Queue tutorial. I will explain step by step to complete this laravel queue series. You have to just follow this below step. 

Step 1 : Download Fresh Laravel App

I will start from scratch. So we have to download fresh laravel 7 app. To download it, run below command.

composer create-project --prefer-dist laravel/laravel QueueRedisHorizon

Step 2 : Create Model and Migration

To complete this tutorial i will use Order model. So we need some order to create laravel queue tutorial. For this reason i need factory also to insert some dummy data quickly.

php artisan make:model Order -fm

After running this command we will get orders migration file and OrderFactory. So open both file and paste it.

database/migrations/create_orders_table.php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class CreateOrdersTable extends Migration
{
    /**
     * Run the migrations.
     *
     * @return void
     */
    public function up()
    {
        Schema::create('orders', function (Blueprint $table) {
            $table->id();
            $table->string('name', 100);
            $table->integer('item_count')->unsigned();
            $table->timestamps();
        });
    }

    /**
     * Reverse the migrations.
     *
     * @return void
     */
    public function down()
    {
        Schema::dropIfExists('orders');
    }
}

Now run php artisan migrate command to save it into databse.

Step 3:  Add Dummy Records

In this we need to add some dummy records to create laravel queue tutorial. So open order factory and paste below code in it.

database/factories/OrderFactory.php

use App\Order;
use Faker\Generator as Faker;

$factory->define(Order::class, function (Faker $faker) {
    return [
        'name' => $faker->name,
        'item_count' => rand(1,10),
    ];
});

now open terminal and paste this

php artisan tinker
//then
factory(App\Order::class, 50)->create();
//
exit

Step 4:  Create Order Mail

In this step i will create Oder mail to send email to users. To create this email i will use markdown mail. So run below command to create it.

php artisan make:mail OrderShipped --markdown=emails.orders.shipped

Now go to Mailtrap and setup your mail connection.

app/Mails/OrderShipped.php

namespace App\Mail;

use Illuminate\Bus\Queueable;
use Illuminate\Mail\Mailable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Contracts\Queue\ShouldQueue;

use App\Order;

class OrderShipped extends Mailable
{
    use Queueable, SerializesModels;

    public $order;

    public function __construct(Order $order)
    {
        $this->order = $order;
    }

    public function build()
    {
        return $this->markdown('emails.orders.shipped');
    }
}

now we need to modify the emails/orders/shipped.blade.php file like this

resources/views/emails/orders/shipped.blade.php

@component('mail::message')
# Introduction

Thank you {{ $order->name }}.  We just shipped {{ $order->item_count }} items.

@component('mail::button', ['url' => ''])
Track Order #{{ $order->id }}
@endcomponent

Thanks,
{{ config('app.name') }}
@endcomponent 

Step 5:  Create Route

Now we need to create our route to send email in laravel with laravel queue. So paste this below code

routes/web.php

Route::get('test', 'MailController@index');

Now create a mail controller to write index method.

php artisan make:controller MailController

Now open mail controller and create an index function in your controller like this:

app/Http/Controllers/MailController.php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

use Illuminate\Support\Facades\Mail;
use App\Mail\OrderShipped;
use App\Order;

class MailController extends Controller
{
    public function index() {

        $order = Order::findOrFail( rand(1,50) );

        $recipient = '[email protected]';

        Mail::to($recipient)->send(new OrderShipped($order));

        return 'Sent order ' . $order->id;
    }
}

Now open your browser and paste this url and hit enter button. Then you will see the below email in your mailtrap inbox.

http://localhost:8000/test

then go to mailtrap and you should see your mail.

Step 6: Setup Queue

Neat, we’re all set to go. Our application can send email, but still it’s not using queues. To demonstrate how to use queue in laravel, let’s create a job table that can be dispatched to a queue.

php artisan make:job SendOrderEmail

now open this file and paste this below code.

app/jobs/SendOrderEmail.php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

use Illuminate\Support\Facades\Mail;
use App\Mail\OrderShipped;
use App\Order;
use Log;

class SendOrderEmail implements ShouldQueue
{
	use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

	public $order;

	public function __construct(Order $order)
	{
		$this->order = $order;
	}

	public function handle()
	{
		$recipient = '[email protected]';
		Mail::to($recipient)->send(new OrderShipped($this->order));
		Log::info('Emailed order ' . $this->order->id);
	}
}

Now time to rewrite our MailController to dispatch SendOrderEmail job instead of sending the email directly. So open and rewrite it like below

app/Http/Controllers/MailController.php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

use Illuminate\Support\Facades\Mail;
use App\Jobs\SendOrderEmail;
use App\Order;
use Log;

class MailController extends Controller
{
    public function index() {

        $order = Order::findOrFail( rand(1,50) );
        SendOrderEmail::dispatch($order);

        Log::info('Dispatched order ' . $order->id);
        return 'Dispatched order ' . $order->id;
    }
}

Now reload this below url. Then you should see same message in your mailtrap browser that is sending via Laravel Queue.

http://localhost:8000/test

and open your Log file you will see the below log info:

storage/log/laravel.log

[2020-05-15 22:22:18] local.INFO: Emailed order 20
[2020-05-15 22:22:18] local.INFO: Dispatched order 20

Step 7 : Setup Database Queue

Now in this we are going to use laravel database queue to send email. Now looking at your .env file you’ll see that QUEUE_DRIVER is set to sync. In config/queue.php you should see that all the queue driver options: sync, database, beanstalkd, sqs, redis, null. Let’s use a real queue to send email now. Run below command to create queue table

php artisan queue:table
php artisan migrate

And open .env and change QUEUE_CONNECTION from sync to database.

.env

QUEUE_CONNECTION=database

The job is ready for us to make a queue work and to pick it up and process it. run:

php artisan queue:work

And reload this url

URL

http://localhost:8000/test

Then you will see the output looks like this:

[2020-05-15 22:35:35][1] Processing: App\Jobs\SendOrderEmail
[2020-05-15 22:35:36][1] Processed:  App\Jobs\SendOrderEmail

and you will get mail from queue in your mailtrap mailbox.

Step 8 : Find Out Failed Jobs

If you see the jobs table now, it must be empty. You know that every time a job is processed, it is removed from the jobs table. What if something goes wrong? We’ll need a failed jobs table to handle failed job.

php artisan queue:failed-table
php artisan migrate

 now add this to check failed job in below path

app/jobs/SendOrderEmail.php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

use Illuminate\Support\Facades\Mail;
use App\Mail\OrderShipped;
use App\Order;
use Log;

class SendOrderEmail implements ShouldQueue
{
	use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

	public $order;

	public function __construct(Order $order)
	{
		$this->order = $order;
	}

	public function handle()
	{
		$recipient = '[email protected]';
		Mail::to($recipient)->send(new OrderShipped($this->order));
		Log::info('Emailed order ' . $this->order->id);
        throw new \Exception("I am throwing this exception", 1);
	}
}

Now let’s send another email by visiting http://localhost.8000/test. This time, let’s add some more parameters to our queue workers. Run:

php artisan queue:work --tries=3

In the terminal you should see something like this:

Go ahead and stop queue:work with CTRL+C. Run the command:

php artisan queue:failed

It shows all the failed jobs that are no longer in your queue.

Go back to SendOrderEmail.php and remove the exception you added.

throw new \Exception("I am throwing this exception", 1);

Now on the command line run:

php artisan queue:retry 1

Here 1 is the ID of the failed_job. It pushes the job back onto the queue. Run the queue again:

php artisan queue:work --tries=3

Now this time the job was processed successfully and your email is in mailtrap!

Step 9 : Setup Redis

Firstly run below command to install Redis via composer.

composer require predis/predis

In the above example we sent mail using laravel queue with database connection. But now we will send mail with the connection of Redis. Now time to setup redis in our machine. So install redis in your machine and after running redis in your machine follow below step.

QUEUE_CONNECTION=redis

Now open database.php file and go to the bottom and make changes like below.

 'redis' => [

        'client' => env('REDIS_CLIENT', 'predis'),
 ],

Now start your redis server and check like below. I am using windows 10.

127.0.0.1:6379> PING
PONG
127.0.0.1:6379>

Now redis and everything is set to go. Why we should use Redis for our Laravel queue connection? Redis offers us clustering and rate limiting. Let’s look at an example of rate limiting from laravel docs.

Open up your SendOrderEmail job and rewrite it to the following:

app/jobs/SendOrderEmail .php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

use Illuminate\Support\Facades\Redis;
use Illuminate\Support\Facades\Mail;
use App\Mail\OrderShipped;
use App\Order;
use Log;

class SendOrderEmail implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $order;

    public function __construct(Order $order)
    {
        $this->order = $order;
    }

    public function handle()
    {
        // Allow only 2 emails every 1 second
        Redis::throttle('any_key')->allow(2)->every(1)->then(function () {

            $recipient = '[email protected]';
            Mail::to($recipient)->send(new OrderShipped($this->order));
            Log::info('Emailed order ' . $this->order->id);

        }, function () {
            // Could not obtain lock; this job will be re-queued
            return $this->release(2);
        });
    }

}

In the line Redis::throttle('any_key')->allow(2)->every(1)->then( I am using the Redis::throttle command and passing in any_key. We could pass in any key we want. Redis will take and remember the key.

Now let’s send another email by visiting http://localhost.8000/test. This time, email is sent by Redis connection.

Step 10 : Setup Horizon

Horizon provides us a awesome dashboard and code-driven configuration for your Laravel powered by Redis queues. Horizon allows us to easily monitor key metrics of your queue system such as job throughput, runtime, and job failures.

Run below command to install Laravel Horizon via composer.

composer require laravel/horizon
php artisan vendor:publish --provider="Laravel\Horizon\HorizonServiceProvider"

Look at the config/horizon.php file. At the bottom of horizon.php. The environment key must match the APP_ENV variable in your .env. For example, I have APP_ENV=local, so your application should use the local settings. I changed my local settings as like below:

'local' => [
      'supervisor-1' => [
          'connection' => 'redis',
          'queue' => ['email'],
          'balance' => 'auto',
          'processes' => 6,
          'tries' => 3,
    ],
],

There are three balance options. They are simple, auto and false. Simple splits jobs evenly between queue worker processes. Auto adjusts the number of worker processes depending on which queue is the biggest. False will not balance queues; whichever queue a job is dispatched to will run the job.

Let’s be specific about the way we dispatch to our queues. Update MailController to look like this:

app/Http/Controllers/MailController.php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

use Illuminate\Support\Facades\Mail;
use App\Jobs\SendOrderEmail;
use App\Order;
use Log;

class MailController extends Controller
{
    public function index() {

        for ($i=0; $i<20; $i++) { 

        $order = Order::findOrFail( rand(1,50) ); 
          SendOrderEmail::dispatch($order)->onQueue('email');
            
        }

        return 'Dispatched orders';
    }
}

Let’s go and start working the queue, but instead of using queue:work we have to use:

php artisan horizon

Now we can finally test our rate-limiting code by visiting http://localhost.8000/test. This will instantly queue up 20 emails. 

Step 11 : Horizon Dashboard

To work with horizon you must ensure that your queue connection is set to redis in your queue configuration file.

Horizon gives us a dashboard for monitoring our queue jobs. Now open this AppServiceProvider.php and paste this below code to open horizon dashboard. 

app/Providers/AppServiceProvider.php

namespace App\Providers;

use Illuminate\Support\ServiceProvider;
use Laravel\Horizon\Horizon;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        Horizon::auth(function ($request) {
            // Always show admin if local development
            if (env('APP_ENV') == 'local') {
                return true;
            }
        });
    }

    public function register()
    {
        //
    }
}

i hope you like this article.