yumupgrades for production use, this is the repository for you.
Active subscription is required.
Magento 2 relies on message queues for some functions.
In a nutshell, this whole approach allows Magento to process some events in a timely, efficient and asynchronous way.
Message consumers are logical groups for of message processors.
They are defined within Magento code (usually declared in
To get a list of consumers, you can run
bin/magento queue:consumers:list. The default list as of Magento 2.3.4:
Each consumer does message processing for queues known to it.
Example of how message queues work
So for example, if you run a built-in export from Magento admin, a message is created in
queue_message MySQL table.
This message will contain requested export details.
Then, in a default setup, within a minute, your Magento cron job, will run a task
consumers_runner, which runs subprocesses for each defined consumer.
So one of them,
exportProcessor will check that MySQL table, and pick up the message about export, and create the export file for you.
All this happens in the background, without any wait for admin page, so there are no timeouts or other issues experienced in admin.
In this simple example, MySQL is the message exchanger software.
A message exchanger is simply put, a place where messages are stored until they are consumed.
RabbitMQ and message queues
Some features need a more sophisticated message exchanger than MySQL.
The Bulk API is one of the Magento features that relies on message queues.
However, it’s different in that it requires RabbitMQ to be the message exchanger.
Whether you use the bulk API endpoints or not, unless you configure RabbitMQ and set up its connection details, your
var/log/system.log will be riddled with the message:
main.INFO: Consumer “async.operations.all” skipped as required connection “amqp” is not configured. Unknown connection name amqp  
There are quite many bad solutions for this message all over the internet. For example, The answer from Rafael Corrêa Gomes (and any to the point that mentions either
'cron_run' => false or
'consumers' => ['async.operations.all'] in the configuration are quite outrageous because they make many folks copy-paste and destroy their message queues from functioning.
cron_run set to
false in configuration means that the message consumers are not going to be launched by the Magento cron. It means that you will have to run them using other means like SupervisorD or SystemD. And if not (and those answers don’t mention that at all), you will have broken stuff beyond words “message queues”, which includes, at very minimum, data export in Magento admin.
The documentation that is being referred to when posting such configuration has incorrect heading below this section with words:
It is a sample, and not a standard. Neither it is a recommendation to solve anything.
But, while the docs are at fault, it doesn’t excuse skipping the lines and not reading further, then posting destructive solutions online.
The real solution is either disabling the Bulk API which most installations don’t use or, configure RabbitMQ if you want the bulk API (highly unlikely, depends on specific use case).
Solution 1 (acceptable in most cases)
If you don’t intend to use bulk API (in all probability, you don’t care, because this is applicable to stores with a lot of visitors / specific use case),
you can disable the related module, and this, in turn, will disable the related “async.operations.all” consumer.
php bin/magento module:disable Magento_WebapiAsync
No more annoying message, at the cost of a disabled function, which didn’t work either way.
Solution 2. Setup and configure RabbitMQ
Step 1. Install RabbitMQ
If you have a powerful server, you may choose to set up RabbitMQ on the same host where you run Magento 2.
Magento 2 supports only the fairly recent branch of RabbitMQ, 3.8.x.
We will use the official RabbitMQ RPM repository to set it up.
The following commands will allow you to install RabbitMQ on either CentOS/RHEL 7 or CentOS/RHEL 8.
First, only on CentOS 7, you need to run the following:
curl -s https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash
This brings in required Erlang 22 (which is too old in the base repositories).
Next commands will be the same for both CentOS 7 and 8:
sudo yum -y install epel-release curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash sudo dnf -y install rabbitmq-server sudo systemctl enable --now rabbitmq-server
If you haven’t configured FQDN as your hostname or it otherwise does not resolve to an IP, you might receive:
Job for rabbitmq-server.service failed because the control process exited with error code.
and the following in
HOSTNAME rabbitmq-server: ERROR: epmd error for host HOSTNAME: timeout (timed out)
The solution (and this is what you have to do in a well-configured server) is either defining FQDN as the hostname and/or ensuring entry for hostname in
You can check the status of your RabbitMQ node by running
sudo rabbitmqctl status.
The default configuration file location is
You will want to create it in case you intend to use a cluster setup.
For single-server setups where Magento 2 and RabbitMQ are on the same machine, it is sufficient to use the default RabbitMQ credentials:
Step 2. Configure Magento 2 with RabbitMQ connection details
app/etc/env.php file with connection details applicable to your RabbitMQ instance.
'queue' => [ 'amqp' => [ 'host' => '127.0.0.1', // IP address of RabbitMQ instance. Change it, if it is not running on the same machine with Magento 'port' => '5672', // Port on which RabbitMQ running. 5672 is default port 'user' => 'guest', // RabbitMQ user name 'password' => 'guest', // RabbitMQ password 'virtualhost' => '/' // The virtual host for connecting to RabbitMQ. The default is /. ], ],
Verify that you have positioned the above config correctly in the file (it should be coming just above the trailing
]; by running
php -l app/etc/env.php.
bin/magento setup:upgrade --keep-generated to apply the changes and create the required queues in RabbitMQ.
Verify your setup. Run:
bin/magento queue:consumers:start async.operations.all
If it stays on without error, it means that the queue consumer has successfully established a connection to the message broker (RabbitMQ) and is waiting for a message to appear.
You can cancel it out with Ctrl+C now. It will be launched later automatically by cron.
You can then examine the output of
sudo rabbitmqctl status.
It will indicate a positive number of connections, e.g.:
Connection count: 1
Queue count: 1
Virtual host count: 1
Finally, minutes later, after you’re sure that Magento cron has run, you can check the process list:
ps aux | grep async.operations.all
You will see the consumer process:
/usr/bin/php path/to/magento2/bin/magento queue:consumers:start async.operations.all –pid-file-path=…
It is normal that this process stays on all the time.
That’s it! Now both Bulk API feature is functional and there is no annoying message about it.