Magento 2 / Server Setup

Magento and ElasticSearch? How about Sphinx!

by ,


We have by far the largest RPM repository with dynamic stable NGINX modules and VMODs for Varnish 4.1 and 6.0 LTS. If you want to install NGINX, Varnish and lots of useful modules for them, this is your one stop repository to get all performance related software.
You have to maintain an active subscription in order to be able to use the repository!

Magento 2 and its “poor” choice of technologies

Magento 2 as an E-Commerce framework, is capable of empowering very large and high-traffic webshops.
It is designed in a way that critical functions like search can scale to dozens of servers.

But Magento 2 absolutely doesn’t want you to run it on a single server, by default. How?

ElasticSearch is the default engine in Magento 2 as of recently. It may be a great software, it may scale to many servers, but being Java-based, it has quite some memory requirements,
Those requirements will push you into buying a very powerful server. However, there’s little chance that your webshop has the traffic the size eBay or YouTube.

There’s really no point in using memory-hungry ElasticSearch when you’re running any medium-size business.

Magento 2 is developed without understanding the simple critical fact:

The winning majority of Magento 2 installations are not the size of actual eBay and are seeking to run everything in one mid-sized server.

The defaults should allow Magento 2 to run on a single server. But now this isn’t true unless you’re running a server with a lot of RAM.

Otherwise, you will soon find that things are way too slower than they should be.

And it’s often not because of the slow server, but because Magento could embrace different technologies to make it more efficient in all-in-one server scenarios.

Yes, Magento 2 could have defaulted to a different, more efficient search engine too. There’s a way to fix the unwelcome Elasticsearch default!

The following instructions are applicable to CentOS/RHEL 6, 7, and 8.

Sphinx search plugin for Magento 2

What’s more efficient than the Java-based Elasticsearch? Virtually anything that is based on C/C++.

The Sphinx search engine not only satisfies that requirement but also has tremendously small memory and CPU footprint when compared to ElasticSearch.

To empower Magento 2 with the ability to use the Sphinx search engine, you must first grab the corresponding search plugin.

While being a paid plugin, Advanced Sphinx Search Pro by Mirasvit will save you money in the long run.
Purchasing a plugin is a one-off expense. But paying for a high-end server is typically a monthly cost that you will have to incur throughout your Magento 2 instance lifetime.

The installation of the plugin is the usual routine of either copying the files manually or (recommended) installing them via composer.

Let’s review the steps for composer-based installation.

First, let’s install clearmage2 that allows us zero-downtime installation of modules:

sudo yum -y install https://extras.getpagespeed.com/release-latest.rpm
sudo yum install clearmage2

Now, connect to your Magento SSH user, and navigate to the root directory of the Magento 2 store.

If you bought the extension in the mirasvit.com store, copy only the composer config installation instruction from the page My Downloadable Products / View & Download to the SSH console.

composer config ...

Then run the following to install and enable the module:

clearmage2 --install mirasvit/module-search-ultimate:* --enable Mirasvit_Core,Mirasvit_Search,Mirasvit_SearchMysql,Mirasvit_SearchSphinx,Mirasvit_SearchAutocomplete,Mirasvit_Misspell,Mirasvit_SearchLanding,Mirasvit_Report,Mirasvit_SearchReport,Mirasvit_SearchElasticNative

If you bought the extension in the Magento Marketplace, run the command:

clearmage2 --install mirasvit/module-search-ultimate --enable Mirasvit_Core,Mirasvit_Search,Mirasvit_SearchMysql,Mirasvit_SearchSphinx,Mirasvit_SearchAutocomplete,Mirasvit_Misspell,Mirasvit_SearchLanding,Mirasvit_Report,Mirasvit_SearchReport,Mirasvit_SearchElasticNative

Now, reindex search spell-correction index

php -f bin/magento indexer:reindex mst_misspell

Go to System / Search Indexes, configure Product index, and run the reindex for it.

Once the plugin is installed, you can proceed to install the Sphinx search engine software.

Sphinx search engine with libstemmer

Libstemmer controls which characters are accepted as valid and which are not, and how the accepted characters should be transformed (eg. should the case be removed or not).
If the charset table is configured for special language chars, then search phrases “kottkvarn” and “k√∂ttkvarn” will return the same result.

As unfortunate as it sometimes is with enterprise OS (full irony intended), system packages of the Sphinx search engine (in EPEL or elsewhere) have a huge oversight of not shipping sphinx with the libstemmer support.

This contributes to my frustrations with much of the RHEL packages from the base repositories.
They are often built with the very minimum functionality and are there just to put a checkmark that they exist. No more, no less.

We need something usable, don’t we? As usual, we bring the packages you love to install and use:

sudo yum -y install https://extras.getpagespeed.com/release-latest.rpm
sudo yum install sphinx

Now there you have it, Sphinx search engine package that doesn’t suck.

It is built with support for Google RE2, libstemmer and everything you need for a bright day :-).

After Sphinx Engine is installed, you need to the Magento search module to it.

Connect with Sphinx Engine

  • in Magento admin, go to System -> Search Management -> Settings and proceed to Search Engine Configuration.
  • In field Search Engine select External Sphinx Engine option, and fill the following fields.
  • Sphinx Host127.0.0.1
  • Sphinx Port9312
  • Sphinx Bin Path/usr/bin/

Now, run full reindex on the command line:

php -f bin/magento mirasvit:search:reindex

Configure search morphology

To configure morphology and charset tables for additional languages, perform the following actions:

  • Open file /vendor/mirasvit/module-search-sphinx/src/SearchSphinx/etc/conf/index.conf.
  • Configure morphology variable. Standard language codes you can find here. Use two-letter codes from 639-1 column.

For example, to add German to the list of supported languages, you need to set in index.conf:

morphology = stem_enru, libstemmer_de

Read more about morphology here.

  • Add charsets to the charset_table variable. List of codes for different charset tables can be found here

For example, to add English and Russian characters, variable should look as shown below:

charset_table = 0..9, A..Z->a..z, _, a..z, \
                U+410..U+42F->U+430..U+44F, U+430..U+44F, U+401->U+451, U+451

Read more about character tables here.

It’s worth noting that you can install the Sphinx search engine on a separate server. But here we covered the most common way which is fine enough for the majority of Magento 2 installations.

With Sphinx search, you have a powerful, language-aware full-text search that is CPU and memory efficient and doesn’t eat your budget every month.

  1. Patrick Paterson

    How about Manticore Search? Sphinx is not open source any more and no recent changes for years while Manticore is being actively developed. We migrated off Sphinx to Manticore Search recently to use their replication functionality and besides that it’s not crashing any more and the performance is slightly better too.

    Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.