Thursday, August 1, 2013

How to setup Sphinx to use multiple instances of searchd for different vhosts on one server (centOS)

If you've spent any sort of time working with start ups, chances are you've had to cram a few sites on to one VPS or Dedicated Server. This is always my preferred method over buying a dedicated box before I know if the boat's gonna float. The practice of running multiple vhosts on one box via one ip is what brought me up against this fun problem today and I'd like to share my solution with you...

The question... How do I get sphinx to store separate sets of indexes for each of my vhosts and serve them all in the right places?

A bit more about my architecture:

  1. RackSpace Cloud - 2GB Ram, 40GB HD, Dual Core 2000mHz CPU
  2. LAMPhp Stack on centOS 6.4
  3. MySQL Server on the same box
  4. Sphinx Server on the same box
  5. (n) vHosts - Solution easily scales to your n(th) vhost
SOLUTION:

Apache Configuration: Right now I'm going to assume you already know how to setup multiple vhosts in apache. If I get requests for help with this I'll happily write another post going into the topic.

MySQL Setup: I'm going to assume you already have mysql installed and have at least one db created that sphinx will be pointing to.

Sphinx Configuration: Here's where we need to get a little custom and make sure to keep our system very organized.
  1. The directory structure
    1. .conf files
      1. /etc/sphinx/myfirstsite.com/sphinx.conf
      2. /etc/sphinx/mysecondsite.com/sphinx.conf
      3. etc...
    2. data files -
      1. /var/lib/sphinx/myfirstsite.com/{data files will go in here}
      2. /var/lib/sphinx/mysecondsite.com/{data files will go in here}
      3. etc...
    3. binlog files -
      1. /var/data/sphinx/myfirstsite.com/{binlog files will go in here}
      2. /var/data/sphinx/mysecondsite.com/{binlog files will go in here}
      3. etc...
    4. log files -
      1. /var/log/sphinx/myfirstsite.com/{log files will go in here}
      2. /var/log/sphinx/mysecondsite.com/{log files will go in here}
      3. etc...
  2. The Conf File - I'm just going to go over the variables you need to change for a multi-searchd setup to work:
    1. index {} -
      1. Set path = data files path
    2. searchd{} -
      1. Listen (Must be different ports for each instance)
        1. listen = 9312, listen = 9306:mysql41
        2. listen = 9313, listen = 9307:mysql41
        3. etc...
      2. Set log = log files path/searchd.log
      3. Set query_log = log files path/query.log
      4. Set pid_file = log files path/searchd.pid
      5. Set binlog = binlog files path
  3. Indexing your data - Now that you have your .conf files for each instance you need to build your first set of indexes as follows
    1. indexer -c /etc/sphinx/myfirstsite.com/sphinx.conf --rotate --all
    2. searchd -c /etc/sphinx/myfirstsite.com/sphinx.conf
At this point if you've done everything correctly you should see something like this:
Sphinx 2.0.7-release (r3759)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinx/myfirstsite.com/sphinx.conf'...
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index 'idx_my_index'
precaching index 'idx_my_index_delta'                          
precached 2 indexes in 0.016 sec

Now that Sphinx is hopefully listening on your desired ports, you just need to point your desired api to the correct port and you're ready to go. I'm currently doing this with php, so I'll show you the php example of changing the port:


PHP
$s = new SphinxClient;
$s->SetServer("127.0.0.1", 9312);
//etc.. the rest of your search code below


If you have any comments, concerns, etc... please leave them below.
I hope you all enjoyed my first post on this blog. I sure enjoyed writing it.

Until next time :)