Hello,
I was in charge of a pretty big search and photo project that is currently running. I'm running into some issues with the search results pages and was hoping for some suggestions. We're running Drupal 6 with apachesolr.
We have multiple servers; lb1 (squid), www1 (apache pre-fork/memcache/solr), app01 (memcache/ftp) and db1 (mysql).
The main issue is that when a user goes from page to page, things run smoothly. But when a user does a search, it generally sits for several seconds (upto 5 seconds), then renders the page. Often times it renders the entire page at once. Our search results are 60 thumbnail images, between the size of 30k and 150k. (I know 150k is really big).
The weird part about this whole thing is more about the time it takes to start loading the next page, does any one have a suggestion?
I've made the suggestion of an ajax search results.
I've also been considering dropping drupal for the search results all together and just loading the search results through another system.
A little side note, i install "SolrExplorer" which is a java front-end provided at the Solr site.
If I do a search through that system for the word "friend", my results are Found 1331 items (took 0.02 sec.)
It doesn't show the images, but the query is lighting fast.
If I do a search on the site it takes 3.5 seconds to get to the next page, then to download the images, it a total of 19seconds for the whole page.
Does anyone have suggestions? We do have cacherouter and boost modules. cacherouter runs memcache across 2 servers.
Comments
Did you check your MySQL
Did you check your MySQL server and see the process list during a search? It might just be your database is slowing the entire process down. If so, just get another db server and perform a slave replication and have the search run off that.
Try Acquia Search for free for 30 days
Hi, it's possible you've got resource contention between your Web Server and your Solr/Java servers.
You could try Acquia Search, just add the connector modules. Docs are here: http://acquia.com/documentation/acquia-search/activation . It's free for 30 days.
If the search results improve then, you've got to provide more resources to run both servers on the same box. Service isolation is a common technique for both debugging and increasing scalability. It's what we do on Drupal.org.
Cheers,
Kieran
If it takes several seconds,
If it takes several seconds, but then the page renders all at once it sounds like you may also have front-end performance issues. Take a look at YSlow for tips here:
http://developer.yahoo.com/yslow/
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Profile
The most important thing you need to do is profile your search pages. If you don't know what causes the delay there is no (efficient) way of fixing it. Indeed try YSlow, PageSpeed or something similar. The profile functionality of XDebug combined with WebGrind (or KCacheGrind) is also a great way to figure out what exactly is going on.
A very crude/poor mans way to test this would be inserting "exit('forced exit');" in your Drupal code at a strategic place. I would suggest putting it right before the search gets executed, then right after executing the search, just before generating the results and right after. When "exit();" is placed right before the search your page should load almost instantly, move the exit around until you find the part that adds seconds of delay. Depending on your PHP skills you can of course also add some timers instead around the suspected code.
I appreciate all your
I appreciate all your suggestions. There we're a lot of changes done on the site to increase the speed. We went from 8-9 second search results speed (sometimes as high as 18-20seconds) to less then 3 seconds.
I'm not sure what was key to the speed change, but here are some of the things I did:
1lb, 1 apache + memcache, 1 memcache + app (ftp/photo processing), 1 db server, 1 solr server
But its incredible how much faster things have become. Thanks for your help.
Memcache stats:
*yeah giving to much memory
APC stats:
*going to increase