Crazy fast AJAX search suggest in CakePHP

The fastest AJAX call is the one that is never made.

Network latency makes using AJAX applications like Google instant search, or even the common autocomplete field, feel like running in treacle*. Having the browser run to the server for more data every time the user pushes a key just doesn’t work well. Many times the data doesn’t even change that often, so one solution is to let the browser cache it. This is how you do it in CakePHP.

One way to implement the search suggest feature is with the Scriptaculous AJAX autocomplete field. While you type it sends frequent requests to the server. In a high-latency setting you may have to wait seconds to obtain the response. Worse still, sometimes the responses arrive out of order.

You get much better responsiveness if you preload the data and use Autocomplete.Local. It works a little differently from AJAX autocomplete in that the data has to be a Javascript array. If you create a controller action /search/suggestData that returns all data as JSON array, you can load the “local” autocomplete data with AJAX. So you could add this to your default layout:

<input id="search" autocomplete="off"/>
<div id="search_suggest" style="display:none;"></div>
<script type="text/javascript">
new Ajax.Request('<?php echo $html->url('/search/suggestData') ?>', {
    method: 'get', // Important! Only GET requests are cached.
    onSuccess: function(response) {
	var suggestData = response.responseJSON;
	new Autocompleter.Local('search_autocomplete', 'search_suggest', suggestData, { });
    }
});
</script>

But that’s not the full story. A search suggest feature should have thousands of suggestions to be useful. Adding all of that for every single page view amounts to significant increase in bandwidth use. Since the data doesn’t change that often, you can make the browser store it in its local cache.

Basically all you need is the Expires HTTP header and make sure you make GET requests. POST requests are not cached. The next time the call is made the browser will serve it from disk and it will be ultra fast. For example, to generate the JSON necessary for the search suggest feature you could use this controller action and view:

// app/controllers/search_controller.php
class SearchController extends AppController {
    function suggestData() {
        header('Expires: '.date('r', strtotime('+1 day')));
        header('Content-type: application/json; charset=utf-8');
        // The top 5000 search words as an array()
        $this->set('data', $this->Search->top5kSearches());
    }
}

// app/views/search/suggestData.ctp
<?php echo json_encode($data) ?>

If the data doesn’t change at all you can set the cache expiry date a lot further into the future. Here it’s set to 1 day.

How many search suggestions should you include, then? As many as possible, but bear in mind that Autocomplete.Local finds its matches by doing a linear search through the data array. With 5000 entries in the array you still get decent performance. It’s not hard to replace the search logic to use e.g. a binary search though.

I have observed this trick to shorten response times from several seconds to about zero for users that have a slow link to the web server. It makes a world of difference for autocomplete fields.

*) Oh, by the way. Apparently it’s easier to swim in treacle than in water. Go figure.

One Trackback

  1. [...] Ajax search suggest with cakephp [...]

2 Comments

  1. Hugo Cornejo
    Posted 2011/03/09 at 23:04 | Permalink

    Cool, maybe you can join this with HTML5 offline capabilities?

  2. Joni
    Posted 2011/03/10 at 00:04 | Permalink

    That’s a really good idea, I’ll look into it.

A penny for your thoughts

(Your email is never shared.)