Crawlable AJAX Applications

Posted on: July 26, 2015 by Dimitar Ivanov

The AJAX revolution makes web to evolve. Nowadays faster loading of web apps is a must. The dynamic content is served in a non-blocking manner, thus provides great user experience. But how googlebot sees the content of hash-bang links? Is they automatically will be indexed? In order to avoid hurting the SEO of your web page, you should take care in proper way. Let the crawlers know about your ajax-driven web app. But how to accomplish it? Let's see how in details below.

Hash fragment

Hash fragment is the string after the hash mark #, since hash-bang fragment is a fragment starting with an exclamation mark !. It's part of client's specification so browser doesn't send it to the server.

<a href="!home"></a>
<a href="!about"></a>
<a href="!contact"></a>

Fragment meta tag

In case your web page has not the #! hash fragment, you need to inform the crawler to request the page appending the _escaped_fragment_ parameter. To do that you should simply add the fragment meta tag to the HEAD part of your HTML web page.

<meta name="fragment" content="!" />

URL Transformation

Since web servers can't read the fragment content, the crawler transform pretty URLs, those with #!, so the hashbang content is passed to the server as _escaped_fragment_ parameter appended to the query string. Keep note that search engines displays the pretty url's in the search results. Also links with _escaped_fragment_ found in HTML are not followed by the crawler.!home!about!contact

# The crawler will transform the above URLs as follow:

Handling the _escaped_fragment_

You should handle properly incoming requests to your web server. Keep note that you should return the same content to your visitors and to the crawler. Thus way doesn't hurt your SEO. Let's see an example:

// page.php
if (isset($_GET['_escaped_fragment_'])) {
    switch ($_GET['_escaped_fragment_']) {
        case 'about':
            // display about page
        case 'contact':
            // display contact page
        case 'home':
            // display home page

Crawling Website URL's

To crawl your ajax-driven URL's, a spider should be able to find them. There are two ways to do that, I'm recommending using both of them:

  • linking your pretty URLs, those with #! hash fragments, from already indexed web page(s)
  • adding your pretty links to sitemap

In modern browsers you can also consider using the HTML5 pushState method.

Make your website more secure by using the HTTP Headers for Wordpress, and never face a cross-origin issue again. Oh yes, it's FREE.
See also
Social sharing

If you have questions about crawlable ajax applications, drop me a comment below. Thanks for reading.


Comments are closed