Harmonising Link Prefetching And Logging

Relates to PHP and Apache, Firefox and Co

As part of my ongoing redevelopment of this website and in particular my weblog itself I have been building a set of classes to provide visitor statistics. The objective is a simple lightweight solution to summarise referrers, page counts, countries and other similar information per request and without the overhead of trawling the Combined Log Format. With PEAR_DB providing the database abstraction, storing the data is a simple case of compacting all the informative variables and running an auto-execution statement. Of course on the first test drive in Firefox, doubling up of log entries reared its head. This is due to the use of the <link rel="next" href="" title=""/> element providing accessible navigation to the next entry. All the Mozilla browsers (since the days of Netscape 7.x and Mozilla 1.2) will send an HTTP request for the URI in the href attribute to improve performance, thus appending an additional, yet misleading, log entry.

Fortunately there is a simple solution to this which can be resolved in the PHP routine. The browser sends a custom header to tell the server that this is a prefetch request:

X-Moz: prefetch

So in the PHP routine, prefetched pages can be excluded with the following conditional:


if (isset($_SERVER["HTTP_X_MOZ"]) &&
    $_SERVER["HTTP_X_MOZ"] == "prefetch") {
  // do not log the request
  return false;      
}
else {
  // log the request
}

This discovery, all thanks to Live HTTP Headers has been a great salvation to me, since the double logging had caused me many headaches in the past. Accurate logs are essential for good SEO and providing valuable and relevant future content to visitors. But with the current growth in Mozilla based browsers (Firefox in particular) I have found logs becoming more and more distorted. Neither removing relative links nor resorting to Internet Exploder are desirable solutions.

Information on Link Prefetching seems to be sparse but here are the main pages from Mozilla:

Unfortunately this solution does not resolve the general quibble I had with overstated logs and Webalizer. The last release of Webalizer actually preceded the introduction of the prefetch HTTP header. This is a hugely popular log analyzer that seems to be present across a wide range of web servers and hosting packages - makes me wonder how many people are optimistically interpreting over-stated logs? I suppose, with accessibility still on a gradual advance to the mainstream, and the use of the link element only appearing on progressive and forward thinking sites, probably not actually that many.

I have currently been test driving an excellent and very detailed alternative log file analyzer called AW Stats. Well worth a test drive.

Posted on Monday, Jul 12, 2004 at 02:33:14.

Comments on Harmonising Link Prefetching And Logging (1)

α comment

"makes me wonder how many people are optimistically interpreting over-stated logs" I doubt there are many sites out there that actually use the link elements.

Posted by Hemebond
Monday, Jul 12, 2004 at 23:34:55

Breadcrumbs Trail

[ Home ] -> TW Blog -> Jul 04 -> Harmonising Link Prefetching And Logging
Site Map

The Severn Solutions website achieves the following standards:

[ XHTML 1.0 ] [ CSS 2 ] [ WAI AA ] [ Bobby AA ]

Page compiled in 0.009 seconds