Bug #1886
closedTest reliability of firestats plugin
0%
Description
A user reports that firestats shows over 400,000 hits for his blog, while google analytics suggests that the number is more like 44000. Any way we can test the reliability of the firestats plugin? We should remove it if it's not providing accurate numbers.
Updated by Dominic Giglio over 12 years ago
Boone, I don't have any experience with Firestats, but do you think it could be something like unique versus returning visitors? Firestats might calculate and display those totals differently than GA?
Updated by Boone Gorges over 12 years ago
- Status changed from Assigned to Rejected
From what I can see, Firestats is working correctly.
Dom's theory is one possible explanation for the discrepancy. Google Analytics collects a ton of data, which can be parsed in any number of ways. They have a bunch of algorithms for combining what appear to be discrete pageviews and visitors. It could be that something like this is happening here.
It could also be that Google is being smarter about not counting non-human visitors. Search engines and other bots may crawl a site many, many times per day. Firestats should be ignoring these hits (and in most cases, it appears to be attempting to do so), but it's possible that it's being outsmarted by Google in this case.
Another thought is that GA's results are misleadingly small, because Google isn't being properly pinged on every pageload. For performance reasons, we put GA tracker code at the bottom of every page, and it typically doesn't phone home until the entire page is rendered. Thus, very short pageviews may not be consistently tracked by GA. Firestats, on the other hand, loads in the head of the document, and so will not miss this kind of pageview.
In any case, it's not practical to expect that these kinds of stat counters will be very accurate, given the variable nature of pageloads on the internet, and unclear definitions of what counts as a "hit". The best use for such tools is relative monitoring: you can't really compare GA to FS on a given day, but you can compare today's FS stats to yesterday's.
Unless there are objections, I'm rejecting this ticket, as everything seems to be working as expected, and no action need be taken.
Updated by Dominic Giglio over 12 years ago
I think the most logical conclusion is that the discrepancy is a combination of the possibilities Boone listed, and probably more.
It would be difficult to put Firestats' analytic engine up against Google's PHD designed algorithms.
There's no objection from me as to marking this rejected.
Updated by Matt Gold over 12 years ago
Thanks, all. A few thoughts:
1. Boone, when you say that the plugin appears to be working correctly, I'm assuming you mean not that you've verified its system but just that it's not throwing any obvious errors. Is that correct?
2. I am reminded here of the differences one sees when one looks at Awstats numbers on a server and then compares those numbers to GA. Here, I think Boone's points about what is being counted and when the script is being called make a lot of sense.
3. Regarding ticket status, rejected is fine with me.
Thanks very much for your work on this.
Updated by Boone Gorges over 12 years ago
Boone, when you say that the plugin appears to be working correctly, I'm assuming you mean not that you've verified its system but just that it's not throwing any obvious errors. Is that correct?
I mean I opened the plugin code to see how it works, and did some cross checks against the live environment to see if it works in the way it advertises itself, and it seems to match up.