Note: My Web pages are best viewed with style sheets enabled.

Browser Sniffing:
Detecting It, Dealing With It, and Defeating It

Copyright © 2010 by David E. Ross

When you tried to view a Web page …

Did you get a message that you need the latest version of Flash — or perhaps some other browser-helper application — when you know you already have the latest version correctly installed?
Were you told to update your browser to the latest version, but there is no newer version than the one you are using?
Did you see the suggestion that you need a different brand of browser?
Did the page say:
This operation is not currently available.
Please contact your System Administrator and try again later.
We are sorry for this interruption in your service.
something incomprehensible such as
XML Parsing Error: mismatched tag. Expected: </input>.
Location: http://store.spruebrothers.com/recent-arrivals-c373.aspx
Line Number 60, Column 3:</form>
--^
or some other unexpected text?

All of these — and other anomalies — are symptoms of browser sniffing. Sniffing is a method by which a Web server determines what browser you are using. Problems with developing Web sites with sniffing are detailed under Sniffing in my "Professional" Web Developers. A well-designed Web page that complies with internationally recognized specifications published by the non-profit World Wide Web Consortium (W3C) and that addresses the recommendations of the Viewable With Any Browser Campaign should not require sniffing.

Unfortunately, too many Web developers create poorly designed pages that fail to comply with the specifications. Instead, Web developers often attempt to enhance Web pages by taking advantage of non-standard capabilities of various browsers. To accomplish this, a developer must create multiple versions of each Web page and then sniff for your browser in order to send you the correct version of the page. Frequently, the implementation of sniffing is incomplete — omitting some browsers — or otherwise incorrect. For example, if you are using a browser that the Web developer did not anticipate, the result can be unreadable garbage.

*** Begin Right Sidebar ***

On this page, some terms appear in bold where they are first used. The meaning of those terms should be clear from that first use.

The first use of other terms are links to pages where they are either defined or discussed in sufficient detail to make their meanings clear. Where those links at to my Internet Glossary, the page will appear either in a new window or a separate tab.

Some of this page's content is quite technical. If you don't understand it, contact me for help.

*** End Right Sidebar ***

Detecting Browser Sniffing

Dealing with Incorrect Browser Sniffing

Defeating Browser Sniffing

Detecting Browser Sniffing

Conclusively detecting whether a Web server is sniffing might be difficult.

The easiest step is if the Web page caused your browser to display a message that you need a newer version of Flash. Just go to Adobe's page for testing your Flash installation. If the box "Version Information" indicates you have the latest version, you are most likely a victim of sniffing. (A similar test can be done for Java.)
Otherwise, you will likely have to either use a different browser or use your browser to spoof another one.
- (For Gecko-based browsers, see Defeating Browser Sniffing for details on spoofing.) When a page fails to display properly with your browser and then does display properly when you spoof another browser, that is a conclusive proof of sniffing. I usually then try the Web page one more time, again without any spoofing; if this time the page fails to display properly, the proof of sniffing is even stronger. Note that many instances of sniffing include the setting of one or more cookies to indicate what kind of browser was detected by the Web server. Before spoofing a different browser, it is necessary to delete any cookies set by the affected Web site; if you are going to then try the site again without spoofing, you must again delete the site's cookies.
- If you do not have a spoofing capability, you must then try to view the affected Web page with another browser. It is better to try a different brand of browser than merely a different version of the same browser; that is, if you have a problem using Firefox 3.6.3, try Internet Explorer 7 and not Firefox 3.5.0. However, a Web page might indeed display properly with one browser and not with another without the involvement of any sniffing. Thus, resolving the problem by changing browsers does not conclusively prove that incorrect sniffing occurred. Instead, there might be a different kind of error in the Web page. You should still deal with this problem as described in Dealing with Incorrect Sniffing; merely change the emphasis of your communications to indicate the page does not display properly without mentioning sniffing.
If you are familiar with JavaScript (often used in sniffing operations) and HTML, you might examine the source of the affected Web page. There, you might see exactly how the Web developer tried to implement sniffing. The two most obvious errors in that implementation are (1) a failure to provide for unexpected browsers and (2) sniffing for the wrong user agent (UA) string.

Dealing with Incorrect Browser Sniffing

The most important thing to remember when you encounter incorrect sniffing is that your browser is not broken. Instead, the problem lies within the Web page. Thus, you must communicate the problem back to the Web site's developer and owner.

Test the affected page at W3C's HTML Validation Service. Note the number of errors (if any).
If you can get into the Web site at all, try to find an E-mail address or Web-mail page for feedback. Report the problem, indicating your UA string. If there were any HTML or XHTML errors, also report the number of errors. NOTE: Your UA string is
CCBot/1.0 (+http://www.commoncrawl.org/bot.html)
If the page is part of a commercial Web site, send a postal letter to the company's CEO. (The name and address of a company's CEO can usually be found on the Web.) Point out that restricting a commercial Web site to a limited set of browsers might be a violation of the Americans with Disabilities Act (ADA); you might even cite the case of the Target Corporation. For such a letter, avoid excessive technical details but do point out that the company's Web site cannot be viewed by your browser.
If you receive a response that you should be using Internet Explorer (IE), reply that approximately half of those who access the Web do so with browsers that are NOT IE. For a commercial Web site, ask if they are willing to ignore half their potential customer base.
If you are using a Gecko-based browser (e.g., Firefox, SeaMonkey, Camino, Fennec, Flock, Galeon, K-Meleon):
- When reporting the problem to the broken Web site, cite Gecko is Gecko and note that — if sniffing can be justified — the Web site should be sniffing for Gecko and not Firefox.
- If you receive a response that you should be using Firefox — not SeaMonkey or another non-Firefox, Gecko-based browser — reply that the Firefox developers assert that sniffing for Firefox is wrong. Include a link to Gecko is Gecko in your reply even if you already cited it in your original report.
- Check the list of sniffing bugs to see if a "Tech Evangelism" bug report has been filed.
  If no such bug report exists, file one at bugzilla.mozilla.com. Bug reports about invalid sniffing should be marked as blocking bug #334967, the general bug report that tracks sniffing bugs.
  If you don't have a bugzilla.mozilla.com account, report the problem in the appropriate news.mozilla.org newsgroup and request someone else to file the bug report on your behalf. Indicate what your UA string is and that you did indeed report the problem to the broken Web site.

Defeating Browser Sniffing

The following is oriented towards Gecko-based browsers. However, some other browsers might have features that could make this information somewhat useful.

Gecko-based browsers can spoof other browsers by sending fake UA strings. Unless a Web site requires specialized applications peculiar to only one browser (e.g., ActiveX used by IE), spoofing is usually successful in defeating incorrect sniffing. This capability is especially important for Gecko-based, non-Firefox browsers. Many sniffing Web sites that work correctly for Firefox fail to work for other Gecko-based browsers because those sites wrongly sniff for Firefox instead of Gecko.

Users can install a browser extension that can temporarily change your UA string for spoofing. Two such extensions are PrefBar and UserAgentSwitcher. Both of these are easy to use. Changing and inserting new UA strings in these extensions is not difficult. Both also have the feature that, if you forget to do so, they restore your true UA string when you terminate and then relaunch your browser. Sources of UA strings to add to the spoofing capabilities of PrefBar and UserAgentSwitcher include:

Note: In general, users should NOT spoof by changing their UA string via setting a browser's preference variables. This can result in a permanent spoof, which will hide the true identity of your browser. In the end, Web developers will then claim your browser never visits their Web sites and thus they don't have to accommodate your browser when sniffing.

However, some Web developers — especially those at financial services companies — absolutely refuse to accept the idea that Gecko is indeed Gecko. For my own online banking, I created a special profile where I did indeed set a preference variable for permanent spoofing of Firefox, which the banking Web sites recognize. I use that profile only for online banking. In that profile, I entered the following in file user.js:

user_pref("general.useragent.extra.spoof", "NOT Firefox/3.6.3");
// spoof Firefox

I choose to make all preference variable changes in the user.js file so that I can annotate them (the second line, beginning with two slashes). This results in a UA string that is

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.9) Gecko/20100317 SeaMonkey/2.0.4 NOT Firefox/3.6.3

at the time this was written. Note carefully what this UA string says: "This is really SeaMonkey and not Firefox." This successfully spoofs Firefox because sniffing almost always looks for merely a fragment of a string (Firefox in this case) and does not parse and evaluate the entire UA string (thus ignoring the NOT). This permanent spoof does not hide the true identity of my browser (SeaMonkey). In a few cases, I successfully argued with a Web developer that sniffing for Firefox was wrong by having him look at his server log, which showed that I actually used SeaMonkey. (Note well: Preference variables set via user.js do not become effective until your browser is terminated and relaunched.)

Some Web sites, however, do some parsing when sniffing. For example, Micro$oft's Get Silverlight looks for Firefox to be the first non-numeric string after Gecko. In this case, it is necessary to spoof with something like

Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.9) Gecko/20100317 Firefox/3.6.3 (NOT - it's really SeaMonkey! Stop browser sniffing) SeaMonkey/2.0.4

In some sniffing cases, it becomes necessary to omit any mention of a non-Firefox browser. Both PrefBar and UserAgentSwitcher have the capability of maintaining lists of spoofs — including user-created UA strings that do not mention the actual browser — so that the user can choose whichever spoof is appropriate for a given sniffing situation.

Updated 15 June 2010

Browser Sniffing: Detecting It, Dealing With It, and Defeating It

Copyright © 2010 by David E. Ross

Detecting Browser Sniffing

Dealing with Incorrect Browser Sniffing

Defeating Browser Sniffing

Browser Sniffing:
Detecting It, Dealing With It, and Defeating It