Comcast Domain Helper or DNS Highjacking

Comcasts new "Domain Helper" has been labeled as DNS Highjacking, and has the webmaster community in an uproar.

-----

From incrediBILL at WebmasterWorld

Comcast is currently targeting Firefox users in the SF Bay Area with DNS Hijacking, or "Domain Helper" as they call it, and showing pages of advertisments when inactive domains are accessed.

"The new product, which has been tested in trial markets since July 9, redirects nonexistent URLs like www.example.com/clinteckergoatbonedbyhisnewbicycle to a search page slathered in advertising instead of returning the proper DNS error to the browser. Readers began reporting the change to us yesterday."

Just happened to me today for the first time so I thought I'd report it since it has gone live. This whole mess scared me at first because I just upgraded to the latest FF 3.5, perfect timing with a new FF release, and thought maybe it was a new "feature" and I couldn't find any way to disable it. Tested on a couple of machines with both FF 3.0 and FF 3.5, same results, no change for MSIE 7.

So I go check example.com to see what happens and we got ads, which is amusing because example.com technically responds with the following:

"You have reached this web page by typing "example.com", "example.net", or "example.org" into your web browser. These domain names are reserved for use in documentation and are not available for registration. See RFC 2606, Section 3."

If you simply change your user agent to be MSIE 7 the "Domain Helper" behavior stops.

Just to see how much hijacking is going on, I tried CURL from my desktop command line to access a non-existent domain and got the proper error:

"curl: (6) Could not resolve host: example333.com; No data record of requested type"

So Comcast is definitely targeting just the smaller, yet substantial subset, of Firefox users for this test.

This will most likely interfere with any Firefox plug-ins that link check your bookmarks or anything of this nature.

Gee thanks Comcast.

-----

From jlivinggood (JL):
@incrediBILL - This is *not* just a FF thing, this would work on any browser and you can test this yourself if you have not already opted out. If you want to know what URL patterns would be affected --> see http://networkmanagement.comcast.net/DomainHelperLogic.htm. Basically, it must have "www." and the domain must be invalid. (If you have already opted out and wanted to test using nslookup, you can find the server IPs to use here: http://dns.comcast.net/dns-ip-addresses.html.)
Also, and importantly, I am not sure where the statement that we redirect "www.example.com" comes from. If this is the case, I would like to see the DNS query response, as redirect should not occur. Why not - because a valid A record exists. What you did provide was a link to the resulting search engine with "www.example.com" at the end of the URL string. But it is just a simple search engine and you could modify it with any search in that URL string -- such as the URL of this site (http://search2.comcast.com/?cat=dnsr&con=ds&url=www.webmasterworld.com). Just because you can perform a search on that site with that FQDN appended to the URL string does not mean the Domain Helper service would have performed the redirect and sent you there.

Regards
JL
Comcast

-----

From incrediBILL:

From JL: "This is *not* just a FF thing, this would work on any browser and you can test this yourself if you have not already opted out."

Per my original post I tested it on multiple computers and multiple browsers here and the only browser showing Comcast ads using my test criteria was Firefox.

From JL: "Basically, it must have "www." and the domain must be invalid."

That explains a LOT...

I never type "www." in front of anything and Firefox attempts to insert "www." in front of the domain name if it fails without the "www." and some of my other browsers don't do that by default.

Had I realized that it was that feature of Firefox I would've titled this thread differently!

From JL: "I am not sure where the statement that we redirect "www.example.com" comes from."

I entered it into Firefox without the www. and your ads popped up so you're intercepting more than you think with some browsers, I'm sure it's a Firefox quirk.

Looks like something changed on your end because I tested this on 2 computers yesterday and Firefox showed a Comcast ad page for "example.com" however I can't reproduce that today.

Good to know it's only triggered by typing "www." since I never do that so I'll never see those Comcast ads again as soon as I disable this behavior of auto-adding "www." in Firefox.

Most websites are running "www." free these days and to make sure their domain is canonicalized in the search engines actively redirect from the "www." version to the shorter non-www version, so Comcast is going to lose a ton of type in traffic if you only trigger based on the presence of "www.".

Thanks for clearing it all up and now I know that my domain typing behavior combined with Firefox's automated DNS resolving feature is why it appeared you targeted Firefox.

-----

From jlivinggood (JL):
@incrediBILL re: "Most websites are running "www." free these days and to make sure their domain is canonicalized in the search engines actively redirect from the "www." version to the shorter non-www version, so Comcast is going to lose a ton of type in traffic if you only trigger based on the presence of "www."."
In my personal opinion, I think folks recognize that we'll lose some traffic but with "www." we know with a high degree of certainty that it is http or https, and so not as likely to cause technical problems in edge cases. It is a more conservative and less lucrative approach but less controversial and less potentially problematic.

BTW, **very** interesting that FF added that www - that really does explain the difference you saw. It had us all scratching our heads here - mystery solved. :-)

JL

-----

What do you believe?