Let me ask you a question dear reader, have you ever visited a website that one of your friends posts a link to only to find that the site requires you to register for an account before you can see the content? Or perhaps you visited a web site on your computer and then also from a smart phone and noticed that the web site redirected you to the “mobile” version of the site for your phone. If so, then someone has been using your User-Agents to customize your web experience.

What is a User-Agent?

Since you are reading this, I’m guessing that you know that you can use a web browser to visit a web page. Not rocket science, I know. But let me pull the curtain away from that simple process of surfing the web and show you a little bit of what your browser sends.

The User-Agent is a “string” that a web browser (or other web-savvy tool) sends to a web site in your request for a page. WHOA Micah! What was that? Let me show you.

I’m going to start Mozilla’s Firefox web browser (http://www.mozilla.com/firefox) and surf to a web site that I control. I’m going to visit http:/example.com/micah.php (a fake website…don’t try to visit it). I type this into the URL bar and hit enter. On my web server, I see this:

Let’s break down what this tells a web site that I visit. To do this, I’m going to use the description at https://developer.mozilla.org/en-US/docs/Web/HTTP/Gecko_user_agent_string_reference.

• Mozilla/5.0 – This is something that all modern browsers send. Nothing neat here.
• Windows NT 6.3; WOW64; rv:32.0 – The “6.3” after the “Windows NT” tells sites I’m using a Windows 8.1 operating system. These versions are easily decoded in http://msdn.microsoft.com/en-us/library/windows/desktop/ms724832(v=vs.85).aspx
• Gecko/20100101 – The browser is based on the Gecko engine. Nothing neat here.
• Firefox/30.0 – I’m using version 30.0 of Firefox. Interesting huh? Now, as of this writing, Firefox is at version 33.0 so my browser is out of date and may be vulnerable to exploitation.

So, to recap, web sites get our operating system and the browser/version in the User-Agent string every time we request a web page (or request a piece of the web page like an image).

What is it used for?

Web site owners can use this information for a variety of purposes. One is to better understand who is visiting their site. For instance, when you visit this blog, Blogger.com collects and analyzes your User-Agents and then allows the blog owner, moi, to see them. It looks like this:

Kinda of interesting who is visiting my blog and what they are using to read/download it.

OK. Metrics is one thing web site owners use User-Agent strings for. They also use them to provide you a different look for the site depending on if you are using one browser or another. What happens if you visit the CNN main web site from an iPad? Well, check it out below:

You see that banner at the top? “Get the CNN App for iPad”…how’d they know I’m using an iPad? User-Agent string. Interesting, no?

Having fun

Want in on a little secret? YOU have control over what your browsers send as your User-Agent! That’s right! You can make your Windows 8.1 computer running Firefox look like an iPhone Safari web browser. And it is so easy to do too. In fact, I don’t really own an iPad. The CNN screenshot I took by spoofing my User-Agent string and visiting their site from the same Firefox 30.0 browser on my Windows 8.1 system. Let me show you.

Many of you know that our web browsers can have extensions, add-ons and plugins to add functionality to them. There are wonderful security add-ons like AdBlock and Ghostery (available for Firefox, Chrome and IE). We are going to use an add-on called User Agent Switcher in Firefox (http://chrispederick.com/work/user-agent-switcher/). In Firefox, visit https://addons.mozilla.org/en-US/firefox/addon/user-agent-switcher/ and the add-on will appear. Click the “Add to Firefox” button. Now, we need to get a whole bunch of well-organized User-Agents.

Visit http://techpatterns.com/downloads/firefox/useragentswitcher.xml in a web browser and you should see a weirdly-formatted document of XML. Right click on it and SAVE it to your computer. We will import that into the add-on. In Firefox, go to the add-ons and press the OPTIONS button next to the User Agent Switcher add-on like in the picture below:

Now we need to tell the add-on to overwrite the existing User-Agents and then to IMPORT that file you just saved.

Your list of User-Agents should now look like this:

Press the OK button and you are ready to start experimenting. The way to switch between User-Agents is by going into the Tools menu or in the add-on itself and selecting the browser you want to become.

Anything else?

Why yes. Yes there is.

You see, it is not just your web browsers that send these User-Agents. Anyone ever use Google/Yahoo/Bing/DuckDuckGo? Sure you have. How do they get all their results? They send out “bots” or web robots to visit all the pages on the Internet. How do they get in to all the places they do? Well, web sites have special rules that, if a User-Agent for the Googlebot is presented, then the browser can visit a certain set of pages. If the User-Agent is a regular browser, then it can visit other pages. Let me illustrate with a more concrete example.

Let’s say your friend has posted a link to a web page in Facebook and it looks interesting. Something like the one below perhaps:

Looks enticing no? Well, I thought so. I clicked the link and saw the following web page:

Aw man. I have to register to gain access to this cool article that I want to read. But wait. How did Facebook get that picture and the blurb about the site? Hmm. I wonder what would happen if I changed my browser’s User-Agent to the Googlebot one I spoke about above?

Then revisit the page.

Success!

Disclaimer

I am in no way saying that you should steal service from or otherwise circumvent authentication on any web site that you do not own or have explicit permission to do so on using these well known methods. I’m just demonstrating that it is possible and leave it to you to decide how to use this information.