[UPDATED – 10/2013 with blacksheepwall content]
The Scenario
In my line-o-work, we need to not only discover hosts on the network to assess but also the domain names of the websites hosted on those servers. See, there are these things called “Virtual Hosts” (http://httpd.apache.org/docs/2.2/vhosts/) where a server administrator, to save time/money/effort, can configure a single IP address to respond to multiple domain names. They then listen on that IP address for a request for one of their domains and that server will serve the pages. Cool! One IP with multiple domain names. For example: IP address 192.168.1.100 could resolve to www.example.com, mail.example.com and videos.example.com. When a browser requested www.example.com, the server would have the web application configured to listen for that request, serve up some content.
Now, let’s say that you are given a subnet of hosts (192.168.1.0/24 in CIDR notation or a class C in Ye Olde Fashioned lingo) to assess during a pentest. You discover that there is a host (192.168.1.100) that has TCP port 80 open and is running a web server on it. Awesome! You open up your web browser, type in http://192.168.1.100 and get a weird error about not getting any content or a default page was not set up and, well, no website! It may be that your system is using virtual hosts and you need to visit the site using its virtual host name and not the IP.
This post about some tools that you can use to find those host names within DNS servers and on the Internet. This class of tools is sometimes referred to as DNS Brute Forcers because they submit *.example.com where “*” is substituted by a word list. So if your word list had apple, admin, email, and test in it, these tools can send apple.example.com, admin.example.com, email.example.com and test.example.com as it iterates through your list of words. I’d be doing a disservice to these tools and their creators if I implied that that was all these scripts/applications do. I urge you to read up on each and decide which is right for you and your needs.
The Tools
- fierce2 – http://trac.assembla.com/fierce
- theharvester – https://code.google.com/p/theharvester/
- DNSRecon – https://github.com/darkoperator/dnsrecon
- blacksheepwall – https://github.com/tomsteele/blacksheepwall
- apple.com (large, international)
- dhs.gov (US Gov’t <– always of interest!)
- sans.org (a smaller domain)
Word Lists
Now, we need to use the same word list across all tools. Since the DNSRecon tool is in front of me now, we’ll use its https://github.com/darkoperator/dnsrecon/blob/master/namelist.txt file for all of the tools. With this, we should expect to see all the tools’ brute-force the same number of hosts. To be fair to the developers, we’ll also use each tool’s own word list to see if one list is “better” than another. Below is some of the output from the “analysis” of the default word lists. Fierce had the biggest list and theHarvester had the smallest.
root@kali:~/dns# ll
total 32
-rw-r–r– 1 root root 11957 Aug 5 22:21 dnsrecon_namelist.txt
-rw-r–r– 1 root root 13191 Aug 5 22:21 fierce_hosts.txt
-rw-r–r– 1 root root 3370 Aug 5 22:45 harvest-dns-names.txt
root@kali:~/dns# wc *
1907 1907 11957 dnsrecon_namelist.txt
2062 2062 13191 fierce_hosts.txt
563 563 3370 harvest-dns-names.txt
Each tool has other switches to control threads and output and so many other things. I’m not touching any of that.
DNSRecon
Setup
Carlos Perez’s (@Carlos_Perez) tool is actively developed and full of DNS-reaping features. git clone https://github.com/darkoperator/dnsrecon.git to grab the tool. I just got version 0.8.7 and chose to run it with:
# ./dnsrecon.py -D ~/dns/dnsrecon_namelist.txt -d domainname] -t brt -c outfile.csv
where -D namelist.txt is the default brute-forcing word list that ships with the tool. This is the file I’ll use on the other tools too. The -t brt tells the tool to use the word list and try to brute-force the DNS.
Results
| Domain | Records Found using DNSRecon Word list |
Records Found using Own List |
|---|---|---|
| apple.com |
142
|
– |
| dhs.gov |
48
|
– |
| sans.org |
22
|
– |
Comments
Couple of comments here. DNSRecon discovered DNS6 (IPv6) entries in the DNS system, which was really cool. DNSRecon completed very quickly. It also tagged each entry with the type of DNS record. One issue to watch for was that it marked a bunch of sans.org domains as valid but the IP they “resolved to” was my DNS server. A bunch of false positives. See the CSV snippet of output below:
Type,Name,Address,Target,Port,String
CNAME,tw.apple.com.pub.vitrue.com,,deejay.cloud.vitrue.com,
A,deejay.cloud.vitrue.com,50.16.57.100
A,deejay.cloud.vitrue.com,50.16.7.33
A,uk.apple.com,17.172.224.31
A,uk.apple.com,17.149.160.31
CNAME,webcast.apple.com,,salesguide.apple.com,
A,salesguide.apple.com,17.254.20.77
CNAME,www.apple.com,,www.isg-apple.com.akadns.net,
CNAME,www.isg-apple.com.akadns.net,,www.apple.com.edgekey.net,
fierce2
Setup
The first hurdle you need to jump for fierce is the installation. It was real simple on a Linux system using the instructions http://trac.assembla.com/fierce/wiki/Installing. I used version 2.0-r420 for this test. Last commits to the tool seemed to be from 12/20/2012.
Let’s go ahead and kick off a scan:
# ./fierce -dns [domainname] -prefix ~/dns/dnsrecon_namelist.txt -all
where the -dns [domainname] is the target and -prefix ~/namelist.txt is the list (in our case from DNSRecon) with the subdomains you’d like to brute-force The -all tells the tool to run all the tests. Fierce does come with its own word list (hosts.txt).
Results
| Domain | Records Found using DNSRecon Word list |
Records Found using Own List |
|---|---|---|
| apple.com |
358
|
358 |
| dhs.gov |
22
|
22 |
| sans.org |
50
|
50 |
Comments
I’d like to point out here that because this went so slow, I got to watch the domain names scroll by. Apple has some very interesting ones:
- Spices
- Found Node! (17.254.17.106 / salt.apple.com)
- Found Node! (17.254.17.107 / pepper.apple.com)
- Found Node! (17.254.17.108 / cumin.apple.com)
- Sports
- Found Node! (17.254.17.176 / rugby.apple.com)
- Found Node! (17.254.17.177 / rugger.apple.com)
- Found Node! (17.254.17.178 / soccer.apple.com)
- Found Node! (17.254.17.179 / kicker.apple.com)
theHarvester
Setup
Results
| Domain | Records Found using DNSRecon Word list |
Records Found using Own List |
|---|---|---|
| apple.com |
72
|
72 |
| dhs.gov |
93
|
88 |
| sans.org |
57
|
53 |
Comments
The tool ran very quickly and organized the results well. I liked how this tool ID’d the virtual hosts. Unfortunately apple.com results had several IPs that were shared by other customers of the web hosting provider and lots of false positives came out in the results. They were all flagged in the Virtual hosts section and were easy to remove. Other tools found the hosts but didn’t flag/separate them in the output. Default output snippet is below:
[+] Hosts found in search engines:
————————————
66.35.59.202:www.sans.org
66.35.59.249:isc.sans.org
66.35.59.8:dns21b.sans.org
[+] Virtual hosts:
==================
66.35.59.202 www.sans.org
66.35.59.202 www2.sans.org
66.35.59.249 isc.sans.edu
66.35.59.249 iscold.sans.edu
blacksheepwall [updated 10/2013]
Setup
Tom Steele’s (@_tomsteele) tool was brought to my attention recently and so I thought I’d add it to the evaluation. It uses node.js which will need to be installed on your system prior to use. Installation is written out in the README.md file in the git repository. Get the tool with:
git clone https://github.com/tomsteele/blacksheepwall.git.
I chose to run it with:
# ./cmd.js -d ~/dns/dnsrecon_namelist.txt -t [domainname]
NOTE – If your system uses OpenDNS or some other DNS service that redirects invalid DNS lookups to itself or a search page, you will need to use a different DNS sever such as Google’s at 8.8.8.8.
I chose to run it with:
# ./cmd.js -d ~/dns/dnsrecon_namelist.txt -t [domainname]
Results
Domain
Records Found
using DNSRecon Word list
Records Found
using Own List
apple.com
70
–
dhs.gov
22
–
sans.org
21
–
| Domain | Records Found using DNSRecon Word list |
Records Found using Own List |
|---|---|---|
| apple.com |
70
|
– |
| dhs.gov |
22
|
– |
| sans.org |
21
|
– |
Comments
root@kali:/pentest/blacksheepwall/bin# ./cmd.js -b -d words.txt -t sans.org –cleanbsw started at Mon Oct 07 2013 18:35:31 GMT-0400 (EDT)bsw finished at Mon Oct
07 2013 18:36:38 GMT-0400 (EDT)204.51.94.43:blogs.sans.org127.0.0.1:nt4.sans.orgbsd.sans.orglinux1.sans.orgsolaris.sans.orglocalhost.sans.orglinux2.sans.org10.10.10.10:oracle.sans.org204.51.94.201:portal.sans.org204.51.94.215:admin.sans.org204.51.94.41:search.sans.org204.51.94.173:files.sans.org66.35.59.202:www2.sans.org204.51.94.202:www3.sans.orgwww.sans.org66.35.59.44:mail.sans.orgmail2.sans.org66.35.59.50:spam.sans.org204.51.94.233:labs.sans.org208.255.174.6:defiant.sans.org204.51.94.40:lists.sans.org
Conclusions
Best Word List
Best Tool
Results
| Domain | Best DNSRecon Run | Best Fierce2 Run | Best theHarvester Run | Best blacksheepwall Run |
|---|---|---|---|---|
| apple.com |
142
|
358 | 72 | 70 |
| dhs.gov |
48
|
22 | 93 | 22 |
| sans.org |
22
|
50 | 57 | 21 |
So which tool will I use in the future? All of them. They all were solid and produced results. With the domain names you have, perhaps one tool finds more hosts than others. Try all three and make your own decision.