Warning!

Downloading unknown content from random web services on the Dark Web can lead to unwanted images and content being stored and displayed on your/your company’s computer.

Before using this or any automated technique, ensure that this is what you want and that you are prepared to take action in the event that something that has to be reported to the authorities is downloaded.

Intro

Yeah…you can say that I’m a fan-boy of Justin Seitz’s (@jms_dot_py) python and OSINT work. He has started publishing a list of Dark Web sites through his Hunchly Twitter account. The tweets are sent out daily and look like the one below.

Clicking on the top “buff.ly” short link will take you to an XLSX file on Justin’s Dropbox. I downloaded it to see what juicy info he scraped from the Dark Web today. I found about 44 systems that his scripts found new on the Tor network. A pic of the file is shown below:

The main column I want is that “A” column with all the *.onion systems. I want to evaluate what web services are running on all those systems but do so, in an efficient manner.

First, I need to copy the content from the “A” column in the XLSX and put it into a text file named hunchly_dark.txt.

On a linux system, I can create a while loop from a Bash shell/terminal window that will pull a line from a file and open it in a web browser. When I close the browser, the script will re-open the browser and surf to the next site from the list.

The Bash loop might look something like:

[code language=”bash”]

$ while read site; do firefox $site; done < hunchly_dark.txt

[/code]

While this can work, and is usable for maybe 1-4 sites, it is very inefficient to launch and close the web browser for every one of the 40 sites.

Automation

I’ve long been a fan of Chris Truncer’s (@christruncer) work too. Chris has a tool called EyeWitness (https://github.com/ChrisTruncer/EyeWitness) that you can feed a list of hosts and it will take screenshots of what is being hosted on those systems. It does way more than just web screenshots too, and I’ll leave that for you to investigate.

So I:

1. Moved the Hunchly XLSX file to a Kali Linux virtual machine (VM) I was using
2. Extracted the host data from column “A” and put it into the hunchly_dark.txt text file
3. Installed EyeWitness on that VM (see https://github.com/ChrisTruncer/EyeWitness for details)
4. Launched tor from the command line
5. Then I ran Eyewitness using the following command:

[code language=”bash”]

$ ./EyeWitness.py –headless -f hunchly_dark.txt -d output_dir1
[/code]

I watched EyeWitness rapidly (TOO rapidly) work through all the sites and braced for the failure I knew was going to appear in the output file. Sure enough, looking at the output HTML file that EyeWitness produces (excerpt below), I saw that none of my Tor sites resolved to their destinations.

From Failure to Success

I knew what happened. EyeWitness tried to resolve and visit those Tor hosts on the clear Internet not over Tor. I needed to find a method to send those requests across Tor.

Enter TorGhost.

TorGhost (http://www.khromozome.com/torghost-v2-0-with-instant-ip-switching/ and  https://github.com/susmithHCK/torghost) proxies normal system traffic over Tor.

I git clone’d it, ran the installer script and then started it up using:

[code language=”bash”]

$ sudo torghost start

[/code]

That started just fine.

Now I re-ran my EyeWitness command (as I had done before) and VOILA! It worked! Each row in the HTML table of the EyeWitness output had a screenshot of the web site at the Tor node. See below for excerpt.

Conclusion

Having cool friends that make it easy for you to do neat stuff is amazing!