I’ll be the first to say it, I’m no programmer. Sure I can hack a few lines together and craft some adequate regular expressions to get ‘er done but I’m no formal coder. So it was with awesome joy that I found the Recon-ng framework. Tim Tomes (@LaNMaSteR53) created/is creating an easy python-based framework that even I can make useful modules for.
Learning Git
I’ve done my fair share of “git clone [insertprojecthere]” and “git pull” but never really got into the “pushes”, “branches” and other features of the Git system. Recon-ng uses the git system over at bitbucket.org. You need to get comfortable with pulls, pushes, adds, commits, forks, branches, and also create an account at bitbucket.org. Use ‘the Googles’ to get git resources and tutorials. Lots out there including http://www.lynda.com/Git-tutorials/Git-Essential-Training/100222-2.html and http://git-scm.com/doc.
The Hard Part…What Should Your Module “Do”
Inspiration is a fickle thing and figuring out what my module should do usually takes most of my time. I always think back to the Toy Story movie during this part of the process as, like the toys in Andy’s room, I want to make modules that will get played with and used. I’ll usually scan Twitter or something to find some web site that does something cool. I then look at the web calls I sent to the site and the response page to see if they are easy to send and parse, respectively. Recon-ng handles all the intricacies of requesting and retrieving the data from the remote website. The https://bitbucket.org/LaNMaSteR53/recon-ng/wiki/Home#!web-request-method page details this.
For this blog (and a NovaHackers presentation), I decided to make a module that would scrape a Google Analytics “UA” code from a given page and then use that response to search for other domains with the same value. Why? Well, let’s think like a company, government or web development firm that has a number of web sites/domains that they use Google Analytics on. They want to be able to use a single UA code to aggregate all the data into so that they just log into one single place to view all the sites they code/manage. So, what this module will do is to discover all the other sites that are managed/coded/maintained by the same group. Perhaps this gives you other targets for a penetration test or brings you some amusement.
Requests and Responses
Here is the nitty-gritty of the module…the core of what it does. I found several web sites that allowed me to input a UA code and perform the search I wanted. The ewhois site seemed like it was reputable and going to be around for a while (this is important since if they change the URL or page, this module will break). So, I went to several web sites and looked in the page source for UA codes. Below is a shot from http://www.xssed.com:
 |
| A web page and source from xssed.com. |
I’ve highlighted the line in the source with the UA code (UA-999804-2) in it. All we need to do is pull that out and insert into the ewhois page and submit it (http://www.ewhois.com/analytics-id/UA-999804/):
 |
| ewhois response to xssed.com’s UA code. |
In this figure, we can see that there are other domains that also use this UA code. Just as we thought! Now, looking at the source of the page (lower part of the picture), I’ve highlighted the unique portions of the page that we can make a regular expression (regex) for to gather these other domains. Sweet! That is all we need to make our module.
Writing the Module
I like to use a mixture of RTFM for recon-ng (https://bitbucket.org/LaNMaSteR53/recon-ng/wiki/Home), RTFM for python (http://docs.python.org/2.7/) and “what have others done?” (i.e., looking at the source code for other modules in recon-ng already) when coding. My source for this module is (as of right now) still a “pull request” here https://bitbucket.org/LaNMaSteR53/recon-ng/pull-request/59/adding-google-analytics-modules/diff if you want to skip to what it looks like in its submitted form. I like simple tools for simple projects. People will prolly yell that I should use an IDE or something else to write these modules in but really, gedit or your favorite editor will work just fine. I use gedit in kali linux but you can use your favorite tools.
 |
| Developing in Kali Linux using Gedit. |
I also like to walk before I run. So this module started out, when I began to write it, as “get and parse a user-supplied web page for a UA code”. Then I moved on to the submission of this code and the scraping of the domains from the response. Here is what the current version of the module looks like when executed:
 |
| The completed module with output. |
As you can see, just entering the “http://xssed.com” URL into the tool and pulling back its UA code allowed me to discover a bunch of other sites that are managed/coded by these same people. There are some interesting ones in there (mesotheliomasymptoms.info?). So, the module works. Now I have to go through my code and clean up debug items and stray white space. Once that is done, I “git push” my changes to my repository on bitbucket and do a “pull request” on bitbucket to send my new file over to be reviewed by the project owners.
This module, from start to finish, took about 1 hour to create and submit to the recon-ng project. The framework is really simple once you start using it.
Waiting…altering…merging
Now we wait for the project owner/maintainers to review my code, provide comments, and generally make the code that I made better and more in line with what they want recon-ng’s code to look like and work. The people that do this for recon-ng are very patient and helpful and have helped me tackle some coding challenges. Once your code is working and has been modified appropriately, they will merge your module into the main project.
Now you just have to think about what your next module will be and start working on it. Lather. Rinse. Repeat.
[Update]
- As I said above, Tim and the recon-ng people did provide some suggestions to make the code tighter and fit in better. I made the changes and the module as “pulled” into the main trunk of the tool’s source.
- This module was renamed as the “google_ids” module.
- Since writing it originally, I added the ability to not only use the Google Analytic “UA-” codes but also the Google AdSense “pub-” codes. That is why it was renamed.
- The Powerpoint presentation of this can be found at http://www.slideshare.net/webbreacher/making-modules.