The Ratrap (pronounced "rat trap") is a program designed to detect and ban malicious robots and leeches.
A Leech is a program designed to download all of a specific content, such as all picture or audio files, or a web site in it's entirety. There are times where individuals are interested in the sites content and they will employ a leeching program to download some or all the content off the website. The problem is, these programs often open multiple connections and put a heavy strain on the server bandwidth. In the case of photo galleries, they also download all thumbnail sizes as well, further waisting bandwidth.
Often, spammers will use automated programs ("robots" or "bots") to search web sites for e-mail address they can spam. Many bots are not malicious in nature, such as search engines. These bots can be told not to go certain places using the "robots.txt" file. (For more information on robots.txt, see http://www.robotstxt.org/) E- mail harvesting bots often ignore the robots.txt file, going where ever they feel like and waisting your bandwidth.
Other bots look for forms they can fill out with the hope the information put into the form will show up somewhere. Such bots are used for advertising. Again, these bots generally ignore the robots text file.
The Ratrap can help by posing as part of your website. It generates pages of random words, links and e-mail addresses. The links simply point back to the script, generating more random pages. When a bot stumbles into the Ratrap, it will end up request more and more pages of garbage. When a bot spends enough time in the Ratrap, the address can then be banned from making further requests to your website. You win on two counts: banning a bad bot and saving bandwidth for real people. And two: flooding a malicious robots with tons of bad e-mail addresses, making their harvest bogus.
If enabled, the Ratrap can ban offending IP addresses if they spend too much time wondering around in the trap. Ratrap can call an external program, passing the IP of the address to block. This make it quite easy to setup on a Linux systems employing iptables. The tracking is done using PHP Data Objects such as an SQLite database or MySQL database.
Released June 10, 2014
MD5 hash: 6d5846a72d1d49afef5f14ab12e8551b
SHA1 hash: ef1a826ca32f180a3ce65e324961205665dd4379
MD5 hash: feb2b2a16fab08e3779786375bc6bdd5
SHA1 hash: 828f2e71018672d79f5ba07933d86bbde45a9337
Installation is a little complicated, since the Ratrap works in conjunction with a few other programs. The main body of the script is in ratrap.php. This script will run without any of the following, but it will be unable to prevent offenders from being banned. The script has been designed for Linux systems running Apache web server. It is possible to use this script on other operating systems with other web server, but such procedures will not be disused here.
Start by unpacking the archive in your web site directory
Modify Apache's httpd.conf and add the following. This will make an alias that will direct everything to /ratrap/ to the Ratrap script. By doing this, the Ratrap script looks like a directory.
Modify (or create) robots.txt in the root directory of the web site. This rule will tell search engines (well behaved search engines anyway) not to recurse anything in the /ratrap/ directories.
Add an invisible link to one or more of your web pages to the Ratrap directory. There are several ways to hide a link and this is just one way. This link will act like bate to malicious bots. Note that we append index.html to the path. This could be any made up file name, because the alias redirects any request made to /ratrap/ to the Ratrap script.
If you have iptables enabled, you can have the Ratrap block offending IPs. By default, the Ratrap is setup to call iptables using sudo. You need to grant your web server premission to use iptables. NOTE: If you don't want the Ratrap to be able to call iptables directly, you can create an intermedit script, but you will have to edit RatrapDatabase.php to specify the script path.
To give your web server premission to run iptables, edit /etc/sudoers and make an addition similar to this. Be sure to change webpage_user to the user the web server runs as.
WARNING: If everything works, at this point, the web server will lock out any IP address that access the Ratrap script more then 5 times. That include you! If you lock yourself out, you are not only locked out of web site, but any access to the server.
The Ratrap can produce emptry images to detect image leeching programs. Any call to the Ratrap with a file extension of .jpg or .jpeg will generate a blank image. If IP banning is enabled, such requests act the same as any Ratrap request, therefore banning the IP address after too many requests have been made. To make use of this fetcher, simply add hidden links to the Ratrap scripts ending in .jpg.
You can view a demo of the Ratrap's output and administrative fetchers with the demo link. Note that actual banning has been disabled, so although the script may indicate an IP address has been banned, no actual action has been taken.
You can monitor the status of banned IP addresses and unban IP addresses by accessing the Ratrap script with the following parameters:
The default password is 'password', but is easily changed in RatrapDatabase.php. NOTE: There is no indication of error if the URL and password are wrong. Invalid requests are simply treated like any Ratrap request, which means too many attempts will result in banning.
The Ratrap is free, open-source software released under the GNU license.
Ratrap is written and maintained by Andrew Que. To get in touch with Andrew Que, visit his contact page
(C) Copyright 2006,2014 by Andrew Que.