hbock / torbel Goto Github PK

About TorBEL
============

TorBEL is a server that attempts to answer one thing and one thing
only for its clients:

  Does this traffic originate from the Tor network?

TorBEL continuously maintains a database of currently running exit
nodes and their exit policies.  It allows clients to query for any
valid IP address and port combination and will return whether or not
this combination represents a recently active Tor exit node.

TorBEL also continuously connects to these exit nodes to ensure that
traffic is able to exit from the node and that traffic indeed exits
from the IP address advertised by the node.

Software Requirements
=====================

The following are required to run TorBEL on your system:

 - Tor 0.2.2.30-rc or higher recommended, with an open
   control port.
 - Python (>= 2.5 is required, >= 2.6 is recommended and
           required for JSON support)
 - Twisted (>= 8.1.0 tested, >= 10.0.0 recommended)
 - Recent Git version of TorCtl
 - flup (if using checkweb.py)
 - ipaddr-py (for query and dnsel)

Network Requirements
====================

TorBEL's controller tests a number of ports by connecting back to
itself over the Tor network.  It is recommended to run the TorBEL
controller on a DMZ interface where any and all traffic can reach
it, otherwise active testing will not work properly.

It is not recommended to use the Tor instance controlled by TorBEL for
other purposes, to maximize performance and reliability.  Using a
normal Tor client won't work at all while TorBEL is running, as it
sets __LeaveStreamsUnattached to manage its own SOCKS connections and
circuits.  If you wish to run a relay on the same machine as TorBEL,
it is imperative that you run two separate instances of Tor.

TorBEL sets the following Tor configuration options independent of what
is available in your torrc:
   - __LeaveStreamsUnattached 1
   - FetchUselessDescriptors 1
   - FetchDirInfoEarly 1
   - FetchDirInfoExtraEarly 1   (Tor >= 0.2.2.13-alpha)
   - LearnCircuitBuiltTimeout 0 (Tor >= 0.2.2.14-alpha)
   - WarnUnsafeSocks 0          (Tor >= 0.2.2.14-alpha)
   - FetchV2Networkstatus 1     (Tor >= 0.2.2.26-beta)
These options significantly affect Tor's operation and thus you should
not run other applications that depend on Tor's normal operation with
the same instance used by TorBEL.

The TorBEL controller must be able to bind to privileged ports (<
1024) if test_port_list contains privileged ports in config.py.  On
Linux, a process must be given the capability CAP_NET_BIND_SERVICE to
bind privileged ports.  This is difficult to set for Python scripts,
therefore it is recommended to run TorBEL as root.  TorBEL will drop
privileges to the user specified in config.py immediately after
binding to privileged ports.

Running TorBEL
==============

Resource Limit Considerations
-----------------------------

When running TorBEL as an active testing suite (why aren't you? :)),
TorBEL's performance is limited directly by the maximum number of file
descriptors it can open (queried via getrlimit(2)) and the maximum
number of circuits you configure it to use. You should set the hard
limit of the 'nofile' resource to a large number, perhaps related to
the maximum number of files open system-wide.  On Linux, this
information is available in the /proc filesystem:

 $ cat /proc/sys/fs/file-max

You can increase the hard limit of file descriptors by editing
/etc/security/limits.conf; you should do this for the root user if you
need to start TorBEL as root as your 'nofile' resource limits retain
their root-specific values even after calling set(e)gid/set(e)uid.  If
you run TorBEL as an unprivileged user entirely, you should set the
limits for that user.  An example /etc/security/limits.conf
configuration:

root             hard    nofile          65535

Please note that TorBEL doesn't care about your soft limits.  It will
always increase the 'nofile' soft limit to its hard limit, even if you
don't want it to.  But you really do want it to.

The default value of max_running_circuits is 200; if you have a fast
machine and a fast ISP, setting this even higher is recommended for
faster throughput.  500 is a good number for a fast connection.

System/Library Path Considerations
----------------------------------

TorBEL must be able to find the TorCtl and torbel modules.  It does
not (at the moment) require any other part of the torflow suite.  Make
sure your PYTHONPATH environment includes the path of the TorCtl and
torbel source trees.  It is recommended to put them in the same root
directory and set your PYTHONPATH accordingly:

PYTHONPATH=/path/to/torbel_and_torctl

TorBEL Configuration
--------------------

After cloning the TorBEL source tree, copy the example configuration
file to config.py:

  $ cp config.py.example config.py

And edit it.  At present, the TorBEL controller does not support
authentication methods other than HashedControlPassword.  This will
be fixed in a later revision.

The TorBEL controller can be executed directly from the source tree
as follows:

  $ ./torbel

If torbel is invoked via sudo, it may not inherit your PYTHONPATH
environment variable.  In this case, you should set it explicitly
at runtime:

  $ sudo PYTHONPATH=/path/to/tor/modules ./torbel

Logging
-------

If the output of TorBEL is too noisy, feel free to increase the
minimum logging severity in config.py.  The default is NOTICE, which
is appropriate for operating in a production environment.  Set to INFO
if you'd like to see more test results, and DEBUG if you'd like
something that's usable for development and testing.  VERBOSE[12]
should only be used if you have a lot of disk space and something is
really broken :)

The default behavior in config.py.example has TorBEL log to syslog
only (using the daemon facility) if it is running as a daemon
(daemonize = True) and to standard output only if it is not.

You can configure TorBEL to log to any combination of stdout, syslog,
and a (non-rotated) log file.  Logging to file is extremely useful for
debugging.  Be warned that if your log_level is set to DEBUG or lower,
your daemon.log (or wherever your syslog ends up) will get utterly
spammed.  You have been warned!