jazzband / django-defender Goto Github PK

First: I am not using django admin at all so I am not importing django defender urls.

Any idea why I'm getting following error?

[07/Aug/2015 13:46:43] ERROR [django.request:256] Internal Server Error: /panel/domain/list/
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 164, in get_response
    response = response.render()
  File "/usr/local/lib/python2.7/dist-packages/django/template/response.py", line 158, in render
    self.content = self.rendered_content
  File "/usr/local/lib/python2.7/dist-packages/django/template/response.py", line 135, in rendered_content
    content = template.render(context, self._request)
  File "/usr/local/lib/python2.7/dist-packages/django/template/backends/django.py", line 74, in render
    return self.template.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 209, in render
    return self._render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 201, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 135, in render
    return compiled_parent._render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 201, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/base.py", line 903, in render
    bit = self.render_node(node, context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/debug.py", line 79, in render_node
    return node.render(context)
  File "/usr/local/lib/python2.7/dist-packages/django/template/defaulttags.py", line 507, in render
    six.reraise(*exc_info)
  File "/usr/local/lib/python2.7/dist-packages/django/template/defaulttags.py", line 493, in render
    url = reverse(view_name, args=args, kwargs=kwargs, current_app=current_app)
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 579, in reverse
    return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
  File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 496, in _reverse_with_prefix
    (lookup_view_s, args, kwargs, len(patterns), patterns))
NoReverseMatch: Reverse for 'defender.decorators.decorated_login' with arguments '()' and keyword arguments '{}' not found. 0 pattern(s) tried: []

On Python 3.4 (Django 1.7) admin view /admin/defender/blocks throws TypeError "'str' does not support the buffer interface".

This is because with Python3 redis .keys() returns a bytes list while utils.strip_keys() expects a string list. I have a simple fix on my fork https://github.com/nephridium/django-defender/tree/decode_redis_data. If you like I can do a PR?

Is there a reason the hiredis package is required in setup.py? It looks like the app will run fine without it.

Is there interest in a locked out signal that can be dispatched when a lockout happens for a user?

For example, we'd like to catch and log these with the rest of our audit logs and a signal is probably going to be the cleanest way to do so.

I'd be happy to implement this.

On django admin >> access attempt everyone gets 127.0.0.1 under ip address. Showldn't I see the user's IP? and also, how do I block an offender?

Thanks

Greetings,

The current PyPI package is version 0.4.2, which has a urls.py file that breaks with current versions of Django.

  File "/home/debian/.virtualenvs/test/lib/python3.4/site-packages/defender/urls.py", line 1, in <module>
    from django.conf.urls import patterns, url
ImportError: cannot import name 'patterns'

I see that this is already fixed in the master branch. Any chance of updating the package on PyPI?

How many of these features are actually available?

Log all login attempts to the database
support for reverse proxies with different headers for IP addresses
rate limit based on:
username
ip address
use redis for the blacklist
configuration
redis server
host
port
database
password
key_prefix
block length
number of incorrect attempts before block
95% code coverage
full documentation
Ability to store login attempts to the database
Management command to clean up login attempts database table
admin pages
list of blocked usernames and ip's
ability to unblock people
list of recent login attempts
Can be easly adapted to custom authentication method.

it would be nice to have a set of admin pages where you can manage the list of currently blocked users, and remove from the list, or manually add people to it. Also browse the recent logins.

To make it works add BasicAuthenticationDefender to DEFAULT_AUTHENTICATION_CLASSES in your settings.py.

BasicAuthenticationDefender should be above other auth methods in DEFAULT_AUTHENTICATION_CLASSES

https://www.mehmetince.net/bypassing-django-defender-ip-based-trackban/

right now we don't have any migrations, which is ok, but if we decide to change the database model it will make it a pain, we should add south migrations so that we can handle this issue in the future easily.

Needs to be compatible with Django 1.7 as well.

Hi there,

Having spent the last few days reading about how to provide a (somewhat) reliable REMOTE_ADDR environment variable to Django and associated modules in a reverse proxy setup (nginx + gunicorn on socket + django), I have the feeling that there is a potential security issue with defender's default reverse proxy setup.

Just switching the BEHIND_REVERSE_PROXY to True will result in the first IP in the X-Forwarded-For header (default header for reverse proxy setups in defender and many other projects) to be used here:
ip_address = request.META.get(config.REVERSE_PROXY_HEADER, '') ip_address = ip_address.split(",", 1)[0].strip()
This header is subject to IP spoofing, and the first IP is not guaranteed to be the user's.

It seems to be a widespread issue. django-axes attempts at mitigating this issue by an additional configuration option: the number of trusted proxies in the X-Forwarded-For, and using it to unstack the IP addresses in the header from right to left. Other projects like gorouter are considering a list of trusted proxies that can be used to unstack the IPs in X-Forwarded-For from right to left.

While acknowledging finding a generic solution is hard as there are a variety of proxy setups and evolving security practices around the Forward headers, I thought I'd just flag it as an issue for a module whose purpose is to prevent brute force attacks (and blocking by IP).

For this reason, I ended up turning off the reverse proxy setup of defender and coding an "unproxy" decorator applied to the WSGI application, based on fluent comments' recommendation for IP detection.

From what I read, the discussion on finding an appropriately generic solution to getting a user IP is ongoing. It might have to be on the Django middleware or WSGI side (rather than in individual libraries - comments, geo-ip, defender, axes, ...) and for libraries to trust the REMOTE_ADDR has been setup properly upstream.

Any thoughts on this issue?
Cheers!

We need to test how defender compares against axes and not using anything, and post the results.

The access attempts table could get very big if you have lots of login attempts. We should create a management command that will take a parameter that will delete any records older then X number of days. This will allow us to keep the table size down.

If you ever get tired of maintaining this or just want to join forces 😄 (Looks very nice!) https://github.com/django-pci

there are some unit tests now, but it would be good to add more, and improve the code coverage.

Currenly setup.py pins the django version to 1.6.8. We should relax that requirement so we only require the django major version 1.6.x.

We'd like to lock users after 3 consecutive failed log in attempts. We do not want to immediately block their IP address though (multiple users behind same public IP). However - we do want to block the IP address after a multitude of failed log in attempts from the same IP, but we want this threshold to be much higher to avoid false positives. Is this out of scope for this app?

I've tried django-defender with Django 1.8 and it works fine.
Please update pypi package dependencies so it could be installed with Django 1.8

I tray to install django-defender but I receive the following error

$ pip install django-defender
Collecting django-defender
Using cached django_defender-0.5.1-py2-none-any.whl
Collecting mockredispy<3.0,>=2.9.0.11 (from django-defender)
Exception:
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/req/req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/download.py", line 809, in unpack_url
unpack_file_url(link, location, download_dir, hashes=hashes)
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/download.py", line 715, in unpack_file_url
unpack_file(from_path, location, content_type, link)
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/utils/init.py", line 599, in unpack_file
flatten=not filename.endswith('.whl')
File "/home/ubuntu/.local/lib/python2.7/site-packages/pip/utils/init.py", line 484, in unzip_file
zip = zipfile.ZipFile(zipfp, allowZip64=True)
File "/usr/lib/python2.7/zipfile.py", line 770, in init
self._RealGetContents()
File "/usr/lib/python2.7/zipfile.py", line 811, in _RealGetContents
raise BadZipfile, "File is not a zip file"
BadZipfile: File is not a zip file

Hi, I'd like to use the package for protecting a login API url, the internals should be adjusted a bit (like checking against 401 instead of 302) but I think passing an argument to the decorator should make it possible. Also I'd really love to add the message on failed request to the db as a celery task.

@kencochrane thoughts? If you think it's ok I could work on making this happen

The docs still list django 1.7 as the max. Does this support newer versions? If so, can you update the readme to the latest tested. If not, what's the rub?

it would be nice to add this to one of the CI services that allow free unit test running for open source projects, so we can make sure all tests always pass.

Did you consider increasing request time instead of blocking user?

Increasing request time is a good way to protect from brutforce and doesn't block user account.

Django 1.11.4

I am using class-based views for my login.

From the Django docs - "The login function-based view should be replaced by the class-based": https://docs.djangoproject.com/en/1.11/topics/auth/default/#django.contrib.auth.views.LoginView

Since i can't find a documentation or figure out how to decorate a class-based view, how would i do it in urls.py ?

urls.py

from defender.decorators import watch_login
urlpatterns = [
...
url(r'^login/$', auth_views.LoginView.as_view(), name='login'),
]

Regards

Hello there. I have a little suggestion for the package. It is not so cool to re-declare your Redis url when you already have it on the settings. I have a quick workaround and would like your opinion.

First, add a new setting to the package: DEFENDER_REDIS_NAME. Is the name of the cache you will use, taken from the CACHES django setting.

Second, some code:

• defender/connection.py

from django.core.cache import caches

def get_redis_connection():
    ...
    elif config.DEFENDER_REDIS_NAME:
        return caches[config.DEFENDER_REDIS_NAME].get_master_client()
    ...

Tried it and worked fine. If you are not against it, I can create a PR really quick.

How to make the defender work on a custom login view?

Thanks!

In my case, I'd like to send an email to users when their account is locked. It would be nice to be able to register a signal listener for when django-defender issues a username/IP block.

Is there a reason to fix version requirements for redis, hiredis, mockredispy?
At least minor version changes should be allowed.

Currently this causes issues when using in apps that use newer version of dependencies (e.g. redis==2.10.5).

This is a design proposal for adding white and black lists to defender. The goal of adding these lists is so that we can always accept or block a set of usernames/ip's in a fast and flexible way.

Two ways to store the White/Black lists

In order to keep this flexible, we will have two ways to store white/black list, a static way using settings variables, and a dynamic way using Redis.

In order to make a change to one of the settings you will need to restart your application to take the changes, this isn't ideal, and should only be used for settings that you know won't change.

Adding values to Redis will allow us to change the settings while the application is running, with no reloads. It also allows you to get fancy with your blocking or accepting by programmatically adding or removing items from the Redis list when needed.

How it will work.

When a request comes in, it will check if the white/black list is enabled, if so, it will take the list from the Django settings and merge them with the list from Redis. The checks will be performed before we process the request, and if there are any white list matches, they will pass before we do any other checks. If there is a black list match then the request is blocked before they are processed.

There will be 6 new settings added:

DEFENDER_ENABLE_WHITELIST Boolean defaulted to False. If enabled it will look in the white list to see if the username or IP is in the white list before evaluating request. If it is in, it will bypass all checks.

DEFENDER_ENABLE_BLACKLIST Boolean defaulted to False. If enabled it will look in the black list to see if the username or IP is in the black list before evaluating request. If it is in, it will automatically block the request.

DEFENDER_IP_WHITELIST default empty list. The list of IP's that you want to skip the checks for.

DEFENDER_IP_BLACKLIST default empty list. The list of IP's that you want to always block.

• We should support CIDN: https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing so that it is easy to add a whole subnet at once.
• We should try and support IPv6 from day one, if it doesn't make this more complicated.

DEFENDER_USERNAME_WHITELIST default empty list. The list of username's that you want to skip the checks for.

DEFENDER_USERNAME_BLACKLIST default empty list. The list of username's that you want to always block.

There will be 4 new Redis keys

There will be 4 new keys to store white and black lists, they will not have an expiration on them.

• prefix:whitelist:ip : list
• prefix:whitelist:username : list
• prefix:blacklist:ip : list
• prefix:blacklist:username : list

Django admin changes

To make managing the white and black lists easier, we will add a new page to the Django admin where you can see the current white and black lists (settings and Redis) and the ability to edit (add and remove) the Redis lists.

This will give admins one place to see all of the settings, and make changes when needed.

Questions?

• Which check should be first, white or black?
• Do we want to log the whitelist or black list request to the database?
• Should we add support for regular expressions to match username's, or just straight username strings?

Calling utils.record_failed_attempt(some_ip, None) currently creates an entry with key defender:failed:username:None in Redis. The count is updated as expected for repeated calls.

The problem is that neither utils.reset_failed_attempts or utils.unblock_username can be used to clear the counts:

def unblock_username(username, pipe=None):
    """ unblock the given Username """
    do_commit = False
    if not pipe:
        pipe = REDIS_SERVER.pipeline()
        do_commit = True
    if username:
        pipe.delete(get_username_attempt_cache_key(username))
        pipe.delete(get_username_blocked_cache_key(username))
        if do_commit:
            pipe.execute()

The explicit check for username means that neither delete call gets made.

IMHO username=None should not result in a Redis entry being made. This means that the utils.record_failed_attempt function should be modified:

    user_block = False
    if username and not config.DISABLE_USERNAME_LOCKOUT:
        user_count = increment_key(get_username_attempt_cache_key(username))
        # if over the limit, add to block
        if user_count > config.FAILURE_LIMIT:
            block_username(username)
            user_block = True

Note the more restrictive if username and not config.DISABLE_USERNAME_LOCKOUT: check.
The same would apply for IP addresses, but it is less likely to present itself since IP addresses are typically available in the request.

I don't think we need ViewDecoratorMiddleware, by removing this, it keeps the code small, and makes it easier to maintain. if we are going to remove it, best to do it now.

@shin- @marcusmartins what do you think?

once stable upload to pypi for easy installing

I can see django-defender with pip search defender but installation fails saying no package meets the requirements as none are listed here.

I have Django-Defender installed and apparently working, but it does not register bad login attempts in the DB. Redis is also running. I think this may be caused by the employment of a custom user model. Is there any way of configuring Django-Defender so that I can tell the package which is my User model? Thanks

We should add a config option that will enable/disable the writing of the login attempt to the database, so that people can turn this feature off if they don't want it.

I've used django-defender in my application and it works well for standard login form, but it's not enough.
I have problem, because I'm also using django-rest-framework, so someone can still use rest api to to compromise passwords with brutforce attack.

I've tried to override rest_framework.authentication.BasicAuthentication class, but there's problem with some of defender.utils functions:

• def is_already_locked(request):
• def check_request(request, login_unsuccessful):
• def add_login_attempt_to_db(request, login_valid):

All these functions unloads username like this:
username = request.POST.get(config.USERNAME_FORM_FIELD, None)

Unfortunately, getting username from rest_framework's requests isn't that simple and requires more complicated code.
I had no problems with other request parameters such as user_agent or ip_address.

Of course I can copy-paste some lines from defender.utils, but I think it's not the best idea.

I suggest to add to every function which uses request to unload userid optional argument: get_username_from_request.

What do you think about that?

I'm trying to work on support for Django 1.11 making the necessary changes to work with the LoginView class-based view.

The following change passes all the tests. In middleware.py:

from django.contrib.auth import views as auth_views
from django.utils.decorators import method_decorator

from .decorators import watch_login

class FailedLoginMiddleware(object):
    def __init__(self, *args, **kwargs):
        super(FailedLoginMiddleware, self).__init__(*args, **kwargs)

        # watch the auth login
        # Django 1.11
        try:
            from django.contrib.auth.views import LoginView
            LoginView.dispatch = method_decorator(watch_login)(LoginView.post)
        except ImportError:  # Django < 1.11
            auth_views.login = watch_login(auth_views.login)

If you watch closely you see we are replacing LoginView.dispatch with a decorated version of LoginView.post, which is probably wrong. If I replace the dispatch method with the decorated version of the same LoginView.dispatch method I get a strange behavior: If I run all the tests, many of them fail because the user blocking triggers too early - it's like if one attempt was being logged as many attempts. However if I run any one of the failing tests in isolation, for example with:

PYTHONPATH=$PYTHONPATH:$PWD django-admin.py test defender.tests.AccessAttemptTest.test_cooling_off --settings=defender.test_settings

it passes! It is like if the cache were not being cleaned up correctly between tests - weird because the DefenderTestCaseMixin.tearDown method does exactly that.

I have no real clue of what could be going on. Does this ring any bell for you, @kencochrane ?

Thanks!

AccessAttemptAdmin() class references existing fields when viewing a AccessAttempt instance on the django admin interface.

django.core.exceptions.FieldErrordefender_accessattempt_change
Unknown field(s) (get_data, post_data) specified for AccessAttempt. Check fields/fieldsets/exclude attributes of class AccessAttemptAdmin.

https://github.com/kencochrane/django-defender/blob/master/defender/admin.py#L37

Right now we only support redis, but it would be nice to have pluggable backends where people can use something other then redis, assuming it supports all the features we need.

creating a new issue for this, breaking it out from #85 so it is easier to track.

It would be nice if we had an option to add the message on failed request to the db as a celery task. This would be helpful for people who are already using celery.

I don't want to make celery a requirement, so it should be something that is configurable. If they have celery enabled, then we will use celery, if not, we will do what we do today, and hit the database directly.

/cc @MattBlack85

middleware.py

                our_decorator = watch_login()
                watch_login_method = method_decorator(our_decorator)

Whats the point of this mess? Besides it throwing TypeError: watch_login() takes exactly 1 argument (0 given) can't it be just one line like it was before?

Like this: watch_login_method = method_decorator(watch_login)

Also for django-defender to work with the new django MIDDLEWARE I need to do the following to the middleware:

try:
    from django.utils.deprecation import MiddlewareMixin as MIDDLEWARE_BASE_CLASS
except ImportError:
    MIDDLEWARE_BASE_CLASS = object

Add this on top of the middleware.py and then swap the class FailedLoginMiddleware(object): to class FailedLoginMiddleware(MIDDLEWARE_BASE_CLASS):

Without doing these changes to the current middleware.py django runserver doesn't even "start". I am on django 1.11.2

Is the current 0.5 version actually running on someones django 1.11.x project? 😂

With 30M rows the access Attempts admin page doesn't load since the page takes too long, and times out. This is because there are filters that try and list all of the ip addresses and usernames, etc. Best to remove those, since they aren't very helpful when you are dealing with that many records.

In install_requires it's written Django <=1.11... so this package uninstalls higher then 1.11 versions and install 1.11.

Right now we have a readme, it would be nice to have better documentation and add it to readthedocs.org

Thanks for the great App!
where can I edit whitelist/blacklist?

thanks

I'm using a system where we use emails as usernames.

If an username like [email protected] gets blocked, then the rendering of /admin/defender/blocks/ fails with the following error:

Reverse for 'defender_unblock_username_view' with arguments '(u'[email protected]',)' and keyword arguments '{}' not found. 1 pattern(s) tried: [u'admin/defender/blocks/username/(?P[A-Za-z0-9-._@]+)/unblock$']

Obviously that's because the + is not in the group of allowed chars for the regexp. Can we make the regexp a bit more permissive? I understand a + sign is OK in an URL and as part of a username.

We are using Django 1.11.4, which should totally work, but the installation is failing due to a conflict:

error: Django 1.11.4 is installed but Django<=1.11,>=1.8 is required by set(['django-defender'])

Seems to be ok with master -- but we'd need something from pypi. Thanks!

Hi, i have a problem in production mode

DataError: invalid input syntax for type inet: "b''"
LINE 1: ...estamp", "path") VALUES ('[email protected]', 'b'''''::i...
                                                             ^

  File "django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
DataError: invalid input syntax for type inet: "b''"
LINE 1: ...estamp", "path") VALUES ('[email protected]', 'b'''''::i...
                                                             ^

  File "django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "admin_honeypot/views.py", line 27, in dispatch
    return super(AdminHoneypot, self).dispatch(request, *args, **kwargs)
  File "django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "django/views/generic/edit.py", line 185, in post
    return self.form_invalid(form)
  File "admin_honeypot/views.py", line 51, in form_invalid
    path=self.request.get_full_path(),
  File "django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 399, in create
    obj.save(force_insert=True, using=self.db)
  File "django/db/models/base.py", line 796, in save
    force_update=force_update, update_fields=update_fields)
  File "django/db/models/base.py", line 824, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "django/db/models/base.py", line 908, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "django/db/models/base.py", line 947, in _do_insert
    using=using, raw=raw)
  File "django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 1045, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "django/db/models/sql/compiler.py", line 1054, in execute_sql
    cursor.execute(sql, params)
  File "raven/contrib/django/client.py", line 114, in execute
    return real_execute(self, sql, params)
  File "django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)

python 2.7
Django 1.8.13
django-defender 0.4.2

I cannot cleanup username filed, because defender handle post data before form validation, so if user submit non ascii username - defender raise UnicodeEncodeError exception.

"utils.py", line 68, in get_username_blocked_cache_key
    return "{0}:blocked:username:{1}".format(config.CACHE_PREFIX, username)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)