unexceptable / wagtail-robots Goto Github PK

Django 3.2 has relevant changes that are incompatible with the current release of wagtail-robots. The change in AutoField from django 3.1 to 3.2 LTS creates the behavior of migration that generates an ever-missing migration, and makemigrations want to create this AutoField change, and it will not go away without a release that addresses the new AutoField in django 3.2.

wagtail-robots seems to have drifted away from django-robots enough that attempts to migrate to django-robots will generate a catastrophic database migration path, given that they seem to use the same table names.

Django-robots commit jazzband/django-robots@70da760 addresses and fixes this issue for them.

Because the underlying models changed quite a bit, I didn't move over the tests from django-robots.

As such, we need to write a bunch of tests to actually confirm this does as intended beyond manually checking each time.

Django-robots have fairly good translation support, we should try and emulate that, or potentially for the strings which are still the same, pull in some of the existing translations from the other project.

I have submitted a PR to update Wagtail 3 import paths, can this be reviewed please.

#15

Thanks

setup.py and setup.cfg by default aren't quite as nice as using PBR, and also allows us to use tags for versioning.

As such, we should switch to PBR.

https://docs.openstack.org/pbr/latest/

I'm using django-redis with serialization using MsgPack as described here. I'm also using Zlib Compression as described here. My settings are as follows:

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "KEY_PREFIX": "key_prefix_here",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            "SERIALIZER": "django_redis.serializers.msgpack.MSGPackSerializer",
            "COMPRESSOR": "django_redis.compressors.zlib.ZlibCompressor",
        },
    }
}

SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"

When I enable caching of robots.txt, an exception is raised when /robots.txt is accessed, as follows:

TypeError: can not serialize 'TemplateResponse' object
File "django/core/handlers/exception.py", line 34, in inner
  response = get_response(request)
File "django/core/handlers/base.py", line 145, in _get_response
  response = self.process_exception_by_middleware(e, request)
File "django/core/handlers/base.py", line 143, in _get_response
  response = response.render()
File "django/template/response.py", line 108, in render
  newretval = post_callback(retval)
File "django/utils/decorators.py", line 156, in callback
  return middleware.process_response(request, response)
File "django/middleware/cache.py", line 106, in process_response
  lambda r: self.cache.set(cache_key, r, timeout)
File "django/template/response.py", line 93, in add_post_render_callback
  callback(self)
File "django/middleware/cache.py", line 106, in <lambda>
  lambda r: self.cache.set(cache_key, r, timeout)
File "django_redis/cache.py", line 36, in _decorator
  return method(self, *args, **kwargs)
File "django_redis/cache.py", line 76, in set
  return self.client.set(*args, **kwargs)
File "django_redis/client/default.py", line 127, in set
  nvalue = self.encode(value)
File "django_redis/client/default.py", line 354, in encode
  value = self._serializer.dumps(value)
File "django_redis/serializers/msgpack.py", line 8, in dumps
  return msgpack.dumps(value)
File "__init__.py", line 35, in packb
  return Packer(**kwargs).pack(o)
File "msgpack/_packer.pyx", line 286, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 292, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 289, in msgpack._cmsgpack.Packer.pack
File "msgpack/_packer.pyx", line 283, in msgpack._cmsgpack.Packer._pack

When I disable caching, the problem goes away. Any suggestions on how I could still enable caching of robots.txt, but with serialization disabled just for wagtail-robots?

I encountered some issues when upgrading my software to
Django 4.0
Wagtail 2.16

The issues were within wagtail-robots (at the time of writing).

I modified the following to resolve the issue (in development environment):

1. models.py

line 5:
from django.utils.translation import ugettext_lazy as _

change to
from django.utils.translation import gettext_lazy as _

2. urls.py

line 1:
from django.conf.urls import url

change to:
from django.urls import re_path

line 6:
url(r'^$', rules_list, name='robots_rule_list'),

changed to:
re_path(r'^$', rules_list, name='robots_rule_list'),

wagtail sites do not need to have a site_name, but wagtail-robot breaks if there is a rule with a site assigned that does not have a name:
https://github.com/Adrian-Turjak/wagtail-robots/blob/26f12c919cc1f617122f340b426c18cf5f944b6f/robots/wagtail_hooks.py#L19

sequence item 0: expected str instance, NoneType found

Steps to reproduce:

1. Create a wagtail Site without a name
2. Create a rule assigning this site
3. Save