gregmuellegger / django-autofixture Goto Github PK

Ability to add default photos from custom folder path

It would be great if it could auto integrate with faker lib so that test data is realistic w/ minimum effort.

My model:

class Product(models.Model):
name = models.CharField(max_length=100)
description = models.TextField()
cost = models.FloatField(blank=True, null=True)
price = models.FloatField()

I try with:

(venv)vagrant@lucid32:~/project$ ./manage.py loadtestdata app.Product:3
Traceback (most recent call last):
File "./manage.py", line 14, in
execute_manager(settings)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/init.py", line 438, in execute_manager
utility.execute()
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/init.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(_args, *_options.dict)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(_args, *_options)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/transaction.py", line 217, in inner
res = func(_args, *_kwargs)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/management/commands/loadtestdata.py", line 208, in handle
autofixture.create(model, count, *_kwargs)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/init.py", line 92, in create
return autofixture.create(count)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/base.py", line 478, in create
instance = self.create_one(commit=commit)
File "/usr/local/venv/local/lib/python2.7/site-packages/autofixture/base.py", line 456, in create_one
instance.save()
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/base.py", line 553, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, *_kwargs)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1436, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/usr/local/venv/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
django.db.utils.IntegrityError: null value in column "price" violates not-null constraint

Any tip?

Hello, great package.

In my opinion it lacks ability to easily set max length instead of getting it from model.

geodjango allows you to define point fields it will be great to be able to generate data with them with auto fixtures.
thank you for your this App

Add the ability to provide a Python regular expression string to the StringGenerator constructor to only allow strings that match the regular expression to be inserted into the database.

For example, in autofixtures.py:

class EntryAutoFixture(AutoFixture):
    field_values = {
        'blog': generators.StringGenerator(
            chars=string.ascii_letters + string.digits + '-',
            match='^[a-zA-Z0-9][a-zA-Z0-9\-]*$',
            min_length=1,
            max_length=40
        ),
    }

This would, in effect, limit strings generated to only containing alphanumeric characters and dashes and the string cannot begin with a dash.

This would be useful where there are regex validators present in Django url configurations and on Django models.

I can provide a Pull Request if this proposal is acceptable.

Thanks for creating this, it saved me tons of time as you pointed out but (being new) it did take me a while to figure out how to use it. I was so happy to find something to autopopulate for me, I can't tell you how much i wanted to hug you.

I've forked and cleaned up the readme just a bit and added a bit to the examples to make them a little more clear to new devs. I'd like to submit a pull!

Thanks again!

from django.contrib.auth.models import UNUSABLE_PASSWORD

no longer exists in Django 1.6, causing django-autofixture to crash.

There appears to be a new alternative

from django.contrib.auth.hashers import UNUSABLE_PASSWORD_PREFIX

There is also set_unusable_password() function on the auth model as well which is probably a better substitute.

Mat

Hello,

I am would like ask why you use custom ppa to provide python 2.6.,2,7.33 and pypy.

All of them are supported by Travis http://docs.travis-ci.com/user/languages/python/ now.

If you drop ppa you can drop sudo and move to container-infrastructure which work faster ( http://docs.travis-ci.com/user/migrating-from-legacy/ ).

Greetings,

CreateInstanceError: Cannot solve constraints for "supporters.Person", tried 1000 times. Please check value generators or model constraints. At least the following fields are involved: user

the model:

class Person(models.Model):
    user = models.ForeignKey(User, unique=True)

or

class Person(models.Model):
    user = models.OneToOneField(User)

Seems like this should be easy to handle...not sure why it is having this failure.

As test for #73 shows django-autofixture isn't django 1.9 compatible.

Since Django 1.7 we can optionally add AppConfig subclass path to INSTALLED_APPS, like:

INSTALLED_APPS = [
    'my_core.apps.CoreAppConfig',
    # ...
]

Docs on that:
https://docs.djangoproject.com/en/1.8/ref/applications/

def get_generator(self, field):
    if field.name == 'content_type':
            return generators.ChoicesGenerator(
                   values=[ContentType.objects.get_for_model(m)
                           for m in [ModelOne, ModelTwo, ModelThree]])
    elif field.name == 'object_id':
        # where to pick right model class? ¯\(°_o)/¯

I think, for that case, instances of GeneridForeignKey are also could be passed to get_generator(), and being skipped in base implementation, so user will can handle them in subclass method.
Or another hook like prost_process_instance could be defined, which must be called before instance being saved first time.

I'm trying to create model instance like this:

AutoFixture(User, field_values={'date_joined': joined})

Here I'm explicitly telling AutoFixture class, that I want custom value for
date_joined field. But it still uses default value.

This can be fixed using overwrite_defaults=True:

AutoFixture(User, field_values={'date_joined': joined},
            overwrite_defaults=True)

But then, all fields loose their default values.

I think, that if field_values are specified, then it must be used
overwriting defaults.

(Pdb) from autofixture import AutoFixture
(Pdb) fixture = AutoFixture(Post, generate_fk=True)
(Pdb) entries = fixture.create(10)
*** RuntimeError: maximum recursion depth exceeded
(Pdb) Post.objects.all()
[]

Post model

Run on project Opps CMS

I got AttributeError: 'NoneType' object has no attribute '_meta' exception in process_m2m.

through = field.rel.through #returns None!
auto_created_through_model = through._meta.auto_created # 'NoneType' object has no attribute '_meta'

So I changed it for :

through = field.rel.through 
auto_created_through_model = through and through._meta.auto_created 

Its executing (right now ;-) ), but does it eligible solution?

This will override the default value for AutoFixture, esp. noticable when default is set on a subclass.

The Link class in base.py is an arcane relict from four years ago and hard to understand.

We should try to remove the class in incorporate the logic into the Autofixture class.

Hello, i noticed that autofixtures are missing UUIDField support, so the any fixture of the model with UUID field raises IntegrityError in case when that uuid field on the model is required.

For now i'am monkey patching the base class and everything works fine:

import uuid

from django.db.models import fields

from autofixture import AutoFixture, generators
from autofixture.base import AutoFixtureBase


class UUIDGenerator(generators.Generator):

    def generate(self):
        return uuid.uuid4()



def monkey_patch_autofixture():
    AutoFixtureBase.field_to_generator[fields.UUIDField] = UUIDGenerator

I have defined my field_values as below:

field_values = {
'name': namelist[randint(0,len(namelist)-1)][0],
}

with namelist is defined as a list of username.

But when i launch the manage.py with python manage.py loadtestdata webapp.user:50, i have 50 users with the same name...

For starters, amazing utility and just what I was looking for. Thank you for writing and maintaining this project.

If a model has a DateTimeField() and settings.py contains USE_TZ=True then I am receiving the following error:

/home/me/.pythonbrew/venvs/Python-2.7.2/myproject/lib/python2.7/site-packages/
django/db/models/fields/__init__.py:827: RuntimeWarning: DateTimeField
received a naive datetime (2010-12-22 18:30:51.443334) while time zone
support is active. RuntimeWarning)

ValueError: astimezone() cannot be applied to a naive datetime

I've got a workaround by doing the following in generators.py:

class DateTimeGenerator(Generator):
    from django.utils.timezone import now

    min_date = now() - datetime.timedelta(365 * 5)
    max_date = now() + datetime.timedelta(365 * 1)

The Django version of now() [1] returns a naive or aware datetime object according to the value of USE_TZ.

[1] https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#naive-and-aware-datetime-objects

Hi,
I'm very new to Django and to this plugin too, so please be patient.

I wanted to report this error when using the package with Django 2. I have a minimal working example here. The error can be reproduced following these steps

git clone https://github.com/davethecipo/autofixture-django2-bug.git
cd autofixture-django2-bug
python3 -m venv venv35
## activate virtual environment, then run
cd project
pip install -r requirements.txt
./run.sh

I may be watching at it wrong, but I have a problem even using generate_fx=True, because I think it will only create fk of the model being fixtured. In my example I have Equipment, with Device, With Sector, with Facility, all foreign keys, being device and sector m2m. and I would like to create an Equipment, without manually creating all the rest of the setup.
As discussed here, #58 , sometimes it would be great to recursively generate foreign key relations. maybe there could be a keyword that would specify the recursive limit or something

This is a great package. Thank you, @gregmuellegger, for creating this!

It would be nice to generate capitalized or titlecased titles. Maybe for fields called "title", or maybe some other rule.

No pressure, just something on my wish list. If you think this would be a good idea, I can take a stab at implementing it.

I typed the command as below:
$ python manage.py loadtestdata entry.Entry:3

but it's now working. I got this:
Unknown command: 'loadtestdata'
Type 'manage.py help' for usage.

Maybe something wrong?

Is there any plan to add support to Django 2.0?

I have the following model at my app

BLOG_POST_STATUS = (
    ('U', 'Waiting for triage'),  # This is the status after we receive the blog post draft.
    ('R', 'Waiting to be reviewed'),  # Blog post is assigned to one staff to be reviewed.
    ('G', 'Waiting to be proofread'),  # Blog post is assigned to be proofread by the community officer.
    ('L', 'Waiting to be published'),  # Blog post will be publish by the community officer.
    ('P', 'Published'),  # Blog post is published and have a URL at the website.
    ('D', 'Declined'),  # Blog post submitted by mistake.
    ('O', 'Out of date'),  # Blog post that wait too long to be publish for any reason.
    ('X', 'Remove'),  # When the fellow decided to remove their request.
)

class Blog(models.Model):
    status = models.CharField(
        choices=BLOG_POST_STATUS,
        max_length=1,
        default="U"
    )
    title = models.CharField(
        max_length=MAX_CHAR_LENGTH,
        null=True,
        blank=True
    )
    published_url = models.CharField(
        max_length=MAX_CHAR_LENGTH,
        null=True,
        blank=True
    )
    tweet_url = models.CharField(
        max_length=MAX_CHAR_LENGTH,
        null=True,
        blank=True
    )

    added = models.DateTimeField(auto_now_add=True)
    updated = models.DateTimeField(auto_now=True)

    def save(self, *args, **kwargs):  # pylint: disable=arguments-differ
        if self.published_url:
            self.status = 'P'
        super(Blog, self).save(*args, **kwargs)

After I run

$ python --version
Python 3.6.0 :: Continuum Analytics, Inc.
$ python manage.py shell
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 12:22:00) 
Type 'copyright', 'credits' or 'license' for more information
>>> import django
>>> django.__version__
'1.11.3'
>>> import autofixture
>>> autofixture.__version__
'0.12.1'
>>> exit()
$ python manage.py loadtestdata myapp.Blog:120

I ended up with 120 Blog entries but all has a not null published_url and status equals to P. Would be great to have a more random generator.

So i have a model called Usage, with a many to many to another model called Reservation and this reservation has a foreign key to another named Offer

How can i create test data with the fk and m2m automatically generating the new objects?

this is what i have so far:

from autofixture import AutoFixture
from .models import Usage


def run(verbose=True):
    fixture = AutoFixture(
        Usage, generate_fk=['card', 'reservations__offer'],
        generate_m2m={'reservations': (1, 1)}
    )
    user_card_usages = fixture.create(2)
    print ("Finished loading fixtures: ", user_card_usages)

The reservations__offer is not being generated (i am not sure about the syntax)

I used this in tests, there's almost 50 unit tests now. But when running test with nosetests command, sometimes I got TransactionManagementError. Full traceback is listed below:

======================================================================
ERROR: test suite for <class 'trades.tests.test_pingpp.TestPingpp'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/suite.py", line 209, in run
    self.setUp()
  File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/suite.py", line 292, in setUp
    self.setupContext(ancestor)
  File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/suite.py", line 315, in setupContext
    try_run(context, names)
  File "/usr/local/Cellar/numpy/1.11.0/libexec/nose/lib/python2.7/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/Users/windrunner/bxzz/phantasm/phantasm/tests/base.py", line 43, in setUpClass
    AutoFixture(model).create(10)
  File "/usr/local/lib/python2.7/site-packages/autofixture/base.py", line 550, in create
    instance = self.create_one(commit=commit, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/autofixture/base.py", line 498, in create_one
    process = self.check_constraints(instance)
  File "/usr/local/lib/python2.7/site-packages/autofixture/base.py", line 454, in check_constraints
    constraint(self.model, instance)
  File "/usr/local/lib/python2.7/site-packages/autofixture/constraints.py", line 37, in unique_constraint
    if model._default_manager.filter(**check).exists():
  File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 586, in exists
    return self.query.has_results(using=self.db)
  File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 484, in has_results
    return compiler.has_results()
  File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 811, in has_results
    return bool(self.execute_sql(SINGLE))
  File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 59, in execute
    self.db.validate_no_broken_transaction()
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 327, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

1. Installed django-autofixture using pip install
2. Added "autofixture" to INSTALLED_APPS. I know its installed because it does NOT complain when running python manage.py runserver.
3. In the virtualenv, running django-admin loadtestdata .... results in 'Unknown Command'

Am I missing any steps?

Support I want to generate a car lists with random brand. But brands are from my given list.
Can I do that in your autofixture?

Current implementation of post_process_instance implies that user, that would override it must save manually modified instance inside method of child class.
But actually Im not sure - is it really intended by author to be as that. Old documentation documentation says that this method should return "modified instance" in "last possible moment" which might mean moment before saving.
post_process_instance in UserFixture illustrates what Im talking about, take a look, with this implementation any modifications done by post_process_instance will be lost during creation, including password, because instance not being saved.

Unfortunately, there is no way to input a minimum and maximum value for a FloatGenerator.

My development environment depends on Django 1.10. Builtin tests does not cover anything for 1.10. Is it working on 1.10 or is there any plan to migrate to higher versions of Django?

When generating an object and some of its foreign keys at once by calling e.g. AutoFixture(Book, generate_fk=['author']), any custom autofixture registered against Author will be ignored, and instead a default generated autofixture will be used.

This behaviour surprised me, is it a deliberate design decision or a bug?

I wanted to create a user with predefined password for a test so i did this:

    import autofixture
    class Test(TestCase):
        def test(self):
            user = autofixture.create_one(User, username='test', password='test')

This raises an exception : TypeError: init() got an unexpected keyword argument 'username'

To fix this i had to import autofixture.autofixures like this:

    import autofixture
    import autofixture.autofixtures
    class Test(TestCase):
        def test(self):
            user = autofixture.create_one(User, username='test', password='test')

which worked

Hi,

I installed django-autofixture on Ubuntu 12.10 32-bit with Python 2.7.3 and django 1.4.3. After installation when I type the following command I get the subsequent errors:

django-admin.py loadtestdata

Error:
unknown command: 'loadtestdata'
Type 'django-admin.py help' for usage.

So I looked up django-admin help and there is no loadtestdata command. BUT there is a 'loaddata' listed. So then I tried the following command:

django-admin.py loaddata myapp.Merchant:3

This lead to the following error:
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.

I installed django-fixtures using pip in a virtualenv. Please help!

i am pretty new to this but i am running into a RuntimeError: maximum recursion depth exceeded while calling a Python object with my foreign key

python manage.py loadtestdata tourist_card.UserCardUsage:1 --generate-m2m=reservations:1:1 --generate-fk=ALL --settings=settings_local

auth.User(pk=3744): J_X0vN_vb9g7bShriTHQmp
RuntimeError: maximum recursion depth exceeded while calling a Python object
`` 

What am i doing wrong?

In init.py:

if isinstance(model_or_iterable, (list, tuple, set)):

should be:

if not isinstance(model_or_iterable, (list, tuple, set)):


def unregister(model_or_iterable, fail_silently=False):
    '''
    Remove one or more models from the autofixture registry.
    '''
    from django.db import models
    if isinstance(model_or_iterable, (list, tuple, set)):
    model_or_iterable = [model_or_iterable]
    for model in models:
    if isinstance(model, basestring):
        model = models.get_model(*model.split('.', 1))

I noticed today that this doesn't work. Nasty, because my tests now failed randomly, which was hard to debug. Is this something that can be fixed?

autofixture.generators.InstanceGenerator fails when you defined your model with a Q object in limit_choices_to. InstanceGenerator assumes that limit_choices_to is always a dictionary.

It looks like InstanceSelector has the same issue.

I got it on Mac OS 10.10.1

File "./manage.py", line 10, in
execute_from_command_line(sys.argv)
File "lib/python2.7/site-packages/django/core/management/init.py", line 399, in execute_from_command_line
utility.execute()
File "lib/python2.7/site-packages/django/core/management/init.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(_args, *_options.dict)
File "lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(_args, *_options)
File "lib/python2.7/site-packages/django/db/transaction.py", line 339, in inner
return func(_args, *_kwargs)
File "lib/python2.7/site-packages/autofixture/management/commands/loadtestdata.py", line 227, in handle
autofixture.create(model, count, *_kwargs)
File "lib/python2.7/site-packages/autofixture/init.py", line 131, in create
return autofixture.create(count, *_create_kwargs)
File "lib/python2.7/site-packages/autofixture/base.py", line 535, in create
instance = self.create_one(commit=commit, **kwargs)
File "lib/python2.7/site-packages/autofixture/base.py", line 482, in create_one
self.process_field(instance, field)
File "lib/python2.7/site-packages/autofixture/base.py", line 381, in process_field
value = self.get_value(field)
File "lib/python2.7/site-packages/autofixture/base.py", line 377, in get_value
value = generator()
File "lib/python2.7/site-packages/autofixture/generators.py", line 70, in call
return self.get_value()
File "lib/python2.7/site-packages/autofixture/generators.py", line 66, in get_value
value = self.generate()
File "lib/python2.7/site-packages/autofixture/generators.py", line 639, in generate
ContentFile(get_placeholder_image(width, height))
File "lib/python2.7/site-packages/autofixture/placeholder.py", line 22, in get_placeholder_image
font = ImageFont.truetype(font, size=fontsize, encoding=encoding)
File "lib/python2.7/site-packages/PIL/ImageFont.py", line 218, in truetype
return FreeTypeFont(filename, size, index, encoding)
File "lib/python2.7/site-packages/PIL/ImageFont.py", line 134, in init
self.font = core.getfont(file, size, index, encoding)
File "lib/python2.7/site-packages/PIL/ImageFont.py", line 34, in getattr
raise ImportError("The _imagingft C module is not installed")
ImportError: The _imagingft C module is not installed

autofixture/init.py", line 62, in unregister
for model in models:
TypeError: 'module' object is not iterable

when I use sqllite as my django databases when I run this command
python manage.py loadtestdata -v0 product.TrustProduct:10
it works perfectly well
But when I use Mysql as my database;

Django throwout this error message:

django.db.utils.OperationalError: (1118, 'Row size too large (> 8126). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline.')

Providing a max_value to the FloatGenerator fails with:

__init__() got an unexpected keyword argument 'max_value'

There simply is a typo in generators.py, I will provide a pull request shortly.

As I understood from autofixture / base.py there is no support for this.
It would be nice to have support for imagefields. The obvious problem is what images should be used? Well place a kitten is a start: http://placekitten.com/ or something like this http://placehold.it/

so the process of auto filling imagefield is not that hard:

1. download random set of images with random resolutions
2. fill the fields with random files

Possible problems:

1. where to put the images? Media folder? actually imagefield has to have the upload attribute so here we go.

Any thoughts?

It seems that if a field is specified to be None in field_values, django-autofixture assumes that an incorrect value was generated for that field and therefore re-generates the instance (which will of course fail again). Is there a way to specify that a field should actually be None?

I just spent a while digging through the source to figure out how to limit the set of objects that would be set as a foreignkey. I got it to work, but I think that is such a cool feature that it should get documented in the README.

(pull request forthcoming!)

I ran into errors when trying to loadtestdata --generate-fk=ALL for a model that has defined a generic relation. See my test case for an example ericbuckley@47c0caf.

I suggest add support for django-filebrowser fields. It should detect accepted extensions, connect it to supported extensions of placeholer and return right fixtures.