The changes from f64d69e for will_not_clone in versions/admin.py broke compatibility with Django 1.7. The invocation of each_context in Django 1.7 doesn't have request as argument. See django/contrib/admin/sites.py on line 264.

Some suggestions:

• Use tox for testing against multiple versions of Python
• Use Travis CI or Circle CI for continuous integration tests
• Use Coveralls to measure code coverage
• Advertise test status and code coverage in README file with badges
• Add a CONTRIBUTING.rst file
• Add Sphinx documentation
• Support Python 3

Some links with more advise:

• https://github.com/pydanny/cookiecutter-djangopackage
• http://treyhunner.com/2013/07/maintaining-an-open-source-project/

An object's history should also be accessible from non-current versions (via the top-right History button)

If you have two models

class OrderPosition(Versionable):
    …

class OrderPosition(models.Model):
    order = VersionedForeignKey(
        Order,
        verbose_name=_("Order")
    )
    status = models.CharField(…)

a lookup like

 OrderPosition.objects.filter(order__status=Order.STATUS_PAID)

will fail with ValueError("joined_alias not set") (triggered in versions/models.py:271). When I change the OrderPosition model to be versionable as well (which I now did, do avoid further confusion), all works well. However, for the future, one might want to fix this, if it is possible without too much effort.

Encountered with python 2.7.9, Django 1.7.4 and CleanerVersion 1.3.3 - 1.4.1. Did not try 1.3.2 or earlier.

How to reproduce (this sequence is based on the process described in the Django tutorial https://docs.djangoproject.com/en/1.7/intro/tutorial01/ )

django-admin.py startproject foo
cd foo
./manage.py migrate
./manage.py startapp bar

• add bar in foo.settings.INSTALLED_APPS
• edit bar/models.py so that it looks like this:

from django.db import models
import versions.models as vmodels

class City(vmodels.Versionable):
    name = models.CharField(max_length=40)

class Office(vmodels.Versionable):
    name = models.CharField(max_length=40)
    city = vmodels.VersionedForeignKey(City)

class Employee(vmodels.Versionable):
    name = models.CharField(max_length=40)
    offices = vmodels.VersionedManyToManyField(Office, related_name='employees')

Then run:

./manage.py makemigrations bar && ./manage.py migrate

This gives following output:

Migrations for 'bar':
  0001_initial.py:
    - Create model City
    - Create model Employee
    - Create model Office
    - Alter unique_together for office (1 constraint(s))
    - Add field offices to employee
    - Alter unique_together for employee (1 constraint(s))
    - Alter unique_together for city (1 constraint(s))

Operations to perform:
  Apply all migrations: admin, contenttypes, bar, auth, sessions
Running migrations:
  Applying bar.0001_initial...Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/venv/cv/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/venv/cv/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/venv/cv/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/venv/cv/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/venv/cv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 161, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/venv/cv/lib/python2.7/site-packages/django/db/migrations/executor.py", line 68, in migrate
    self.apply_migration(migration, fake=fake)
  File "/venv/cv/lib/python2.7/site-packages/django/db/migrations/executor.py", line 102, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/venv/cv/lib/python2.7/site-packages/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/venv/cv/lib/python2.7/site-packages/django/db/migrations/operations/fields.py", line 37, in database_forwards
    field,
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/sqlite3/schema.py", line 176, in add_field
    self._remake_table(model, create_fields=[field])
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/sqlite3/schema.py", line 144, in _remake_table
    self.quote_name(model._meta.db_table),
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/schema.py", line 102, in execute
    cursor.execute(sql, params)
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/venv/cv/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/venv/cv/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 485, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: table bar_employee__new has no column named offices

When trying to do this (where Foo is subclass of Versionable), an infinite loop happens:

f = Foo.objects.current.defer('version_start_date').get(pk=some_pk)
print("Execution never reaches here")

This test should illustrate the issue:

class FilterOnRelationTest(TestCase):
    def test_filter_on_relation(self):
        team = Team.objects.create(name='team')
        player = Player.objects.create(name='player', team=team)
        t1 = get_utc_now()
        sleep(0.1)
        l1 = len(Player.objects.as_of(t1).filter(team__name='team'))
        team.clone()
        l2 = len(Player.objects.as_of(t1).filter(team__name='team'))
        self.assertEqual(l1, l2)

For the first Player.objects.as_of()-query, we get the player object; for the second, we get no results.

My understanding is that VersionedForeignKey.get_extra_restriction() adds WHERE-clauses as appropriate wrt. version_start_date and version_end_date for the active querytime. (Or for the "current" version if there is no active querytime.)
With the JOIN constructed by Django, we get a join on primary key, which would match only the newest version, and if the newest version is not the appropriate one for the querytime the WHERE-clause discards it, and we get no result where one was expected.

Unfortunately, I don't have any good solution to this; modifying logic that sets up joins so that as_of()-queries might join on identity instead of id (as the existing get_extra_restriction() should ensure that we get only one) where appropriate, without breaking table creation, without breaking the currently working queries through versioned many-to-many relations, would seem to require overriding a lot of methods on Query with only minor changes... That might work, though it sounds rather messy... Hopefully, you can come up with a better idea?

With the latest release of your nice plugin (which I am very grateful you have provided to the community), I encountered a problem using the VersionedManyToMany field.

My model setup is like so:

Cassette(Versionable):
{{details}}

CassetteGross(Versionable):
cassettes = VersionedManyToManyField(Cassette)

When I attempt to access the cassettes from my CassetteGross object (cg below), I get the following error:

cg.cassettes.current.all()
Traceback (most recent call last):
File "", line 1, in
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\models\query.py", line 116, in repr
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\models\query.py", line 141, in iter
self._fetch_all()
File "C:\dev\pythonenv\LIS\lib\site-packages\versions\models.py", line 375, in _fetch_all
self._result_cache = list(self.iterator())
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\models\query.py", line 265, in iterator
for row in compiler.results_iter():
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\models\sql\compiler.py", line 700, in results_iter
for rows in self.execute_sql(MULTI):
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\models\sql\compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\backends\utils.py", line 81, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\backends\utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "C:\dev\pythonenv\LIS\lib\site-packages\django\utils\six.py", line 658, in reraise
raise value.with_traceback(tb)
File "C:\dev\pythonenv\LIS\lib\site-packages\django\db\backends\utils.py", line 65, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column case_cassettegross_cassettes.cassette_id does not exist
LINE 1: ...settegross_cassettes" ON ( "case_cassette"."id" = "case_cass..

I was wondering if you could give me some feedback as to whether this is a problem on my end of things so that I may go about fixing this. I am also wondering if you could comment back with the release information for the new version if possible?

Thanks for your continued support, I hope I can help resolve this issue.

Regards,

Mike

Trying out the brand-new Django 1.8a1, I run into the following issue:

  File "/daten/proj/tixl/src/tixlbase/models.py", line 681, in get_all_variations
    for var in all_variations:
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/query.py", line 163, in __iter__
    self._fetch_all()
  File "/daten/proj/tixl/src/versions/models.py", line 380, in _fetch_all
    self._prefetch_related_objects()
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/query.py", line 583, in _prefetch_related_objects
    prefetch_related_objects(self._result_cache, self._prefetch_related_lookups)
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/query.py", line 1506, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(obj_list, prefetcher, lookup, level)
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/query.py", line 1617, in prefetch_one_level
    all_related_objects = list(rel_qs)
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/query.py", line 163, in __iter__
    self._fetch_all()
  File "/daten/proj/tixl/src/versions/models.py", line 375, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/query.py", line 239, in iterator
    results = compiler.execute_sql()
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 815, in execute_sql
    sql, params = self.as_sql()
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 369, in as_sql
    from_, f_params = self.get_from_clause()
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 601, in get_from_clause
    clause_sql, clause_params = self.compile(from_clause)
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 342, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/sql/datastructures.py", line 87, in as_sql
    extra_sql, extra_params = compiler.compile(extra_cond)
  File "/daten/proj/tixl/env/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 342, in compile
    sql, params = node.as_sql(self, self.connection)
  File "/daten/proj/tixl/src/versions/models.py", line 204, in as_sql
    for lhs, table, join_cols in _query.join_map:
AttributeError: 'VersionedQuery' object has no attribute 'join_map'

I would have tried to fix it myself, but it took my to long to get an understanding of what exactly is going on in VersionedWhereNode.

If I have a VersionedManyToManyField on a model, I can dump it to a fixture fine, but loaddata fails with this message from versions/models.py line 961:

Related values can only be directly set on the current version of an object

I'll follow up with more information and hopefully a solution.

When converting an existing models.Model model class to a Versionable model class, defaults are not provided, therefore the user must make decisions as part of makemigration migration step. For the 2 examples below, Versionable should be able to handle these sensibly. For identity fields, it should copy the table's PK field, as this is what it does for new rows in the table. For date fields (example 2) it should be the database "now()" at the time it is run on the database, not a hard-coded value (like timezone.now()) in the migration.

Example 1: Handle new identity column in existing table

You are trying to add a non-nullable field 'identity' to assessment without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows)
 2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now()
>>> ''

Example 1 Migration File Output

migrations.AddField(
    model_name='assessment',
    name='identity',
    field=models.CharField(default='', max_length=36),
    preserve_default=False,
),

Example 2: Handle new version date column in existing table

You are trying to add a non-nullable field 'version_birth_date' to assessment without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
 1) Provide a one-off default now (will be set on all existing rows)
 2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now()
>>> timezone.now()

Example 2 Migration File Output

migrations.AddField(
    model_name='assessment',
    name='version_birth_date',
    field=models.DateTimeField(default=datetime.datetime(2015, 7, 20, 15, 30, 4, 971732, tzinfo=utc)),
    preserve_default=False,
),

<Model>.objects.current.all().delete() does not apply the _delete_at() procedure defined in Versionable (which is to terminate each version), but deletes objects from the DB (including referencing objects; so, CASCADING is applied here). In this Django-default case, deletion is done via django.db.models.deletion.Collector.
The current workaround is to do a for-loop over all instances and to call .delete() on each one of the instances. This terminates versions in the sense of CleanerVersion.

This issue is partially related to #42.

I did a first draft integrating the propagate_querytime() function directly to the filtering function in branch https://github.com/swisscom/cleanerversion/tree/integrate_query_time_propagation

However, this new integration seems like it's not stressed enough by the unit tests yet. So, there's clearly some potential for better testing. See 5f69b77#diff-32744b51015e429d1e825c232dba9d4cR975

Also, some cleanup could be done, for example the return value of function 'path_stack_to_tables' should be adapted to be a list of tuples and not a list of lists; see:
5f69b77#diff-20afbe5785bb1d7a9595465020589f5fR407

@jczulian: fyi

I see that the documentation specifically mentions PostgreSQL support. Should I infer that MySQL will not work? I ran the tests on my application, and had 31 failures. Just wondering if this is because I'm using MySQL. If so, it would be nice if you explicitly stated that MySQL will fail in the documentation. Thanks.

Since we have the Admin, would you guys want Selenium tests on the Admin interface?

You clone the object in get_object before the forms validation take place.

I think the correct place to do this would be in save_model() taking into consideration that it also affects save_related().

During running an auto-generated migration, I run into

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/commands/sqlmigrate.py", line 30, in execute
    return super(Command, self).execute(*args, **options)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/core/management/commands/sqlmigrate.py", line 61, in handle
    sql_statements = executor.collect_sql(plan)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/migrations/executor.py", line 82, in collect_sql
    migration.apply(project_state, schema_editor, collect_sql=True)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/migrations/operations/fields.py", line 139, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/backends/schema.py", line 445, in alter_field
    return self._alter_many_to_many(model, old_field, new_field, strict)
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/backends/sqlite3/schema.py", line 229, in _alter_many_to_many
    old_field.rel.through._meta.get_field_by_name(old_field.m2m_reverse_field_name())[0],
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/utils/functional.py", line 17, in _curried
    return _curried_func(*(args + moreargs), **dict(kwargs, **morekwargs))
  File "/daten/proj/pretix/env/lib/python3.4/site-packages/django/db/models/fields/related.py", line 2221, in _get_m2m_reverse_attr
    return getattr(self, cache_attr)
AttributeError: 'VersionedManyToManyField' object has no attribute '_m2m_reverse_name_cache'

The migration looks like

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import versions.models
import pretixbase.models


class Migration(migrations.Migration):

    dependencies = [
        ('pretixbase', '0002_auto_20150211_2031'),
    ]

    operations = [
        migrations.AlterField(
            model_name='quota',
            name='items',
            field=versions.models.VersionedManyToManyField(verbose_name='Item', to='pretixbase.Item', blank=True, related_name='quotas'),
            preserve_default=True,
        ),
    ]

I'm not 100% sure but I'm fairly certain that the migration was created because I explicitely set the related_name attribute, while it was auto-generated before. Do you think there is an easy way to solve this?

With list_display_show_identity = True in admin

Django 1.8 introduces UUIDField, which should turn out to be a good choice for cleanerversion's primary keys. However, it's very unclear to me whether there might be migration issues (it should work) and how to maintain compatibility to Django <1.8.

Let CleanerVersion do the following:

• validate that id, if provided, is a valid UUID v4 string
• set identity = id (and of course generate id if not provided)

Here's some code that validates uuid4: https://gist.github.com/ShawnMilo/7777304 (but would need to validate that input id is '^[-\w]+$', and strip '-' characters before doing that comparison)

Since the id is the primary key column, the database will enforce uniqueness.

It is possible, if a really bad random number generator is used, that uuid collisions will happen. Add a clear warning in the docs.

When running 'python manage.py test versions' in a Django app using CleanerVersion, I get the error 'ImportError: No module named six.' This appears to be due to the fact that CleanerVersion uses six without it being in the install_requires list in setup.py. I believe that it should either be added to that list, or CleanerVersion should use Django's internal copy by replacing 'import six' by 'from django.utils import six.'

It appears that cleanerversion allows only keys to the latest version of particular entity with VersionedForeignKey allowing to check version for specific time.
But what if I want to have a link to some specific version? E.g. I want to store the state of some entity just like it was at that particular time. I can't use id/identity as those get overriden. I could use id + datetime but that seems to work OK only for single entity and constructing 1 query to select some "events" with corresponding historical versions of objects those event were fired appears to be not a trivial task.
Is having something like this out of the scope of cleanerversion and I need to use something else?
Or I just miss something obvious?

Add the possibility to clear the filter on a model's view (currently one can't return to the initial view, listing ALL objects and not only filtered ones by either the as_of or the is_current filter)

The first beta for Django 1.9 has just been released, no more features will come to 1.9, only bugs will be fixed. One could therefore start evaluating what needs to change in cleanerversion to support Django 1.9.

At a first look, I noticed that some pretty things of the ORM implementation pretty essential to cleanerversion have been changed:

• ValuesQuerySet and ValuesListQuerySet have been removed.
• In django.db.models.fields.related, the members ReverseSingleRelatedObjectDescriptor, ReverseManyRelatedObjectsDescriptor, ManyRelatedObjectsDescriptor, create_many_related_manager, ForeignRelatedObjectsDescriptor have been refactored into new classes at django.db.models.fields.related_descriptors.

At the moment, code like this:

from django.db import models
from versions.models import Versionable, VersionedForeignKey


class Beer(Versionable):
    name = models.CharField(max_length=50)


class BeerDrinker(Versionable):
    name = models.CharField(max_length=50)
    now_drinking = VersionedForeignKey(Beer, null=True, blank=True, related_name='drinkers')

beer1 = Beer.objects.create(name="Chopsfab Hell")
drinker1 = BeerDrinker.objects.create(name="Joe", now_drinking=beer1)
drinker2 = BeerDrinker.objects.create(name="Mary", now_drinking=beer1)

t0 = get_utc_now()
beer1 = beer1.clone()
beer1.drinkers = []
print len(Beer.objects.as_of(t0).first().drinkers.all())

will print "0" - which seems wrong, because there were two drinkers at time t0.

I'm not sure how this should be handled at the moment. Two ideas:

• At direct assignment time, automatically clone the referencing records
• At clone time, also clone referencing records

Does anyone else have some input / ideas?

The VersionedAdmin class has custom boolean fields list_display_show_identity, ..._end_date, ..._start_date that can be overwritten in subclasses to have the admin show fewer versioning descriptors. There should be a place in the docs that explains them as well as the admin as a whole. I failed to think of this when making the original pull request. I'll work on it over the next week.

Multi-table inheritance ( https://docs.djangoproject.com/en/1.7/topics/db/models/#multiple-inheritance ) with Versionable is not possible.

Just to make it clear, multi-table inheritance is when you don't declare abstract=True in your superclasses, and results in multiple tables being created (for the example below, adding a Spaceship object would create an entry in an appname_spaceship table, as well as a record in appname_vehicule, and these two tables would be linked with a foreign key.

For example, this code

    class Vehicule(Versionable):
        name = models.CharField(max_length=80)
        make_year = models.IntegerField()

    class Spaceship(Vehicule):
        wormhole_ready = models.BooleanField(default=False)

fails with

django.db.utils.ProgrammingError: column "id" named in key does not exist'

(Django 1.7)

I'm not super motivated to try to fix this issue, because I've never used the multi-table inheritance feature, and I've seen it's use discouraged.

I just wanted to point out that it doesn't work. Perhaps if someone one day has a need for it they will make it work.

The VersionedForeignKey should do either of those:

• raise a warning if the on_delete attribute is set, as it has no effect
• implement on_delete correctly and thus Verionable.delete should do things as cascade deletion, deletion protection, setting references to NULL, …

Currently, Django 1.6 has reached end of life.

Should we not create a Django 1.6 branch of CleanerVersion and let it rest in peace?

This would also mean:

• documenting what version of CleanerVersion works with what release of Django
• removing 1.6 dependant code from CleanerVersion to ease development going further
• removing 1.6 from the Travis test environments

Soon enough (early 2016) Django 1.7 will reach end of life;

I'm suggesting this to avoid, as much as possible, if VERSION >= clauses littering the code, and to let code take advantage of new Django APIs without worrying about unsupported Django versions. It also creates a burden on developers if all Django versions since 1.6 must be supported; this burden increases with every release.

To reproduce:

./manage.py syncdb && ./manage.py syncdb
OR
./manage.py migrate && ./manage.py migrate

The error generated is like this:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 533, in handle
    return self.handle_noargs(**options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/syncdb.py", line 27, in handle_noargs
    call_command("migrate", **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 155, in handle
    changes = autodetector.changes(graph=executor.loader.graph)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/autodetector.py", line 40, in changes
    changes = self._detect_changes(convert_apps, graph)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/autodetector.py", line 108, in _detect_changes
    self.new_apps = self.to_state.render()
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/state.py", line 67, in render
    model.render(self.apps)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/state.py", line 311, in render
    body,
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 171, in __new__
    new_class.add_to_class(obj_name, obj)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 300, in add_to_class
    value.contribute_to_class(cls, name)
  File "/vagrant/versions/models.py", line 382, in contribute_to_class
    name)
  File "/vagrant/versions/models.py", line 425, in create_versioned_many_to_many_intermediary_model
    to: VersionedForeignKey(to, related_name='%s+' % name),
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 110, in __new__
    model_module = sys.modules[new_class.__module__]
KeyError: u'__fake__'

When models including M2M-relationships are written to the DB, column name cases are mixed up between CamelCase and lowercase.

Keep everything in lowercase.

Reported by @gidoca (Thanks!)

Exception message verbosity for related (and failed) lookups could be enhanced by adding the source object's type, identity and, if set, the querytime used for the lookup.
Note that the DoesNotExist class is bound to the related model's class.
So, the following code would raise an Article.DoesNotExist exception if no article is found:

class Journal(Versionable):
   name = CharField(...)

class Article(Versionable):
   name = CharField(...)
   journal = VersionedForeignKey(Journal)

# ...some code happens here...

# Assume there's no current journal assigned to article, this would raise a Journal.DoesNotExist exception saying: "DoesNotExist: Journal matching query does not exist."
journal = article.journal

At the moment it works like this:

obj.restore(fk_field_name=foreign_object_pk)

But it should really work like everywhere else with Django Models:

obj.restore(fk_field_name=foreign_object_instance)

and

fk_column = fk_field_name + '_id'
obj.restore(fk_column=foreign_object_pk)

Queries for objects during a time interval should be possible.

Right now, objects at a specific point in time can be queried. If one needs objects that lived during a specific time interval, it gets difficult. CleanerVersion should help to do this by adding the following query possibility on VersionedManager and VersionedQuerySet:

Model.objects.in_interval(start, end, unique=<bool>, inclusive=<bool>).filter(...)

Arguments are:

• start: a timestamp indicating the start of the time interval
• end: a timestamp indicating the end of the time interval
• unique: a boolean; if True, return the latest valid version of an object during a time interval; if False, return all versions of an object during a time interval; Default: False
• inclusive: a boolean: if True, versions that either start or end (or both) during a time interval are considered; if False, only versions that start AND end during a time interval are considered; Default: True

The return value is, as usual an object of type VersionedQuerySet.

Some more details:
inclusive set to True translates to the filtering expression version_start_date < interval_end_date AND version_end_date > interval_start_date, i.e. a version is part of the resulting QuerySet if it starts, ends or exists only during the interval.
inclusive set to False translates to the filtering expression version_start_date >= interval_start_date AND version_start_date < interval_end_date AND version_end_date > interval_start_date AND version_end_date <= interval_end_date, i.e. a version is part of the resulting QuerySet if it starts AND ends during the specified time interval.

Related objects (either O2M, M2O or M2M) should also be limited by the above mentioned filters.

CleanerVersion does not currently integrate with the Django admin, but it may someday. As suggested in #54, here are some features or behaviors that might be considered when integrating CleanerVersion with the Django admin:

• Create a new version on every edit (or optionally with "create new version" checkbox on edit form)
• View only current revisions and/or set the "as_of" date via filter on changelist
• View all versions of an instance
• Revert to an old version, perhaps by editing that version

previous_version, next_version, and current_version already exist. It would make sense to add a method to get the given object at a specific time, returning None if no such version existed.

Then it would be possible to write:

Foo.objects.time_version(foo_object, as_of_time)

Instead of:

Foo.objects.as_of(as_of_time).filter(identity=foo_object.identity).first()

And it would allow optimizing it to return the same object if the asked-for time lies in the given object's validity period.

Django 1.7 introduced migrations. CleanerVersion has been adapted to support these (in PR #24).
However, has not been tested whether CleanerVersion on Django 1.6 supports migrations with South.
If it does not, the according fixes should be applied if possible.

Initial note by @jczulian:
"Since Django migration is inherited from South (migration system used before 1.7) I wonder what is happening when using Django 1.6.X together with South? Does South do something similar to Django 1.7? Therefore should we test alongside the Django's version number if South is present and act in the same way as we do when running under Django 1.7"

In https://github.com/swisscom/cleanerversion/blob/master/versions/models.py#L549 the docstring says that a querytime can be assigned None. If this is done, however, the DB query fails.
So, instead, assign a Querytime instance with 'active' being set to False.

For example in the terminal:

from versions_tests.models import City
A = City(name="A")
A.save()
B = A.clone()
B.name = "B"
B.save()
A = City.objects.filter(name = "A")
A.restore()
B = City.objects.filter(name = "B").get()
B.restore()
As = City.objects.filter(name = "A") #There are more than one now as expected
for a in As:
     print a.is_current #prints out True, False; so one is still current 

And we have two objects with the same identity that are current. Am I using .restore incorrectly?

Has this been encountered before? I was working on integrating .restore into the admin and I thought this seemed wonky.

I have an object X, which is a Versionable model. I want to make a new object Y that has all the same field values as X, but has an unrelated history (i.e. no history).

In projects without versioning, I can do this:

o = MyModel.objects.get(name='x')
o.id = None
o.pk = None
o.save()

But a similar approach gets me an integrity error with CleanerVersion:

IntegrityError: NOT NULL constraint failed: mymodel.id

Any advice? I'll post back here when/if I find a solution. Thanks!

I am just learning CleanerVersion and I think I just learned that one should use VersionedForeignKey to point to a Versionable model, but use ForeignKey to point to an unversioned model.

I didn't get this from the docs. Maybe there is a way to mention it in there.

I also didn't get this from the error message I get when trying to display that field in the admin:

/lib/python2.7/site-packages/versions/models.py", line 619, in __get__
raise TypeError("It seems like " + str(type(self)) + " is not a Versionable")
TypeError: It seems like <class 'versions.models.VersionedReverseSingleRelatedObjectDescriptor'> is not a Versionable

I think at least, this message should indicate which field and model are set up incorrectly.

(Or maybe I'm missing something and I'm all wrong ...)

Sorry, I don't have a patch for either of these now. Maybe soon. :-)

Im currently working with Django 1.8 and I would like to use the cleanerversion.
Is there any release date for the support?

I'm trying to write a data migration for my app, and receiving the "not a Versionable" error when I read any VersionedForeignKey fields from models retrieved with the apps.get_model() recommended in Django docs. For example, this code is inside a RunPython migration:

Thing = apps.get_model('myapp', 'Thing')
mything = Thing.objects.current.first()
thing.vfk_field # throws TypeError("It seems ...")

I'm not sure whether this is the fault of Django migrations or of CleanerVersion.

I set a breakpoint inside versions/models.py at line 619 and I find that my target objects have made-up types:

(Pdb) type(current_elt).mro()
[<class 'Color'>, <class 'django.db.models.base.Model'>, <type 'object'>]
(Pdb) apps.get_model('rms', 'color').mro()
[<class 'rms.models.Color'>, <class 'versions.models.Versionable'>, <class 'rms.models.Term'>, <class 'django.db.models.base.Model'>, <type 'object'>]

Additionally, in this context there is no Model.objects.current and no Model.objects.as_of because objects is not a VersionedManager.

I don't know what the right thing to do is here. In my project, I can avoid writing a data migration today but I will probably need one later.

Hi there,
as per https://docs.djangoproject.com/en/1.8/topics/i18n/timezones/#code it seems like the date-time passed to the version_end_date and version_start_date are being passed a naive datetime (that is a datetime without a timezone).

[...] refactor your code wherever you instantiate datetime objects to make them aware. This can be done incrementally. django.utils.timezone defines some handy helpers for compatibility code: now(), is_aware(), is_naive(), make_aware(), and make_naive().

I think this happens because of line 51 in models.py but i'm not sure.

Thanks!

Currently, it is not possible to manage non-versionable models with the Admin-interface integration that comes with CleanerVersion.
Adapt such that it is possible to mix them.

When using VersionedManyToManyField without specifying an intermediary model by means of the through argument, invalid SQL is generated. It will generate queries like SELECT "demo_secondmodel_other"."FirstModel_id" FROM ..., when the database column is actually called firstmodel_id in lowercase. This causes all queries involving the many-to-many relationship to fail. Note that this problem only manifests itself in DBMS' whose column names are case sensitive, like PostgreSQL. In SQLite, for instance, it does not. Also, having many-to-many relationships using an explicit intermediary model works as intended.

I created a minimal Django app which reproduces the problem. It can be found at gidoca/versioned-m2m@2a2b644. A working version with an explicit intermediary model can be found at gidoca/versioned-m2m@0c627a4.

How to reproduce.

1. Create a Versionable object having relation (m2m or o2m) with another Versionable. Save this first version.
2. Create a new version of this model.
3. Using the latest version call on it the previous_version() method to get back the initially created version.
4. Navigate the relation of that origin version and see that you don't get anything back as you would expect.

Possible solution:
Add an optional as_of parameter to previous_version() and next_version(), so that the object returned will propagate this given as_of time to the queries generated for related objects. If no as_of time is given, set the object's query_time to it's version_end_date minus 1 microsecond (or to None if it's the current object)

In the case that the given as_of lies outside of the version's validity period, an Exception will be raised.

However:
How common is the use case that we ask for a previous version and know it's expected validity period? Perhaps previous_version / next_version should not have an additional as_of parameter - they should just set the returned object's query_time to it's version_end_date minus 1 microsecond (or to None if it's the current object). The caller can then adjust the object's as_of time themselves, if needed.

Currently it does not seem to be possible to insert strings including unicode characters ("äöüé..."). Try to insert a city like "Zürich" to reproduce.

Hi,

Apologies for opening an issue when what I really have is a question.

I want to implement immutable / append-only models in our system (no UPDATEs only INSERTs), so we can store full history for our data. In other platforms I've done this with an integer versionId with a unique constraint on (versionId, id) columns. All saving code would try to insert rows with (versionId + 1, id). This also gave me concurrency control - if 2 clients both try to insert a row with the same (versionId, id) pair then it would fail and I could show a nice error page and not have the 'lost update' problem when 2 clients both try to update the same record.

cleanerversion looks very close to being what I need :) except that it uses timestamps instead of integers for versioning. I'm thinking of forking the project and changing the timestamps to integers but I wanted to ask - do you see any problems with this idea, and is there maybe a better way?

Thank you.

Create an exclusion constraint for version_start_date and version_end_date for objects with the same identity. ( see http://www.postgresql.org/docs/9.3/static/rangetypes.html#RANGETYPES-CONSTRAINT).

This will only be usable if the btree_gist postgresql extension is installed (this requires superuser permissions to install).