yapstudios / yapdatabase Goto Github PK

From YapDatabase.m:274

// There are a few reasons why the database might not open.
// One possibility is if the database file gets corrupt.
// In the event of a problem, we simply delete the database file.
// This isn't a big deal since we can just redownload the data.

I don't expect corrupt database files to be too common, but not all projects will be able to re-download data.

I might be going about this wrong, but I feel like a merge operation could be useful. Something like:

- (void)mergeObject:(id)object forKey:(NSString *)key inCollection:(NSString *)collection withMergeHandler:(id (^)(id currentValue, id newValue))merge;

The method would load the previous value (nil if none was found), execute the block passing in both values, take the result of the block and store that in the database.

Any thoughts on this? I could implement it outside of YapDB, but it would be a little more performant if I modified the current setObject method (leaving the API the same, just creating a new internal method).

Hi,

If you needed to do fast searching for specific values, not just a key lookup, how would you accomplish that? For example, doing a substring search for "Term" within a Movies database and having it return a list of movies with the Movie Titles that contain the term "Term".

For example:

Movie Title:

The Terminator
Terminator 2: Judgement Day
The Terminal

Right now I'm using SQLCipher with the FTS3 full text search engine in SQLite. But there's an overhead in maintaining the separate FTS search engine that I'd like to remove.

I'm assuming that your system would require me to fetch all data into memory and then do an in-memory filter using something like an NSPredicate. There would be a big memory and performance impact in doing so I would think.

I have a YapCollectionsDatabaseView on my YapCollectionsDatabase to view the teams in the database. I'm also displaying this data in a UITableView and watching for updates. I noticed the examples in the wiki (https://github.com/yaptv/YapDatabase/wiki/Views) appear to be out of date at the moment, so I did my best to adjust the code appropriately.

When the UITableView is first presented, the teams are displayed correctly. However, when the view is updated, it appears that the changes information in the notification is incorrect, resulting in an assertion failure in UITableView when those changes are applied.

As best as I could tell, this is the notification created and sent by YapAbstractDatabaseConnection:

NSConcreteNotification 0x8e7bfd0 {name = YapDatabaseModifiedNotification; object = <YapCollectionsDatabase: 0x9c4a880>; userInfo = {
    connection = "<YapCollectionsDatabaseConnection: 0x99a56e0>";
    extensions =     {
        teams =         {
            changes =             (
                "<YapDatabaseViewRowChange: Update pre(0) post(0 -> 0) group(teams -> teams) key(<YapCollectionKey[0x9aa2dc0] collection(teams) key(1)>)"
            );
        };
    };
    metadataChanges = "<YapSet: 0x9aa3af0>";
    objectChanges = "<YapSet: 0x9aa39a0>";
}}

And this is the notification received by my notification observer:

NSConcreteNotification 0x8e7bfd0 {name = YapDatabaseModifiedNotification; object = <YapCollectionsDatabase: 0x9c4a880>; userInfo = {
    connection = "<YapCollectionsDatabaseConnection: 0x99a56e0>";
    extensions =     {
        teams =         {
            changes =             (
                "<YapDatabaseViewRowChange: Move indexPath(2147483647, 0) newIndexPath(0, 0) group(teams -> teams) key(<YapCollectionKey[0x9aa2dc0] collection(teams) key(1)>)"
            );
        };
    };
    metadataChanges = "<YapSet: 0x9aa3af0>";
    objectChanges = "<YapSet: 0x9aa39a0>";
}}

It seems strange that the change would have gotten modified from an 'update' to a 'move' and that the original section for the move would be the same as INT32_MAX. Is there more information that would be helpful or does this suggest that I might be doing something wrong?

I have a situation where I was attempting to compare an object previously retrieved from the database to a newly retrieved object (with a LongLivedReadTransaction in between). However, I noticed that the previously retrieved object was already updated with the current values before I began the LongLivedReadTransaction.

My guess is that the object was cached and therefore was automatically updated with the current values (because it was still in the cache). This surprised me because I had been thinking the cache would be snapshotted in sync with the LongLivedReadTransaction. But it looks like caches are updated automatically and if I want to find out about changes for an object, I should use one of the -hasChange methods.

Is that understanding of caches updated automatically (not in-sync with LongLivedReadTransactions) correct?

Is this currently included in the codebase? I can't seem to find any trace of it...
https://github.com/yaptv/YapDatabase/wiki/Relationships

Hi,
What's the YapDatabase best-way to search and return a set of objects with a particular array of keys?
Something like SELECT * FROM ... WHERE myKey IN ('a','b','c'...).

Where is the all-in-one header file like <Foundation/Foundation.h>, <CoreData/CoreData.h> ?

Should I import standard extension separately?

The wiki talks about calling beginLongLivedReadTransaction, and there is a public API for endLongLivedReadTransaction.

Does this need to be called at some point from outside the connection? How should I clean up a connection which is being used by a view controller / database view?

Hi guys! I think YapDatabase is pretty cool looking, so I threw together a little hello world project to test it out, and figured I would let you know so you could use it as an example, if you wished. You can find it here.

Sometimes one needs to build a view of a very big collection, and sorting & filtering on a database level would be very useful.

How hard would it be to implement such view with current codebase?

Little bit of context: I'm trying to get YapDatabase to work for an email app. Currently we do everything with SQLite for threading (though messages are stored in standard mail formats). Basically, we have to group all of the messages matching a query and display them in a table view where there are potentially many groups (conversation threads), and each group may contain many messages.

In reading over the Wiki, views appear to be a great tool for this kind of stuff, but I don't see an obvious API for how ordering of groups works. I see how ordering of objects within groups works. I could of course add another layer of abstraction on top of my messages in order to accomplish this, but was hoping something simple already exists.

So, is this possible with vanilla collections views with Yap Database?

Thanks for the awesome lib. Had a blast over the past couple days reading your code and trying to digest this.

I'm working on an app to be submitted in early November. If you were starting a new project now, would you start with the master branch or the 2.1 branch?

Full-Text Search is not a requirement.

Also, is there a mailing list for discussing YapDatabase with others that are using it?

Thanks!

* Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '* -[NSKeyedUnarchiver initForReadingWithData:]: incomprehensible archive (0x62, 0x70, 0x6c, 0x69, 0x73, 0x74, 0x30, 0x30)'

I'm still trying to wrap my head around views, so I may be missing something fairly obvious.

I'd like to store multiple orderings for objects. As a parallel situation, I'd like the user to store books, but display them in multiple lists, each with their own arbitrary ordering (rather than being sorted by title, for example).

In other words, for books A, B, C, D, E, a user should be able to have the following lists:
List 1: C, B, E
List 2: A, B, C
List 3: E, D, C, B

One way I can imagine solving this is by having an ordering property on each book, with a reference to the list (something like book C - @{ @"list1": 1 , @"list2": 3, @"list3" : 3 ). Then when sorting list1, I'd simply compare the values of the corresponding ordering property for a book.

Is there a better way to accomplish this with views?

Can't find anything relevant in the docs about this.

Adding 'id' as a secondary index silently wipes out the other secondary indexes.

Any advice (other than "don't do that")?

Hi,

I was just looking into this database. The idea looks awesome. I was wondering why there's no SQLite index on the key column in your database file? For small datasets that wouldn't be a problem, but for very large datasets with multiple thousands of records, wouldn't an index on the key increase the speed of fetching?

The wiki covers basic migrations such as adding a new property etc... but what about more complicated things such as renaming a class, renaming a property on a class, or changing a property from one type (say UIImage) to another (say NSData)?

It seems like there could be an extension or something that could handle heavy migrations like these. Or is there a strategy that you guys have used in situations like these?

Method: [NSDictionary sharedKeySetForKeys] is not available for platform ios 5.0. Is there any solution or advise for me to help with it?

Due to some optimizations, in my code am storing objects by archiving them myself and then calling setPrimitiveData: forKey: withPrimitiveMetadata: instead of setObject:...

This leads to one problem when I use views, where none of the the groupingBlock get called (Unless I change the version)

I understand that you need an 'object' to call the YapCollectionsDatabaseViewGroupingWithObjectBlock, but when using setPrimitiveData: grouping blocks of type
YapCollectionsDatabaseViewGroupingWithKeyBlock and YapCollectionsDatabaseViewGroupingWithMetadataBlock should be called.

As a result, newly inserted data never shows up while enumerating the group.

Using a YapDatabaseFilteredView I am seeing the sortingBlock of a YapDatabaseFilteredViewTransaction being invoked. With the following stack trace:

#0  0x002d0df3 in __98-[YapDatabaseViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:]_block_invoke at /Users/jackson/Projects/codereview/ios/codereview/Pods/YapDatabase/YapDatabase/Extensions/Views/YapDatabaseViewTransaction.m:1687
#1  0x002cfc32 in -[YapDatabaseViewTransaction insertRowid:collectionKey:object:metadata:inGroup:withChanges:isNew:] at /Users/jackson/Projects/codereview/ios/codereview/Pods/YapDatabase/YapDatabase/Extensions/Views/YapDatabaseViewTransaction.m:1802
#2  0x002560c1 in -[YapDatabaseFilteredViewTransaction handleUpdateObject:forKey:inCollection:withMetadata:rowid:] at /Users/jackson/Projects/codereview/ios/codereview/Pods/YapDatabase/YapDatabase/Extensions/FilteredViews/YapDatabaseFilteredViewTransaction.m:759
#3  0x0029964f in -[YapDatabaseReadWriteTransaction     setObject:forKey:inCollection:withMetadata:] at /Users/jackson/Projects/codereview/ios/codereview/Pods/YapDatabase/YapDatabase/YapDatabaseTransaction.m:4641
#4  0x00297618 in -[YapDatabaseReadWriteTransaction setObject:forKey:inCollection:] at /Users/jackson/Projects/codereview/ios/codereview/Pods/YapDatabase/YapDatabase/YapDatabaseTransaction.m:4439
#5  0x000535ee in -[YapDatabaseReadWriteTransaction(CRVDatabaseExt) setObject:ifNotEqualToData:forKey:inCollection:] at /Users/jackson/Projects/codereview/ios/codereview/codereview/YapDatabase+CRVDatabaseExt.m:191
#6  0x00053124 in -[YapDatabaseReadWriteTransaction(CRVDatabaseExt) mergeObject:forKey:inCollection:withMergeHandler:] at /Users/jackson/Projects/codereview/ios/codereview/codereview/YapDatabase+CRVDatabaseExt.m:157
#7  0x0005c203 in __41-[CRVGitHubPullRequestReviewable refresh]_block_invoke_7 at /Users/jackson/Projects/codereview/ios/codereview/codereview/CRVGitHubPullRequestReviewable.m:166
#8  0x0023725b in __44-[YapDatabaseConnection readWriteWithBlock:]_block_invoke_2 at /Users/jackson/Projects/codereview/ios/codereview/Pods/YapDatabase/YapDatabase/YapDatabaseConnection.m:1345

It looks like YapDatabaseFilteredViewTransaction handleUpdateObject invokes its super's YapDatabaseViewTransaction::insertRowid but the YapDatabaseViewTransaction doesn't have a sortingBlock set because its a FilteredView.

When manually create edge between to collections, Relationships extension doesn't check that edge for these two keys is already created.

Steps to reproduce:

1. Create two collections, "Collection1" and "Collection2"
2. Create edge between two collections:
   YapDatabaseRelationshipEdge* edge =
   [YapDatabaseRelationshipEdge edgeWithName:@"Collection1ToCollection2"
   sourceKey:@"object_from_collection1_id"
   collection:@"Collection1"
   destinationKey:@"object_from_collection2_id"
   collection:@"Collection2"
   nodeDeleteRules:YDB_DeleteDestinationIfAllSourcesDeleted | YDB_DeleteSourceIfAllDestinationsDeleted];
3. Try to create the edge for the same @"object_from_collection1_id" and @"object_from_collection2_id".
4. Notice edge is created successfully. When enumerating edges from the "Collection1ToCollection2" relationship there are two edges.

Expected results:
Relationships extension doesn't allow to create duplicate edges.

Hey

Any ideas as to why the formatting on some pages in the wiki are broken?

For example, on the Full Text Search page the formatting seems to break about half way through, making it difficult to read.

I've taken a look at the markdown and I can't see anything wrong.

Is iCloud sync in the roadmap?

What's the recommended direction if I want to implement syncing? Use the plugin architecture?

Hi,

Would it be possible to use SQLCipher (http://www.sqlcipher.net) instead of the built-in SQLite engine? I don't see why not, but I'm just asking for clarification.

I cannot figure out where I've gone wrong with registering the YapDatabaseView extension. The application crashes stating that

-[YapDatabase registerExtension:withName:]: unrecognized selector sent to instance

I've been trying it with both a YapDataBase object and a YapCollectionsDatabase object.
Here's my code (assertions to prove that nothing is nil):

YapDatabaseViewBlockType groupingBlockType = YapDatabaseViewBlockTypeWithMetadata;
YapDatabaseViewGroupingWithMetadataBlock groupingBlock = ^NSString*(NSString *key, id metadata){
    return @"";
};

YapDatabaseViewBlockType sortingBlockType = YapDatabaseViewBlockTypeWithMetadata;
YapDatabaseViewSortingWithMetadataBlock sortingBlock = ^(NSString *group, NSString *key1, id metadata, NSString *key2, id metadata2){
    return [metadata compare:metadata2];
};

YapDatabaseView *view = [[YapDatabaseView alloc] initWithGroupingBlock:groupingBlock
                                                     groupingBlockType:groupingBlockType
                                                          sortingBlock:sortingBlock
                                                      sortingBlockType:sortingBlockType];

YapDatabase *db = [[YapDatabase alloc] initWithPath:databasePath];

NSParameterAssert(db);
NSParameterAssert(view);

[db registerExtension:view withName:@"view"];

I user NSOperationQueue to set a queue with maxConcurrentOperation of 10. every operation opens a new connection and execute a write into database. then it crashs. bug info is:
2013-09-06 14:05:01.335 MeePo[1303:f203] -[NSPathStore2 noteCommittedChanges:]: unrecognized selector sent to instance 0x1e9a4580
2013-09-06 14:05:01.338 MeePo[1303:f203] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSPathStore2 noteCommittedChanges:]: unrecognized selector sent to instance 0x1e9a4580'
*** First throw call stack:
(0x312d82a3 0x38f7d97f 0x312dbe07 0x312da531 0x31231f68 0x103a63 0x3939511f 0x3939499b 0x39394895 0x393a3215 0x393a33b9 0x393c9a11 0x393c98a4)

what's that about? how can i solve this problem?
thanks

Hi,

just started playing around with the code yesterday and I'm really liking it so far. But one thing I'm struggling with on how to do it conceptually is a simple edit workflow. Basically what I'm trying to achieve is to be able to present a modal view controller on iOS to start editing one or many items and their relationships and to have an explicit Save action which stores everything and a Cancel which leaves the existing data as it was before starting to edit.

My problem is that I currently have to manage the state for the UI AND the state on what to do in the database and then perform all the operations in a readWrite block when saving. If I only have a few known objects this isn't a problem, but as soon as I want to manage relationship as well this gets messy.

Example:
I have the model objects Group and User. Users can be part of the group. When editing a Group I want to be able to add and remove Users from that group but only save it at the very end when the user hits Save.
Naturally I would want to use a view I create temporarily in that viewController to get a list of the users for one specific group and to get nice update notifications so I can update the UI. But for this to work I have to actually save the data in the database (otherwise I don't get a YapDatabaseModifiedNotification). But then how would I roll back if the user taps Cancel?

Any recommendations on how to achieve this?

Thanks,
Martin

I stand up the database in the test setup and add some data.

Then in the tear down I set the database and the connection to nil... but the database isn't being deallocated, which means [YapDatabaseManager deregisterDatabaseForPath:] isn't being called, so every subsequent test fails because self.database is nil.

Here is a gist with the test and the manual call to YapDatabaseManager deregisterDatabaseForPath:].

https://gist.github.com/jonnolen/8918335

Is there something that I'm missing here that is preventing the database from being deallocated?

I have one view that I setup as the application launches. This view works all the time. At runtime the user may select some options that cause another view to be registered. This other view only works the first time it is registered and only for that application's lifecycle or if I programmatically unregister the view. So if I delete the application out of memory or call unregisterExtension: then that view will never populate again, however, the view registered at startup remains fine.

Here is what my registration process looks like for the main view that always works.

+ (void)registerRanchBlockSortedView
{
    NSDate *theMidnightHour = [[NSCalendar currentCalendar] dateFromComponents:[[NSCalendar currentCalendar] components:(NSYearCalendarUnit|NSMonthCalendarUnit|NSDayCalendarUnit) fromDate:[NSDate date]]];

    YapCollectionsDatabaseViewBlockType groupingBlockType = YapCollectionsDatabaseViewBlockTypeWithObject;
    YapCollectionsDatabaseViewGroupingWithObjectBlock groupingBlock;
    groupingBlock = ^NSString *(__unused NSString *collection,
                                __unused NSString *key,
                                WorkScheduleRecord *object){

        NSParameterAssert(object.work_date.length == 13);

        NSDate *date = [[NSDate dateWithTimeIntervalSince1970:(object.work_date.doubleValue/1000.0f)] dateByAddingTimeInterval:-[[NSTimeZone timeZoneWithName:@"PST"] secondsFromGMT]];
        NSTimeInterval interval = [date timeIntervalSinceDate:theMidnightHour];

        if (interval >= 0 && interval < 60*60*24)
        {
            /*
             *  The WorkScheduleRecord is for today's date so we will include it in
             *  our view.
             *  We will group crews by location.
             */
            return [object location];
        }
        else
        {
            /*
             *  This means that the WorkScheduleRecord is not for today and therefore we will
             *  exclude it from our view.
             */
            return nil;
        }
    };

    YapCollectionsDatabaseViewBlockType sortingBlockType = YapCollectionsDatabaseViewBlockTypeWithObject;
    YapCollectionsDatabaseViewSortingBlock sortingBlock;
    sortingBlock = ^NSComparisonResult(__unused NSString *group,
                                       __unused NSString *collection1,
                                       __unused NSString *key1,
                                       WorkScheduleRecord *object1,
                                       __unused NSString *collection2,
                                       __unused NSString *key2,
                                       WorkScheduleRecord *object2) {

        /*
         *  First we compare ranches (numerical values) and sort lowest to
         *  highest. In the case that both objects come from the same ranch
         *  we then sort by block (alphanumeric values).
         *
         *  Example:    ranch = 120
         *              block = B
         *              block = B1
         */

        if (object1.ranch.integerValue > object2.ranch.integerValue)
        {
            return NSOrderedDescending;
        }
        else if (object1.ranch.integerValue < object2.ranch.integerValue)
        {
            return NSOrderedAscending;
        }
        else
        {
            return [object1.block compare:object2.block];
        }
    };

    YapCollectionsDatabaseView *databaseView =
    [[YapCollectionsDatabaseView alloc] initWithGroupingBlock:groupingBlock
                                            groupingBlockType:groupingBlockType
                                                 sortingBlock:sortingBlock
                                             sortingBlockType:sortingBlockType];

    [[ThaweDatabase database] registerExtension:databaseView withName:@"RanchesWorkSchedule_Today_Ranchblock"];
}

And here is the function I call to register a view on-the-fly.

+ (void)registerCrewViewForWorkScheduleRecord:(WorkScheduleRecord *)record
{
    NSDate *theMidnightHour = [[NSCalendar currentCalendar] dateFromComponents:[[NSCalendar currentCalendar] components:(NSYearCalendarUnit|NSMonthCalendarUnit|NSDayCalendarUnit) fromDate:[NSDate date]]];

    YapCollectionsDatabaseViewBlockType groupingBlockType = YapCollectionsDatabaseViewBlockTypeWithObject;
    YapCollectionsDatabaseViewGroupingWithObjectBlock groupingBlock;
    groupingBlock = ^NSString *(__unused NSString *collection,
                                __unused NSString *key,
                                WorkScheduleRecord *object){

        NSParameterAssert(object.work_date.length == 13);

        if (object.crew_number.integerValue == record.crew_number.integerValue)
        {
            NSDate *date = [[NSDate dateWithTimeIntervalSince1970:(object.work_date.doubleValue/1000.0f)] dateByAddingTimeInterval:-[[NSTimeZone timeZoneWithName:@"PST"] secondsFromGMT]];
            NSTimeInterval interval = [date timeIntervalSinceDate:theMidnightHour];

            if (interval >= 0 && interval < 60*60*24)
            {
                return @"";
            }
        }
        return nil;
    };

    YapCollectionsDatabaseViewBlockType sortingBlockType = YapCollectionsDatabaseViewBlockTypeWithObject;
    YapCollectionsDatabaseViewSortingBlock sortingBlock;
    sortingBlock = ^NSComparisonResult(__unused NSString *group,
                                       __unused NSString *collection1,
                                       __unused NSString *key1,
                                       WorkScheduleRecord *object1,
                                       __unused NSString *collection2,
                                       __unused NSString *key2,
                                       WorkScheduleRecord *object2) {

        /*
         *  First we compare ranches (numerical values) and sort lowest to
         *  highest. In the case that both objects come from the same ranch
         *  we then sort by block (alphanumeric values).
         *
         *  Example:    ranch = 120
         *              block = B
         *              block = B1
         */

        if (object1.ranch.integerValue > object2.ranch.integerValue)
        {
            return NSOrderedDescending;
        }
        else if (object1.ranch.integerValue < object2.ranch.integerValue)
        {
            return NSOrderedAscending;
        }
        else
        {
            return [object1.block compare:object2.block];
        }
    };

    YapCollectionsDatabaseView *databaseView =
    [[YapCollectionsDatabaseView alloc] initWithGroupingBlock:groupingBlock
                                            groupingBlockType:groupingBlockType
                                                 sortingBlock:sortingBlock
                                             sortingBlockType:sortingBlockType];

    [[ThaweDatabase database] registerExtension:databaseView withName:record.crew_number];
}

As you can see, both methods are nearly identical. On first registration the grouping and sorting blocks are invoked, every subsequent registration doesn't invoke the blocks anymore. If I am in the same application lifecycle as the first registration then the view is still populated when I re-register it even though the blocks are not invoked. When I re-launch the application and register a view that had been previously registered then the blocks are not invoked and the view is empty. Of course, all of this pertains to just the second method, since the first method is my main view and always works.

The answer may be obvious, but I am having a hard time figuring it out. Any help would be appreciated.

Is there any guidance on using YapCollectionsDatabaseFullTextSearch with a UISearchDisplayController?

Best I can tell there is no simple way to create a mapping from the FTS extension, but maybe I'm missing something obvious.

I tried key-value observing an object from YapDatabase today but never received any changes via -observeValueForKeyPath:ofObject:change:context: when changes were made.

The object was part of a table view (populated with a view) and I passed that object to the next view controller where I observed it.

Digging into things a bit, it looks like when I update a table view with a view, I get a unique object instance representing that key with the view's -objectAtIndex:inGroup:. That suggests to me that the first object instance that I passed to the view controller is no longer valid and not updated, which would explain why KVO didn't work.

Does this behavior seem correct or does it suggest that I have a problem elsewhere?

I realize you have a hello world , but it gives no instructions on actually installing this lib.

Could we drop it to 6.0? I can't see any major SDK change.

Is YapDatabse using FTS3 or FTS4?

The lifecycle of a YapDatabase instance isn't what I'd call simple right now. Perhaps I'm missing something?

You can only have one instance registered per path right now, so why doesn't the framework allow for a singleton? You need to create new connections off a YapDatabase instance, so the database needs to be available in multiple objects, but in order to access it you either need to pass the instance around or you need to create the singleton yourself. Given that it already acts as a singleton, why not have this available at the framework level?

It looks like currently there is no support to force the re-creation of secondary indexes if block and/or indexed columns change (this is supported for the Views extension).

There could possibly be a way to avoid even needing a version if the column names are used to check if the definition has changed. That said, block changes would not be detected and so there will still need to be a way for the user to force a recreation if that happens.

FYI - I'm working on https://github.com/kareemk/yapper a RubyMotion ORM built on top of YapDatabase.

I couldn't find pod spec here. I went here CocoaPods/Specs@396da9d and added pod configuration but it gives lot build errors and warnings??

I understand that when encoding the root object - all sub objects will also be encoded. I am wondering what happens if there is a large object graph where only portions changed. Must the whole graph be encoded and written to the DB? I am storing large collections of collections of collections :) of images with metadata and considering this DB as an option - could you provide some guidelines? Thanks!

I know this topic has been covered, but that topic has been closed. I commented there, but wasn't sure if it would be seen, so forgive me for opening a new issue.

I just wanted to see if there's any progress or best practices on implementing a generic sync solution for YapDatabase.

Currently I use iCloud sync by creating a structurally equivalent copy of my main sqlite file that contains only the incremental changes since the last time a sync had been performed. I store those change files in my iCloud container and then process it on the other side when I get a notification that they've been downloaded. After processing them, I trigger refreshes of my UI when processing has finished.

It would be great if there was some solution built-in to YapDatabase to handle syncing changes between devices (Mac & iOS).

Is there anything inherent in YapDatabase that would ensure there would be no conflicts between primary keys generated on different devices at different times? I use a UUID for all my primary keys, so there's no conflicts when I generate data on any device. Looking into a YapDatabase it looks like you're using integer keys, so that might be a problem unless there's a secondary key which can uniquely identify a device.

It takes me to a "create a wiki" page...?

Hi, not sure if this is a CocoaPods issue or not exactly. But, because the pod spec includes,

s.library      = 'sqlite3'

then, when building a static library which depends on YapDatabase, it fails to link, because it tries to link against sqlite. Personally, I think perform such library linking in Xcode directly, not via CocoaPods.

I'm not really sure of the best way of fixing this for my own use case. I've either got to maintain a fork (and private spec) of YapDatabase with this change, or write a post install script into my own library to find "-lsqlite3" and remove it... Any thoughts?

Is there a way to use NSPredicate or something similar to pull out a subset of the data in the database? I realize that with caching, after the first time of pulling out everything it's not a big deal to do it a second time. I just wondered if this was in the API somewhere or there was an extension already written that does this.

Github markdown parser doesn't like first code block here: https://github.com/yaptv/YapDatabase/wiki/LongLivedReadTransactions

Tried some workarounds but no luck so far.

Looks like Github bug for me, probably we just have to wait.

or at least ask someone at yaptv to verify my changes. I feel like I stepped on toes, just going in and editing! Well, the intention was good anyway.

There were references to the YapCollectionsDatabase, but I see that is no longer used (or, it's replaced YapDatabase). So I thought I would be a "good citizen" and remove those references. I hope my changes are decent, but I also hope someone double checks 'em.

Great tool, and great documentation. Thanks.

There's a new CocoaPods podspec for YapDatabase. Check it out when you get a chance and make sure eveything''s kosher.

Hi, I'm in the process of migrating an app over from core data to YapDatabase. I'm currently using Views to replace my NSFetchedResultsController and I've run into a snag. I have a list of events that are grouped by day. So I need a section, or group, for each day for which there is an event. The problem is that mappings require you to know beforehand the names of all the groups. Is there a way to do this using views and mappings?

Thanks.

autoFlushMemoryLevel is if-def'd out.

#if TARGET_OS_IPHONE
@property (atomic, assign, readwrite) int autoFlushMemoryLevel;
#endif

However, the usage of this property isn't defined out. This seems to be occurring here and here.

Could this fix be tagged as a new release? I would like to add OS X support to the podfile, but it would be great to have a new tag for this that isn't broken on OS X.

I did some benchmarks with the recent version of YapDatabase vs. CoreData.
The test was to insert 10k objects in to CoreData and YapDatabase on main thread. I was expecting the performance of YapDatabse significantly better than CoreData but CoreData took ~18.38 seconds whereas Yap took ~30.23 seconds on an iPhone 5 running iOS 7.0.4

The setup was:

• CoreData + NSFetchRequestController
• YapDatabase with View (no search)
• However If don't use Yapdb View and just insert these objects it took ~15.57 sec.

Then I added full text search to YapDatabse and it took ~32.29 sec. for the insertions.

I searched for a word with NSPredicate in CoreData and FullTextSearch in Yapdb:

• CoreData with NSPredicate ~0.37 sec.
• FTS in YapDatabase (Without snippets) ~0.83 sec.

YapDatabse's wiki states the performance of full text search almost 6x better than CoreData.

The test rig is available at: https://github.com/chaitanyapandit/YapBenchmarks

I'm getting this error printed a lot. Appears inconsistent:

YapCollectionsDatabaseConnection: Error creating 'getRowidForKeyStatement': 1 no such table: database2

To my knowledge, I have no 'database2' table.