Giter Club home page Giter Club logo

objective-leveldb's Introduction

CircleCI

Introduction

An Objective-C database library built over Google's LevelDB, a fast embedded key-value store written by Google.

Installation

By far, the easiest way to integrate this library in your project is by using CocoaPods.

  1. Have Cocoapods installed, if you don't already

  2. In your Podfile, add the line

     pod 'Objective-LevelDB'
    
  3. Run pod install

  4. Make something awesome.

How to use

Creating/Opening a database file on disk

LevelDB *ldb = [LevelDB databaseInLibraryWithName:@"test.ldb"];
Setup Encoder/Decoder blocks

By default, any object you store will be encoded and decoded using NSKeyedArchiver/NSKeyedUnarchiver. You can customize this by providing encoder and decoder blocks, like this:

ldb.encoder = ^ NSData * (LevelDBKey *key, id object) {
  // return some data, given an object
}
ldb.decoder = ^ id (LevelDBKey *key, NSData * data) {
  // return an object, given some data
}
NSMutableDictionary-like API
ldb[@"string_test"] = @"laval"; // same as:
[ldb setObject:@"laval" forKey:@"string_test"];

NSLog(@"String Value: %@", ldb[@"string_test"]); // same as:
NSLog(@"String Value: %@", [ldb objectForKey:@"string_test"]);

[ldb setObject:@{@"key1" : @"val1", @"key2" : @"val2"} forKey:@"dict_test"];
NSLog(@"Dictionary Value: %@", [ldb objectForKey:@"dict_test"]);

All available methods can be found in its header file (documented).

Enumeration
[ldb enumerateKeysAndObjectsUsingBlock:^(LevelDBKey *key, id value, BOOL *stop) {
    // This step is necessary since the key could be a string or raw data (use NSDataFromLevelDBKey in that case)
    NSString *keyString = NSStringFromLevelDBKey(key); // Assumes UTF-8 encoding
    // Do something clever
}];

// Enumerate with options
[ldb enumerateKeysAndObjectsBackward:TRUE
                              lazily:TRUE       // Block below will have a block(void) instead of id argument for value
                       startingAtKey:someKey    // Start iteration there (NSString or NSData)
                 filteredByPredicate:predicate  // Only iterate over values matching NSPredicate
                           andPrefix:prefix     // Only iterate over keys prefixed with something 
                          usingBlock:^(LevelDBKey *key, void(^valueGetter)(void), BOOL *stop) {
                             
    NSString *keyString = NSStringFromLevelDBKey(key);
    
    // If we had wanted the value directly instead of a valueGetter block, we would've set the 
    // above 'lazily' argument to FALSE
    id value = valueGetter();
}]

More iteration methods are available, just have a look at the header section

Snapshots, NSDictionary-like API (immutable)

A snapshot is a readonly interface to the database, permanently reflecting the state of the database when it was created, even if the database changes afterwards.

LDBSnapshot *snap = [ldb newSnapshot]; // You get ownership of this variable, so in non-ARC projects,
                                       // you'll need to release/autorelease it eventually
[ldb removeObjectForKey:@"string_test"];

// The result of these calls will reflect the state of ldb when the snapshot was taken
NSLog(@"String Value: %@", [snap objectForKey:@"string_test"]);
NSLog(@"Dictionary Value: %@", [ldb objectForKey:@"dict_test"]);

All available methods can be found in its header file

Write batches, atomic sets of updates

Write batches are a mutable proxy to a LevelDB database, accumulating updates without applying them, until you do using -[LDBWritebatch apply]

LDBWritebatch *wb = [ldb newWritebatch];
[wb setObject:@{ @"foo" : @"bar" } forKey: @"another_test"];
[wb removeObjectForKey:@"dict_test"];

// Those changes aren't yet applied to ldb
// To apply them in batch, 
[wb apply];

All available methods can be found in its header file

LevelDB options
// The following values are the default
LevelDBOptions options = [LevelDB makeOptions];
options.createIfMissing = true;
options.errorIfExists   = false;
options.paranoidCheck   = false;
options.compression     = true;
options.filterPolicy    = 0;      // Size in bits per key, allocated for a bloom filter, used in testing presence of key
options.cacheSize       = 0;      // Size in bytes, allocated for a LRU cache used for speeding up lookups

// Then, you can provide it when initializing a db instance.
LevelDB *ldb = [LevelDB databaseInLibraryWithName:@"test.ldb" andOptions:options];
Per-request options
db.safe = true; // Make sure to data was actually written to disk before returning from write operations.
[ldb setObject:@"laval" forKey:@"string_test"];
[ldb setObject:[NSDictionary dictionaryWithObjectsAndKeys:@"val1", @"key1", @"val2", @"key2", nil] forKey:@"dict_test"];
db.safe = false; // Switch back to default

db.useCache = false; // Do not use DB cache when reading data (default to true);
Concurrency

As Google's documentation states, updates and reads from a leveldb instance do not require external synchronization to be thread-safe. Write batches do, and we've taken care of it, by isolating every LDBWritebatch it inside a serial dispatch queue, and making every request dispatch synchronously to it. So use it from wherever you want, it'll just work.

However, if you are using something like JSONKit for encoding data to JSON in the database, and you are clever enough to preallocate a JSONDecoder instance for all data decoding, beware that this particular object is not thread-safe, and you will need to take care of it manually.

Testing

If you want to run the tests, you will need Xcode 5, as the test suite uses the new XCTest.

Clone this repository and, once in it,

./setup-test.sh
cd Tests && open Objective-LevelDB.xcworkspace

Currently, all tests were setup to work with the iOS test suite.

License

Distributed under the MIT license

objective-leveldb's People

Contributors

cybertk avatar emilwojtaszek avatar hoisie avatar iamdoron avatar looping avatar lsm avatar matehat avatar readmecritic avatar russellhancox avatar xinmyname avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

objective-leveldb's Issues

pod 'Objective-LevelDB' compile error

we use Objective-LevelDB in our project, after we add pod 'Objective-LevelDB' success, we build our project, there are compile errors as below
f2662256-0db8-4c72-a436-c8a6651040ca

please help, these problems have puzzled me for Several Days

support for Carthage

will there be an option to install this via Carthage?
Cocoapods is super easy, but id much prefer to just drop a Framework into my project, and Cartage makes this super easy.

LevelDB instance gets overreleased under high contention

As someone else already mentioned in one of the other issues, LevelDB.mm seems to be missing some retains. I was getting unexplainable crashes for about 0.3% of my users and it turns out that the db ivar in LevelDB.mm is getting released even though a strong reference to its parent , and in turn itself exists when used across a multitude of threads with a very high volume of reads. Ironically, the issue gets worse when writes are synced on the wrapper.

The issue does get resolved when I move the three .mm files to ARC - which I thought would be easier to do than further debug the overrelease. Maybe the .mm files should just be moved to arc for good?

.../CURRENT: Operation not permitted

"IO error: /var/mobile/Containers/Data/Application/B8C7E31C-AE41-4C00-AA807458AFA6024E/Documents/ShareStore/DataBase/CURRENT: Operation not permitted"

Errors encountered when opening the database,Is there any solution?

setNilValueForKey

[<CGUserDataModel 0x280304ff0> setNilValueForKey]: could not set nil as the value for the key sex111

adding attributes dynamically, reloading stored data, and reporting program errors

prefix not working?

why this snippet output nothing? what is the correct way of using this method?

    LevelDB *db = [LevelDB databaseInLibraryWithName:@"test.ldb"];
    db.encoder = ^ NSData *(LevelDBKey *key, id object) {
        return [NSKeyedArchiver archivedDataWithRootObject:object];
    };
    db.decoder = ^ id (LevelDBKey *key, NSData *data) {
        return [NSKeyedUnarchiver unarchiveObjectWithData:data];
    };

    [db setObject:@"NewYork" forKey:@"foo:1"];
    [db setObject:@"Paris" forKey:@"foo:2"];
    [db setObject:@"Shenzhen" forKey:@"foo:3"];

    [db enumerateKeysBackward:YES startingAtKey:nil filteredByPredicate:nil andPrefix:@"foo" usingBlock:^(LevelDBKey *key, BOOL *stop) {
        NSLog(@"key:%@", NSStringFromLevelDBKey(key));
        NSString *value = [db objectForKey:NSStringFromLevelDBKey(key)];
        NSLog(@"value:%@", value);
    }];

Not able to use LDBWritebatch

It seems the class LDBWritebatch is not recognised in an Objective-C file. When i try to make the same tests you make in the doc an error appears.
Even if i add @LDBWritebatch in the imports zone i cannot access to the class methods.

add default encoder and decoder for convenience.

db.encoder = ^ NSData *(LevelDBKey *key, id object) {
    return [NSKeyedArchiver archivedDataWithRootObject:object];
};
db.decoder = ^ id (LevelDBKey *key, NSData *data) {
    return [NSKeyedUnarchiver unarchiveObjectWithData:data];
};

Broken Podspec?

I'm trying to fork this repo for use with Facebook/RocksDB, but I'm having problems verifying that this podspec passes lint.

I clone the repo, and init/update the submodule. Then...
$ pod lib lint --verbose

(6 failures)
-> Objective-LevelDB (2.1.0)
- ERROR | [Objective-LevelDB/leveldb] [xcodebuild] /Users/i/Downloads/Objective-LevelDB/leveldb-library/util/arena.cc:5:10: fatal error: 'util/arena.h' file not found
- ERROR | [Objective-LevelDB/leveldb] [xcodebuild] /Users/i/Downloads/Objective-LevelDB/leveldb-library/table/block.cc:7:10: fatal error: 'table/block.h' file not found
- ERROR | [Objective-LevelDB/leveldb] [xcodebuild] /Users/i/Downloads/Objective-LevelDB/leveldb-library/table/block_builder.cc:29:10: fatal error: 'table/block_builder.h' file not found
- ERROR | [Objective-LevelDB/leveldb] [xcodebuild] /Users/i/Downloads/Objective-LevelDB/leveldb-library/util/bloom.cc:8:10: fatal error: 'util/hash.h' file not found
- NOTE | [Objective-LevelDB/leveldb] [xcodebuild] BuildHeaders/Objective-LevelDB/leveldb/slice.h:97:43: warning: implicit conversion loses integer precision: 'const size_t' (aka 'const unsigned long') to 'const int' [-Wshorten-64-to-32]
- NOTE | [Objective-LevelDB/leveldb] [xcodebuild] BuildHeaders/Objective-LevelDB/leveldb/slice.h:97:53: warning: implicit conversion loses integer precision: 'const size_t' (aka 'const unsigned long') to 'const int' [-Wshorten-64-to-32]
- ERROR | [Objective-LevelDB/leveldb] [xcodebuild] /Users/i/Downloads/Objective-LevelDB/leveldb-library/db/builder.cc:5:10: fatal error: 'db/builder.h' file not found
- ERROR | [Objective-LevelDB/leveldb] [xcodebuild] /Users/i/Downloads/Objective-LevelDB/leveldb-library/util/cache.cc:10:10: fatal error: 'port/port.h' file not found

I've tried to work around this by:

  • adding a header_mappings_dir to the subspec
  • adding /util, /db, /table, etc. to source_files
  • adding additional dirs to HEADER_SEARCH_PATHS

I feel like I'm missing something bigger... Thoughts?

pod --version, 0.33.1

Multi-process support on iOS

Hi,

Apologies for posting a query as a bug.
Does this support multiple process since iOS (on iPhone) spawns multiple processes trying to access the same db?

Regards,
Nirendra

Swift usage error about encodeWithCoder

I am using Objective-LevelDB in my Swift project, I implementation the encoder and decoder like this, there is nothing error with it.

self.levelDB.encoder = {(key: UnsafeMutablePointer<LevelDBKey>, object: ImplicitlyUnwrappedOptional<AnyObject>) -> NSData! in
     return NSKeyedArchiver.archivedDataWithRootObject(object)
} as LevelDBEncoderBlock

self.levelDB.decoder = {(key: UnsafeMutablePointer<LevelDBKey>, data: ImplicitlyUnwrappedOptional<NSData>) -> AnyObject! in
     return NSKeyedUnarchiver.unarchiveObjectWithData(data)!
} as! LevelDBDecoderBlock

But when I run this code, it crashes.

let object = NSObject()
self.levelDB.setObject(object, forKey: "key")

The error message is :

2016-01-13 19:41:15.157 LevelDBDemo[24442:9950819] -[NSObject encodeWithCoder:]: unrecognized selector sent to instance 0x7f968e0c28a0
2016-01-13 19:41:15.166 reason[24442:9950819] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSObject encodeWithCoder:]: unrecognized selector sent to instance 0x7f968e0c28a0'

How to deal with it?

Has a minimum requirement of iOS 7.0 - OS X 10.7?

Are you serious, next update version requires iOS 7.0?
By using pod 'Objective-LevelDB', :git => 'https://github.com/matehat/Objective-LevelDB.git', I got an error shows below when running pod update,

[!] The platform of the target `Pods` (iOS 6.0) is not compatible with `leveldb (1.17.1)` which has a minimum requirement of iOS 7.0 - OS X 10.7.

I don't think leveldb needs iOS 7 APIs.
This issue should be well fixed.

disable assert in release mode

There are a lot of asserts in LevelDB, and XCode doesn't disable these asserts in release mode by default.
IMO, Objective-LevelDB should add NDEBUG=1 definition in podspec to disable these asserts.

Propose adding TTL for values

What are your thoughts on adding TTL to this project? I'm using this in my own app and have something I could merge upstream in the next few weeks.

I think adding a timeout on keys would be a huge feature, and while a little high level imo, fits the scope of this library. What do you think?

Cannot `git clean`

When invoking git clean in project dir, failed with

fatal: Not a git repository: Pods/Objective-LevelDB/leveldb-library/../.git/modules/leveldb-library

Linker command failed - duplicate symbol _main for architecture x86_64

I'm getting this linking issue even with a default empty project.

Deployment target 8.3
Using Objective-LevelDB (2.1.4) installed via Cocoapods
Using leveldb-library (1.18.1) installed via Cocoapods

Here's the Podfile:

platform :ios, '8.0'
use_frameworks!

target 'levelDB_test' do
  pod 'Objective-LevelDB'
end

target 'levelDB_testTests' do
  pod 'Objective-LevelDB'
end

Here's the error log:

duplicate symbol _main in:
    /Users/luca/Library/Developer/Xcode/DerivedData/levelDB_test-gqscfpqoazdhylfthcxwbyirjqjw/Build/Intermediates/Pods.build/Debug-iphonesimulator/Pods-levelDB_test-leveldb-library.build/Objects-normal/x86_64/db_bench.o
    /Users/luca/Library/Developer/Xcode/DerivedData/levelDB_test-gqscfpqoazdhylfthcxwbyirjqjw/Build/Intermediates/Pods.build/Debug-iphonesimulator/Pods-levelDB_test-leveldb-library.build/Objects-normal/x86_64/leveldb_main.o
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Any suggestions will be appreciated.

Thanks.

Cheers,
Luca

Enumeration that are both prefixed and offset aren't position correctly

Here is an example of the fully parametrized enumeration API:

- (void) enumerateKeysAndObjectsBackward:(BOOL)backward
                                  lazily:(BOOL)lazily
                           startingAtKey:(id)key
                     filteredByPredicate:(NSPredicate *)predicate
                               andPrefix:(id)prefix
                              usingBlock:(id)block;

The startingAtKey and prefix parameters can both be used to offset the starting key used in enumerating a leveldb instance. Their semantic is different though: prefix also restricts the end of the enumeration, so that the enumerated keys always contain the prefix, while startingKey only restricts the key where the enumeration begins.

In the current behavior, if a prefix is provided, the starting key is ignored. This is shouldn't be the case if the starting key honors the prefix restriction. In that case, it should be possible to specify where, in the prefixed region to start enumeration.

When filtering / enumerating the objects that don't match the predicate are not released

I am trying to enumerate a database finding only the values that match a given predicate.

Here is my code:

        NSPredicate *predicate = [NSPredicate predicateWithFormat:@"ANY SELF.names BEGINSWITH[cd] %@", searchText];
        @autoreleasepool {
            [self.datastore enumerateKeysAndObjectsBackward:NO
                                                     lazily:NO
                                              startingAtKey:nil
                                        filteredByPredicate:predicate
                                                  andPrefix:nil
                                                 usingBlock:^(LevelDBKey *key, id value, BOOL *stop){
                                                         [self.searchResults addObject:value];
                                                 }];
        }
        [[NSOperationQueue mainQueue] addOperationWithBlock:^{
            [self.searchDisplayController.searchResultsTableView reloadData];
        }];

The thing is that while enumerating Objective-LevelDB keeps all values in memory, even the ones that don't match the predicate. Not even lazy evaluation solves the issue here.

Am I doing something wrong? Can something be done to improve on memory usage?

The dbName is ignored

The dbName is ignored and LevelDB treats the path as a place to store file.
Thus when delete db,the whole path is deleted

Calling encoder block twice!

When using - (void) setObject:(id)value forKey:(id)key, encoder block would be called twice. It's inefficiency when encoding a large data.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.