rxswiftcommunity / rxgrdb Goto Github PK
View Code? Open in Web Editor NEWReactive extensions for SQLite
License: MIT License
Reactive extensions for SQLite
License: MIT License
Observation (fetch) on one thread is blocked by write operations on another thread. Only after all write operations are finished the SELECT is invoked at the very end.
Actually in old version: (Using GRDB.swift (0.109.0) Using RxGRDB (0.3.0) Using RxSwift (3.3.1)) if subscribing is performed in the same block (thread) then after each INSERT there is a SELECT as it should be. In the same situation in the new version it's not working as expected anymore.
Using GRDB.swift (2.10.0)
Using RxGRDB (0.10.0)
Using RxSwift 4.1.2
Example code:
import UIKit
import GRDB
import RxGRDB
import RxSwift
class Test : Record {
var id: Int?
var title: String?
required init(row: Row) {
id = row["id"]
title = row["title"]
super.init(row: row)
}
init(title: String) {
self.title = title
super.init()
}
override class var databaseTableName: String {
return "Test"
}
override func encode(to container: inout PersistenceContainer) {
container["id"] = id
container["title"] = title
}
}
class ViewController: UIViewController {
private let disposeBag = DisposeBag()
@IBOutlet weak var myLabel: UILabel!
override func viewDidLoad() {
super.viewDidLoad()
let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first! as NSString
let databasePath = documentsPath.appendingPathComponent("db.sqlite")
var config = Configuration()
config.trace = { print($0) }
let dbPool = try! DatabasePool(path: databasePath, configuration: config)
try! dbPool.write { db in
try db.execute("""
CREATE TABLE IF NOT EXISTS test (
id INTEGER PRIMARY KEY,
title TEXT NOT NULL)
""")
}
let processingQueue = DispatchQueue(label: "processing", qos: .userInitiated)
processingQueue.async {
do {
try dbPool.write { db in
for _ in 1...50000 {
let test = Test(title: "London")
try test.insert(db)
let londonId = db.lastInsertedRowID
print(londonId)
}
}
}
catch let error as NSError {
print( error.localizedDescription)
}
}
processingQueue.async {
Test.filter(Column("title") == "London").order(Column("id").desc).rx
.fetchOne(in: dbPool)
.subscribe(onNext: { test in
self.myLabel.text = "\(test?.id ?? 0)"
})
.disposed(by: self.disposeBag)
}
}
}
Hey @groue!
I've attached a sample project that illustrates the bug I'm describing here.
Run the test testExample
in the tests target of the project. You'll see that the call to fetching all Foo
objects succeeds outside before the PromiseKit done
block but it fails inside the block (there's a crash).
Here are some other behaviours I've noticed that should help:
Foo.all().rx.fetchAll(in: dbPool)
in the fooObservableViaPool
method does work as intended.let foos = try! database.fooObservableViaPool().toBlocking().first()!
let foos = try! database.fooObservableViaPool().subscribe(onNext:{ foos in /.../ })
0.7.0
to 0.10.0
. This issue didn't exist in 0.7.0
.Let me know if there's any more information you need from me. Thank you for your time! ๐
Hey, I've attached a sample project that demonstrates my use case: I'm trying to access objects from the database in the onNext
event of an observable (that I get from RxGRDB). This causes the app to crash with the 'Database is re-entrant' error.
This project doesn't crash though but the GRDB version I'm using with my Swift 3.2 project does. I was wondering why this is? Does the onNext
block hold on to a db object internally? If so, how can I access values from the database within a subscription's onNext
?
TestingRxRentrance.zip
When I build scheme use Carthage, it show error
This might be something I have missed in the documentation and sorry if it is.
I know in RxSwift, you can bind a UITextField to variable via:
textField.rx.text.orEmpty
.bind(to: self.someVariable)
.addDisposableTo(disposeBag)
My question is, does RxGRDB have a native way to handle a binding from a SQLite request to a textiField or should I just use a combination of the two?
I would like to share a pattern I am using to safely and easily wait for database initialization. I hope by sharing this it will help inform GRDB's design based on how it's being used and also catch anything that could be done by better utilizing GRDB's existing features.
I don't want any queries to execute before the db migration or loading of initial data set is complete. I also don't want to naively block the whole app while waiting. Without reactive programming this could get very messy. To accomplish this I wrap access to the db queue through an observable/signal so that the only way to get a queue is by observing the signal that creates it.
final class AppDatabase {
let queue: DatabaseQueue
// This is the only way to get an instance of the database queue
// This signal never completes and only fires one event which contains an instance of ready AppDatabase
// If you have an instance of AppDatabase then it's safe to query against it
static func create(
path: String,
wipeDatabase: Bool = false,
application: UIApplication? = nil
) -> Signal<AppDatabase, AnyError> {
return SafeSignal.trying { try AppDatabase(path, wipeDatabase, application) }
.executeOn(.global(qos: .background))
.map(migration)
.map(importData)
.shareReplay(limit: 1)
}
private init(
_ path: String,
_ wipeDatabase: Bool = false,
_ application: UIApplication? = nil
) throws {
print("Database path: " + path)
if wipeDatabase {
print("Wiping away previous database...")
try? FileManager.default.removeItem(atPath: path)
}
var config = Configuration()
#if DEBUG
config.trace = { print($0) }
#endif
self.queue = try DatabaseQueue(path: path, configuration: config)
if let application = application {
self.queue.setupMemoryManagement(in: application)
}
}
}
private func migration(db: AppDatabase) throws -> AppDatabase {
var migrator = DatabaseMigrator()
...
try migrator.migrate(db.queue)
return db
}
private func importData(adb: AppDatabase) throws -> AppDatabase {
...
return adb
}
// All queries as defined via extensions to AppDatabase
extension AppDatabase {
func fetchDailyActivity(to date: Date) -> Signal<[DailyActivity], AnyError> {
...
}
}
Now all db queries look something like this:
lazy var fetch = Env.database // once migration is complete we get a db instance (Env is a global, this should also have been done via dependency injection)
.combineLatest(with: self.today) // input for the query, so when this changes we end up fetching again
.flatMapLatest { db, day in db.fetchDailyActivity(to: day) } // The actual query
In the above, the query wont be executed until the database is ready. Additionally, since the input is also a signal, the query will be re-run when the input changes (like when significantTimeChange fires for example).
Some migrations may be fast, some larger migrations may take a while. A polite way to proceed is to only show a loading indicator if the migration takes "long enough". If the migration completes before the deadline no loading indicator should be shown. This will prevent the "quick flash" loading screen many apps have. The signal below is used in the root view controller to display a full screen loading indicator when a large migration takes place.
private let deadline = Property(())
.toSignal()
.delay(interval: 0.5)
/*
Signal that the database is currently busy so that the UI can be blocked.
This signal emits a value only if the database isn't ready before the deadline.
If the debase becomes ready before the deadline the signal simply completes.
*/
lazy var databaseBusy = deadline.take(until: Env.database)
.materialize()
See initial request from @hartbit: groue/GRDB.swift#211
I'm trying to use module RxGRDB, but I have a problem. My project deployment target is 10.0, but requirements of module says that minimum is ios8.0
Requirements: iOS 8.0+ / OSX 10.10+ / watchOS 2.0+ โข Xcode 8.3+ โข Swift 3.1
Error log:
Error:(10, 8) module file's minimum deployment target is ios10.3 v10.3: /Users/.../Carthage/Build/iOS/RxGRDB.framework/Modules/RxGRDB.swiftmodule/x86_64.swiftmodule
Does RxGRDB support joined requests? For example, I have 2 structures called Flight and Aircraft as defined below,
struct Flight {
var FLIGHT_ID: String
var AC_ID: String
...
}
struct Aircraft{
var AC_ID: String
var AC_REGISTRATION: String
...
}
I would like to bind the value of AC_REGISTRATION to a textField based on a given value of FLIGHT_ID.
Is there any way to do this in RxGRDB?
Hey!
I'm using a piece of code that looks something like this:
dbQueue.rx.changes(in: [Project.all()])
.observeOn(ConcurrentDispatchQueueScheduler(qos: .userInitiated))
.map { db in
let foo = Row.fetchAll(db, mySQLQuery)
...
}
I get a crash that reads 'Database was not used on the correct thread'. From what I understand, I am using the db
object on a separate thread than the one it was initiated on (dbQueue
).
But I wanted to know if there is a way to achieve what I'm trying to do because the work I'm doing in my map
block is something I'd like to get off the main thread.
Any help is appreciated, thanks!
In case you are interested in collaborating or taking a look, I made a quick and dirty port of RxGRDB for ReactiveKit.
Since the libraries aren't that different a lot of work was mechanical though some areas required more thought such as dealing with the explicit error type in ReactiveKit (I took the lazy way out for now and defined an AnyError
type). I got all unit tests to pass except for testPrimaryKeyDiffScanner
(still trying to figure out what's goin on with that one).
When we upgraded from 0.12.0 to 0.13.0 we started experiencing slowness in simple queries. What can I do to troubleshoot this or what more information could I provide?
Attempting to add RxGRDB to a medium sized macOS app causes errors to methods that use FTS5:
"Use of unresolved identifier 'FTS5'" and "Value of type 'Database' has no member 'makeFTS5Pattern'"
The application tests, compiles and runs without RxGRDB.
Hi @groue , Thx for your awesome work . Do you have any plan to update the RxGRDB to Swift4.0?
This question can also apply to plain GRDB without Rx.
I have a lot of complex queries that involve multiple statements in the form of "setup", "query" then "teardown". For example, many times I find myself creating a few temp tables, adding indexes, querying, then dropping the tmp tables.
var result: [MyRecordType] = []
try db.inTransaction {
try db.execute("""
CREATE TEMPORARY TABLE tmpTable1 ...;
CREATE TEMPORARY TABLE tmpTable2 ...;
CREATE INDEX tmpTable1 ...;
CREATE INDEX tmpTable2 ...;
""")
result = try MyRecordType.fetchAll(db, "SELECT * FROM tmpTable1 JOIN tmpTable2 ...;")
try db.execute("""
DROP TABLE IF EXISTS tmpTable1;
DROP TABLE IF EXISTS tmpTable2;
""")
return .commit
}
return result
There's no way to neatly pack the whole thing into a fetch request or even a SQLQuery
so I end up putting the above code into a get(_ db: Database)
method somewhere. This felt clumsy but worked until I wanted to use the Rx extensions to get notified of changes.
How would I do this with RxGRDB and ensure the temp tables are created and destroyed each time the query is ran for a change?
Hi @groue, sorry for bothering you, but I wonder if you know a possible workaround for a problem I'm facing with using RxGRDB as a git submodule:
๐ฅ Build phase [CP] Check Pods Manifest.lock
fails with
diff: /Podfile.lock: No such file or directory
diff: /Manifest.lock: No such file or directory
error: The sandbox is not in sync with the Podfile.lock. Run 'pod install' or update your CocoaPods installation.
โ
But if I delete this build phase ([CP] Check Pods Manifest.lock
), all is good, everything compiles just fine (after I add and build RxSwift and GRDB.swift submodules).
But that means that I modify the RxGRDB
submodule's source code, which I'm not sure is the cleanest approach ...
UPDATE
I've also found this stackoverflow thread https://stackoverflow.com/questions/31901347/restkit-giving-error-after-installation-as-a-git-submodule which suggested some workarounds, will try them out and report back.
pod repo remove master && pod setup
didn't work (not even sure it should've)pod update
inside RxGRDB
submodule causes the build to fail with Apple Mach-O Linker Error
ld: warning: directory not found for option '-F/Users/asd/Library/Developer/Xcode/DerivedData/demo-fwpwyrqayjjgnsabixvjddmcgyto/Build/Products/Debug-iphonesimulator/GRDB.swift-iOS'
ld: warning: directory not found for option '-F/Users/asd/Library/Developer/Xcode/DerivedData/demo-fwpwyrqayjjgnsabixvjddmcgyto/Build/Products/Debug-iphonesimulator/RxSwift-iOS'
ld: framework not found Pods_RxGRDBiOS
clang: error: linker command failed with exit code 1 (use -v to see invocation)
pod install
would probably cause a linker error just like 2.Hey @groue :) I feel like I'm bothering you more than usual in the last few days, but I think I've found a possible minor bug in RxGRDB specifically.
In my test target, I inject an in-memory DatabaseQueue
to fetch some order information. I noticed that fetchOne
never emits anything specifically in test targets, but it does in the main target. This is even though the order exists (as confirmed by imperatively fetching it).
I tried switching from in-memory to path-based but that didn't provide any change
print("queue -> \(databaseQueue)")
let orderObj = try! databaseQueue.read { db in
return try Order.fetchOne(db, key: orderID)
}
print("order -> \(orderObj)")
let order = Order.filter(key: orderID)
.rx.fetchOne(in: databaseQueue)
.debug("fetch")
... more irrelevant code down here ...
Prints out
queue -> GRDB.DatabaseQueue
order -> Optional(MyApp.Order(id: "CE37F0B2-E25E-4898-942D-8DC5D00C5D0F", ... more info ...))
fetch -> subscribed
As you can see, reading from the database queue imperatively returns the correct object, but observing with fetchOne
doesn't seem to product the desired effect.
Even if the object wouldn't exist, I would expect a nil
emission instead of nothing.
Any thoughts on what could cause this?
Thanks!
Shai.
Hi Groue,
I've been making fair use of your library so far, implementing an example for the DatabaseRegionConvertible protocol to have observable regions for joined queries. And it has come to my attention that v0.13.0 it no longer supports the fetch:from: method. Instead it now tells us to use the ValueObservation struct.
I had the DatabaseRegionConvertible protocol implement in a few places and naturally the breaking changes made me have to revisit these to implement them using the ValueObservation struct.
My first reaction however was to port back, using Cocoapods, to a previous version of RxGRDB to keep using the DatabaseRegionConvertible protocol and look at a way to implement the struct later.
However the podspec v0.11.0 to 0.12.1 of RxGRDB have a dependency on GRDB or GRDBCipher for version '~> 3.0', which picks up the latest version being 3.5.
In these versions RxGRDB still has the DatabaseRegionConvertible.swift file which implements the Protocol. But now the GRDB and GRDBCipher versions also contain an implementation of the protocol, which gives a bunch of 'Ambigious' type errors.
So basically my question is if the dependencies in the cocoapod files for these versions could be set to a specific version of the GRDB or GRDBCipher.
I suppose I could have made a pull request for this change, but I don't feel comfortable doing this yet. And it seems to me this change would need to be pushed towards the cocoapods reposit as well. My apologies for this. I hope my story made some sense and didn't mess up the terminology of the project to much.
With Xcode now having built-in support for the Swift Package Manager, it would be nice if this project had a Package.swift
so that it is installable using that (both RxJS and GRDB already have Swift Package Manager support).
After the recent increase in the iOS deployment target of GRDB.swift, RxGRDB doesn't compile
Compiling for iOS 10.0, but module 'GRDB' has a minimum deployment target of iOS 11.0
Probably, the deployment target for RxGRDB should be raised as well
GRDB.swift (5.12.0)
RxGRDB (2.0.0)
Xcode 13.0 (13A233)
COCOAPODS: 1.11.2
Hi @tonyarnold
I have a table with 3 columns but its model has 4 properties, What should I do in this case? because when I set insert query I got this error:
Fatal error: 'try!' expression unexpectedly raised an error: SQLite error 1 with statement
INSERT INTO "student" ("id", "name", "lastName", "age") VALUES (?,?,?,?)
: table student has no column named age!
My table columns are: id, name, lastName
My model properties: id, name, lastName, age
is there any protocol that I can fix this problem?
With the release of GRDB 4.1.0 a few days ago, the framework seems to no longer compile, with an error in ValueObservation+Rx.swift
on line 17 saying:
Type 'ValueObservation<Reducer>' does not conform to protocol '_ValueObservationProtocol'
The prompt attempts to insert the func start(i: _, onError: _, onChange: _)
defined in the _ValueObservationProtocol
. This protocol requirement appears to mirror a similar function in GRDB found in ValueObservation.swift
which was updated with the release of 4.1.0
to no longer throw an error, which looks to have broken protocol conformance.
Hi
First, thank you for this amazing library, it is really good and helps a lot.
I would like to know on which thread the actual fetching of the data is made ?
Actually in our project we are using GRDB.
Now I am working on integrating RxSwift and RxGRDB.
let say that I have this:
//On the main thread.
SQLRequest<String>(sql: query)
.rx
.fetchAll(in: databasePool)
.do(onNext: { /* do amazing stuff */ย })
.subscribe()
.dispose(by: disposableBag)
I understand that the emited values will be on the main thread which is good to me and even if it is not, the observeOn operator can help me change this :).
So I am wandering, Is the fetching is done on the main thread it self ? Main thread asynchronously ?
I try something like this after:
//On main thread
print("BEFORE")
SQLRequest<String>(sql: queryBuilder.query)
.rx
.fetchAll(in: databasePool)
.do(onNext: { _ in print("DONEXT") })
.subscribe()
.dispose(by: disposableBag)
print("AFTER")
I got printed:
BEFORE
AFTER
DONEXT
I wanted an answer to be sure about what I am doing and be sure about what is going on under the hood :).
for information I am user DatabasePool for the DB connection.
Thank you for the help
Got this error
Compiling for iOS 9.0, but module 'GRDB' has a minimum deployment target of iOS 10.0: /Users/josephcheung/Library/Developer/Xcode/DerivedData/NavBarExperiment-bnwxzmahhyddhzeiwgfjglhjhare/Build/Products/Release-iphonesimulator/GRDB.swift/GRDB.framework/Modules/GRDB.swiftmodule/x86_64-apple-ios-simulator.swiftmodule
I think we need to bump ios target in RxGRDB to 10.0 as well https://github.com/RxSwiftCommunity/RxGRDB/blob/master/RxGRDB.podspec#L13
I'm using SQLCipher with GRDB via cocoapods:
pod 'GRDB.swift/SQLCipher'
pod 'SQLCipher', '~> 4.0'
(from my Podfile)
This works fine. When I add RxGRDB to my setup:
pod 'RxGRDB'
I get this error during database initialization:
SQLite error 21: GRDB is not linked against SQLCipher. Check https://discuss.zetetic.net/t/important-advisory-sqlcipher-with-xcode-8-and-new-sdks/1688
I followed the referenced manual but unfortunately it doesn't help.
Any ideas how this can be fixed?
I'm trying to represent an ordered list of persons in a UITableView section based on rxGRDB's events. The integer order table field defines a sorting order, the table uses soft-delete to handle changes. Following the example in rxGRDB's readme, the PrimaryKeyDiffScanner informs about changes in the persons table.
Does its implementation limit order by id only? How feasible is it to make a SimpleDiffScanner
? Would the differ in GRDB (used to represent UITableView as well) be a good start for diffing collections?
I don't understand the result for the following code snippet; I would expect 0 inserts and deletes, while 4 inserts and deletes are printed (4 being the db table actual row count).
// ideally I want to use this request to get an ordered list of non-delete
// let personsRequest = Person.order(Column("order")).filter(Column("deleted") == nil)
// but first try to get a simple to work
let personsRequest = Person.order(Column("id"))
var initialPersons: [Person]?
do {
initialPersons = try DelayDb.queue.inDatabase { database in
try personsRequest.fetchAll(database)
}
let scanner = try DelayDb.queue.inDatabase { database in
try PrimaryKeyDiffScanner(
// **** I would expect inserting the initial table would result in no insertions/deletions later ****
database: database, request: personsRequest, initialRecords: initialPersons!)
}
let rowRequest = personsRequest.asRequest(of: Row.self)
// The scanner is designed to feed the built-in RxSwift `scan` operator:
rowRequest.rx
.fetchAll(in: DelayDb.queue)
.scan(scanner) { (scanner, rows) in
// print(rows.map { $0.description } ) // I'm indeed seeing the right 'Row' values printed ...
return scanner.diffed(from: rows)
}
.subscribe(onNext: { scanner in
let diff = scanner.diff
print("inserted \(diff.inserted.count) records") // expect 0 iso 4
print("updated \(diff.updated.count) records")
print("deleted \(diff.deleted.count) records") // expect 0 iso 4
})
Output:
inserted 4 records
updated 0 records
deleted 4 records
Thanks for any clarifications on what I do wrong ... (using the cocoa pod version of rxGRDB)
Hey,
is there a reason why the scheduler overloads are restricted to a SerialDispatchQueueScheduler?
I want to setup a unit test with an inMemory SQLite GRDB instance and somehow i can't get it to work. I'm not too familiar with GRDB/Scheduler internals so i don't know if this is even possible? Do you have any thoughts on this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.