Giter Club home page Giter Club logo

rethinkdb-rs's Introduction

RethinkDB Rust driver

Overview

What is RethinkDB?

RethinkDB is the first open-source scalable database built for realtime applications. It exposes a new database access model -- instead of polling for changes, the developer can tell the database to continuously push updated query results to applications in realtime. RethinkDB allows developers to build scalable realtime apps in a fraction of the time with less effort.

rethinkdb-rs's People

Contributors

allensh12 avatar asakatida avatar atnnn avatar brettgriffin avatar chipotle avatar coffeemug avatar dalanmiller avatar danielcompton avatar danielmewes avatar deontologician avatar dependabot-support avatar dineyw23 avatar epc avatar linkyndy avatar mafrosis avatar mglukhovsky avatar mlucy avatar neumino avatar pilwon avatar raspo avatar rushmorem avatar tavurth avatar thejsj avatar totalkrill avatar tryneus avatar ultrabug avatar untra avatar wkich avatar yoeori avatar zackboe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rethinkdb-rs's Issues

client error; invalid type: map, expected a string while reading from a stream

Hi, I'm learning how to use RethinkDB and I ran into a weird error.

Code:

use reql::cmd::connect::Options;
use reql::{r, Error, Session};
use futures::stream::{StreamExt};

const DB_NAME: &str = "my_custom_db";

#[tokio::main]
async fn main() -> Result<(), Error> {
    let mut cfg = Options::default();
    cfg = cfg.db(DB_NAME);
    cfg = cfg.host("localhost");
    cfg = cfg.port(28015);

    println!("Starting...");
    let db = r.connect(cfg).await?;
    println!("Connected!");

    let mut result = r.db_create(DB_NAME).run::<&Session, String>(&db);
    while let Some(x) = result.next().await {
        match x {
            Ok(y) => println!("SUCCESS: {}", y),
            Err(e) => println!("ERROR: {}", e),
        }
    }

    let mut result = r.table_create("users").run::<&Session, String>(&db);
    while let Some(x) = result.next().await {
        match x {
            Ok(y) => println!("SUCCESS: {}", y),
            Err(e) => println!("ERROR: {}", e),
        }
    }

    db.connection()?.close(()).await?;
    Ok(())
}

The output:

Starting...
Connected!
ERROR: client error; invalid type: map, expected a string
ERROR: client error; invalid type: map, expected a string

And the weird part: the database and table are being created:
image

Am I doing something wrong?
Thanks!

order_by with desc order #question

Hello,

Is it possible to pass args to order_by to retrieve desc order by index?
Something lik4 r.order_by(index = r.desc('time')) // python

Thank you,

next branch delivery plan

I noticed next branch supports futures ^0.3. Is there any plan to release an updated crate with such improvements?

How to connect with non default params

I've tried to connect with non default params but I can't create a Config object since the retries field is private. Also I can't implement a constructor since I'm working outside the crate

fn load<'a>(socket: SocketAddr, db: &'a str, port: &'a str, password: &'a str, retries:u64) -> Config<'a> {
    Config {
        servers: vec![socket],
        db: "test",
        user: "admin",
        password: "",
        retries: retries,
        #[cfg(feature = "tls")]
        tls: None,
    }
}

Is there something that I'm missing?

`tableList` missing

I can't find tableList or table_list anywhere in the r struct. Is it possible to use?

Can not set options to create a table

I'm not sure if I'm missing something, db("abc").table_create only takes one argument and it's a string whereas rethinkdb should normally take the table name and options. I'm trying to set the primary key, if there's another way please let me know.

Docs

Having a hard time with the docs and examples. Can anyone point me to examples or tutorials for newbies like me. Like creating a database and table using the rethinkdb-rs driver. That would really help

How to correctly use order_by and limit?

Hi.

I'm trying to write a fairly simple query that'll fetch the 2000 latest "records" (new to rethink so excuse me if I'm using incorrect terminology).

Lack of documentation made composing the query a bit challenging, but I came up with the following:

r.db("dbname").table("tablename").order_by(r.desc("time")).filter(reql::func!(|doc| -> reql::Command {
	doc.get_field("time").lt(r.now())
})).limit(2000).run(&session).try_next().await;

However, this doesn't compile because Command::limit doesn't accept an integer of any kind as a parameter, and Command::order_by doesn't accept whatever r::desc returns.

Could anyone provide a working example of how to use these methods?

Thanks!

driver client buggy

when trying to update something or replace an entry , I get Driver(Other ("Client is buggy (failed to serialize query.")))

Error during execution of `cargo metadata`: error: could not find `Cargo.toml`

After making an clean setup on Ubuntu 18.04 and trying to build the project the message bellow appears:

$ cargo make
[cargo-make] INFO - cargo make 0.28.0
[cargo-make] INFO - External file not found or is not a file, skipping.
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: default
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: init
[cargo-make] INFO - Running Task: pre-format
[cargo-make] INFO - Running Task: format
[cargo-make] INFO - Execute Command: "cargo" "fmt"
Error during execution of `cargo metadata`: error: could not find `Cargo.toml` in `/home/jaswdr/rethinkdb` or any parent directory

cargo-fmt 1.4.12
This utility formats all bin and lib files of the current crate using rustfmt.

USAGE:
    cargo fmt [FLAGS] [OPTIONS] [-- <rustfmt-options>...]

FLAGS:
        --all        Format all packages (only usable in workspaces)
    -q, --quiet      No output printed to stdout
    -v, --verbose    Use verbose output
        --version    Print rustfmt version and exit

OPTIONS:
        --manifest-path <manifest-path>      Specify path to Cargo.toml
        --message-format <message-format>    Specify message-format: short|json|human
    -p, --package <package>...               Specify package to format (only usable in workspaces)

ARGS:
    <rustfmt-options>...    Options passed to rustfmt

[cargo-make] ERROR - Error while executing command, exit code: 1
[cargo-make] WARN - Build Failed.

Platform I'm building on:

Ubuntu 18.4.1 x86_64

What I expected to happen:

Build finish successfully

What actually happened:

Stops on error

How to reproduce the problem:

  • make a fresh install of rust with rustup
  • install make and deny dependencies
  • try to run cargo make

Other informations

cargo version: cargo 1.43.0-beta (bda50510d 2020-03-02)
rustfmt version: rustfmt 1.4.12-beta (a828ffe 2020-03-11)

How to close a changes request?

My ws client can request a multiple changes query. I have a shared connection between all possible requests, which means that I have only 144 available connections to use. I'm pretty sure that connection is not freed when I have stopped consuming events from the changes query because when I have utilized all connections from the pool, I cannot open a new changes request.

My questions are - How can I close the changes connection? How can I limit pool size?

Code snipped is following:

let mut query = self.rethink_client.db(db_name).table(table_name);
let mut data_stream = query
            .changes()
            .with_args(args!({include_initial: true}))
            .run::<Change<InMessageRaw, InMessageRaw>>(self.rethink_connection)
            .unwrap()
            .compat();

  loop {
            tokio::select! {
                _ = &mut cancel_rx => {
                  info!("closing data_stream...");
                  break
                },
                Some(doc) = data_stream.next() => {
                      match doc {
                        Ok(doc) => {
                            match doc {
                                Some(reql::Document::Expected(change)) => {
                                    let message = change.new_val.unwrap();
                                    debug!("processing message for topic - '{}' with offset '{}'", message.topic, message.offset);
                                    in_tx.send(message).await;
                                }
                                Some(reql::Document::Unexpected(_)) => break,
                                None => break
                            }
                        }
                        Err(doc) => {
                            error!("{:?}", doc);
                            break
                        }
                    }
                }
            }
        }

Example of how to send struct using Arg::into_arg trait

Would be helpful to know how to implement the trait for a personal struct with mixed member types, i have not yet figured out how to do an insert into my database.

I saw that 'str' has IntoArg implemented as such i expected

let call = r.db("test")
    .table("table")
    .insert( "hello")
    .run::<ServerStatus>(conn).unwrap();

printing call yields:

Some(Err(Runtime(QueryLogic("Cannot convert STRING to SEQUENCE"))))

Aborting a call to changes() somehow.

Hi again

So I ran into another issue, which i cannot seem to figure out. I want to listen for changes to a table, but only for a while, then I want to stop and do other things.

Looking at the code I found that there is Response.poll() , which should return a status, my plan was to use that and poll for data, getting the data when possible, or otherwise just timeout after enough time had passed, this way i could offload checking the entire table for changes to the data.

So i tried this, and it compiles:

    let r = Client::new();
    // Create a connection pool
    println!("Connecting");
    let conn = r.connect(conf).unwrap();



    let now = Instant::now();

    let mut query = r.db("rethinkdb")
        .table("server_status")
        .changes()
        .run::<Change<ServerStatus, ServerStatus>>(conn).unwrap();

    loop {
        println!("loop");

        match query.poll() {
            Ok(n) => {
                println!("{:?}", n);
            },
            _ => break,
        }
        if now.elapsed().as_secs() > 30 {
            break;
        }
    }

But it crashes with:

thread 'main' panicked at 'no Task is currently running', libcore/option.rs:1010:5

Idea: ACID transactions support

How to create an index?

Hello,

I have spent some time but was unsuccessful in figuring out how to specify arguments to create an index.

The particular one I need to create:

r.db("registry").table("messages").indexCreate("index", [r.row("last_name"), r.row("first_name")])

Appreciate any help:)

RebirthDB

Hey, listen! I don't know if your driver already supports fully the 2.3 version, but we've forked RethinkDB to release a 2.4 version and all drivers will have separate repositories under our organization, discussion here: https://spectrum.chat/rebirthdb

So I'm inviting you to join us, there you can have more visibility for your project, what do you think?

Rust Library Not Documented

Using reql 0.10.4 with Rust trying to figure out to set a primary index on a table and more importantly how to use secondary indices when querying. It's telling me order_by for example only takes a string as an input but the notation on the RethinkDB website is order_by(index: r.desc("column")). I tried using r.args() but I can not get this to work, can someone please post a usable example for an indexed query?

Can not nested command in json object

can not using reql::Command in serde:json! because serde::Serialize behavior

let command = r
  .table("users")
  .insert(json!({
    "id": r.uuid(),    // error
    "age": r.expr(16), // error
    "created": r.now() // error
  }));

this problem also hard to solve because we can not serialize json multiple time in rethinkdb

// original body
{ "tags": ["love", "boy", "hate"] }

// after serialize query
{ "tags": [2, ["love", "boy", "hate"]]}

i also try another approch like using r.object([["key", "value"], ["key2", r.uuid()]]) and r.hashMap is not implemented yet.

any suggestion on this please.

Client.connect throws a runtime error

When attempting to open a connection, I receive the following error ERROR 2018-06-03T21:04:08Z: r2d2: driver

The only place I can see this being thrown is when I call connect on my client struct. I wasn't able to find much information beyond this due to how vague the error is.

How do I do multi document insert?

Here is what I'm trying to accomplish:

  1. If a database[with intended db name] does not exist, create a new db
  2. Do a multi document insert. I'm talking about inserting something like the following:

{ name: "William Adama", tv_show: "Battlestar Galactica",
posts: [
{title: "Decommissioning speech", content: "The Cylon War is long over..."},
{title: "We are at war", content: "Moments ago, this ship received word..."},
{title: "The new Earth", content: "The discoveries of the past few days..."}
]
}

Insertion is a much higher priority.

I'm sorry I couldn't follow along with the documentation on crates.io since I don't know if this is just me, but some of the documentation example is not even in rust. For example:
var p = r.connect({
host: 'localhost',
port: 28015,
db: 'marvel'
});
p.then(function(conn).

Support between queries

Any plans for supporting between queries?

Edit: I just realized all the docs and examples are outdated compared to the current state of the repo. I cannot seem to make heads or tails of how to run queries based on my reading of the code. I will pull it down and see if I can figure out how to do the same queries I had with the one on crates.io. That might help me figure out how the current state works a bit and might be able to find between queries.

Timeout error when performing queries in a timed loop

I'm working with rethinkdb 2.3.6 both in prod and dev, I've got a loop which looks for data every minute. I can see that code work fine on dev but on prod it seems to end up with Driver(GetTimeout(GetTimeout(Some("driver")))) error.

I reduced the time to sleep in that loop to 20 seconds and it seems to work fine. Any ideas what parameter I should pass to connect method so that it does not timeout?

store datetime #question

Hello,
Congratulation about your decision to continue rethinkdb. + Happy the rust crate looks active.

Are there any plans to support datetime?

Thank you,

Changefeeds inevitably overruns their stack

So after some testing where i start a changefeed, the stack will always get overrun.

I am going to guess it is due to fact that the data is never ended, thus, never discarded in the background.

*snip*
    let query = r.db("database")
        .table("changing_table")
        .changes()
        .run::<Change<DataStruct, DataStruct>>( conn ).unwrap();

    let mut wait = query.wait();
    loop {
        let val = wait.next();
        match  val {
 *snip*

This is snippet of the code I am using. Is there any other way to discard data once it is moved out of the iterator?

build fails with musl target

When I run cargo build --target i686-unknown-linux-musl I get the following error

error[E0463]: can't find crate for proc_macro
--> /home/opensourcegeek/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro-hack-0.3.3/src/lib.rs:159:1
|
159 | extern crate proc_macro;
| ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

Seems like proc-macro-hack depends on proc_macro crate - I cannot find it in crates.io, there is only proc-macro2. I'm not sure how the build works fine on gnu target, I'll dig more but thought I'll report here as well.

r.connect() hangs indefinetly

Hi!

I am trying to run the minimal example from the example folders. I have a local rethinkdb running version 2.3.6, started with the --bind all flags. I have compiled the minimal example using rust nightly and it hangs indefinetly after trying to connect to the database

InitializationError when trying to connect

Hey!
I followed the example code and opened a connection to my RethinkDB with my credentials like this:

 let r = Client::new();

 // Create an even loop
 let core = Core::new().unwrap();

 // Create a connection pool
 let conn = r.connect(args!(core.handle(), {host: "host", username: "username", password: "password"})).unwrap();

But when I want to run my program, it panics with the following error

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Driver(Io(Error { repr: Custom(Custom { kind: Other, error: InitializationError(None) }) }))', src\libcore\result.rs:906:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.

The connection to my database and my credentials work.
I am on Windows 10 with rustc 1.21.0 and

futures = "0.1.17"
reql = "0.0.10"
reql-types = "0.0.2"
tokio-core = "0.1.10"

What did I do wrong?
Thanks in advance.

The State of the Library: How Can We Help?

Hey there, I have interest in using this library for at least one project and I was curious what kind of things might be available for helping with to get 0.1.0 released. I can see there are a few issues in the 0.1.0 milestone and I was wondering where we are at with those.

It is very possible that I won't have time to help, but I might get some time so I wanted to get a feel for what the state of the library is right now.

Also, would you advise against using the next branch version? I was curious whether it would be worth trying out the updates in the branch or if it isn't at all usable.

Some information on filter

So I'm trying to write a simple filter to expand the changelog example so that it notes only changes which match our TestItem implementation, however I can't seem to get any filters working:

let query =
        r.db("test")
        .table("test")
        .filter(|doc| {
            doc.has_fields("test").and(doc.test.type_of("NUMBER"))
        })
        .changes()

Gives me:

error[E0619]: the type of this value must be known in this context
 |
 | .filter(|doc| { doc.has_fields("test").and(doc.test.type_of("NUMBER")) })
 |                 ^^^^^^^^^^^^^^^^^^^^^^

error[E0277]: the trait bound `[closure@src/main.rs:55:17: 55:81]: reql::IntoArg` is not satisfied
 |
 |.filter(|doc| { doc.has_fields("test").and(doc.test.type_of("NUMBER")) })
 | ^^^^^^ the trait `reql::IntoArg` is not implemented for `[closure@src/main.rs:55:17: 55:81]`

Any advice on the filter implementation would be appreciated, I'm rather new to Rust at this point.

Try tk-pool

Instead of using r2d2 for our connection pool, we should try tk-pool which integrates directly with Tokio.

Concurrent connections

I'm trying to understand how to use RethinkDB in rust. I'm using RethinkDB to power a GraphQL endpoint (using juniper). Static single document queries seem to not be an issue, but when using Subscriptions in GraphQL (which are commonly WebSocket streams streaming change feeds to the client) I have trouble satisfying more than 1 client. I get driver errors like Driver(ConnectionLocked) and "client error; another query is running a changefeed on this connection". How can I facilitate multiple change feeds?

Currently, I'm trying to open a new session (to the same endpoint) for every client that connects, just to see if that would fix the issue. But this still isn't working correctly. Is it possible to open multiple change feeds on the same session, or do I need to make a new session for every change feed? Is this a limitation of the Rust driver, or RethinkDB itself?

Notes on `slog` usage.

Hi. I'm the author of slog. I sometimes check rev-dependencies on slog and I've noticed that reql is using it.

First of all, thanks for using slog - I've put quite a bit of effort into making it, and I'm happy to see it being used. I hope it will work well for you. Second - I know reql is 0.0.5 and some stuff I might point out you might find obvious, and just made it as shortcuts in reql. Even so - maybe what I wrote will be useful for other people when eg. google will point them here. Third - feel free to ignore anything that you disagree with, I'm just sharing my opinion on how I envisioned things to work, and doing it your own way is fine with me.

Logger is Sync so it does not require wrapping in RwLock. The idea is that Loggers should be worry-free and when creating them you should only be concerned about the "context of logging", and not about your program threading-structure and so on.

Generally libraries should almost never create their own Drains. It's app to application using libraries to setup logging. Only the application can know what are the logging requirements: where, how and what to log. Usually libraries should accept Logger from whatever is using them, potentially falling back to something. Here's the reference example: https://github.com/slog-rs/slog/blob/master/crates/example-lib/lib.rs . In case of reql, I guess corresponding struct to MyLib would be Client. Note that this requires storing Logger in Client and potentially other structs. This is beneficial, as such Loggers can carry the additional data. In reql case, I guess the Client's logger should carry the user, host, db records with respective values. So eg. when someone is using multiple connections, it's clear which message comes from which Client.

It's not slog related but while looking at reql the:

pub const r: command::Client = command::Client;

thing is very non-idiomatic. reql is it's own crate, with it's own namespace. Having a const object, introduces a singleton, which forces locking and other problems. If anything there should be either global static function returning a connection builder, or Client should have a static method, returning a builder, and the user of reql would be responsible for locking etc.

extern create reql;

fn main() {
  let logger = /* ... */
  let reql_db = reql::Client::new(logger).set_db("x").build().expect("couldn't connect to db");
}

I'm not sure if logging errors in error! is a good idea. I mean - it's the data that is being returned to the user of the library and most of the time user is going to log an error themselves. It only makes sense for the library to log things that the user of it is not able to access directly, to allow transparency into what is happening internally.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.