Comments (30)
vincentdchan
commented on May 18, 2024
2
Use newer version will be more safe:
polodb_core = "0.10.2"
polodb_bson = "0.10.2"
from polodb.
vincentdchan
commented on May 18, 2024
1
You need to specific the update operation, so try:
collection
.update(
Some(&mk_document! { "_key": key_name }),
&mk_document! {
"$set": &mk_document! {
"value": value
}
},
)
Other operation can be found at docs/en-US/Update.md
from polodb.
peepo5
commented on May 18, 2024
1
I will soon.
from polodb.
vincentdchan
commented on May 18, 2024
1
I've reproduced successfully, I'll see what happened.
from polodb.
peepo5
commented on May 18, 2024
1
Thank you very much. I will impliment this later. You may also want to add a warning to the readme to not access a database in this way to prevent corruption.
from polodb.
vincentdchan
commented on May 18, 2024
1
It's about database initiation.
from polodb.
peepo5
commented on May 18, 2024
error[E0277]: the trait bound `Value: From<&polodb_bson::Document>` is not satisfied
--> src/db.rs:140:6
|
140 | &mk_document! {
| __________________^
141 | | "$set": &mk_document! {
142 | | "value": value
143 | | }
144 | | },
| |_________________^ the trait `From<&polodb_bson::Document>` is not implemented for `Value`
This is what I get after using identical code as yours.
from polodb.
vincentdchan
commented on May 18, 2024
A little mistake:
collection
.update(
Some(&mk_document! { "_key": key_name }),
&mk_document! {
"$set": mk_document! { // <- do NOT need '&'
"value": value
}
},
)
from polodb.
peepo5
commented on May 18, 2024
π testing now
from polodb.
peepo5
commented on May 18, 2024
Alright that works! As I said in my other issue, It would be nice to have some examples, even just copy paste code to get a better understanding of this module in use. Thanks for the help.
from polodb.
peepo5
commented on May 18, 2024
I compared the database values before and after update and it seems nothing changed. Very weird.
I ran my program twice at once, checking the db while in function.
Before update:
[
Trade {
...
main_id: 88877584825,
sl_id: None,
tp_id: None,
},
]After update (nothing changed):
[
Trade {
...
main_id: 88877584825,
sl_id: None,
tp_id: None,
},
]After function finished:
[
Trade {
_id: Some(
310723,
),
timestamp_open: 1634630287,
filled: true,
risk: 1,
main_id: 88877584825,
sl_id: Some(
90149687,
),
tp_id: Some(
90140261,
),
},
]So this seems working.
Then after however, I tried to insert another entry into the database:
thread 'main' panicked at 'index 1 is greater than length 1', /home/admin/.cargo/registry/src/github.com-1ecc6299db9ec823/polodb_core-0.10.0/page/data_page_wrapper.rs:77:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtraceTo insert, I am using the following function:
collection
.insert(&mut mk_document! {
"_id": id,
"timestamp_open": td.timestamp_open.to_string(),
"filled": td.filled,
"risk": td.risk.to_string(),
"main_id": td.main_id,
"sl_id": if td.sl_id == None {0} else {td.sl_id.unwrap()},
"tp_id": if td.tp_id == None {0} else {td.tp_id.unwrap()},
})
.unwrap();Do I need to specify the insert type to avoid this?
Also on another try, for some reason, when using collection.find_all(...), it gave me an empty result (when it shouldnt) and then it gave me a similar error.
"testa" command runs this function:
pub fn db_get_ftrades() -> Result<Vec<Trade>, Error> {
let mut db = Database::open(database_location().as_str()).unwrap();
let mut collection = db.collection("ftrades").unwrap();
let all_trades = collection.find_all().unwrap();
let mut trade_array: Vec<Trade> = Vec::with_capacity(all_trades.len());
for doc in all_trades {
let sl_id = doc.get("sl_id").unwrap().unwrap_int();
let tp_id = doc.get("tp_id").unwrap().unwrap_int();
trade_array.push(
Trade {
_id: Some(doc.get("_id").unwrap().unwrap_int().to_string().parse::<Decimal>()?),
timestamp_open: /*DateTime::parse_from_str(*/doc.get("timestamp_open").unwrap().unwrap_string().parse::<Decimal>()?/*, "%s")?*/,
filled: doc.get("filled").unwrap().unwrap_boolean(),
risk: doc.get("risk").unwrap().unwrap_string().parse::<Decimal>()?,
main_id: doc.get("main_id").unwrap().unwrap_int() as u64,
sl_id: if sl_id == 0 {None} else {Some(sl_id as u64)},
tp_id: if tp_id == 0 {None} else {Some(tp_id as u64)},
}
);
}
Ok(trade_array)
}It errors on the db.collection("ftrades").unwrap() for reference
from polodb.
vincentdchan
commented on May 18, 2024
This error demonstrates that your db has been corrupted. Could you give me a minimal program or procedure descriptions to reproduceοΌ
from polodb.
peepo5
commented on May 18, 2024
Just a question, do you need to update the database using a command before using the database again in the same function?
from polodb.
vincentdchan
commented on May 18, 2024
Are you saying using collection again or using database again?
It's designed to be allow using collection for multiple times.
Database is designed to be used multiple times too, but limited. Only one instance can read/write database at the same time. Otherwise, the thread will be locked.
from polodb.
vincentdchan
commented on May 18, 2024
But corruption is unexpected, the database should be safe.
from polodb.
peepo5
commented on May 18, 2024
https://github.com/termcrypt/termcrypt/blob/master/src/ftx_exchange/ftx_inter.rs (line 247-576)
https://github.com/termcrypt/termcrypt/blob/master/src/db.rs
These are the relevant files.
I managed to replicate the issue on a test project.
cargo init then change main.rs:
//this is a temporary file to test databases within termcrypt.
use anyhow::{
Error,
//bail
Result,
};
use chrono::{Local};
use polodb_bson::mk_document;
use polodb_core::Database;
use rand::Rng;
use rust_decimal::prelude::*;
use rust_decimal_macros::dec;
use rustyline::Editor;
#[derive(Debug)]
pub struct Trade {
pub _id: Option<Decimal>,
pub timestamp_open: Decimal,
pub filled: bool,
pub risk: Decimal,
pub main_id: u64,
pub sl_id: Option<u64>,
pub tp_id: Option<u64>,
}
fn main() {
let mut line_main = Editor::<()>::new();
loop {
let read_line = line_main.readline("> ");
match read_line.unwrap().as_str() {
"o" => {
new_order().unwrap();
}
"g" => {
println!("{:#?}", db_get_ftrades().unwrap())
}
_ => {}
}
println!();
}
}
pub fn database_location() -> String {
format!("database")
}
pub fn new_order() -> Result<(), Error> {
let mut db = Database::open(database_location().as_str()).unwrap();
let mut collection = db.collection("ftrades").unwrap();
let mut line_main = Editor::<()>::new();
let main_db_id = db_insert_ftrade(Trade {
_id: None,
timestamp_open: Local::now().timestamp().to_string().parse::<Decimal>()?,
filled: false,
risk: dec!(1),
main_id: rand::thread_rng().gen_range(10..9999999), //random number placeholder for real id
sl_id: None,
tp_id: None,
})
.unwrap();
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"sl_id": rand::thread_rng().gen_range(10..9999999)
}
},
)
.unwrap();
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"tp_id": rand::thread_rng().gen_range(10..9999999)
}
},
)
.unwrap();
println!("done");
Ok(())
}
pub fn db_insert_ftrade(td: Trade) -> Result<i64, Error> {
let mut db = Database::open(database_location().as_str()).unwrap();
let mut collection = db.collection("ftrades").unwrap();
let all_trades = collection.find_all().unwrap();
let mut id: i64;
//makes sure id is not already in ftrades
loop {
id = rand::thread_rng().gen_range(10..999999);
let mut failed = false;
for trade in &all_trades {
if id == trade.get("_id").unwrap().unwrap_int() {
failed = true
}
}
if !failed {
break;
}
}
collection
.insert(&mut mk_document! {
"_id": id,
"timestamp_open": td.timestamp_open.to_string(),
"filled": td.filled,
"risk": td.risk.to_string(),
"main_id": td.main_id,
"sl_id": if td.sl_id == None {0} else {td.sl_id.unwrap()},
"tp_id": if td.tp_id == None {0} else {td.tp_id.unwrap()},
})
.unwrap();
Ok(id)
}
pub fn db_get_ftrades() -> Result<Vec<Trade>, Error> {
let mut db = Database::open(database_location().as_str()).unwrap();
let mut collection = db.collection("ftrades").unwrap();
let all_trades = collection.find_all().unwrap();
let mut trade_array: Vec<Trade> = Vec::with_capacity(all_trades.len());
for doc in all_trades {
let sl_id = doc.get("sl_id").unwrap().unwrap_int();
let tp_id = doc.get("tp_id").unwrap().unwrap_int();
trade_array.push(
Trade {
_id: Some(doc.get("_id").unwrap().unwrap_int().to_string().parse::<Decimal>()?),
timestamp_open: /*DateTime::parse_from_str(*/doc.get("timestamp_open").unwrap().unwrap_string().parse::<Decimal>()?/*, "%s")?*/,
filled: doc.get("filled").unwrap().unwrap_boolean(),
risk: doc.get("risk").unwrap().unwrap_string().parse::<Decimal>()?,
main_id: doc.get("main_id").unwrap().unwrap_int() as u64,
sl_id: if sl_id == 0 {None} else {Some(sl_id as u64)},
tp_id: if tp_id == 0 {None} else {Some(tp_id as u64)},
}
);
}
Ok(trade_array)
}then change Cargo.toml:
[dependencies]
polodb_core = "0.10.0"
polodb_bson = "0.10.0"
rand = "0.8.4"
anyhow = "1.0.44"
rust_decimal = "1.16"
rust_decimal_macros = "1.16"
rustyline = "9.0.0"
chrono = "0.4"
dirs = "4.0"then test like how i did in the image. I think the database works fine without the collection.update btw you can test removing both of the updates.
from polodb.
vincentdchan
commented on May 18, 2024
Get it. I will try your program.
from polodb.
peepo5
commented on May 18, 2024
Are you able to reproduce the issue?
from polodb.
vincentdchan
commented on May 18, 2024
Not yet. Sorry I am busy these days, I will try tomorrow.
from polodb.
peepo5
commented on May 18, 2024
π thank you.
from polodb.
vincentdchan
commented on May 18, 2024
What toolchain are you using? I can not compile your proj with stable Rust. @peepopoggers
from polodb.
peepo5
commented on May 18, 2024
1.55
from polodb.
vincentdchan
commented on May 18, 2024
stable/nightly/beta ?
from polodb.
peepo5
commented on May 18, 2024
stable.
try rustup default 1.55.0 or something
from polodb.
peepo5
commented on May 18, 2024
Hi there. I do not want to rush you in any way, but I would like to know if this is a problem with my code's logic or your module. This problem is stunting development for my app and I may have to consider moving database modules if there is no answer soon.
from polodb.
vincentdchan
commented on May 18, 2024
I've located the problem. It seems that you open two DB instances of one file in the same program space, the key is in the function new_order, reorder the codes, and the problem will be fixed:
pub fn new_order() -> Result<(), Error> {
// insert data here, and close the database
let main_db_id = db_insert_ftrade(Trade {
_id: None,
timestamp_open: Local::now().timestamp().to_string().parse::<Decimal>()?,
filled: false,
risk: dec!(1),
main_id: rand::thread_rng().gen_range(10..9999999), //random number placeholder for real id
sl_id: None,
tp_id: None,
})
.unwrap();
// reopen the database, so not conflicts
let mut db = Database::open(database_location().as_str()).unwrap();
let mut collection = db.collection("ftrades").unwrap();
let mut line_main = Editor::<()>::new();
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"sl_id": rand::thread_rng().gen_range(10..9999999)
}
},
)
.unwrap();
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"tp_id": rand::thread_rng().gen_range(10..9999999)
}
},
)
.unwrap();
println!("done");
Ok(())
}from polodb.
vincentdchan
commented on May 18, 2024
Also, the deeper reason to this bug is that, two instances contain different caches of data. So PoloDB itself should have a mechanism to avoid cache conflicts. I will consider some method to avoid this problem. Thanks for your feedback.
from polodb.
vincentdchan
commented on May 18, 2024
Also, I've read your code. You can update an item in one statement:
Original:
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"sl_id": rand::thread_rng().gen_range(10..9999999)
}
},
)
.unwrap();
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"tp_id": rand::thread_rng().gen_range(10..9999999)
}
},
)
.unwrap();After:
line_main.readline("Useless pause. Press enter.")?;
collection
.update(
Some(&mk_document! { "_id": main_db_id }),
&mk_document! {
"$set": mk_document! {
"sl_id": rand::thread_rng().gen_range(10..9999999),
"tp_id": rand::thread_rng().gen_range(10..9999999),
}
},
)
.unwrap();
line_main.readline("Useless pause. Press enter.")?;This is not neccessary, but better code.
from polodb.
peepo5
commented on May 18, 2024
Just out of curiosity, is it an issue with database initiation or collection initiation?
from polodb.
peepo5
commented on May 18, 2024
I have seen the warning for busy database while using it now, that is a great addition.
from polodb.
Related Issues (20)
- Is the "$regex" search operator implemented ? HOT 4
- New Python binding HOT 2
- Feat: aggregation
- Feat: IndexedDB backend
- Doc: help adding content to "Advanced condition" section HOT 2
- Feat: iOS support
- Feat: Android support
- Any plan to support Node.js? HOT 2
- No Database::open_file in wasm HOT 2
- Deletions not persisting to disk HOT 5
- Add `update_or_insert` operation
- Any plans for querying the data with pagination enabled? HOT 1
- Query by enum type HOT 1
- The Regex example in the official documentation does not compile HOT 1
- Null is used as _id when Option<ObjectId> is None.
- Database easily corrupted HOT 2
- Will implement Clone and Debug trait for Database struct like mongodb crate? HOT 1
- Crash and corrupt data base HOT 1
- Aggression or Aggregation? HOT 1
- find_many_with_session crash. HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from polodb.