martimm / bson Goto Github PK

See investigation at rakudo/rakudo#3850 (comment) for details.

When floats/doubles are serialized to JSON then converted to Perl numbers, they're re-interpreted as Rat. Rat can't be handled by the database and gives an error.

Suggested solution: if Rat can be converted to Num without loss of data, do it. This will let BSON work with data that has been serialized without truncating data. (Verify that $n.Num.Rat == $n first.)

It would also be nice if BSON had an option that enabled truncating all floating point types to a type that is allowed in the database, even if some precision is lost. Perhaps $doc.accept-numbers, analogous to accept-hash which permits order information loss.

#23

bson type 0x11 for timestamps. It was assumed that it was for internal use but newer servers are returning a timestamp field in the ismaster now.

Need to be more careful when using threads for encoding and decoding documents. In some project of mine, the process runs into a lock waiting for a promise to finish. An answer from Jnthn at SO. The quick fix mentioned didn't help (yet).

To see the problem, run the following script. If the server is running on localhost, a date created by DateTime.new(now) should be almost the same as a time created by the MongoDB $currentDate operator. This script prints the difference. The desired result should be less than 1 second. The actual result is thousands of years.

#! env perl6

use BSON::Document;
use MongoDB::Client;

my MongoDB::Client $client .= new(:uri("mongodb://127.0.0.1"));
my $db = $client.database('test');

my DateTime $startTime;

sub record-time() {
	$startTime = DateTime.new(now);
	my BSON::Document $req .= new: (
		update => 'timetest',
		updates => [ (
			q => (name => 'time-test',),
			u => ('$currentDate' => (lastModified => True),),
			upsert => True,
		  ),
		],
	);
	my $response = $db.run-command($req);
	say $response;
}

sub check-time {
	my $response = $db.collection('timetest').find(
		:number-to-return(1),
		:criteria(name => 'time-test', lastModified => ('$exists' => True),),
	).fetch;

	say 'Time between request and DB action: '~($response<lastModified>-$startTime);
}

record-time;
check-time;

Replace https://github.com/bbkr/BSON/blob/0d8721d60ca54cc36e3e4e2cd14e2e4798d7542e/lib/BSON.pm#L247-L256

with native pack/unpack methods. It is possible right now but results in performance drop.

After upgrade of perl I have suddenly problems encoding and decoding the Num type to/from BSON Double precision.

Funny enough there was also a bug in the test program and never surfaced until now.

Installed but complained on OSX.

$ panda install BSON
==> Fetching BSON
==> Building BSON
Compiling lib/BSON/Exception.pm6 to mbc
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
Compiling lib/BSON/ObjectId.pm to mbc
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
Compiling lib/BSON/Regex.pm to mbc
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
Compiling lib/BSON/Javascript.pm to mbc
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
Compiling lib/BSON/Binary.pm to mbc
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
Compiling lib/BSON.pm to mbc
===SORRY!===
Missing or wrong version of dependency '&encode_int32' (from 'lib/BSON/Binary.pm')
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
Compiling lib/BSON/EDC-Tools.pm6 to mbc
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Builder.pm:125
==> Testing BSON
t/103-string.t .... ok
t/102-int.t ....... ok
t/101-binary.t .... ok
t/100-double.t .... ok
t/600-extended.t .. ok
t/500-native.t .... ok
All tests successful.
Files=6, Tests=113,  7 wallclock secs ( 0.07 usr  0.02 sys + 17.73 cusr  0.66 csys = 18.48 CPU)
Result: PASS
Use of Nil in numeric context  in block  at /Users/gabor/rakudo-star-2015.06/install/share/perl6/lib/Panda/Tester.pm:36
==> Installing BSON
...
==> Successfully installed BSON

Hi

perl6 -e 'use BSON; BSON.new;'
You cannot create an instance of this type
  in block <unit> at -e:1

Previously BSON was Role and it could be used as standalone object instance.
Now it is a package, but documentation still says in SYNOPSIS that BSON.new is valid usage.

Rakudo now has built-in methods that duplicate custom decodings in BSON code:
https://docs.perl6.org/type/Blob#Methods_on_blob8_only_(6.d,_2018.12_and_later)

They are really fast, so maybe it's worth switching to those methods?
Current methods can be left as fallback until most Rakudo builds (Star, linux, macOS) are at least 2019.01 (there were some issues with endian flags fixed after 2018.12).

In function signatures, sub-elements of objects can be mapped to variables:

sub print-id(% (:_id($id))) { say $id; }
print-id $(_id => 4);
# output: 4
my %doc := BSON::Document.new: (_id => 4,)
print-id %doc;
# output: (Any)

As you can see, this fails. Since documents are used like Hash, they should implement this feature as well. I'll make a pull request that implements this shortly.

Thinking about undefined, it is deprecated in BSON. I can code this in some way or another or leave the key-value out all together. When back in perl it will just show the value as (Any) (a typed undefined value) no matter if the key was in the document or not. So in the last case it will save space and time leaving the pair out.

BSON code has problems with small (e.g. 1E-100) doubles.

in test 300-document.t it seems that a bug is found when handling a very large document.

Whole parallelization can be implemented in e_list type (hashes are also treated as arrays).

multi method _e_list ( @p ) {

    return Buf.new unless @p;
    return [~]( @p>>.map: { self._element( $_ ) } );
}

So far in Rakudo:

• hyperoperator dispatch is in place but no true parallel processing happens, that gives about 10% performance drop compared to serial processing
• ~ reduce bug for Buf occurs

Once those issues are resolved parallelization can be implemented.

When I install this module from panda I get this output with rakudo star 2014.12.1:
https://gist.github.com/isleofmax/772fea4af9d33ef66354

Current BSON implementation is pushed as far as Perl6 Rakudo allows.

• once we get pack/unpack in Rakudo we can implement 8 bytes (64-bit IEEE 754 floating point) and push such MongoDB features like authentication

Hi, your repository contains instances of old is_approx testing function. That function has been deprecated for several months and will be removed entirely from Rakudo in December (tentatively, Dec. 17).

Please upgrade to the new is-approx. If you did not specify custom tolerance argument, simply renaming from is_approx to is-approx is sufficient.

Faild test in Rakudo Start 2012.08:

t/native.t .... 1/18 No such method 'bytes' for invocant of type 'Str'
in method _string at lib/BSON.pm:281
in block at lib/BSON.pm:85
in method _element at lib/BSON.pm:72
in method _e_list at lib/BSON.pm:54
in method _document at lib/BSON.pm:24
in method encode at lib/BSON.pm:8
in block at t/native.t:60